Since January 2005 (Round
6), I have been actively participating in Critical
Assessment of PRedicted Interactions (CAPRI), a community-wide experiment that tests protein-docking methods
(i.e., methods for modeling protein interactions) in blind
predictions.
As
a key member of the Vajda Team, I had been continuously developing and
applying SDU, a protein docking method that explores and exploits the
binding free energy landscape for its global optimization (see the
corresponding section in Research), with
considerable success. Our team ranked
No. 5 among 63 worldwide predictor teams in the 3rd assessment (Rounds 6–12, 2005–2007), and then No. 1 among
64 teams in the 4th assessment (Rounds 13–19, 2007–2009).
I have led my own team since May 2010 (Round 22) and
continued on developing protein-docking methods. In the latest assessed Rounds 22–27 right after the 4th assessment (rounds 20 and 21 were only for scorers instead of
predictors, so were targets 55 and 56), my team tied at No. 5 among 63 teams. (Last
updated on May 10, 2013 according to official summary by Dr. Marc Lensink at the 5th CAPRI Evaluation Meeting.)
Detailed assessments and target information in three rounds
are provided in reverse chronological order as follows.
We thank members of the community for many useful comments and feedbacks.
5th
Assessment, 2010–2013 (Leader of the Shen
Team)
Please be aware that some teams did not participate in all rounds. For instance, the newly-developed SwarmDock server just participated for the last four targets and was the only team that produced at least an acceptable prediction for each of the four. Therefore, please follow a Special Issue of Proteins (December 2013) where teams have more exciting science stories to share than their performance!
Rank |
Group |
T46 |
T47 (regular [water]) |
T48 |
T48 (Trimer) |
T49 |
T49 (Trimer) |
T50 |
T51.1 |
T51.2 |
T51.3 |
T53 |
T54 |
T57 (protein-polysaccharide) |
T58 (with SAXS data) |
Summary: #Targets / *** + **
+ * |
1 |
Bonvin |
* |
*** [**] |
|
* |
|
* |
** |
* |
|
|
** |
|
** |
* |
9 / 1 ***
+ 3 ** + 5 * |
|
2 |
Bates |
|
** [**] |
* |
|
* |
|
* |
|
* |
|
* |
|
* |
** |
8 / 2 ** + 6 * |
|
3 |
Vakser |
|
*** [**] |
* |
* |
* |
* |
* |
|
|
|
|
* |
* |
* |
7 / 1 ***
+ 6 * |
|
4 |
Vajda |
|
*** [**] |
|
** |
|
* |
** |
|
|
|
*** |
|
** |
|
6 / 2 *** + 3 ** + 1 * |
|
5 |
Fernandez-Recio |
|
*** [*] |
|
* |
|
* |
** |
|
|
|
** |
|
|
** |
6 / 1 ***
+ 3 ** + 2 * |
|
5 |
Shen |
|
*** [*] |
** |
** |
** |
** |
* |
|
|
|
** |
* |
|
|
6 / 1 *** + 3 ** + 2 * |
|
|
|
|
|
|
7 |
Zou |
|
*** [***] |
** |
* |
* |
* |
* |
|
|
|
|
|
** |
* |
6 / 1 ***
+ 2 ** + 3 * |
|
8 |
Zacharias |
|
*** [***] |
|
* |
|
* |
* |
|
|
|
* |
|
* |
|
6 / 1 *** + 5 * |
|
9 |
ClusPro |
|
** [ ] |
|
** |
|
* |
** |
|
|
|
** |
|
* |
|
6 / 4 **
+ 2 * |
|
10 |
Eisenstein |
|
*** [**] |
|
** |
* |
* |
** |
|
|
|
* |
|
|
|
5 / 1 *** + 2 ** + 2 * |
|
10 |
Grudinin |
|
*** [**] |
|
|
|
|
** |
|
|
|
* |
|
* |
** |
5 / 1 ***
+ 2 ** + 2 * |
|
12 |
Gray |
|
*** [**] |
|
|
|
|
|
|
|
|
* |
|
* |
** |
4 / 1 *** + 1 ** + 2 * |
|
12 |
HADDOCK |
* |
*** [**] |
|
|
|
* |
|
|
|
|
|
|
** |
|
4 / 1 ***
+ 1 ** + 2 * |
|
12 |
Seok |
|
*** [**] |
|
|
|
|
|
|
|
|
** |
|
* |
* |
4 / 1 *** + 1 ** + 2 * |
|
12 |
Weng |
|
*** [*] |
|
|
* |
* |
* |
|
|
|
** |
|
|
|
4 / 1 ***
+ 1 ** + 2 * |
|
16 |
Nakamura |
|
*** [***] |
|
|
|
|
|
* |
|
|
* |
* |
|
|
4 / 1 *** + 3 * |
|
17 |
SwarmDock |
|
|
|
|
|
|
|
|
|
|
* |
* |
* |
** |
4 / 1 **
+ 3 * |
|
18 |
Wolfson |
|
** [*] |
* |
** |
* |
* |
|
|
|
|
|
|
|
|
3 / 2 ** + 1 * |
|
19 |
PIE/DOCK |
|
|
|
* |
|
* |
** |
|
|
|
|
|
|
|
3 / 1 **
+ 2 * |
|
19 |
Zhou |
|
** [*] |
* |
* |
|
* |
|
|
|
|
|
|
|
|
3 / 1 ** + 2 * |
|
21 |
Cui |
|
*** [ ] |
* |
** |
|
|
|
|
|
|
|
|
|
|
2 / 1 ***
+ 1 ** |
|
22 |
HexServer |
|
*** [ ] |
|
|
|
|
|
|
|
|
|
|
|
* |
2 / 1 *** + 1 * |
|
22 |
Luethy |
|
*** [ ] |
|
|
|
|
* |
|
|
|
|
|
|
|
2 / 1 ***
+ 1 * |
|
24 |
Elber |
|
|
|
* |
|
|
** |
|
|
|
|
|
|
|
2 / 1 ** + 1 * |
|
24 |
Fernandez-Fuentes |
|
|
|
|
|
|
** |
|
|
|
* |
|
|
|
2 / 1 **
+ 1 * |
|
24 |
Mitchell |
|
** [ ] |
|
|
|
|
|
|
|
|
|
|
* |
|
2 / 1 ** + 1 * |
|
27 |
FibreDock |
|
*** [ ] |
|
|
|
|
|
|
|
|
|
|
|
|
1 / 1 *** |
|
27 |
Ritchie |
|
*** [** ] |
|
|
|
|
|
|
|
|
|
|
|
|
1 / 1 *** |
|
27 |
Wang |
|
*** [**] |
|
|
|
|
|
|
|
|
|
|
|
|
1 / 1 *** |
|
30 |
Camacho |
|
|
|
|
|
|
** |
|
|
|
|
|
|
|
1 / 1 ** |
|
30 |
LZerD |
|
|
|
|
|
|
|
|
|
|
** |
|
|
|
1 / 1 ** |
|
30 |
Ten Eyck |
|
|
|
|
|
|
|
|
|
|
** |
|
|
|
1 / 1 ** |
|
33 |
Dokholyan |
|
|
|
|
|
|
|
|
|
|
|
|
* |
|
1 / 1 * |
|
33 |
Kihara |
|
|
|
|
|
|
|
|
|
|
|
|
|
* |
1 / 1 * |
|
33 |
Pal |
|
|
|
|
|
|
* |
|
|
|
|
|
|
|
1 / 1 * |
|
33 |
Poupon |
|
|
|
|
|
|
|
|
|
|
* |
|
|
|
1 / 1 * |
|
33 |
SurFit |
|
|
|
|
|
|
|
|
|
|
* |
|
|
|
1 / 1 * |
|
33 |
Zhang |
|
|
|
|
|
|
|
|
|
|
* |
|
|
|
1 / 1 * |
|
39 |
25 Others |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0 |
|
Notes:
- All assessments are
official results according to the CAPRI website.
Tied teams are given the same rank and alphabetically
ordered.
- For
all targets but T47, predictions are classified as *
(acceptable), ** (medium), and *** (high). Blank space means that
no acceptable predictions were submitted.
- The only, slight exception in
classifying predictions was for T47, where the real challenge is the
prediction of water-mediated interactions
between a given protein sequence and an unbound protein. Here, the
classification is * (fair), ** (good), *** (excellent), and ****
(outstanding).
- T48 and T49 both involve binding of
hydroxylase and ferredoxin. The hydroxylase is a hetero-hexamer
but there was some confusion among participants that led to some
trimer-form predictions. Therefore, the assessment team decided
to evaluate predictions in both forms. Accordingly, the best
model from each team for T47/T48 was reported here, in either hexamer
or timer form,
as previously treated in targets 40 and 42 in the 4th assessment (see
below) where two possible binding modes exist.
- T51 was a multi-domain protein
GH5-CBM6-CBM13-Fn3-CBM62 among which CBM13 had to be
homology-built. Its
assessment was performed for three pairwise interfaces.
- T52 was canceled due to available information from the internet.
4th Assessment, 2009
(Key Member of the Vajda Team)
(SDU) indicates that SDU directly
contributed to our (best) prediction.
|
|
|
|
|
|
|
|
|
Target |
Complex |
PDB
ID |
Bio
Function |
Type
of Modeling |
Our
Prediction |
Best Prediction from
the Community (# of such teams / # of all) |
|
T29 |
Trm8/Trm82 |
2VDU |
RNA
Maturation |
Homology-Bound |
Incorrect |
Medium
(4/39) |
|
T30 |
Rnd1/RBD
of plexin B1 |
2REX |
Signal
Transduction |
Unbound-Unbound |
Incorrect |
Acceptable
(2/35) |
|
T32 |
Savinase/BASI |
3BX1 |
RNA
Maturation |
Unbound-Unbound |
High
(Medium from SDU) |
High
(5/36) |
|
T33 |
Enzyme/RNA |
|
RNA
Maturation |
Homology-Homology |
Incorrect |
Incorrect |
|
T34 |
Enzyme/RNA |
|
RNA
Maturation |
Homology-Bound |
Medium
(SDU) |
Medium
(8/28) |
|
T35 |
CBM22/GH10 |
2W5F |
Domains
of Xylanase Xyn10B |
Homology-Homology |
Incorrect |
Incorrect |
|
T36 |
CBM22/GH10 |
2W5F |
Domains
of Xylanase Xyn10B |
Bound-Homology |
Incorrect |
Acceptable
(1/32) |
|
T37 |
Arf6/LZ2
of JIP4 |
2W83 |
Membrane
Trafficking |
Unbound-Homology |
Incorrect |
High(1/39) |
|
T38 |
Centaurin-α1/KIF13B
FHA |
3FM8 |
Membrane
Trafficking |
Unbound-Homology |
Incorrect |
Incorrect |
|
T39 |
Centaurin-α1/KIF13B
FHA |
3FM8 |
Membrane
Trafficking |
Unbound-Bound |
High |
High
(1/37) |
|
T40.1 |
Trypsin/API-A |
3E8L
(C:A) |
Double-headed
Protease Inhibitor |
Unbound-Bound |
High
(SDU) |
High
(18/38) |
|
T40.2 |
Trypsin/API-A |
3E8L
(C:B) |
Double-headed
Protease Inhibitor |
Unbound-Bound |
High
(Medium from SDU) |
High
(20/38) |
|
T41 |
Colicin
E9 DNase/Im2 |
2WPT |
|
Unbound-Unbound |
Medium
(SDU) |
High
(8/) |
|
T42.1 |
|
(A:B) |
|
|
High
(not refined by SDU) |
High
(6/) |
|
T42.2 |
|
(B:C) |
|
|
Incorrect |
High
(2/) |
|
Overall |
|
|
|
|
4
*** + 2 ** (1st/64) |
4
*** + 2 ** (Vajda Team, 1st/64) |
Notes: T31 was not evaluated due to the difficulty of experimentally
determining the complex structure with NMR.
References:
-
The
Targets of CAPRI Rounds 13–19.
J Janin. Proteins: Structure,
Function, and Bioinformatics. Volume 78, Issue 15, pages 3067–3072, November 2010.
- Docking
and Scoring Protein Interactions: CAPRI 2009. MF Lensink and SJ
Wodak.
Proteins: Structure,
Function, and Bioinformatics. Volume 78, Issue 15, pages 3073–3084, November 2010.
- Achieving
Reliability and High Accuracy in Automated Protein Docking: ClusPro,
PIPER, SDU, and Stability Analysis in CAPRI Rounds 13–19.
D Kozakov, DR Hall, D Beglov, R Brenke, SR Comeau, Y Shen et al. Proteins: Structure,
Function, and Bioinformatics. Volume 78, Issue 15, pages 3124–3130, November 2010.
3rd Assessment, 2007
(Key Member of the Vajda Team)
(SDU) indicates that SDU directly
contributed to our (best) prediction.
|
|
|
|
|
|
|
|
|
Target |
Complex |
PDB
ID |
Bio
Function |
Type
of Modeling |
Our
Prediction |
Best Prediction from
the Community (# of such teams / # of all) |
|
T20 |
HemK/RF1 |
2B3T |
Translation
Termination |
Unbound-Homology |
Incorrect |
Acceptable
(1/29 ) |
|
T21 |
Orc1/Sir1 |
1ZHI |
Transcription
regulation |
Unbound-Unbound |
Acceptable
(SDU) |
Medium
(3/37) |
|
T24 |
Arf1/ARHGAP21 |
2J59 |
Signal
Transduction |
Unbound-Homology |
Incorrect |
Acceptable
(3/37) |
|
T25 |
Arf1/ARHGAP21 |
2J59 |
Signal
Transduction |
Unbound-Bound |
Medium
(SDU) |
High
(1/37) |
|
T26 |
TolB/Pal |
2HQS |
Membrane
Transport |
Unbound-Unbound |
Medium
(SDU) |
Medium
(8/37) |
|
T27.1 |
Hip2/UBC9 |
2O25
(A:D) |
Protein
Processing |
Unbound-Unbound |
Incorrect |
Incorrect |
|
T27.2 |
Hip2/UBC9 |
2O25
(A:C) |
Protein
Processing |
Unbound-Unbound |
Incorrect |
Medium
(2/38) |
|
T28 |
NEDD4L
Dimer |
2ONI |
Protein
Processing |
Unbound-Unbound |
Incorrect |
Incorrect |
|
Overall |
|
|
|
|
2
** + 1 * (5th/63) |
3
** + 2 * (Weng Team, 1st/63) |
Notes: T22 and T23 in Round 8 were cancelled by the organizers due to
leaked experimental information on the internet.
References:
-
The
Targets of CAPRI Rounds 6–12.
J Janin. Proteins: Structure,
Function, and Bioinformatics. Volume 69, Issue 4, pages 699–703, December 2007.
- Docking
and Scoring Protein Complexes: CAPRI 3rd Edition. MF Lensink, R
Mendez, and SJ Wodak.
Proteins: Structure,
Function, and Bioinformatics. Volume 69, Issue 4, pages 704–718, December 2007.
- Docking
with PIPER and Refinement with SDU in Rounds 6–11
of CAPRI. Y Shen, R Brenke, D Kozakov, SR Comeau, D Beglov, and S
Vajda. Proteins: Structure,
Function, and Bioinformatics. Volume 69, Issue 4, pages 704–718, December 2007.
|