Advances in GPCR modeling evaluated by the GPCR Dock 2013 assessment: meeting new challenges. Structure, in press.

Irina Kufareva1, Vsevolod Katritch2, GPCR Dock 2013 participants, Raymond C. Stevens2, Ruben Abagyan1.

1 UCSD Skaggs School of Pharmacy and Pharmaceutical Sciences, La Jolla, CA, 92039, USA.
2 Department of Molecular Biology, The Scripps Research Institute, La Jolla, CA, 92037, USA.


The work was partially supported by NIH grants R01 GM 071872 and U01 GM094612 to RA and U54 GM094618 to RCS.


The community-wide GPCR Dock assessment was established to stimulate and monitor the progress in molecular modeling and ligand docking for GPCRs. The four targets in the present third assessment round presented new and diverse challenges for modelers, including prediction of allosteric ligand interaction and activation states in 5-hydroxytryptamine receptors 1B and 2B, and modeling by extremely distant homology for smoothened receptor. 44 modeling groups participated in the assessment. The results for the four target complexes are shown as independent assesments.

Results tables

This section contains a short description of the parameters displayed in the columns in the table of results. Please, use the manuscript for a more comprehensive description.

Group ID: We referred to participating groups by the abbreviated name of their institution rather than by names. Because some institutions host more than one group, we developed a group nomenclature in which the institution name is followed by the 4-digit group ID assigned on registration.

Model: The rank of the model according to authors (from 1 to 5)

Mean alignment shift error: Each model was superimposed with the TM domains of all GPCR X-ray structures (templates) available in the PDB at the time of the assessment using a sequence-independent structural alignment algorithm in ICM. The structure with the highest superposition quality was chosen and a sequence correspondence (alignment) between the model sequence and the TM domain of the template structure was derived from the superposition. This correspondence was compared to the “correct” alignment of the target to the template in the TM region. Alignment errors in the model were calculated as sequence shifts in each TM helix separately and then averaged over the superimposable part of the TM domain. Models for which no unambiguous structural alignment to the PDB template could be established were excluded from this analysis.

TM backbone RMSD: The protein molecule of each model was superimposed onto the backbone CA, C, and N atoms of the TM helices of the target structure. TM regions were defined by residue stretches 1.30-1.60, 2.37-2.64, 3.22-3.54, 4.38-4.64, 5.35-5.64, 6.28-6.58, and 7.32-7.55 in Ballesteros-Weinstein notation (in this notation, a single most conserved residue among the class A GPCRs is designated x.50, where x is the TM helix number; all other residues on that helix are numbered relative to this conserved position. Superimposition was performed using an adaptive algorithm that iteratively finds the region of higher similarity by assigning distance-dependent Gaussian weights to deviating fragments of the structure. Application of this algorithm ensured that the superimposition quality was not dominated by a single flexible and/or poorly predicted part, e.g. one deviating part of a helix.

TM superimposition quality, Qsuper: The fraction of TM bundle for which high-quality superimposition was found (< 2 Å RMSD).

ECL2 backbone RMSD: With the same superimposition of TM helices, we calculated RMSD of backbone atoms of the model's ECL2 from that of the template. We chose to focus on ECL2 rather than on all extracellular parts of the protein because of its size and the critical role it plays in ligand binding for many GPCRs. Extracellular loops were defined as follows:

Residues 191-196, 340-344 in 5HT1B and residues 198-200 in 5HT2B were omitted from the comparison because they were disordered in the target structures.

Protein prediction Z-score: Protein prediction Z-scores were calculated in the spirit of the previous assessments, GPCR Dock 2008 and 2010. TM domain RMSD values and ECL2 RMSD values were independently converted into Z-scores (the opposite of RMSD, Z-score was taken so that higher values correspond to better models in all cases); the two Z-scores were averaged. The new mean and standard deviation were calculated excluding the low-scoring models that deviated from the old mean by more than two standard deviations (SD), and new Z-scores were found using this corrected mean and SD.

Ligand fingerprint prediction accuracy: Similarity between the predicted and the experimental pocket residue content was assessed by comparing the contact strength fingerprints of the ligand on the residue backbone and side-chains. The reported value is the arithmetic average of the fingerprint prediction recall and precision.

Pocket residue RMSD:Similarity of the pocket residue conformations was evaluated by measuring RMSD between the heavy atoms of the residues that constituted the binding pockets in the target structures. Binding pockets were defined as the sets of residues for which the ligand contact strength fingerprint in the target structures exceeded 1 (the value corresponding to the interatomic distance of 4 A). Specifically, the sets of target pocket residues included:

The optimal superimposition of TM domains was performed prior to the binding pocket comparison as described above. Residue symmetry was taken into account when calculating pocket residue RMSD.

Ligand heavy atom RMSD: RMSD of the ligand non-hydrogen atoms from their respective counterparts in the crystallographic structure was determined after superimposition of the model onto the reference template as described above. Internal ligand symmetry was taken into account for RMSD definition.

Atom (residue) contacts: The strength of contacts that are the same between the target structure and the model is calculated and compared to the total strength of ligand-protein contacts in the target structure (recall) or in the model (precision). As with ligand RMSD, topologically equivalent atom permutations in the ligand were enumerated and side-chain internal symmetry was taken into account. The reported values are: strength in common(number of residues)/strength in model(number of residues)/strength in target structure(number of residues).

Ligand-pocket contact strength prediction accuracy: The reported values are the average of recall and precision.

Correctness: Expected ligand RMSD and contact variation among the experimental structures was analyzed and approximated by an analytical function. For each model, its calculated ligand RMSD from the target structure and its fraction of correctly reproduced contact strengths were compared to the experimental distribution. The percentile of the experimental pairs with at least as high RMSD or at least as low fraction of common contacts was calculated using the analytical approximation. The percentiles were used for the final model ranking.


Models can be visualized and analyzed online using ActiveICM web-browser plugin (see Links section on top of the page).

Download models

A single compressed file (403 MB) consisting of all models in icb format, compatible with the free ICM browser (see Links section on top of the page).

A single compressed file (24 MB) consisting of all models in pdb format.


Ballesteros, J.A., and Weinstein, H. (1995). Integrated methods for the construction of three-dimensional models and computational probing of structure-function relations in G protein-coupled receptors. In Methods in Neurosciences (Academic Press), pp. 366-428.

Kufareva, I., and Abagyan, R. (2012). Methods of Protein Structure Comparison. In Homology Modeling, A.J.W. Orry, ed. (Humana Press), pp. 231-257.

Kufareva, I., Rueda, M., Katritch, V., Stevens, R.C., and Abagyan, R. (2011). Status of GPCR modeling and docking as reflected by community-wide GPCR Dock 2010 assessment. Structure 19, 1108-1126.

Michino, M., Abola, E., participants, G.D., Brooks, C.L., Dixon, J.S., Moult, J., and Stevens, R.C. (2009). Community-wide assessment of GPCR structure modelling and ligand docking: GPCR Dock 2008. Nat Rev Drug Discov 8, 455-463.

Wacker, D., Wang, C., Katritch, V., Han, G.W., Huang, X.-P., Vardy, E., McCorvy, J.D., Jiang, Y., Chu, M., Siu, F.Y., et al. (2013). Structural Features for Functional Selectivity at Serotonin Receptors. Science 340, 615-619.

Wang, C., Jiang, Y., Ma, J., Wu, H., Wacker, D., Katritch, V., Han, G.W., Liu, W., Huang, X.-P., Vardy, E., et al. (2013a). Structural Basis for Molecular Recognition at Serotonin Receptors. Science 340, 610-614.

Wang, C., Wu, H., Katritch, V., Han, G.W., Huang, X.-P., Liu, W., Siu, F.Y., Roth, B.L., Cherezov, V., and Stevens, R.C. (2013b). Structure of the human smoothened receptor bound to an antitumour agent. Nature 497, 338-343.

Wang, C., Wu, H., Evron, T., Vardy, E., Han, G., Huang, X., Hufeisen, S., Mangano, T., Urban, D., Katritch, V., et al. (2014). Structural basis for Smoothened receptor modulation and chemoresistance. Nature Communications accepted.