In the following subsection, the benchmark results of SSD and MSD for ligand- binding design were further analyzed. First, we show that the MSD concept in fact accounts for the performance advantage over SSD. Second, the different sequence preferences of MSD and SSD are studied.
The MSD concept is crucial for performance on BR_EnzBench
The sequence recovery results of the hIFABP benchmark and for BR_EnzBench strongly suggest that MSF:GA:ENZDES is superior to ENZDES in more complex
design applications. However, it was unclear to us, whether the different con- cepts (single-state versus multi-state) or the different optimizers (MC versus GA) contributed most to the performance. Choosing a MSD approach increases computa- tional cost which has to be substantiated by making plausible that the choice of the optimizer has less effect on the performance.
As described before, the performance of MSF:GA:ENZDESon BR_EnzBench was
assessed ensemble-wise by determining for each enskm the nssr scores, which were averaged (Equation (2.12)). Due to the stochastic approach of the Backrub algorithm, which was used to create the conformational ensembles (see Subsection 2.7.1), the conformations that are combined in each of the ensembles enskm are unrelated. As these ensembles contain not more than five conformations each, the nssrMSD(enskm)
values (Equation (2.13)) vary due to the small sample size and one can sort for each prot(k) the four enskm on their nssrMSD(enskm) value. The result is a ranking
enskrank=u(1≤u≤4)of the four ensembles and we created the set ES1that contained
the 16 ensembles (one for each prot(k)) with the lowest nssrMSD(enskm)value. Anal-
ogously, we compiled the sets ES2−ES4; consequently, ES4consisted of those 16
ensembles that had the highest nssrMSD(enskm)value; for details see Subsection 2.7.3.
For these four sets ESi, boxplots of the corresponding nssrSSDand nssrMSDvalues
were determined; see Fig. 3.12. The boxplots characterizing the SSD results are nearly identical; this finding indicates that the conformations allocated to the four sets ES1-
ES4give rise to a similar SSD performance. Moreover, the boxplots representing the
nssrSSD(ES1)and nssrMSD(ES1)values are nearly identical (median values 47.60%
and 47.76%), which indicates that the optimizer GA is not generally superior to MC. Additionally the continuous increase observed for nssrMSD(ES1) → nssrMSD(ES4)
(but not for nssrSSD(ES1) →nssrSSD(ES4)values) supports the notion that it is the
concluded that the MSD approach (and not the optimizer) contributes most to the performance of MSF:GA:ENZDES.
Fig. 3.12 Performance of ENZDESand MSF:GA:ENZDESon a distinct grouping of
conformations. Each of the sets ES1-ES4 contains a quarter of the conformations
from BR_EnzBench, which were grouped according to their nssrMSDvalues as de-
scribed in Subsection 2.7.3. ES1contains all ensembles with the lowest and ES4those
with the highest recovery values. For each set ESi, the corresponding nssrSSD(ESi)
and nssrMSD(ESi)values are represented by two boxplots. (Left) Performance of
ENZDES(blue boxplots). (Right) Performance of MSF:GA:ENZDES(orange boxplots). Whiskers indicate the lowest and the highest datum still within the 1.5 interquartile range.
The residue preferences of ENZDESand MSF:GA:ENZDESdiffer
It is known that ENZDEShas a certain bias in recapitulating native residues [Leaver- Fay et al., 2013]. Therefore it is reasonable to assess and compare the bias introduced by ENZDESand MSF:GA:ENZDES. For the assessment of the ENZDESoutcome, we selected the 13440 sequences representing the best designs on BR_EnzBench and determined nssrSSD(aaj)values. This distribution represents for all amino acids aaj
the fraction of similar residues recovered at all design shell positions. Analogously, the distribution nssrMSD(aaj)was computed that indicates the fraction of similar
residues recovered by MSF:GA:ENZDES; for details see Subsection 2.7.3.
The two distributions that are plotted in Fig. 3.13 indicate that the recovery rates are similar and are below the optimal value of 100% for all residues. Generally, sequence recovery for large polar or charged residues is low, which contributes to the weakness of Rosetta to accurately design hydrogen bonds and electrostatics [Stranges and Kuhlman, 2013]. Interestingly, ENZDESis slightly better in recovering polar and charged residues (D, E, H, K, N, R, S), whereas MSF:GA:ENZDESclearly recovers a higher fraction of hydrophobic residues (A, F, I, L, P, V, W, Y).
Fig. 3.13 Recovery of design shell residues from BR_EnzBench by means of
ENZDES and MSF:GA:ENZDES. The distributions nssrSSD(aaj) (blue bars) and
nssrMSD(aaj)(orange bars) represent for each amino acid aajthe nssr value deduced
from 13440 design sequences. These were created by ENZDESor MSF:GA:ENZDES
for the benchmark proteins BR_EnzBench, respectively. nssr takes into account the recovery of all residues which are similar to the native aaj. For details, see Subsection
2.7.3.
This general trend is most evident in the two benchmark proteins with the largest difference in nssrSSD and nssrMSD values: ARL3-GDP (PDB ID 1fzq) is a distinct
GTP binding protein [Hillig et al., 2000] from Mus musculus and both the ligand and the native binding pocket are considerably polar. Fig. 3.14 A shows that ENZDES
correctly recovered the residues interacting with the guanine group (colored in teal) of GDP, while MSD was less successful. On the other hand, in the glucose binding protein (PDB ID 2b3b) from Thermus thermophilus four tryptophan residues provide tight binding to glucose by shape complementarity. Fig. 3.14 B shows that MSF:GA:ENZDES recovered three critical tryptophan residues (colored in teal) in most designs, whereas ENZDESpreferred small polar residues that do not provide tight packing. It seems that the representation of a protein by means of an ensemble improves hydrophobic packing but not the formation of polar interaction networks. Their design is considerably more difficult than hydrophobic packing due to the partially covalent nature of a hydrogen bond and the geometric requirements for orientations and distances [Boyken et al., 2016; Leaver-Fay et al., 2013].