A challenging RNA test set 16

We developed the ShapeKnots algorithm using a test set of 16 non-pseudoknotted and pseudoknot-containing RNAs that were selected for their complex, and generally difficult to predict, structures (Table 2.1, top). These RNAs included (i) five RNAs with lengths >300 nucleotides, both with and without pseudoknots; (ii) five riboswitch RNAs whose structures only form upon binding by specific ligands, for which thermodynamic rules are obligatorily incomplete; (iii) four RNAs with structures that are predicted especially poorly, with accuracies <60% using nearest-neighbor thermodynamic parameters; and (iv) three RNAs whose structures are probably modulated by protein binding. SHAPE experiments were performed on each of the RNAs in the presence of ligand if applicable but in the absence of any protein. Each of the training set RNAs had SHAPE probing patterns that suggested these RNAs folded in solution into structures generally consistent with accepted secondary structure models based on either X-ray crystallography or comparative sequence analyses.

Table 2.1:Prediction accuracies as a function of algorithm and SHAPE information.

Sensitivities (sens), positive predictive value (ppv), and their geometric average (geo) are shown for four test cases: no pseudoknots allowed and no SHAPE data; no pseudoknots allowed and with SHAPE data (both by free energy minimization); pseudoknots allowed and no SHAPE data; and pseudoknots allowed and with SHAPE data (both using ShapeKnots). Complicating features are ligand (L) and protein (P) binding that are not accounted for in nearest-neighbor thermodynamic parameters. Pseudoknot (PK) predictions are indicated with a checkmark ( ) or X; a checkmark indicates that pseudoknots were predicted correctly and that there were no false-positive pseudoknot predictions. For the ribosomal RNAs (†), regions in which the SHAPE reactivities were clearly incompatible with the accepted structure, as described 18, were omitted from the sensitivity and ppv calculations; for the E. coli 16 rRNA, this included nucleotides 143- 220. The HIV-1 5' leader domain (§) was included as an example of pseudoknot prediction in a large RNA. Because the accepted structure for this RNA is based on SHAPE-directed prediction 19, we did not include sensitivity and ppv for this RNA in the overall Average values; however, the pseudoknot was proven independently 20 and is included.

!""#$%&'()*#+,#-' ./!01%*2-2

!"#! $$% &"' () !"#! $$% &"' () !"#! $$% &"' () !"#! $$% &"' () *++,-.+/0123

(-"4560-78'!97:+;<0!"#$%&'()($ => ? 6 @ABC 6DD EFB6 3 @ABC 6DD EFB6 3 @ABC 6DD EFB6 3 6DD 6DD 6DD ✓ 6DD%/456

G"H'I"-.!"0$!",J'K#':<0;,I.# >E ( 6 >DBD ECBD C>BL 3 @DBD ECBD @EB6 3 6DD 6DD 6DD ✓ 6DD 6DD 6DD ✓ FD

:MN*1.!$3<0/".!: EC O D FCBA FCBA FCBA ✓ FCBA FCBA FCBA ✓ FCBA FCBA FCBA ✓ FCBA FCBA FCBA ✓ LD

G((0-78'!97:+;<#*"#+,)( EF ? D EEB= LCBD L6BD ✓ FCBC LEBC F6B> ✓ EEB= LCBD L6BD ✓ FCBC LEBC F6B> ✓ ED

P*MP0+'-'#.0%7-,!0$!",J'K#':0 LA O 6 @FBA FDBD ELBF 3 @FBA ECBD EAB6 3 @CB> @LBD @@BE 3 L>B@ LLBD L@B= ✓ @D

+/+H7+4J74QR(0-78'!97:+;<#-"#+.,)/01/# FE ? D ECBD EEBL E@B> ✓ LFB= L@BA LEBE ✓ ECBD EEBL E@B> ✓ LFB= L@BA LEBE ✓ CD

P*R0S0-78'!97:+;<#2"#'/34+,34/3$($ 66L ? 6 E>B> LDB@ EEB> 3 E@BF LCBE L6BA 3 E@BF L6B6 EFBD 3 FAB= FEB= F>BL ✓ >D

R4T'U0-78'!97:+;<0!"#$%&'()($0 6C> ? D LEBC F6B= LFB> ✓ LEBC F6B= LFB> ✓ LEBC F6B= LFB> ✓ LEBC F6B= LFB> ✓ =D

(C>@0J'I.7#<08S=0&-',$0S07#:-'# 6CC O D >ABF >>B> >=B@ ✓ F>B@ F@B> FCBC ✓ >ABF >>B> >=B@ ✓ F>B@ F@B> FCBC ✓ AD

?/!7#"0-78'!97:+;<02"#510('(51 6E> ? 6 EEBL L=B6 LDB> 3 EFB> LFB= L>BA 3 L>B6 LABL L=BC ✓ LEB= LLBE LLBD ✓ 6D

Q-',$0S07#:-'#<#67,10+%$0!$B A6> O 6 E=BD ECB> E>BA 3 L6BD LCBD L=BD 3 E=BD ECB> E>BA 3 FAB6 FCB6 F=B@ ✓ D %7#$

V"$.:7:7!0W0%7-,!0SMXP0J'I.7# ==@ O 6 =FB> =LBD =LBE 3 EFBL L@BC L=B6 3 =FB> =@B= =EBL 3 FAB= F@BD F>B6 ✓

Q-',$0SS07#:-'#<08"#(./9/3$($# >6A O 6 LLB@ FEBC FABF 3 E>BA L>BC EFBA 3 LLB@ FEBC FABF 3 F=BA FEB@ FCB> ✓

Q-',$0S0S#:-'#<#2"#'./05,:.()1 >AC O 6 L=BA E>B= ELB@ 3 LEBL LLB@ LLBA 3 L=BA E>B= ELB@ 3 F=BF F6BA FABC ✓

CY0J'I.7#0'Z0A=P0-MN*<0*"#+,)(0[ C66 ?<( D FEBA E=BL L>BE ✓ FEBA E@BL L@B> ✓ FEBA E=BL L>BE ✓ FEBA E@BL L@B> ✓

CY0J'I.7#0'Z06@P0-MN*<#*"#+,)(0[ C=D ?<( D @=B@ CFB6 @6B= ✓ F=BD L=B@ LLBA ✓ @=B@ CFB6 @6B= ✓ F=BD L=B@ LLBA ✓

!"#$%&# 89:8 88:; 8<:= >=:8 >?:8 ><:< 8;:8 88:? 8?:; @A:B @9:@ @=:<

C('-%'(-

\H,'-7J"0-78'!97:+;<#;"#$90(341/ @@ ? 6 C@B= @>B= @DB6 3 @ABC E6B> @@BL 3 F=BL F=BL F=BL ✓ F=BL F=BL F=BL ✓

*J"#7#"0-78'!97:+;<0-"#<%)3(=(+%$ E6 ? D 6DD 6DD 6DD ✓ 6DD 6DD 6DD ✓ 6DD 6DD 6DD ✓ 6DD 6DD 6DD ✓

:MN*1$;"3<#*"#+,)( E@ O D FCBA 6DD FEB@ ✓ 6DD 6DD 6DD ✓ FCBA 6DD FEB@ ✓ 6DD L>BD F6BE 3

CP0-MN*<0*"#+,)( 6AD ?<( D ALB@ ACBD A@BE ✓ LCBE E@BF L6BA ✓ ALB@ ACBD A@BE ✓ LCBE E@BF L6BA ✓

CY0J'I.7#0'Z06@P0-MN*<0>"#<,)+13((0[ >E= ?<( D LCB@ E6BF ELBC ✓ F@BA L=BA LFBC ✓ LCB@ E6BF ELBC ✓ F@BA L=BA LFBC ✓

VS]460CY0$!",J'K#':0J'I.7#0^ CDD O 6 >@BL =CBL >DBF 3 EEBC LLBD LAB@ 3 LDBD LDBD LDBD 3 6DD 6DD 6DDBD ✓

!"#$%&# 8A:9 8=:= 8=:? >>:@ >?:A >8:; >B:? 8>:9 8@:A @;:9 >8:? @9:=

'"#$%(( 8=:B 8?:A 8A:> ><:= >?:? >;:= 8?:@ 88:8 88:9 @A:; @B:@ @=:9

D(2E-4F4-4('%4,E#G&2-4H"(%$4-6%2EE(&-(*%'-I)E-)I('

P7&#.H0-"+'&#7:7'#0$.-:7+H"0MN*<0;,I.# =D6 ?<( D D D D ✓ CFBD CFBD CFBD ✓ D D D ✓ CCBD C=BF C>B> 3

MN.!"0(<0!"#$%&'()($# >DC ?<( 6 CEB> CCBD C@BA 3 E@BC L6BC EFBD 3 CEB> CCBD C@BA 3 ECBE EFBL EEBE 3

Hajdin et al., Table 1

CI24,4,5%'(- ?"#&:; \".:,-"! ()!

O O _ _

The structures of the 16 RNAs in the test set are predicted poorly by a conventional algorithm based on their sequences alone: The average sensitivity (sens; fraction of base pairs in the accepted structure predicted correctly), positive predictive value (ppv, the fraction of predicted pairs that occur in the accepted structure), and geometric average of these metrics are 72, 78, and 74%, respectively (Table 2.1).

In the process of developing this training set, we also analyzed two RNAs – RNase P RNA and the human signal recognition particle RNA – whose in vitro SHAPE reactivities were incompatible with the accepted structures for these RNAs. I include prediction statistics for these RNAs at the bottom of Table 2.1, but do not use these to evaluate our SHAPE-directed modeling algorithm.