Differential Testing

To test for differential usage of alternative events, we use the published tool rDiff [69]. In this context, we treat each alternative event as artificial gene expressing two different isoforms that are defined by the two possible paths through the event-sub-graphs beginning at start-terminal and ending at end-terminal nodes. For instance, the two isoforms of an exon skip event, would be an isoform of three exons, containing the middle exon and an isoform of two exons, skipping the middle exons. The first and the last exon of these two isoforms would be identical. We store all extracted event isoforms in a common event file in GFF3 format that can then be used as input file for rDiff. To account for directionality in the test, that is to identify which of the event’s two isoforms was up- or down-regulated, we modified the rDiff output to take the normalized counts of each isoform into account for reporting the final p-value. To this end, we altered the rDiff sourcecode to take the mean expression values of the two tested isoforms into account, when reporting the p-values. We denote an event as up-regulated, if the normalized read count of the longer isoform increases between the two tested conditions A and B, and we denote the event as down-regulated otherwise. In this context we determine the length of an isoform as the sum of its exonic positions. Depending on the direction of change of the normalized counts, the test p-value is assigned to the respective direction and a value of 1 to the other direction.

2.5.6 Results and Evaluation

We tested SplAdder with both artificial data as well as in application to biological samples. Here, we will focus on the evaluations on simulated data. The performance when applied to several biological datasets and a comparison between augmented and non-augmented annotation is discussed in Chapter 3, Section 3.4.3.

Main goal of this evaluation was to measure how well SplAdder can reconstruct alternative splicing informations from RNA-Seq data if this information is lacking in the annotation. Specifically, we would like to re-construct the same splicing graph where we have access to all isoforms in one case and to only one isoform and additional RNA-Seq data in the other

A

B

Figure 2.21: Evaluation of the SplAdder performance on artificial data. The two panels show precision, recall and F-score for the prediction of exons, internal exons and introns shown in dark green, light green and yellow, respectively. The predictions are compared to the original annotation the reads were sampled from. A: Performance measures based on the PALMapper alignments of the reads. B: Optimal performance based on the ideal alignment set that was created during read simulation.

case. To this end, we took a random set of genes with at least two isoforms and generated RNA-Seq data from all transcripts. We then only kept the first annotated isoform per gene and tried to reconstruct the full splicing graph using SplAdder on the simulated reads. The simulation was implemented as follows.

Pursuing a similar strategy as described in Section 2.4.4, we used FluxSimulator (version

1.1.1-20121103021450) [102] to generate a set of 2 × 106 RNA-Seq reads of 76 nt length.

All reads were sampled from a set of 5,000 randomly chosen genes from the ENSEMBL annotation [85]. The reads were then aligned to the hg19 human reference genome with PALMapper, allowing for up to 10 mismatches and at most 2 gaps. To compute an upper performance limit on the given dataset, we also retained the originally sampled reads in BAM format as ideal input data set.

To generate our testing set, we took the set of 5,000 genes the reads were originally sampled from and retained only the first annotated transcript isoform for each gene. This resulted in an annotation without any alternative events. We then used SplAdder to aug-

ment this truncated annotation based on the RNA-Seq read evidence. For several confidence levels, we compared a set of quality measures, describing how similar the splicing graphs generated by SplAdder were to the ones generated from the not truncated original annota- tion. The evaluation was performed on a subset of 1,491 genes that expressed at least two isoforms. Figure 2.21 shows an overview of the performance evaluation.

We used three different performance measures. The correct augmentation of nodes in the graph is harder than adding single edges. Thus, the exon-level performance measures how many nodes (exons) in the predicted splicing graph match to the graph generated from the not truncated annotation set. Especially terminal exons are difficult to predict, as RNA- Seq alignments are only a poor measure of transcription start- or stop-sites. Hence, the second evaluation measure takes only internal exons into account, as their boundaries can be identified through spliced alignments. The last measure uses the overlap on intron level, which is the easiest task, as the boundaries are completely defined by spliced alignments and existing nodes in the graph.

As shown in Figure 2.21, SplAdder is able to reconstruct almost all intron edges correctly. Notably, with higher confidence levels the precision increases further, which comes at the cost of lower sensitivity, which explains the dropping F-Score measures for more strict filtering.

Although SplAdder detects a large variety of different events, covering a large fraction of the existing variability, there also exist certain limitations to the approach. Especially infor- mation regarding transcript starts and ends is difficult to extract from RNA-Seq data due to the coverage slowly running out towards the transcript borders, which makes it difficult to infer specific sites. Also complex events are difficult to evaluate, for instance, in cases where several alternative exons of an exon skip event overlap or in cases of a coordinated retention of introns. For these specific events, approaches that take full transcripts into account might be better suited.