Traversing the suf ix tree

With the enhanced suf ix array consisting of suf ix array, lcp table, and child table, we are now able to top-down traverse the nodes of the suf ix tree of a text which actually is a node traversal of the corresponding lcp-interval tree.

Algorithm 3.13: L H(𝑠, 𝑝)

input : text 𝑠 and pa ern 𝑝

output : minimal 𝑙 with 𝑝 ≤| |𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] 1 𝑙 ← 0, 𝑘 ← 0 2 𝑙 ← |𝑠|, 𝑘 ← 0 3 while𝑙 < 𝑙 do 4 𝑖 ← 5 𝑘 ← min{𝑘 , 𝑘 } 6 if𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] <| | 𝑝 then 7 𝑙 ← 𝑖 + 1 8 𝑘 ← 𝑘 + lcp 𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] , 𝑝 9 else 10 𝑙 ← 𝑖 11 𝑘 ← 𝑘 + lcp 𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] , 𝑝 12 return𝑙 Algorithm 3.14: U H(𝑠, 𝑝)

input : text 𝑠 and pa ern 𝑝

output : minimal 𝑟 with 𝑝 <| |𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] 1 𝑟 ← 0, 𝑘 ← 0 2 𝑟 ← |𝑠|, 𝑘 ← 0 3 while𝑟 < 𝑟 do 4 𝑖 ← 5 𝑘 ← min{𝑘 , 𝑘 } 6 if𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] ≤| | 𝑝 then 7 𝑟 ← 𝑖 + 1 8 𝑘 ← 𝑘 + lcp 𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] , 𝑝 9 else 10 𝑟 ← 𝑖 11 𝑘 ← 𝑘 + lcp 𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ] , 𝑝 12 return𝑟

Figure 3.8:Binary search with mlr-heuristic. L H and U H return the

same interval boundaries as L and U . A heuristic determines a lower bound on the lcp length between 𝑠𝗌𝗎𝖿𝗍𝖺𝖻[ ]and 𝑝 and thereby reduces the

number of character comparisons on average.

1 5 50 500 5000 0.0 0.4 0.8 1.2 hs_chr2 (Σ =5) pattern length running time (s) LCPE MLR Naive SufTree 1 5 50 500 5000 0.0 0.4 0.8 1.2 sprot (Σ =24) pattern length running time (s) 1 5 50 500 5000 0.0 0.4 0.8 1.2 rfc (Σ =256) pattern length running time (s)

Figure 3.9:Comparison of the ESA based search algorithms available in SeqAn. We com-

pared the running times required to search 100,000 patterns using the naive binary search (Naive), the binary search with mlr-heuristic (MLR), the binary search with lcp-tree (LCPE), and the suf ix tree search (SufTree). As texts we used the irst 100 M characters of a DNA, amino acid, and natural language text. Patterns are random substrings of varying length.

Top-down traversal

We implemented a so-called top-down iterator and functions to go to the root node, to go down to the leftmost child, and to go right to the next sibling of the currently visited node, where the children are lexicographically ordered by their edge labels from left to right. The top-down iterator maintains the values 𝑙𝑏 and 𝑟𝑏, the lcp-interval boundaries of the

Algorithm 3.15: R (𝑖𝑡𝑒𝑟)

input : suffix tree iterator 𝑖𝑡𝑒𝑟

1 𝑛 ← |𝗅𝖼𝗉| − 1 // 𝑛 is the text length 2 𝑖𝑡𝑒𝑟.𝑙𝑏 ← 0 3 𝑖𝑡𝑒𝑟.𝑟𝑏 ← 𝑛 4 𝑖𝑡𝑒𝑟.𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 ← ⊥ Algorithm 3.16: N (𝑖) input : ℓ-index 𝑖 1 𝑗 ← 𝖼𝗅𝖽[𝑖] 2 if𝑖 < 𝑗and𝗅𝖼𝗉[𝑖] = 𝗅𝖼𝗉[𝑗]then 3 returntrue 4 returnfalse Algorithm 3.17: D (𝑖𝑡𝑒𝑟)

input : suffix tree iterator 𝑖𝑡𝑒𝑟 output : returns true on success 1 if𝑖𝑡𝑒𝑟.𝑟𝑏 − 𝑖𝑡𝑒𝑟.𝑙𝑏 ≤ 1then 2 returnfalse;

3 if𝑖𝑡𝑒𝑟.𝑟𝑏 ≠ 𝑖𝑡𝑒𝑟.𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏then 4 𝑖𝑡𝑒𝑟.𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 ← 𝑖𝑡𝑒𝑟.𝑟𝑏

// get 𝗎𝗉 value of right boundary 5 𝑖𝑡𝑒𝑟.𝑟𝑏 ← 𝖼𝗅𝖽[𝑖𝑡𝑒𝑟.𝑟𝑏 − 1]

6 else

// get 𝖽𝗈𝗐𝗇 value of left boundary 7 𝑖𝑡𝑒𝑟.𝑟𝑏 ← 𝖼𝗅𝖽[𝑖𝑡𝑒𝑟.𝑙𝑏]

8 returntrue

Algorithm 3.18: R (𝑖𝑡𝑒𝑟)

input : suffix tree iterator 𝑖𝑡𝑒𝑟 output : returns true on success 1 if𝑖𝑡𝑒𝑟.𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 ∈ {⊥, 𝑖𝑡𝑒𝑟.𝑟𝑏}then 2 returnfalse

3 𝑖𝑡𝑒𝑟.𝑙𝑏 ← 𝑖𝑡𝑒𝑟.𝑟𝑏 4 if N (𝑖𝑡𝑒𝑟.𝑟𝑏)then

// 𝑖𝑡𝑒𝑟.𝑟𝑏 has a succeeding ℓ-index

5 𝑖𝑡𝑒𝑟.𝑟𝑏 ← 𝖼𝗅𝖽[𝑖𝑡𝑒𝑟.𝑟𝑏] 6 else

// 𝑖𝑡𝑒𝑟.𝑟𝑏 is the last ℓ-index

7 𝑖𝑡𝑒𝑟.𝑟𝑏 ← 𝑖𝑡𝑒𝑟.𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 8 returntrue

currently visited node, and 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 the right boundary of the parent node. It starts in the root of the lcp-interval tree which is the interval [𝑙𝑏..𝑟𝑏) = [0..𝑛). For the root node, we initialize 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 with ⊥, see Algorithm 3.15. The intervals in the lcp-interval tree are distinct from each other and two iterators can be compared by comparing their boundary pairs. For leaf nodes hold 𝑟𝑏 − 𝑙𝑏 = 1.

When moving the iterator to the irst child of the current node, the left boundary re- mains the same whereas the smallest ℓ-index in ℓ-indices(𝑙𝑏, 𝑟𝑏) becomes the new right boundary 𝑟𝑏 and 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏 the former value of 𝑟𝑏. If the current node is not the last child of its parent (𝑟𝑏 ≠ 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏), the smallest ℓ-index in ℓ-indices(𝑙𝑏, 𝑟𝑏) is the 𝗎𝗉 value of 𝑟𝑏 stored at 𝖼𝗅𝖽[𝑟𝑏 − 1], otherwise it is the 𝖽𝗈𝗐𝗇 value of 𝑙𝑏 stored only in this case at 𝖼𝗅𝖽[𝑙𝑏]. The corresponding pseudo-code is given in Algorithm 3.17. Moving the iterator to the next sibling is possible iff 𝑟𝑏 ∉ {⊥, 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏}, i.e. for all but the last sibling. Then 𝑙𝑏 becomes the former 𝑟𝑏 and 𝑟𝑏 becomes, if existent, its next ℓ-index or 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏, otherwise (see Algorithm 3.18).

Besides the D function that moves an iterator along the leftmost edge to the irst child, we also implemented variants to go down to a child at the end of an edge starting with a certain character or to go down along the path of string characters. All the traversal functions return a boolean which is true, iff the iterator could be successfully moved.

Random traversal

If only the 3 tables of the enhanced suf ix array are given, it is not possible to move a top-down iterator upwards the tree in 𝒪(1) time. It is however possible to recover the state the iterator had in the parent node by manually maintaing a stack of top-down it-

Algorithm 3.19: N _P (𝑖𝑡)

input : suffix tree iterator 𝑖𝑡 // try go down, if not try go right 1 if not D (𝑖𝑡)and not R (𝑖𝑡)then

2 while U (𝑖𝑡)and not R (𝑖𝑡)do

// go up until we can go right 3 ifI R (𝑖𝑡)then

// end when entering the root node 4 (𝑖𝑡.𝑙𝑏, 𝑖𝑡.𝑟𝑏) ← (−1, −1)

5 returnfalse 6 returntrue

Algorithm 3.20: N _P (𝑖𝑡)

input : suffix tree iterator 𝑖𝑡 1 if R (𝑖𝑡)then

2 while D (𝑖𝑡)do

// try go right and down to leaf

3 else // otherwise go up

4 if not U (𝑖𝑡)then

// end after halting in root 5 (𝑖𝑡.𝑙𝑏, 𝑖𝑡.𝑟𝑏) ← (−1, −1)

6 returnfalse

7 returntrue

erator copies. To provide a more convenient interface we implemented the subclass top-

down history iterator which extends the top-down iterator by a stack. The stack stores

the lcp-interval boundaries of nodes on the path to the root. We adapted the R and D functions to clear the stack and to push the current interval boundary pair (𝑙𝑏, 𝑟𝑏)onto the stack before going down. The new function U simply replaces (𝑙𝑏, 𝑟𝑏) by the topmost pair and removes it from stack. We also need to adapt 𝑝𝑎𝑟𝑒𝑛𝑡𝑅𝑏, which is set to the new topmost 𝑟𝑏 value on stack.

Depth- irst search

While the two iterators above require 3 tables of the enhanced suf ix array, we have seen in Section 3.5.1 that only 2 tables are required to conduct a postorder depth- irst search (DFS). Algorithm 3.9 can be executed to report all nodes. However, in some applications it is more appropriate to have an iterator that can be moved node-by-node. Therefore, we implemented a light-weight bottom-up iterator which maintains 𝑙𝑏 and 𝑟𝑏, the current node boundaries, and provides functions B and N . Like a coroutine [Knuth, 1997], N resumes and suspends the execution of Algorithm 3.9 between two calls of report(𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙) and instead of reporting the interval it sets 𝑙𝑏 and 𝑟𝑏 accordingly. B executes Algorithm 3.9 up to the irst call of report(𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙). The last node the iterator halts in is the root node. If N is called again, it sets the iterator in a de ined end-state and indicates that by returning false.

We provide the same depth- irst search interface for the top-down history iterator but implemented N by means of D , R , and U to traverse all nodes in the order of either a postorder or a preorder depth- irst search. The pseudo-codes of both DFS variants are shown in Algorithms 3.19 and 3.20. In contrast to postorder DFS, in a preorder DFS each node is traversed before its children.

Some applications require to traverse nodes only up to a certain tree or string depth or have a different criteria to skip a whole subtree in the traversal. To meet this require-

ment, we implemented N R and N U . N R skips the subtree of

the current node and proceeds with the next sibling by omitting D in line 1 of Algo- rithm 3.19. Analogously N U skips the subtree and all right siblings of the current node.