Discussion

In the preceding sections, we focused on the Markov property based classifiers that can learn from sequence data using only count queries of the form S(D, s, C = ck). However, one

of the important classifiers in this family is the Hidden Markov Model based classifier (see Rabiner (1990) for an overview). Hidden Markov Model (HMM) based approaches have been used to address a variety of learning tasks from sequence data [Rabiner (1990), Stanke and Waack (2003), Martin et al. (2005), Zhu et al. (2006a)]. We do not address the HMM based approach since the implementation of Baum-Welch algorithm in the statistical query approach is not straightforward. We beleieve it will involve certain simplifying assumptions (say fixing the number of hidden states) and additional type of queries besides S(D, s, C = ck) (say in

the Gene finding approach, the queries to compute the transition matrix between the states). In our setting of computing the Markov Property based class of predictors using statistical queries we assumed that the data source holding the dataset answers queries of the form S(D, s, C = ck). In practice the sequence data is often accessible via the web and the data

source provider often provides a variety of tools to access the data and it is not difficult envision support for such type of queries. However, we show that supporting these types of queries is fairly straightforward for a dataset stored in an RDBMS. Note that we acknowledge existing commercial RDBMS may not be an efficient choice for efficiently storing and querying sequence data [Stonebraker (2005), Stein (2010)] . In fact, there is a pressing need for storage solutions to the massive amount of sequence data that is being generated at an enormous rate. An approach to address a related problem (Web indexing) that generates Petabytes amount of text data is BigTable [Chang et al. (2006)]. However, assuming that dataset is stored in a RDBMS the queries of the form S(D, s, C = ck) can be easily supported. Given

Descs(D), let us assume the data is stored in a RDBMS with the following schema: the data is stored in a table named D with the attributes id, sequenceData, and class where

id is a unique identifier of the sequence (primary key), the attribute sequenceData contains the representation of the sequence as a string and the attribute class specifies the family to which the particular sequence belongs (note that it is fairly straightforward to dump FASTA or XML formatted data into this schema). In such a setting, the support for queries of the format S(D, s, C = cj) and S(D, |s|, C = cj) can be added by the user by writing appropriate

constructs using a procedural language (say PL/SQL) supported by the database. However, in general the data sources holding D may not allow user written code to be executed on their system. In such a case, the support for these queries needs to be provided in terms of standard SQL queries. We propose the use of the regular expression support in SQL queries provided by RDMS. Let R(D, C = cj, expr) represent the SQL query corresponding to the count of the

number of instances in D that have the value of the attribute class as cj and the sequenceData

matches the regular expression expr. The exact syntax of R(D, C = cj, expr) will depend on

the particular RDBMS under use. For Oracle 10g database, the R(D, C = cj, expr) will be the

SQL statement Select count(id) from D where REGEX LIKE (sequence expr) AND class = cj.

Similarly for MySQL R(D, C = cj, expr) be the SQL statement Select count(id) from D where

sequenceData REGEX expr AND class = cj. Consider the regular expression ˆ[.]{n}s which

matches any n characters followed by the string s. Assuming lmax is the maximum possible

length of a sequence, we have

S(D, s, C = cj) =

X

n∈{0,1...lmax}

R(D, C = cj,ˆ[.]{n}s)

Similarly, the regular expressionˆ[.]{n + k} matches any n+k characters starting from the beginning of the line. Then,

S(D, |s|, C = cj) =

X

n∈{0,1...lmax}

R(D, C = cj,ˆ[.]{n + k})

As a result to calculate a single query of the form S(D, s, C = cj) (or S(D, |s|, C = cj),

lmax number of SQL queries are submitted to the database. Hence, the computed Query

Complexity is increased by a factor of lmax. However, the preceding discussion demonstrates

that it is possible to provide support for the type of queries required to build Markov Models using only statistical queries.

Observation: It is possible to compute the answer to the query from answers to the queries of the form S(D, |s|, C = cj) as S(D, |s|, C = cj) =P_x∈Σ|s|S(D, x, C = cj). However,

it requires |Σ|s|| queries as opposed to l_max queries using the approach suggested earlier and may be useful only in cases when results to queries of the form S(D, s, C = cj) are already

available (say in cache).

4.7 Summary and Related Work

Summary: Due to the exponential increase in the rate at which sequence data are being generated, there is an urgent need for efficient algorithms for learning predictive models of sequence data from large sequence databases and for updating the learned models to accom- modate additions or deletions of data in settings where the sequence database can answer only a certain class of statistical queries.

In this chapter we presented an approach to learning predictive models from sequence data using sufficient statistics by posing count queries against a sequence data source. This ap- proach can be used to build the predictive model without access to the underlying data as long the data source is able to answer a class of count queries. In addition, this approach scales well to settings where the dataset is very large in size because it does not need to load the entire dataset in memory. We have also outlined some optimization techniques to minimize the number of queries submitted to the data source. In addition, we showed how the class of Markov model based predictors can be updated in response to addition or deletion of subsets of the data.

Related Work: The approach to learning Markov models and their variants presented in this chapter builds on the statistical query based approach to learning from large datasets (including distributed data sets) introduced by Caragea et al. (2004b). Markov Models have been successfully used in a broad range of applications in computational biology including gene finding (e.g. GeneMark [Borodovsky and McIninch (1993)], GenScan [Burge and Karlin (1997)]), protein classification [Yuan (1999), Yakhnenko et al. (2005), Fischer et al. (2004)] ,

among others. Salzberg et al. (1998) have used Interpolated Markov Models for gene finding. Bejerano and Yona (2001) and Sun and Deogun (2004) have used Probabilistic Suffix Trees for protein classification. Begleiter et al. (2004) discuss Variable order Markov Models. Abouel- hoda et al. (2004) have investigated approaches to reducing the memory requirements of suffix tree construction algorithms. Koul et al. (2008) describe approaches to build Naive Bayes and Decision Trees from databases using SQL queries.