Generative model for Data Generating Motifs

Data: Signal length|x|, motif length|m|, number of motifs nm, motif mixing

parameter γ, and motif parameters{θ_j}n_j=m₁ Result: motif representation of signal

for i =0, . . ._||_mx|_| do

Pick motif mi ∼Cat(γ); for k=0 . . .|m| −1 do Draw observation: xi|m|+k ∼ N (θmi,µk, θ 2 mi,σk); end end

Under this definition, even noisy, poorly conserved subsequences are technically mo- tifs. It is common to include a background motif, assumed to be poorly conserved, to account for such subsequences. Inference with this model is identical to inference with a Gaussian Mixture Model. Each mixture component, representing a motif, is an |m|- dimensional Gaussian with a spherical covariance matrix.

3.2.2

Contextual Motifs- A Novel Extension

We extend the work above to incorporate context. We represent context as a categorical variable ct ∈ C that, at time point t, takes a discrete value ct ∈ {1, . . . , nc}, where nc is the

number of distinct contexts. Contextual motifs are then tuples of the form(mt, ct), repre-

senting a motif and the context under which it occurs. Contextual motif discovery is the task of discovering these tuples. Discovery can also be viewed as the process of discov- ering motifs occurring within similar contexts. This distinction can be important when contexts vary from signal to signal. Without taking context into account, our measure of motif quality may mistake infrequent contexts with infrequent motifs within a context.

For example, a certain quantity of food consumption may always present a distinctive blood glucose pattern in individuals with poor glycemic control. If some signals in our CGM dataset contained large meals, and others did not, contextless motif discovery could fail to recognize the prevalence of this pattern.

We present methods that extend both data-derived and data-generating motifs to con- textual motifs. When extending data-derived motifs, we focus on independently inferring context in a two-stage approach. Data-generating motifs can be extended using either a two-stage approach or by jointly inferring motifs and context. To perform this joint infer- ence, we propose a generative model based on a subclass of dynamic Bayesian networks [73].

Despite the vast literature studying motifs, and motif discovery methods, there is rel- atively limited work on considering abstractions based on motifs. Still, we discuss the related work in this area and how it differs from what we propose.

Van Esbroeck et al. considered representing physiological signals using a bag of motifs (a common approach in motif discovery) [74]. The authors built upon this representation using a topic modeling approach. In their approach, each topic is associated with a dis- tribution over motifs and each signal is then a distribution over topics. These topics are then used as an abstraction to represent the signal. While the idea of a topic is concep- tually similar to context, their method differs from our proposed method in two ways. First, we jointly infer contexts and motifs, whereas they assume that motifs are first dis- covered and then topics are inferred. Second, because of the bag-of-motifs approach the topics they learn assume a static representation of the signal. In contrast, our approach leverages the temporal contiguity of contexts to allow for flexible variation in motif rep- resentations both within and across signals.

In other domains, where the order motifs occur is important (e.g., genetics) researchers have considered leveraging this temporal ordering to discover better motif represen- tations. Lin et al. proposed a method based on hierarchical HMMs where the pres- ence/absence of motifs in one part of a signal affects the presence/absence of motifs in neighboring parts [75]. Again, this is conceptually similar to context, since context can be viewed as a type of inter-motif structure. However, like the method described above, Lin et al.’s method also requires a separate motif discovery step. Our approach combines the discovery steps, enabling joint motif-context discovery and reducing the reliance on prior

knowledge.

Finally, others have looked at non-motif-based approaches to learning abstractions based on time-series data. For example, Saria et al. examined the joint discovery of gen- erating functions and temporal topics in NICU data [76]. Our approach to joint discovery is similar, though we focus on discovering motifs as opposed to generating functions, since we are interested in preserving interpretability.

3.3

Methods

In this section, we present the main technical contributions of this chapter. We begin by discussing contextual motifs applied to data-derived motifs. We introduce methods to discover data-derived contextual motifs when context is observed and when context is unobserved, using a two-stage context inference procedure. We then discuss data- generating contextual motifs. In addition to observed context methods and two-stage context inference, we introduce a method to jointly infer context and motifs.

3.3.1

Data-Derived Contextual Motifs

For completeness, we begin by briefly describing a method to discover contextual motifs when context is observed. In the more likely case of unobserved context, we present techniques to infer context in two stages: first inferring context followed by inferring motifs.

Observed Context

When context is observed, we aim to discover motifs within similar contexts. This results in the simple extension outlined in Algorithm 2

This framing of contextual motifs can be thought of as a specialized instance of multi- variate motifs. However, by framing the problem as we have, each context can be mined for motifs independently of the others. This enables efficient, parallel motif discovery.

Compared to a standard approach that ignores context, this approach may perform worse if a motif does not occur enough times in any particular context to be discovered. However, when there is not enough per context information to discover useful motifs, it