In this chapter, we study whether recurrence plots or recurrence networks are informative enough to be used for further machine learning tasks. We consider weighted recurrence networks as recur- rence images and build advanced convolutional neural network (CNN) models for them to predict labels of original time series. Based on our experiments on benchmark UCR time series datasets, our method has the best performance among 9 advanced classification algorithms.
Our main contribution is that we show recurrence networks are good representations of the original time series and they even capture some information that is hard to retrieve through original time domain. Our other contribution is that our proposed RI-CNN method can be used for many time series classification tasks.
Converting time series to recurrence images has many advantages. However our current con- volutional layers are not perfect since recurrence images are symmetric with respect to main di- agonal. In the future, one interesting research direction is to map a symmetric high dimensional recurrence image to an asymmetric low dimensional image with minimal information loss. We will also consider using alternative filters to scan the symmetric images more efficiently.
Bibliography
H. D. Abarbanel, R. Brown, J. J. Sidorowich, and L. S. Tsimring. The analysis of observed chaotic data in physical systems. Reviews of modern physics, 65(4):1331, 1993.
R. Baragona. A simulation study on clustering time series with metaheuristic methods. Quaderni di Statistica, 3:1–26, 2001.
M. Bartlett. Deterministic and stochastic models for recurrent epidemics. In Proceedings of the third Berkeley symposium on mathematical statistics and probability, volume 4, page 109, 1956.
M. G. Baydogan, G. Runger, and E. Tuv. A bag-of-features framework to classify time series. IEEE transactions on pattern analysis and machine intelligence, 35(11):2796–2802, 2013.
D. J. Berndt and J. Clifford. Using dynamic time warping to find patterns in time series. In KDD workshop, volume 10, pages 359–370. Seattle, WA, 1994.
M. Bicego, V. Murino, and M. A. Figueiredo. Similarity-based clustering of sequences using hidden markov models. In Machine learning and data mining in pattern recognition, pages 86–95. Springer, 2003.
L. Cao. Practical method for determining the minimum embedding dimension of a scalar time series. Physica D: Nonlinear Phenomena, 110(1):43–50, 1997.
Y. Chen, E. Keogh, B. Hu, N. Begum, A. Bagnall, A. Mueen, and G. Batista. The ucr time series classification archive, July 2015. www.cs.ucr.edu/˜eamonn/time_series_data/.
N. Coffey, J. Hinde, and E. Holian. Clustering longitudinal profiles using p-splines and mixed effects models applied to time-course gene expression data. Computational Statistics & Data Analysis, 71:14–29, 2014.
E. Coviello, A. B. Chan, and G. R. Lanckriet. Clustering hidden markov models with variational hem. The Journal of Machine Learning Research, 15(1):697–747, 2014.
Z. Cui, W. Chen, and Y. Chen. Multi-scale convolutional neural networks for time series classifi- cation. arXiv preprint arXiv:1603.06995, 2016.
D. A. Cummings, W. J. Moss, K. Long, C. S. Wiysonge, T. J. Muluh, B. Kollo, E. Nomo, N. D. Wolfe, and D. S. Burke. Improved measles surveillance in cameroon reveals two major dynamic patterns of incidence. International Journal of Infectious Diseases, 10(2):148–155, 2006. M. Dalto. Deep neural networks for time series prediction with applications in ultra-short-term
wind forecasting, 2014.
D. C. De Lucas. Classification techniques for time series and functional data. PhD thesis, Univer- sidad Carlos III de Madrid, 2010.
R. V. Donner, Y. Zou, J. F. Donges, N. Marwan, and J. Kurths. Recurrence networks - a novel paradigm for nonlinear time series analysis. New Journal of Physics, 12(3):033025, 2010. J.-P. Eckmann, S. O. Kamphorst, and D. Ruelle. Recurrence plots of dynamical systems. Europhys.
Lett, 4(9):973–977, 1987.
J. Ernst, G. J. Nau, and Z. Bar-Joseph. Clustering short time series gene expression data. Bioin- formatics, 21(suppl 1):i159–i168, 2005.
C. Faloutsos, M. Ranganathan, and Y. Manolopoulos. Fast subsequence matching in time-series databases, volume 23. ACM, 1994.
B. F. Finkenst¨adt and B. T. Grenfell. Time series modelling of childhood diseases: a dynamical systems approach. Journal of the Royal Statistical Society: Series C (Applied Statistics), 49(2): 187–205, 2000.
O. Frank and D. Strauss. Markov graphs. Journal of the american Statistical association, 81(395): 832–842, 1986.
A. M. Fraser and H. L. Swinney. Independent coordinates for strange attractors from mutual information. Physical review A, 33(2):1134, 1986.
C. J. Geyer and E. A. Thompson. Constrained monte carlo maximum likelihood for dependent data. Journal of the Royal Statistical Society. Series B (Methodological), pages 657–699, 1992. X. Golay, S. Kollias, G. Stoll, D. Meier, A. Valavanis, and P. Boesiger. A new correlation-based
fuzzy logic clustering algorithm for fmri. Magnetic Resonance in Medicine, 40(2):249–260, 1998.
A. Goldenberg, A. X. Zheng, S. E. Fienberg, E. M. Airoldi, et al. A survey of statistical network models. Foundations and Trends in Machine Learning, 2(2):129–233, 2010.R
B. T. Grenfell, O. N. BJøRNSTAD, and B. F. Finkenst¨adt. Dynamics of measles epidemics: scaling noise, determinism, and predictability with the tsir model. Ecological Monographs, 72(2):185– 202, 2002.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016a.
K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pages 630–645. Springer, 2016b.
D. R. Hunter, S. M. Goodreau, and M. S. Handcock. Goodness of fit of social network models. Journal of the American Statistical Association, 103(481), 2008.
T. Jebara, Y. Song, and K. Thadani. Spectral clustering and embedding with hidden markov mod- els. In Machine Learning: ECML 2007, pages 164–175. Springer, 2007.
Y.-S. Jeong, M. K. Jeong, and O. A. Omitaomu. Weighted dynamic time warping for time series classification. Pattern Recognition, 44(9):2231–2240, 2011.
T. Kahveci and A. Singh. Variable length queries for time series data. In Data Engineering, 2001. Proceedings. 17th International Conference on, pages 273–282. IEEE, 2001.
T. Kahveci, A. Singh, and A. G¨urel. Similarity searching for multi-attribute sequences. In Scientific and Statistical Database Management, 2002. Proceedings. 14th International Conference on, pages 175–184. IEEE, 2002.
K. Kalpakis, D. Gada, and V. Puttagunta. Distance measures for effective clustering of arima time-series. In Data Mining, 2001. ICDM 2001, Proceedings IEEE International Conference on, pages 273–280. IEEE, 2001.
H. Kantz and T. Schreiber. Nonlinear time series analysis, volume 7. Cambridge university press, 2004.
M. B. Kennel, R. Brown, and H. D. Abarbanel. Determining embedding dimension for phase-space reconstruction using a geometrical construction. Physical review A, 45(6):3403, 1992.
A. Kom´arek, L. Kom´arkov´a, et al. Clustering for multivariate continuous and discrete longitudinal data. The Annals of Applied Statistics, 7(1):177–200, 2013.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document
recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
C. Li, G. Biswas, M. Dale, and P. Dale. Building models of ecological dynamics using hmm based temporal data clusteringa preliminary study. In International Symposium on Intelligent Data Analysis, pages 53–62. Springer, 2001.
T. W. Liao. Clustering of time series data a survey. Pattern recognition, 38(11):1857–1874, 2005. E. A. Maharaj. Cluster of time series. Journal of Classification, 17(2):297–314, 2000.
N. Marwan, M. C. Romano, M. Thiel, and J. Kurths. Recurrence plots for the analysis of complex systems. Physics Reports, 438(5):237–329, 2007.
N. Marwan, J. F. Donges, Y. Zou, R. V. Donner, and J. Kurths. Complex network approach for recurrence analysis of time series. Physics Letters A, 373(46):4246–4254, 2009.
L. Matassini, H. Kantz, J. Hołyst, and R. Hegger. Optimizing of recurrence plots for noise reduc- tion. Physical Review E, 65(2):021102, 2002.
P. D. McNicholas and T. B. Murphy. Model-based clustering of microarray expression data via latent gaussian mixture models. Bioinformatics, 26(21):2705–2712, 2010.
T. Michael, S. Spiegel, and S. Albayrak. Time series classification using compressed recurrence plots. In Proceedings of ECML-PKDD, 2015.
M. Morris, M. S. Handcock, and D. R. Hunter. Specification of exponential-family random graph models: terms and computational aspects. Journal of statistical software, 24(4):1548, 2008. M. K. Ng and Z. Huang. Data-mining massive time series astronomical data: challenges, problems
and solutions. Information and Software Technology, 41(9):545–556, 1999.
V. Niennattrakul and C. A. Ratanamahatana. Clustering multimedia data using time series. In Hybrid Information Technology, 2006. ICHIT’06. International Conference on, volume 1, pages 372–379. IEEE, 2006.
V. Niennattrakul and C. A. Ratanamahatana. On clustering multimedia time series data using k- means and dynamic time warping. In Multimedia and Ubiquitous Engineering, 2007. MUE’07. International Conference on, pages 733–738. IEEE, 2007.
A. Panuccio, M. Bicego, and V. Murino. A hidden markov model-based approach to sequential data clustering. In Structural, Syntactic, and Statistical Pattern Recognition, pages 734–743. Springer, 2002.
D. Piccolo. A distance measure for classifying arima models. Journal of Time Series Analysis, 11 (2):153–164, 1990.
T. Rakthanmanon and E. Keogh. Fast shapelets: A scalable algorithm for discovering time series shapelets. In Proceedings of the 2013 SIAM International Conference on Data Mining, pages 668–676. SIAM, 2013.
S. Rani and G. Sikka. Recent techniques of clustering of time series data: a survey. Int. J. Comput. Appl, 52(15):1–9, 2012.
G. Robinson and M. Thiel. Recurrences determine the dynamics. Chaos: An Interdisciplinary Journal of Nonlinear Science, 19(2):023104, 2009.
P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65, 1987.
H. Sakoe and S. Chiba. Dynamic programming algorithm optimization for spoken word recogni- tion. Acoustics, Speech and Signal Processing, IEEE Transactions on, 26(1):43–49, 1978.
T. Schreiber and A. Schmitz. Surrogate time series. Physica D: Nonlinear Phenomena, 142(3): 346–382, 2000.
P. Senin and S. Malinchik. Sax-vsm: Interpretable time series classification using sax and vector space model. In Data Mining (ICDM), 2013 IEEE 13th International Conference on, pages 1175–1180. IEEE, 2013.
D. F. Silva, V. M. De Souza, and G. E. Batista. Time series classification using compression distance of recurrence plots. In Data Mining (ICDM), 2013 IEEE 13th International Conference on, pages 687–696. IEEE, 2013.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv:1409.1556, 2014.
P. Smyth et al. Clustering sequences with hidden markov models. Advances in neural information processing systems, pages 648–654, 1997.
T. A. Snijders. Markov chain monte carlo estimation of exponential random graph models. Journal of Social Structure, 3(2):1–40, 2002.
V. M. Souza, D. F. Silva, and G. E. Batista. Extracting texture features for time series classifica- tion. In Pattern Recognition (ICPR), 2014 22nd International Conference on, pages 1425–1430. IEEE, 2014.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
F. Takens. Detecting strange attractors in turbulence. Springer, 1981.
M. Thiel, M. C. Romano, J. Kurths, R. Meucci, E. Allaria, and F. T. Arecchi. Influence of obser- vational noise on the recurrence quantification analysis. Physica D: Nonlinear Phenomena, 171 (3):138–152, 2002.
M. Thiel, M. Romano, P. Read, and J. Kurths. Estimation of dynamical invariants without em- bedding by recurrence plots. Chaos: An Interdisciplinary Journal of Nonlinear Science, 14(2): 234–243, 2004a.
M. Thiel, M. C. Romano, and J. Kurths. How much information is contained in a recurrence plot? Physics Letters A, 330(5):343–349, 2004b.
J. J. Van Wijk and E. R. Van Selow. Cluster and calendar based visualization of time series data. In Information Visualization, 1999.(Info Vis’ 99) Proceedings. 1999 IEEE Symposium on, pages 4–9. IEEE, 1999.
M. Vlachos, J. Lin, E. Keogh, and D. Gunopulos. A wavelet-based anytime algorithm for k-means clustering of time series. In In Proc. Workshop on Clustering High Dimensionality Data and Its Applications. Citeseer, 2003.
S. Wasserman and P. Pattison. Logit models and logistic regressions for social networks: I. an introduction to markov graphs andp. Psychometrika, 61(3):401–425, 1996.
Y. Xia, O. N. Bjørnstad, and B. T. Grenfell. Measles metapopulation dynamics: a gravity model for epidemiological coupling and dynamics. The American Naturalist, 164(2):267–281, 2004. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European