# American Institute of Mathematical Sciences

June  2015, 2(2): 247-265. doi: 10.3934/jcd.2015005

## A kernel-based method for data-driven koopman spectral analysis

 1 United Technologies Research Center, 411 Silver Lane, East Hartford, CT 06118, United States 2 Dept. of Mechanical and Aerospace Engineering, Princeton University, Princeton, NJ 08544 3 Department of Chemical and Biological Engineering & PACM, Princeton University, Princeton, NJ 08544, United States

Received  February 2015 Revised  February 2016 Published  May 2016

A data-driven, kernel-based method for approximating the leading Koopman eigenvalues, eigenfunctions, and modes in problems with high-dimensional state spaces is presented. This approach uses a set of scalar observables (functions that map a state to a scalar value) that are defined implicitly by the feature map associated with a user-defined kernel function. This circumvents the computational issues that arise due to the number of functions required to span a sufficiently rich'' subspace of all possible scalar observables in such applications. We illustrate this method on two examples: the first is the FitzHugh-Nagumo PDE, a prototypical one-dimensional reaction-diffusion system, and the second is a set of vorticity data computed from experimentally obtained velocity data from flow past a cylinder at Reynolds number 413. In both examples, we use the output of Dynamic Mode Decomposition, which has a similar computational cost, as the benchmark for our approach.
Citation: Matthew O. Williams, Clarence W. Rowley, Ioannis G. Kevrekidis. A kernel-based method for data-driven koopman spectral analysis. Journal of Computational Dynamics, 2015, 2 (2) : 247-265. doi: 10.3934/jcd.2015005
##### References:
 [1] S. Bagheri, Koopman-mode decomposition of the cylinder wake, Journal of Fluid Mechanics, 726 (2013), 596-623. doi: 10.1017/jfm.2013.249. [2] S. Bagheri, Effects of weak noise on oscillating flows: Linking quality factor, Floquet modes, and Koopman spectrum, Physics of Fluids, 26 (2014), 094104. doi: 10.1063/1.4895898. [3] G. Baudat and F. Anouar, Kernel-based methods and function approximation, In Proceedings of the International Joint Conference on Neural Networks, IEEE, 2 (2001), 1244-1249. doi: 10.1109/IJCNN.2001.939539. [4] C. M. Bishop et al, Pattern Recognition and Machine Learning, Springer, 2006. doi: 10.1007/978-0-387-45528-0. [5] J. P. Boyd, Chebyshev and Fourier Spectral Methods, Courier Dover Publications, Mineola, NY, 2001. [6] M. Budišić, R. Mohr and I. Mezić, Applied Koopmanism, Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), 047510, 33pp. doi: 10.1063/1.4772195. [7] C. J. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2 (1998), 121-167. [8] A. Chatterjee, An introduction to the proper orthogonal decomposition, Current Science, 78 (2000), 808-817. [9] K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, Journal of Nonlinear Science, 22 (2012), 887-915. doi: 10.1007/s00332-012-9130-9. [10] R. R. Coifman and S. Lafon, Diffusion maps, Applied and Computational Harmonic Analysis, 21 (2006), 5-30. doi: 10.1016/j.acha.2006.04.006. [11] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, 2000. doi: 10.1017/CBO9780511801389. [12] C. E. Elmer and E. S. Van Vleck, Spatially discrete FitzHugh-Nagumo equations, SIAM Journal on Applied Mathematics, 65 (2005), 1153-1174. doi: 10.1137/S003613990343687X. [13] P. Gaspard and S. Tasaki, Liouvillian dynamics of the {Hopf} bifurcation, Physical Review E, 64 (2001), 056232, 17pp. doi: 10.1103/PhysRevE.64.056232. [14] M. S. Hemati, M. O. Williams and C. W. Rowley, Dynamic mode decomposition for large and streaming datasets, Physics of Fluids, 26 (2014), 111701. doi: 10.1063/1.4901016. [15] P. Holmes, J. L. Lumley, G. Berkooz and C. W. Rowley, Turbulence, Coherent Structures, Dynamical Systems and Symmetry, Cambridge University Press, 2nd edition, 2012. doi: 10.1017/CBO9780511919701. [16] M. R. Jovanović, P. J. Schmid and J. W. Nichols, Sparsity-promoting dynamic mode decomposition, Physics of Fluids, 26 (2014), 024103. [17] J.-N. Juang, Applied System Identification, Prentice Hall, 1994. [18] B. O. Koopman and J. von Neumann, Dynamical systems of continuous spectra, Proceedings of the National Academy of Sciences of the United States of America, 18 (1932), 255-163. doi: 10.1073/pnas.18.3.255. [19] B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proceedings of the National Academy of Sciences of the United States of America, 17 (1931), 315-318. doi: 10.1073/pnas.17.5.315. [20] J. A. Lee and M. Verleysen, Nonlinear Dimensionality Reduction, Springer, 2007. doi: 10.1007/978-0-387-39351-3. [21] R. B. Lehoucq, D. C. Sorensen and C. Yang, ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, volume 6. SIAM, 1998. doi: 10.1137/1.9780898719628. [22] A. Mauroy and I. Mezic, A spectral operator-theoretic framework for global stability, In 52nd IEEE Conference on Decision and Control, (2013), 5234-5239. doi: 10.1109/CDC.2013.6760712. [23] I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dynamics, 41 (2005), 309-325. doi: 10.1007/s11071-005-2824-x. [24] I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annual Review of Fluid Mechanics, 45 (2013), 357-378. doi: 10.1146/annurev-fluid-011212-140652. [25] C. E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, 2006. [26] C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, Journal of Fluid Mechanics, 641 (2009), 115-127. doi: 10.1017/S0022112009992059. [27] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, Journal of Fluid Mechanics, 656 (2010), 5-28. doi: 10.1017/S0022112010001217. [28] P. J. Schmid, D. Violato and F. Scarano, Decomposition of time-resolved tomographic PIV, Experiments in Fluids, 52 (2012), 1567-1579. doi: 10.1007/s00348-012-1266-8. [29] P. J. Schmid, L. Li, M. P. Juniper and O. Pust, Applications of the dynamic mode decomposition, Theoretical and Computational Fluid Dynamics, 25 (2011), 249-259. doi: 10.1007/s00162-010-0203-9. [30] B. Scholkopf, The kernel trick for distances, Advances in Neural Information Processing Systems, (2001), 301-307. [31] L. Sirovich, Turbulence and the dynamics of coherent structures. part I: Coherent structures, Quarterly of applied mathematics, 45 (1987), 561-571. [32] G. Tissot, L. Cordier, N. Benard and B. R. Noack, Model reduction using dynamic mode decomposition, Comptes Rendus Mćcanique, 342 (2014), 410-416. doi: 10.1016/j.crme.2013.12.011. [33] J. H. Tu, C. W. Rowley, J. N. Kutz and J. K. Shang, Toward compressed DMD: Spectral analysis of fluid flows using sub-Nyquist-rate PIV data, Experiments in Fluids, 55 (2014), 1-13. [34] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421. doi: 10.3934/jcd.2014.1.391. [35] M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, Journal of Nonlinear Science, 25 (2015), 1307-1346. doi: 10.1007/s00332-015-9258-5. [36] A. Wynn, D. S. Pearson, B. Ganapathisubramani and P. J. Goulart, Optimal mode decomposition for unsteady flows, Journal of Fluid Mechanics, 733 (2013), 473-503. doi: 10.1017/jfm.2013.426.

show all references

##### References:
 [1] S. Bagheri, Koopman-mode decomposition of the cylinder wake, Journal of Fluid Mechanics, 726 (2013), 596-623. doi: 10.1017/jfm.2013.249. [2] S. Bagheri, Effects of weak noise on oscillating flows: Linking quality factor, Floquet modes, and Koopman spectrum, Physics of Fluids, 26 (2014), 094104. doi: 10.1063/1.4895898. [3] G. Baudat and F. Anouar, Kernel-based methods and function approximation, In Proceedings of the International Joint Conference on Neural Networks, IEEE, 2 (2001), 1244-1249. doi: 10.1109/IJCNN.2001.939539. [4] C. M. Bishop et al, Pattern Recognition and Machine Learning, Springer, 2006. doi: 10.1007/978-0-387-45528-0. [5] J. P. Boyd, Chebyshev and Fourier Spectral Methods, Courier Dover Publications, Mineola, NY, 2001. [6] M. Budišić, R. Mohr and I. Mezić, Applied Koopmanism, Chaos: An Interdisciplinary Journal of Nonlinear Science, 22 (2012), 047510, 33pp. doi: 10.1063/1.4772195. [7] C. J. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2 (1998), 121-167. [8] A. Chatterjee, An introduction to the proper orthogonal decomposition, Current Science, 78 (2000), 808-817. [9] K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, Journal of Nonlinear Science, 22 (2012), 887-915. doi: 10.1007/s00332-012-9130-9. [10] R. R. Coifman and S. Lafon, Diffusion maps, Applied and Computational Harmonic Analysis, 21 (2006), 5-30. doi: 10.1016/j.acha.2006.04.006. [11] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, 2000. doi: 10.1017/CBO9780511801389. [12] C. E. Elmer and E. S. Van Vleck, Spatially discrete FitzHugh-Nagumo equations, SIAM Journal on Applied Mathematics, 65 (2005), 1153-1174. doi: 10.1137/S003613990343687X. [13] P. Gaspard and S. Tasaki, Liouvillian dynamics of the {Hopf} bifurcation, Physical Review E, 64 (2001), 056232, 17pp. doi: 10.1103/PhysRevE.64.056232. [14] M. S. Hemati, M. O. Williams and C. W. Rowley, Dynamic mode decomposition for large and streaming datasets, Physics of Fluids, 26 (2014), 111701. doi: 10.1063/1.4901016. [15] P. Holmes, J. L. Lumley, G. Berkooz and C. W. Rowley, Turbulence, Coherent Structures, Dynamical Systems and Symmetry, Cambridge University Press, 2nd edition, 2012. doi: 10.1017/CBO9780511919701. [16] M. R. Jovanović, P. J. Schmid and J. W. Nichols, Sparsity-promoting dynamic mode decomposition, Physics of Fluids, 26 (2014), 024103. [17] J.-N. Juang, Applied System Identification, Prentice Hall, 1994. [18] B. O. Koopman and J. von Neumann, Dynamical systems of continuous spectra, Proceedings of the National Academy of Sciences of the United States of America, 18 (1932), 255-163. doi: 10.1073/pnas.18.3.255. [19] B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proceedings of the National Academy of Sciences of the United States of America, 17 (1931), 315-318. doi: 10.1073/pnas.17.5.315. [20] J. A. Lee and M. Verleysen, Nonlinear Dimensionality Reduction, Springer, 2007. doi: 10.1007/978-0-387-39351-3. [21] R. B. Lehoucq, D. C. Sorensen and C. Yang, ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, volume 6. SIAM, 1998. doi: 10.1137/1.9780898719628. [22] A. Mauroy and I. Mezic, A spectral operator-theoretic framework for global stability, In 52nd IEEE Conference on Decision and Control, (2013), 5234-5239. doi: 10.1109/CDC.2013.6760712. [23] I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dynamics, 41 (2005), 309-325. doi: 10.1007/s11071-005-2824-x. [24] I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annual Review of Fluid Mechanics, 45 (2013), 357-378. doi: 10.1146/annurev-fluid-011212-140652. [25] C. E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, 2006. [26] C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter and D. S. Henningson, Spectral analysis of nonlinear flows, Journal of Fluid Mechanics, 641 (2009), 115-127. doi: 10.1017/S0022112009992059. [27] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, Journal of Fluid Mechanics, 656 (2010), 5-28. doi: 10.1017/S0022112010001217. [28] P. J. Schmid, D. Violato and F. Scarano, Decomposition of time-resolved tomographic PIV, Experiments in Fluids, 52 (2012), 1567-1579. doi: 10.1007/s00348-012-1266-8. [29] P. J. Schmid, L. Li, M. P. Juniper and O. Pust, Applications of the dynamic mode decomposition, Theoretical and Computational Fluid Dynamics, 25 (2011), 249-259. doi: 10.1007/s00162-010-0203-9. [30] B. Scholkopf, The kernel trick for distances, Advances in Neural Information Processing Systems, (2001), 301-307. [31] L. Sirovich, Turbulence and the dynamics of coherent structures. part I: Coherent structures, Quarterly of applied mathematics, 45 (1987), 561-571. [32] G. Tissot, L. Cordier, N. Benard and B. R. Noack, Model reduction using dynamic mode decomposition, Comptes Rendus Mćcanique, 342 (2014), 410-416. doi: 10.1016/j.crme.2013.12.011. [33] J. H. Tu, C. W. Rowley, J. N. Kutz and J. K. Shang, Toward compressed DMD: Spectral analysis of fluid flows using sub-Nyquist-rate PIV data, Experiments in Fluids, 55 (2014), 1-13. [34] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: Theory and applications, Journal of Computational Dynamics, 1 (2014), 391-421. doi: 10.3934/jcd.2014.1.391. [35] M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, Journal of Nonlinear Science, 25 (2015), 1307-1346. doi: 10.1007/s00332-015-9258-5. [36] A. Wynn, D. S. Pearson, B. Ganapathisubramani and P. J. Goulart, Optimal mode decomposition for unsteady flows, Journal of Fluid Mechanics, 733 (2013), 473-503. doi: 10.1017/jfm.2013.426.
 [1] Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391 [2] Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002 [3] Hao Zhang, Scott T. M. Dawson, Clarence W. Rowley, Eric A. Deem, Louis N. Cattafesta. Evaluating the accuracy of the dynamic mode decomposition. Journal of Computational Dynamics, 2020, 7 (1) : 35-56. doi: 10.3934/jcd.2020002 [4] Mojtaba F. Fathi, Ahmadreza Baghaie, Ali Bakhshinejad, Raphael H. Sacho, Roshan M. D'Souza. Time-resolved denoising using model order reduction, dynamic mode decomposition, and kalman filter and smoother. Journal of Computational Dynamics, 2020, 7 (2) : 469-487. doi: 10.3934/jcd.2020019 [5] Mustaffa Alfatlawi, Vaibhav Srivastava. An incremental approach to online dynamic mode decomposition for time-varying systems with applications to EEG data modeling. Journal of Computational Dynamics, 2020, 7 (2) : 209-241. doi: 10.3934/jcd.2020009 [6] Stefan Klus, Christof Schütte. Towards tensor-based methods for the numerical approximation of the Perron--Frobenius and Koopman operator. Journal of Computational Dynamics, 2016, 3 (2) : 139-161. doi: 10.3934/jcd.2016007 [7] Jiang Xie, Junfu Xu, Celine Nie, Qing Nie. Machine learning of swimming data via wisdom of crowd and regression analysis. Mathematical Biosciences & Engineering, 2017, 14 (2) : 511-527. doi: 10.3934/mbe.2017031 [8] Andreas Chirstmann, Qiang Wu, Ding-Xuan Zhou. Preface to the special issue on analysis in machine learning and data science. Communications on Pure and Applied Analysis, 2020, 19 (8) : i-iii. doi: 10.3934/cpaa.2020171 [9] Siwei Yu, Jianwei Ma, Stanley Osher. Geometric mode decomposition. Inverse Problems and Imaging, 2018, 12 (4) : 831-852. doi: 10.3934/ipi.2018035 [10] Jae-Hong Pyo, Jie Shen. Normal mode analysis of second-order projection methods for incompressible flows. Discrete and Continuous Dynamical Systems - B, 2005, 5 (3) : 817-840. doi: 10.3934/dcdsb.2005.5.817 [11] Stefan Klus, Péter Koltai, Christof Schütte. On the numerical approximation of the Perron-Frobenius and Koopman operator. Journal of Computational Dynamics, 2016, 3 (1) : 51-79. doi: 10.3934/jcd.2016003 [12] Yu-Ting Lin, John Malik, Hau-Tieng Wu. Wave-shape oscillatory model for nonstationary periodic time series analysis. Foundations of Data Science, 2021, 3 (2) : 99-131. doi: 10.3934/fods.2021009 [13] Ping Yan, Ji-Bo Wang, Li-Qiang Zhao. Single-machine bi-criterion scheduling with release times and exponentially time-dependent learning effects. Journal of Industrial and Management Optimization, 2019, 15 (3) : 1117-1131. doi: 10.3934/jimo.2018088 [14] Xingong Zhang. Single machine and flowshop scheduling problems with sum-of-processing time based learning phenomenon. Journal of Industrial and Management Optimization, 2020, 16 (1) : 231-244. doi: 10.3934/jimo.2018148 [15] Rakesh Pilkar, Erik M. Bollt, Charles Robinson. Empirical mode decomposition/Hilbert transform analysis of postural responses to small amplitude anterior-posterior sinusoidal translations of varying frequencies. Mathematical Biosciences & Engineering, 2011, 8 (4) : 1085-1097. doi: 10.3934/mbe.2011.8.1085 [16] Ahmad Deeb, A. Hamdouni, Dina Razafindralandy. Comparison between Borel-Padé summation and factorial series, as time integration methods. Discrete and Continuous Dynamical Systems - S, 2016, 9 (2) : 393-408. doi: 10.3934/dcdss.2016003 [17] Mickaël D. Chekroun, Jean Roux. Homeomorphisms group of normed vector space: Conjugacy problems and the Koopman operator. Discrete and Continuous Dynamical Systems, 2013, 33 (9) : 3957-3980. doi: 10.3934/dcds.2013.33.3957 [18] Émilie Chouzenoux, Henri Gérard, Jean-Christophe Pesquet. General risk measures for robust machine learning. Foundations of Data Science, 2019, 1 (3) : 249-269. doi: 10.3934/fods.2019011 [19] Ana Rita Nogueira, João Gama, Carlos Abreu Ferreira. Causal discovery in machine learning: Theories and applications. Journal of Dynamics and Games, 2021, 8 (3) : 203-231. doi: 10.3934/jdg.2021008 [20] Annalisa Pascarella, Alberto Sorrentino, Cristina Campi, Michele Piana. Particle filtering, beamforming and multiple signal classification for the analysis of magnetoencephalography time series: a comparison of algorithms. Inverse Problems and Imaging, 2010, 4 (1) : 169-190. doi: 10.3934/ipi.2010.4.169

Impact Factor: