Representing Time: Towards Pragmatic Multivariate Time Series Modeling

 Time series models are specialized in learning temporal dependencies among observations and interactions between multiple features in a data stream. During  the last decade, the unprecedented success of Deep Learning (DL) models in Computer Vision and Natural Language Processing has steadily permeated to time series tasks. From Recurrent Neural Networks to Transformers, new advancements in  architectural design improved capabilities and performance. Despite this success,  I identify several challenges to adopting current state-of-the-art (SoTA) methods,  including handling distribution shifts and missing data, computational complexity,  and interpretability.  

The success of DL models is usually attributed to their ability to discover helpful  data representations automatically. Multivariate time series models involve high dimensional objects with numerous time series and temporal observations. However,  they often exhibit strong temporal dependencies and inter-feature relations. In  this thesis, I propose to design DL architectures and algorithms for forecasting  and anomaly detection tasks that leverage these dependencies to induce efficient  learning of representations that satisfy desirable properties that can (i) improve the  models’ performance, (ii) improve robustness by favoring domain adaptation, and  (iii) reduce over-parameterization to improve scalability. The completed work is  organized in three parts, presenting seven novel model types and algorithms that  achieve state of the art performance in various tasks while addressing key adoption  challenges.  

In the first part, I explore the dynamic latent space principle and design latent  temporal representations to make robust anomaly detection and forecasting models.  In the second part, I present a novel scalable and interpretable model for multi-step  forecasting based on a novel non-linear frequency decomposition with connections  to Wavelet theory. It also features two extensions on using multivariate exogenous  covariates for high-impact domains, including energy and healthcare. Finally, in  the third part, I present a large-scale study on enabling conditions, on both model  design and data characteristics, for transferability of pre-trained models on time  series tasks