Temporal Deep Learning for Financial Time Series

Abstract

The widespread usage of machine learning in different mainstream contexts has made deep learning the solution of choice in various domains, including finance. However, the real-world application of deep learning in the stock market is still emerging. When practitioners are applying machine learning to the stock market, there are three primary stages, namely (1) data/feature engineering, (2) price forecast, and (3) investment decision and strategy. Considerations across all three stages include the unique characteristics of financial time series data, the need to understand model predictions, and ease of use. Historically, deep learning is considered to not excel under these considerations, hence it is common to see classical statistical and traditional machine learning methods in use by market practitioners. In this Ph.D. thesis, we focus on advancing this area of research by combining the state-of-the-art in both financial statistics and deep learning. In this thesis, we put financial-specific data attributes into consideration to produce novel algorithms based on the temporal Transformer deep learning architecture, a state-of-the-art deep learning approach that draws dependencies between data sequences using a mechanism called attention. By introducing temporal Transformers focused on the financial domain, we illustrate its predictive use case for financial time series. We also incorporate discussions on explainable AI (XAI), to mitigate the black-box nature often associated with such deep learning algorithms. We address the first two stages by introducing similarity embedded temporal Transformer (SeTT) and run-similarity embedded temporal Transformer (r-SeTT) algorithms that combine temporal Transformer architecture with time series forecasting and statistical principles. We employ similarity vectors generated from historical trends across different financial instruments that are used to adjust the weight of the temporal Transformer model during the training process. This approach takes advantage of the conditional heteroscedasticity in financial time series, by using the historical volatility in combination with the attention mechanism in a temporal Transformer deep neural network architecture. Our experimentation shows that by focusing on the historical windows that are most similar to the current window in the attention-tuning process, we outperform both classical financial models and the baseline temporal Transformer model in terms of predictive performance. To effectively utilize an extended history of financial time series data, we further develop an ensemble algorithm called windowing ensemble of temporal Transformers (WETT). Our ensemble algorithm leverages a combination of base models generated from sliding windows of historical timeframes, with additional weight initialization diversification options for a complete experimentation regime. By decomposing the constituent time series of the extended timeframe, we optimize the utilization of the series for financial deep learning. This simplifies the training process while achieving better performance, particularly when accounting for the non-constant variance of financial time series. To address the last stage of applying complex deep learning architectures to the financial market, we examined advancements in XAI and its application in facilitating investment decision-making processes. Our discussion centers on incorporating XAI into our model development pipeline, through the use of surrogate models. This step aims not only to augment our comprehension of the temporal Transformer model but also to improve the model's predictive capabilities by assessing the effectiveness of the input features.