Deep Learning for Time Series

Comparative Analysis of Deep Learning (GRU, LSTM, RNN) and Traditional Time Series Models

In my study, I apply deep learning models to analyze consumer sentiment from 1978 to 2023. This period covers a range of significant economic changes and technological advancements, which greatly affect consumer behavior. By using advanced deep learning techniques, such as LSTM, RNN, and GRU, which are known for their effectiveness in handling time-series data, we aim to uncover how consumer sentiment has changed over these decades. Our goal is to provide valuable insights for economists, policymakers, and businesses. This analysis is not just academically relevant; it’s crucial for understanding consumer trends and making informed decisions in various sectors.

Data Processing

Train shape: (464,)
Test shape: (83,)

RNN

The first image shows the model’s training and validation loss decreasing over epochs, with occasional spikes in validation loss suggesting possible overfitting. The second image reveals that the model’s predictions closely align with actual consumer sentiment data, indicating effective learning. The RNN model is simple yet performs well, with a low number of parameters and good accuracy, as reflected in the train and test RMSE values.

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 simple_rnn (SimpleRNN)      (None, 50)                2600      
                                                                 
 dense (Dense)               (None, 1)                 51        
                                                                 
=================================================================
Total params: 2651 (10.36 KB)
Trainable params: 2651 (10.36 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
1/3 [=========>....................] - ETA: 0s3/3 [==============================] - 0s 2ms/step
1/1 [==============================] - ETA: 0s1/1 [==============================] - 0s 15ms/step
Train RMSE = 0.34722
Test RMSE = 0.33005

LSTM

The LSTM model’s loss graph displays converging training and validation losses, which indicates good learning with minimal overfitting. Predictions closely follow actual sentiment data, showing the model’s effectiveness. Despite its complexity with more parameters, the LSTM has slightly higher RMSE scores than the RNN, which may suggest the need for further model optimization.

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 lstm (LSTM)                 (None, 50)                10400     
                                                                 
 dense_1 (Dense)             (None, 1)                 51        
                                                                 
=================================================================
Total params: 10451 (40.82 KB)
Trainable params: 10451 (40.82 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
1/3 [=========>....................] - ETA: 0s3/3 [==============================] - 0s 2ms/step
1/1 [==============================] - ETA: 0s1/1 [==============================] - 0s 16ms/step
Train RMSE = 0.31734
Test RMSE = 0.35838

GRU

The GRU model’s losses align well, with minimal overfitting. Predictions accurately track the actual sentiment, indicating strong model performance. With 8,001 parameters, the GRU outperforms the LSTM in error metrics, suggesting a better fit for sentiment analysis.

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 gru (GRU)                   (None, 50)                7950      
                                                                 
 dense_2 (Dense)             (None, 1)                 51        
                                                                 
=================================================================
Total params: 8001 (31.25 KB)
Trainable params: 8001 (31.25 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
1/3 [=========>....................] - ETA: 0s3/3 [==============================] - 0s 2ms/step
1/1 [==============================] - ETA: 0s1/1 [==============================] - 0s 15ms/step
Train RMSE = 0.30190
Test RMSE = 0.33236

Deep Learning vs. Traditional Time Series

Deep learning models like RNN, LSTM, and GRU are advanced and able to capture complex, non-linear relationships in data due to their neural network structures. They excel with large datasets and can uncover patterns traditional models might miss. However, they are “black boxes” with less interpretability, and training them requires substantial computational power and data.

Traditional time series models such as ARIMA, ARIMAX, VAR, and ARCH are more straightforward and interpretable. They are designed to work with time series data, explicitly accounting for seasonality, trends, and volatility. These models are computationally lighter and can perform well with smaller datasets. They are often chosen for their interpretability and the clear statistical assumptions they are based on.