Repository files navigation

• Compare the accuracy of various time series forecasting algorithms such asProphet,DeepAR,VAR,DeepVAR, andLightGBM
• (Optional) Usetsfresh for automated feature engineering of time series data.

• The dataset can be downloaded fromthis Kaggle competition.
• In addition to theAnaconda libraries, you need to installaltair,vega_datasets,category_encoders,mxnet,gluonts,kats,lightgbm,hyperopt andpandarallel.
  • kats requires Python 3.7 or higher.

• The M5 Competition aims to forecast daily sales for the next 28 days based on sales over the last 1,941 days for IDs of 30,490 items per Walmart store.
• Data includes (i) time series of daily sales quantity by ID, (ii) sales prices, and (iii) holiday and event information.
• Evaluation is done throughWeighted Root Mean Squared Scaled Error. A detailed explanation is given in the M5 Participants Guide and the implementation is atthis link.
• For hyperparameter tuning, 0.1% of IDs were randomly selected and used, and 1% were used to measure test set performance.

• Prophet can incorporate forward-looking related time series into the model, so additional features were created with holiday and event information.
• Since aProphet model has to fit for each ID, I had to use theapply function of thepandas dataframe and instead usedpandarallel to maximize the parallelization performance.
• Prophet hyperparameters were tuned through 3-fold CV using theBayesian Optimization module built into theKats library. In this case,Tweedie was applied as the loss function. Below is the hyperparameter tuning result.

• In the figures below, the actual sales (black dots), the point predictions and confidence intervals (blue lines and bands), and the red dotted lines representing the test period are shown.

• SinceVAR is a multivariate time series model, the more IDs it fits simultaneously, the better the performance, and the memory requirement increases exponentially.

• DeepAR can incorporate metadata and forward-looking related time series into the model, so additional features were created with sales prices, holiday and event information. Dynamic categorical variables were quantified throughFeature Hashing.
• As a hyperparameter, it is very important to set the probability distribution of the output, and here it is set as theNegative Binomial distribution.

• In the case ofDeepVAR, a multivariate model, what can be set as the probability distribution of the output is limited (i.e.Multivariate Gaussian distribution), which leads to a decrease in performance.

• I usedtsfresh to convert time series into structured data features, which consumes a lot of computational resources even with minimal settings.
• ALightGBMTweedie regression model was fitted. Hyperparameters were tuned via 3-fold CV using theBayesian Optimization function of thehyperopt library. The following is the hyperparameter tuning result.

• The sales forecast for day D+1 was used recursively to predict the sales volume for day D+2 through feature engineering, and through this iterative process, 28-day test set performance was measured.

As a result,DeepAR was finally selected and submitted its predictions to Kaggle, achieving a WRMSSE value of 0.8112 based on the private leaderboard.

• Taylor SJ, Letham B. 2017. Forecasting at scale.PeerJ Preprints 5:e3190v2
• Prophet: Forecasting at Scale
• Stock, James, H., Mark W. Watson. 2001. Vector Autoregressions.Journal of Economic Perspectives, 15 (4): 101-115.
• David Salinas, Valentin Flunkert, Jan Gasthaus, Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks,International Journal of Forecasting, 36 (3): 1181-1191.
• David Salinas, Michael Bohlke-Schneider, Laurent Callot, Roberto Medico,Jan Gasthaus. 2019. High-dimensional multivariate forecasting with low-rank Gaussian Copula Processes.In Advances in Neural Information Processing Systems. 6827–6837.
• Kats - One Stop Shop for Time Series Analysis in Python
• GluonTS - Probabilistic Time Series Modeling