Dynamic ensemble for probabilistic time-series forecasting via deep reinforcement learning

Ensembles have long been a cornerstone in improving performance by integrating black-box base learners. The primary approach, “stacked generalization” is inherently static, lacking adaptability to changes in input data post-training. This limitation is more pronounced in time series forecasting, where these methods struggle to manage temporal correlations or non-stationarity. Given base learners may perform better under various data conditions and timeframes, combining them naively could hurt the desirable properties of probabilistic forecasters for subsequent tasks.

To address these challenges, we introduce a novel “dynamic ensemble policy.” This policy employs a deep Reinforcement Learning framework to ensemble multi-horizon probabilistic forecasters. By incorporating intermediate ensembled samples as feedback to each base learner, we mitigate temporal error accumulation that plagues base auto-regressive learners. The resulting ensemble forecaster is dynamic, adapting to forecast start times and unseen data, while preserving the advantageous properties of auto-regressive models in terms of samplings and quantile estimates. To train this policy, we design a Markov Decision Process (MDP) by defining appropriate state, action, and transition dynamics. An environment, TS-GYM, simulates this MDP, enabled by the interactions between the agent that ensembles forecasters, offline datasets, and base forecasters. The effectiveness of the proposed framework is demonstrated on multiple synthetic and real-world experiments against several ensemble baselines.