Time-MOE: Billion-Scale Time Series Foundation Models With Mixture of Experts

In recent decades, deep learning has achieved notable advances in time series forecasting. However, unlike the domains of language and vision, where large-scale pre-training has driven major breakthroughs, time series models remain constrained by limited scalability and considerable computational costs. These challenges hinder the creation of more powerful forecasting systems suitable for practical deployment. To overcome these barriers, this chapter introduces Time-MoE, a unified and scalable architecture designed to pre-train large forecasting foundation models while reducing inference overhead. Leveraging a sparse mixture-of-experts (MoE) mechanism, Time-MoE improves efficiency by activating only a subset of experts for each prediction, thereby cutting computation while maintaining high model capacity. This design allows the model to scale without proportionally increasing inference costs. Time-MoE consists of decoder-only transformer variants trained in an auto-regressive manner, capable of handling flexible context lengths and prediction horizons. Training is performed on our newly curated large-scale time series corpus, Time-300B, which spans over 9 domains and comprises more than 300 billion time points. For the first time, the authors scale a time series foundation model to 2.4 billion parameters, yielding significant gains in forecasting accuracy. Their findings validate the applicability of scaling laws to the time series domain. Compared with dense architectures with equivalent activated parameters or computational budgets, Time-MoE achieves consistently superior performance. Collectively, these contributions establish Time-MoE as a state-of-the-art approach for real-world time series forecasting, combining scalability, accuracy, and efficiency.