Zero-shot traffic flow prediction with foundation models: a comparison with deep learning approaches

Abstract

Traffic flow prediction plays an important role in managing urban transportation systems, helping to reduce congestion and improve road safety. Although existing deep learning models improve their predictive accuracy with complex architectures, they require large datasets for task-specific training. Recently, the rapidly developed foundation models have shown outstanding performance in time series prediction. Motivated by the development, we apply two foundation models, Lag-Llama and Chronos, for zero-shot traffic flow prediction and compare their accuracy with that of deep learning models. Our results show that foundation models outperform deep learning models in traffic flow prediction under both normal conditions and disruptive events. Unlike deep learning models, which require large-scale historical data and extensive training time for each task, pre-trained foundation models can be directly applied to datasets with varying data sizes, traffic dynamics, and context lengths. We also find that foundation models with longer context lengths and larger model sizes achieve higher prediction accuracy but require increased inference times. Selecting an appropriate foundation model is also crucial—models trained on a comprehensive dataset are more likely to achieve superior zero-shot performance, making them a practical and efficient choice for real-world traffic prediction applications.

Introduction

Traffic flow prediction is a critical task of intelligent transportation systems (ITS)^1,2, focusing on predicting future traffic flow conditions based on historical and real-time data^3–6. Accurate traffic prediction plays a crucial role in urban planning, infrastructure management, and traffic control, which helps to reduce traffic congestion, enhance road safety, and lower environmental impacts^6–9. Due to the rapid growth of urban populations and vehicle ownership worldwide, cities face increasing challenges to maintain smooth traffic flow¹⁰. Effective prediction of traffic flows provides transportation authorities, city planners, and travellers with timely information that enables better decision-making, more efficient resource allocation, and an improved quality of life^2,11,12.

The traffic flow prediction models have evolved from traditional statistical approaches to deep learning models in recent years⁵. Initially, statistical models such as Autoregressive Integrated Moving Average (ARIMA) dominated the field due to their simplicity, interpretability¹³, and effectiveness in modelling linear¹⁴ and stationary time-series data. However, these models often struggled to capture complex, non-linear relationships in real-world traffic conditions¹⁵. To overcome these limitations, deep learning models have been applied broadly, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and more recently, Transformer-based architectures^2,6,15–18. CNNs are good at capturing local temporal patterns in traffic data by applying convolution operations along the time dimension¹⁹, while LSTMs are effective in modelling long-term temporal dependencies because of their gated mechanisms^20–22. Transformer-based models further improve on prediction accuracy by using self-attention mechanisms¹⁸ to capture both short- and long-term dependencies in traffic data more efficiently and flexibly. Despite their improved accuracy and adaptability to complex scenarios, deep learning models typically require extensive computational resources and large datasets for training¹¹, which presents challenges for practical implementation and real-time applications. Besides, these models are often developed for specific tasks²³, which may lead to overfitting and reducing their ability to generalise effectively to new or unseen data.

In the context of government mobility interventions and large-scale disruptive events, such as COVID-19, traffic flow prediction becomes even more challenging. The imposed restrictions, such as lockdowns and social distancing measures, caused rapid changes in travel behaviour, leading to increased irregularities and deviations from historical traffic trends^24–30. Most existing traffic prediction models are trained only on pre-pandemic data^12,13,22,31, which limits their ability to predict traffic flows during periods of significant disruption accurately. Limited models have attempted to address this challenge by considering external factors, such as the effects of imposed response measure³² and the evolving status of COVID-19³³. Another study decomposed irregular traffic flows into distinct attributes and predicted them separately to improve model performance during the pandemic³⁴. However, these models often rely on extensive, labeled datasets that capture the effects of mobility restrictions and event status over time, making them difficult to implement in real time and across regions with differing policies.

Recently, the development of Large Language Models (LLMs) has stimulated interest in developing ‘foundation models’ for time serie²³ and introduced new opportunities for traffic flow prediction^5,35. Initially designed for natural language processing tasks, LLMs learn extensive general knowledge by pre-training on large amounts of textual data^36–38. A distinctive strength of pre-trained LLMs is their ability to perform zero-shot prediction, enabling them to perform various tasks without requiring task-specific training examples^39–41. Models such as GPT have demonstrated impressive zero-shot performance across numerous language understanding and generation tasks from different domains^36,42–44. Motivated by these advances, researchers have started exploring foundation models for time series prediction^40,41,45. By specifically pre-training existing transformer-based model architectures on large-scale time-series datasets, these models learn to effectively capture temporal patterns and dynamics in sequential data^23,46. Inspired by this capability, we are motivated to apply time-series foundation models to zero-shot traffic flow prediction tasks, enabling accurate forecasting of traffic conditions without the need for large amounts of task-specific data.

The overarching goal of this research is to present a comprehensive analysis comparing the performance of deep learning models and time-series foundation models for traffic flow prediction under normal conditions and during disruptive events. The contribution of this paper is threefold. First, it bridges the research gap in applying foundation models to traffic flow prediction by comparing the performance of two groups of time series foundation models and deep learning models on the SCOOT dataset⁴⁷. Second, it evaluates prediction accuracy on heterogeneous and unusual traffic patterns, an area that has been sparsely explored in previous research. It utilises a long-term traffic flow dataset that includes a unique global pandemic period, which allows models to capture long-term traffic trends, seasonal fluctuations, and emergency-related variations, contributing to more robust predictive performance. Third, it highlights the performance gap between time-series foundation models with different model sizes and the diversity and temporal coverage of their training data. A well-trained foundation model with comprehensive datasets is more likely to achieve superior zero-shot performance, making it a practical and efficient choice for real-world traffic flow prediction applications. The research outputs will support city planners in integrating time-series foundation models into intelligent traffic control systems, thereby enhancing their ability to respond effectively to both routine traffic conditions and unexpected disruptions. It is valuable in helping transportation authorities and urban policymakers make informed, data-driven decisions in traffic management for future large-scale emergencies.

Background

Traffic flow prediction

Traffic prediction aims to forecast key factors such as vehicle flow, speed, and congestion levels^48–54. Traffic flow prediction is one of the most fundamental and widely studied tasks in ITS. Traditional approaches typically rely on statistical models, such as ARIMA⁵⁵ or Kalman filter⁵⁶, to capture traffic patterns and seasonalities, often serving as a strong baseline when data exhibit relatively stable trends¹⁴. While these models are relatively straightforward and interpretable¹³, they may struggle with irregular fluctuations in large-scale transportation networks¹⁵. To address these complexities, researchers introduced machine learning models such as Random Forests⁵⁷ and Support Vector Machines⁵⁸. By integrating a richer set of input features, these methods can account for additional factors like weather conditions, special events, or road incidents⁵⁹. Although more flexible than purely statistical techniques, they often struggle to achieve consistently robust performance across diverse traffic scenarios.

In recent years, deep learning approaches have shown significant promise due to their ability to automatically extract features and handle complex dependencies. Convolutional Neural Networks (CNNs), traditionally used for image data, have been adapted for time series prediction by applying convolutional filters along the temporal dimension¹⁹. This allows CNNs to extract local features, detect short-term patterns, and reduce noise in traffic data. Recurrent Neural Networks (RNNs) further enhance sequence modelling by processing time series data, capturing the dynamic behaviour of traffic flow^60–62. To overcome limitations such as vanishing gradients in standard RNNs, Long Short-Term Memory (LSTM) networks, an advanced type of RNN, employ gating mechanisms to maintain long-term dependencies^63–67, making them especially effective for predicting complex temporal patterns, such as rush-hour peaks or irregular traffic flows.

More recently, transformer-based models have gained attention for traffic flow prediction due to their self-attention mechanism⁶⁸, which enable efficient modelling of long-term temporal dependencies and global interactions without relying on recurrent structures. By capturing both short- and long-term patterns in parallel, transformers have demonstrated strong performance and scalability in large-scale traffic prediction tasks. Among these models, Informe¹⁸ is a representative transformer-based architecture for long-sequence time series forecasting that improves computational efficiency through a probabilistic, sparse self-attention mechanism while maintaining the ability to capture long-term temporal dependencies.

Large language models

Recent progress in computer hardware and the availability of large text datasets have led to the development of transformer-based LLMs that demonstrate impressive performance on various natural language processing tasks^11,37,69,70. Language models are designed to predict the next token in a sequence by estimating the probability of each token based on those that have already appeared²³. Tokens may be characters, subwords⁷¹, or words from a vocabulary. The transformer architecture⁷² was initially developed as an encoder-decoder system for machine translation²³ and is currently applied in many popular models, such as BART⁷³ and T5⁷⁴. In these models, the input text is first converted into a continuous representation using an encoder, after which the decoder generates output tokens sequentially based on the representation and previous tokens. Alternatively, a decoder-only architecture, used in models like GPT-3³⁶ and Llama 2⁶⁹, only considers tokens before the current token when making predictions. This architecture simplifies the model’s design while still achieving robust performance.

LLMs are trained on extensive collections of text and can have millions to hundreds of billions of parameters ^74,75. Researchers have found that increasing the number of parameters in these models leads to better performance ³⁶. When the number of parameters becomes large enough, LLMs perform traditional language tasks more accurately and show new abilities that smaller models lack¹¹. Zero-shot generalisation is one such ability where the model makes predictions on tasks it was not explicitly trained for²³. For example, Brown et al.³⁶ demonstrated that as the number of parameters increases, LLMs acquire the ability to handle new tasks without additional task-specific training. This connection between model parameters and zero-shot generalisation highlights that LLMs not only improve their flexibility and power in language understanding and generation but also become capable of tackling challenges such as forecasting time-series data.

Large language models for time-series prediction

LLMs have recently developed as useful tools for time series forecasting by using their powerful sequence modelling and pattern recognition capabilities³⁹. PromptCast⁴⁵ first treats time series forecasting as a natural language generation task, converting numerical inputs and outputs into textual prompts, thus allowing general-purpose language models to serve as core forecasting engines. However, PromptCast often requires carefully designed prompts, which can be time-consuming in complex or domain-specific scenarios. LLMTime⁴⁰ addresses these limitations by directly tokenising time-series data and treating forecasting as next-token prediction. This tokenisation strategy not only avoids extensive prompt engineering but also enables pre-trained LLMs, such as GPT-3 and LLaMA, to produce robust zero-shot forecasts across a variety of benchmark datasets^23,40. However, LLMTime can be computationally and memory-intensive due to the large size of the models, and it requires careful data rescaling to handle varying magnitudes and precision.

Time series foundation models

Unlike PromptCast and LLMTime, which repurpose large pre-trained LLMs with textual or digit-based prompts, researchers have further developed time series foundation models by training them on large, diverse time series datasets. Rasul et al. (2024) propose a foundation model (Lag-Llama) designed explicitly for univariate time series forecasting. Built on a decoder-only transformer architecture that uses lag features as covariates, Lag-Llama is pre-trained on a broad collection of real-world time series across multiple domains, including energy, transportation, economics, environmental science, air quality and cloud operations. This large-scale pre-training process enables it to learn a wide range of time-series patterns, resulting in strong zero-shot generalisation performance. Recent concurrent work, Chronos²³, offers a similarly broad framework for pretrained time series forecasting but adapts transformer-based language model architectures T5⁷⁴ to treat real-valued time series as discrete tokens. Using scaling and uniform binning, Chronos converts continuous sequences into a fixed vocabulary. Once tokenised, it trains a language model on an extensive collection of public and synthetic time-series datasets, thus learning to model a wide range of temporal patterns. Chronos demonstrates superior performance across 42 benchmark datasets, outperforming in-domain and zero-shot scenarios.

Data and methods

Datasets

This section details the dataset employed to evaluate the predictive performance of deep learning and foundation models, with real-world traffic data collected via a Split Cycle Offset Optimisation Technique (SCOOT) based Urban Traffic Control system⁴⁷. The SCOOT uses a network of sensors to capture traffic flow data across the road network. The dataset includes traffic flows from the Glasgow City Council area over four consecutive years, from 1 October 2019 to 30 September 2023, which includes the COVID-19 pandemic period⁹. The SCOOT dataset contains 470 sensors that record traffic flows at 60-min intervals.

Figures 1 and 2 compare the attributes of the SCOOT dataset with those of other traffic flow datasets used in recent traffic flow prediction research from 2022 to 2024^{15,22,31,54,76–85}. Most datasets cover no more than one year during normal periods and use time intervals of less than 30 min^21,86–96, while the SCOOT dataset covers longer than many previous studies. Although one study utilised a seven-year traffic datase⁷⁹, which is longer than the SCOOT dataset, it only covers a period of stable traffic conditions. In contrast, the SCOOT dataset provides traffic flows before, during, and after COVID-19, providing valuable insights into the drastic changes in human mobility patterns in response to government mobility interventions during a period of significant disruption. Its long-term coverage enables models to learn long-term traffic trends, seasonal fluctuations, and emergency-related variations, thereby improving predictive performance. Additionally, the bubble size in Fig. 1 corresponds to the number of data points, and the SCOOT dataset contains a relatively large number of observations. This large volume of data enhances deep learning and foundation models by improving their ability to learn complex traffic patterns and reducing the risk of overfitting.

Fig. 1.

Time span, time interval, and data volume of traffic flow datasets.

Fig. 2.

Time scope of traffic flow datasets.

Deep learning and foundation models

We select three widely used deep learning models for time series analysis, CNN⁹⁷, LSTM⁹⁸ and the transformer-based model Informe¹⁸, as well as two recently developed time series foundation models, Lag-Llama⁴⁶ and Chronos²³, to assess the performance of traffic flow prediction. The details of the models are outlined as follows:

Convolutional neural network (CNN)

CNN⁹⁷ captures temporal patterns by applying convolutional filters to sliding windows of sequential data. This approach effectively detects local trends and short-term dependencies, enhancing prediction accuracy. In our CNN model, two one-dimensional convolutional layers are employed, with each utilising the ReLU activation function. The first layer specifies the input shape based on the sequence length and extracts initial local features from the traffic flow data. A subsequent convolutional layer further refines these features. A max-pooling layer then reduces the dimensionality of the resulting feature maps, preserving essential representations while reducing computational cost. Finally, the network is flattened and regularised with a dropout layer to prevent overfitting, followed by a dense layer that produces forecasts over 6 time steps.

Long short-term memory (LSTM)

LSTM⁹⁸ effectively models long-term dependencies and sequential relationships in time series data via gated memory cells that retain relevant historical information while addressing vanishing gradient issues. Our LSTM model includes two hidden layers. The first LSTM layer, configured with the tanh activation function and set to return sequences, processes the input sequence to extract temporal features and passes the entire sequence to the subsequent layer. The second LSTM layer further refines these features using the tanh activation function. Each LSTM layer is followed by a dropout layer to reduce the risk of overfitting. Finally, a dense layer with six neurons is employed for multi-step prediction.

Informer

Informe¹⁸ is a transformer-based model specifically designed for long-sequence time series forecasting. Unlike standard transformers, which suffer from quadratic computational complexity with respect to sequence length, Informer introduces a probabilistic sparse self-attention mechanism that selects the most informative query–key pairs, significantly reducing memory usage and computational cost. In addition, Informer applies distilling operations between attention layers to progressively reduce sequence length while preserving salient temporal information. Informer incorporates a generative decoder structure that produces multi-step forecasts in a single forward step rather than step-by-step, significantly improving inference speed for multi-step prediction tasks. In this study, the Informer model contains two encoder layers and one decoder layer, with a batch size of 32.

Lag-Llama

Lag-Llama⁴⁶ is a foundation model for univariate probabilistic time series prediction, built on a decoder-only transformer architecture, LLaMA³⁸. Lag-Llama was released on 7 February 2024 and contained 2.45 million parameters. Lag-Llama tokenises input data by constructing lagged feature vectors using historical observations at predetermined lag intervals. These intervals include multiple standard frequencies such as quarterly, monthly, weekly, daily, hourly, and second-level frequencies. Each token also incorporates temporal covariates derived from date-time features such as hour-of-day, day-of-week, and month-of-year, enriching the representation and providing contextual information to the model. The input tokens, composed of lagged features and temporal covariates, are projected into a hidden representation and passed through a series of causally masked transformer decoder layers, employing RMSNorm and Rotary Positional Encoding (RoPE) at each attention layer. The final output from the transformer decoder is fed into a distribution head that predicts the parameters of a Student’s t-distribution (degrees of freedom, mean, and scale) for probabilistic forecasting. Detailed definitions of the lag structures and temporal covariates, including the specific lag intervals and feature construction process, are provided in the original Lag-Llama paper⁴⁶.

Lag-Llama applies a robust scaling procedure using median and interquartile range (IQR) normalisation to handle numerical scale variations across different time series, significantly improving training stability and forecast accuracy. During training, Lag-Llama minimises the negative log-likelihood of the forecast distribution for future values. Lag-Llama is pre-trained on 27 datasets categorised into six domains: air quality, transportation, economics, nature, energy, and cloud operations. The pre-training corpus comprises 7,965 univariate series totaling about 352 million data tokens. This extensive and diverse corpus improves Lag-Llama’s ability to generalise and deliver strong zero-shot forecasting performance.

Chronos

Chronos²³ is a pre-trained probabilistic forecasting framework designed specifically for time series, built on transformer-based language models. The core innovation of Chronos is its approach to treating time-series forecasting similarly to natural language modeling. It achieves this by tokenising continuous time series data into discrete tokens using a two-step approach: scaling and quantisation. Firstly, Chronos tokenises time series data by scaling each series individually using mean scaling, which normalises the data based on the mean of absolute historical values. Then, the scaled data are quantised into discrete bins, forming tokens from a fixed-size vocabulary. This vocabulary includes numerical bins and special tokens such as PAD (for padding sequences to equal lengths) and EOS (end-of-sequence). Detailed descriptions of the tokenisation and scaling procedures, including bin construction and scaling strategies, are provided in the original Chronos paper²³.

The Chronos modelling framework applied in this study was released on 13 March 2024. Chronos primarily employs variants of the T5 family of transformer-based language models, ranging from smaller models with approximately 8 million parameters to larger models of up to 710 million⁷⁴. These models are trained in 5 sizes, named Tiny (8 M), Mini (20 M), Small (46 M), Base (200 M) and Large (710 M), using a cross-entropy loss function, effectively framing regression as a classification task over discrete quantised bins. Chronos models provide probabilistic forecasts by autoregressively sampling from learned categorical distributions and subsequently mapping the sampled tokens back to continuous numerical values via dequantisation and inverse scaling. To enhance training, Chronos utilises data augmentation methods: TSMixup, which creates augmented series through convex combinations of existing series, and KernelSynth, which generates synthetic series using Gaussian processes. Chronos was pre-trained on 28 datasets, comprising publicly available datasets spanning transport, retail, energy, finance, healthcare, and climate science, complemented by synthetic datasets. The comprehensive benchmark evaluation involved 42 datasets to assess in-domain and zero-shot forecasting performance.

Model implementations

To evaluate the model’s performance on usual traffic patterns and unusual traffic dynamics, we divide the SCOOT dataset into two subgroups: the entire dataset, including the pandemic period, and the post-COVID-19 dataset. Based on the Stringency Index, the entire dataset contains hourly traffic flow data from 1 October 2019 to 30 September 2023, while the post-COVID-19 dataset includes data from 3 June 2022⁹⁹. Each subgroup is chronologically divided into training (60%), validation (20%), and testing (20%) sets, with a 60‑minute interval for both training and prediction. To assess the impact of context length on prediction accuracy, we train models with varying context lengths. Specifically, the context length is set to 24 × n hours, where n ranges from 1 to 21, limited by the available computational memory. These context lengths are used to predict traffic flow over the next 6 h, a common forecasting horizon in existing research^{20,86,100,101}.

We conduct experiments with different hyperparameters for each context length and dataset to train deep learning models, selecting the best-performing configuration for comparison. Informer is trained using its default model configuration. For all models, the evaluation metrics are computed at each forecast step and averaged across the six prediction steps to obtain the reported results. For deep learning models that produce deterministic point predictions, the evaluation metrics are further averaged across 10 independent inference runs to reduce the randomness of individual training runs. For foundation models, Lag-Llama generates 100 samples from its predictive distribution for each forecast step, while Chronos produces predicted quantile values at 10 fixed probability levels (0.1, 0.2, …, 0.9). The evaluation metrics for foundation models are therefore computed from aggregated predictions. The Adam optimizer is used for training all deep learning models, with CNN and LSTM models trained for 100 epochs and the Informer model trained for 20 epochs. The best hyperparameter settings for CNN and LSTM are summarised in Tables 1 and 2, respectively, while the variance of evaluation metrics for each model is recorded in Table 3. Mean Squared Error (MSE) is used as the loss function during the model training¹⁰²:

1

Table 1.

Hyperparameters of CNN.

Context Length	Model Hyper-parameter Dataset	CNN
	Filters		Kernel Size		Dropout Rate		Learning Rate		Batch Size
	Post-COVID	Entire	Post-COVID	Entire	Post-COVID	Entire	Post-COVID	Entire	Post-COVID	Entire
24	128	128	3	3	0.3	0.3	0.0005	0.0001	64	64
48	64	128	3	3	0.0	0.3	0.0005	0.0001	128	128
72	64	128	5	5	0.3	0.1	0.001	0.0001	64	64
96	64	64	3	5	0.0	0.3	0.0005	0.0001	128	32
120	64	64	3	5	0.1	0.2	0.0005	0.0005	128	128
144	32	64	2	5	0.3	0.3	0.0005	0.0001	32	128
168	32	64	2	5	0.1	0.3	0.0005	0.0005	64	64
192	128	32	3	5	0.2	0.3	0.0001	0.001	32	128
216	64	64	3	5	0.2	0.3	0.0001	0.0001	32	64
240	32	32	5	3	0.0	0.3	0.0001	0.0001	32	32
264	128	64	3	3	0.3	0.3	0.0005	0.0001	32	128
288	32	128	3	2	0.3	0.3	0.0001	0.001	32	128
312	32	32	2	3	0.1	0.3	0.0005	0.0005	128	128
336	64	32	2	3	0.2	0.3	0.0001	0.0001	32	128
360	32	32	2	2	0.3	0.2	0.0001	0.0001	64	32
384	32	32	2	3	0.1	0.3	0.0001	0.0001	128	64
408	32	32	2	3	0.1	0.3	0.001	0.0001	128	64
432	32	32	3	2	0.0	0.2	0.0001	0.0001	32	64
456	32	32	3	2	0.0	0.3	0.0001	0.0001	64	128
480	64	32	2	3	0.2	0.3	0.0001	0.0001	32	128
504	32	32	2	2	0.1	0.2	0.0001	0.0001	64	64

Table 2.

Hyperparameters of LSTM.

Context Length	Model	LSTM
	Hyper-parameter	Hidden Units		Dropout Rate		Learning Rate		Batch Size
	Dataset	Post-COVID	Entire	Post-COVID	Entire	Post-COVID	Entire	Post-COVID	Entire
24		32	64	0.2	0.2	0.001	0.001	32	32
48		128	256	0.2	0.2	0.01	0.01	64	32
72		256	256	0.0	0.1	0.001	0.001	32	128
96		64	256	0.2	0.4	0.01	0.001	32	32
120		32	128	0.2	0.1	0.001	0.001	32	64
144		128	256	0.3	0.2	0.01	0.001	32	64
168		32	256	0.0	0.3	0.0005	0.001	32	64
192		64	256	0.2	0.4	0.01	0.001	128	32
216		128	256	0.1	0.4	0.01	0.001	32	32
240		128	64	0.1	0.4	0.01	0.005	32	64
264		128	256	0.3	0.4	0.01	0.001	32	32
288		128	64	0.0	0.3	0.01	0.001	32	64
312		64	32	0.2	0.2	0.01	0.001	64	32
336		128	256	0.3	0.1	0.01	0.01	32	64
360		128	32	0.2	0.1	0.001	0.001	32	32
384		64	256	0.2	0.3	0.01	0.001	32	64
408		128	256	0.2	0.4	0.01	0.001	32	64
432		64	128	0.3	0.4	0.01	0.001	32	64
456		64	128	0.3	0.0	0.01	0.001	32	32
480		32	256	0.4	0.1	0.01	0.001	32	64
504		128	128	0.4	0.0	0.01	0.001	32	32

Table 3.

Variance of evaluation metrics for each model.

Model	Dataset	Post-COVID			Entire
	Metrics (Variance across 10 runs)	MAE	RMSE	MAPE	MAE	RMSE	MAPE
CNN		5.424	5.156	3.074	0.083	0.039	2.722
LSTM		4.796	6.401	3.238	0.311	0.732	4.795
Informer		9.641	11.948	8.718	0.285	0.236	8.297
CNN		1.876	2.778	1.402	5.997	16.718	24.111
LSTM		11.191	16.746	7.581	8.020	21.333	29.672
Informer		5.950	8.721	6.945	6.225	18.310	19.957
Lag-Llama		29.133	39.589	28.083	58.478	88.254	216.793
Chronos (Tiny)		1.624	3.104	1.200	7.087	23.276	9.922
Chronos (Mini)		1.339	2.623	0.947	6.798	24.966	8.710
Chronos (Small)		1.451	2.741	1.104	6.690	24.741	8.924
Chronos (Base)		1.573	3.001	1.219	6.292	23.642	8.725
Chronos (Large)		1.445	2.727	1.204	6.148	23.358	8.047

All models are implemented in Python 3.12.4 and executed on a 64-bit Ubuntu server with an Intel Xeon Gold 6334 8-Core Processor × 2 @ 3.60 GHz CPU, 125 GB of RAM, and an NVIDIA A100 GPU with 24 GB of memory. The deep learning models are developed using TensorFlow 2.17.0, while Informer and the foundation models are implemented using PyTorch 2.3.1.

Evaluation metrics

The accuracy of traffic prediction models is typically evaluated using performance metrics that quantify their ability to forecast traffic conditions. In this research, we employ three widely recognised metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). RMSE and MAE assess absolute errors, while MAPE evaluates relative errors¹⁰³. Lower values in all metrics indicate better prediction performance. The formulas are as follows:

2

3

4

where and represent the ground truth and the predicted value for the th traffic flow sample. is the total number of the prediction samples.

Results

Model performance comparison

This study compares the performance of traffic flow prediction models across the entire dataset and the post-COVID-19 dataset at different input lengths (context lengths) for both deep learning and foundation models (Fig. 3). The evaluation results clearly distinguish between model performance when trained on the post-COVID-19 dataset and when trained on the entire dataset. Across all models, evaluation metrics (MAE, MAPE and RMSE) are consistently lower for the post-COVID-19 dataset. This suggests that deep learning and foundation models perform better with stable traffic patterns. Specifically, the improvements in deep learning models are moderate, with slight decreases in RMSE and MAPE when evaluated on the post-COVID-19 dataset, suggesting that deep learning models may be less sensitive to different data patterns. In contrast, foundation models exhibit a noticeable performance gap across datasets. When predicted on post-COVID-19 data, the reduction in MAE, MAPE and RMSE is more evident than deep learning models, particularly for Lag-LLaMA, indicating improved model adaptability to stable traffic dynamics.

Fig. 3.

Comparison of the post-COVID-19 and entire dataset performance (MAE, RMSE, MAPE) of models across different input lengths.

According to the context length, increasing it improves prediction performance for foundation models, whereas only limited improvements are observed for deep learning models. CNN, LSTM and Informer generally exhibit poorer performance as the context length increases. Although LSTM maintains a relatively stable trend, CNN and Informer show greater performance fluctuations. In particular, CNNs exhibit pronounced variability at longer context lengths, suggesting potential overfitting or inefficiencies in capturing long-term dependencies. Foundation models consistently reduce MAE, MAPE and RMSE as context length increases, although with slight fluctuations. Lag-LlaMA, in particular, demonstrates the most considerable improvement, reinforcing its ability to apply long historical sequences effectively. These findings highlight the superior capacity of foundation models to process and utilise long-term dependencies in time-series prediction.

Although foundation models generally benefit from longer context lengths, their performance declines when the context is short—often falling behind deep learning models. In particular, Lag-LLaMA consistently yields higher MAE and RMSE values than deep learning models across all context lengths when evaluated on the full dataset. This can be attributed to the zero-shot nature of these pre-trained models, which rely on broadly learned universal patterns from large-scale, high-quality data rather than task-specific training¹¹. The entire dataset, which includes more heterogeneous and unusual traffic patterns, would make zero-shot prediction more demanding for these models. In contrast, the post-COVID-19 dataset exhibits more stable, universal traffic flow dynamics, enabling foundation models to utilise their extensive pre-training more effectively and outperform traditional deep learning models. Besides, the consistent performance improvements observed with longer context lengths demonstrate the importance of providing foundation models with sufficient historical information to enhance their zero-shot predictions in time series prediction.

Training time and inference time analysis

Figure 4 illustrates the trade-off between training time and Mean Absolute Error (MAE) for deep learning models on the post-COVID-19 and entire datasets. For the post-COVID-19 dataset, CNN demonstrates the fastest training time at less than 1 s per epoch while achieving the best prediction performance. LSTM requires a moderately longer training time of around 6 s per epoch, but delivers poorer performance than CNN. Notably, Informer exhibits the longest training time, yet paradoxically achieves the poorest prediction performance. This indicates that with relatively stable traffic patterns, increased model complexity and training time do not necessarily lead to improved accuracy, with the most computationally intensive model showing the weakest performance. In contrast, the entire dataset exhibits a different pattern. Although the training time trend remains consistent across models, both LSTM and Informer show modest performance improvements over CNN, with all models achieving MAE values of around 28. This suggests that when dealing with more complex and irregular traffic patterns, the sophisticated temporal modelling capabilities of LSTM and Informer provide limited but observable benefits, though these improvements come at the cost of substantially increased computational demands.

Fig. 4.

Average training time and MAE of CNN, LSTM and Informer across all context lengths (measured on NVIDIA A100 GPU).

We also compare the inference time for all the models, including deep learning and foundation models (Fig. 5). It is important to note that Lag-Llama is intentionally omitted from this figure due to its exceptionally high inference times⁵. It requires approximately 132 s per epoch on the post-COVID-19 dataset and 875 s per epoch on the entire dataset, substantially longer than the inference times observed for the other models. For both post-COVID-19 and entire datasets, deep learning models exhibit relatively short inference times but moderately higher MAEs, while the Chronos models show a broader range of inference times that generally increase with model size. Unlike the training time comparison, where Informer required significantly longer training times than CNN and LSTM, all three deep learning models show similar inference durations. This is because they all produce predictions through a single forward procedure during inference. While Informer’s training is computationally intensive due to its complex self-attention mechanisms, its generative-style decoder enables efficient multi-step forecasts comparable to those of CNNs and LSTMs.

Fig. 5.

Average inference time and MAE of models across all context lengths (measured on NVIDIA A100 GPU).

For Chronos models, larger configurations tend to achieve lower MAEs at the cost of progressively longer inference durations. This trade-off stems from Chronos’s autoregressive forecasting method, where future values are generated sequentially by sampling from learned probabilistic distributions²³. This sequential generation process, combined with increased parameter counts in larger models, leads to substantially extended inference times. A key observation is that Chronos (Small) achieves shorter inference times and better performance than Chronos (Mini), indicating that this configuration effectively balances computational efficiency and predictive accuracy. However, beyond Chronos (Small), increasing model size brings only marginal performance improvements while substantially increasing inference time, suggesting diminishing returns for larger Chronos variants in traffic flow prediction tasks.

Model size analysis

Figure 6 compares the average performance across different context lengths as a function of model size, measured by the number of parameters. Specifically, the number of parameters in deep learning models depends on their architecture, hyperparameters, and context length, whereas foundation models maintain a fixed parameter count. From Fig. 6, it is clear that deep learning models and foundation models demonstrate different performance trends as their sizes change. For foundation models, increasing the number of parameters generally improves prediction accuracy on both the post-COVID-19 dataset and the entire dataset, suggesting that additional parameters help learn complex traffic patterns. In contrast, deep learning models show a negative correlation between model size and prediction accuracy on the post-COVID-19 dataset. Specifically, CNN achieves the best performance, followed by LSTM, while Informer performs the worst despite having substantially more parameters than both CNN and LSTM. Across the entire dataset, all three deep learning models converge to similar performance levels regardless of model size.

Fig. 6.

Comparison of the post-COVID-19 and entire dataset performance of models across different model sizes, averaged across all context lengths.

Discussion

In this study, we compare the performance of deep learning models and cutting-edge foundation models for traffic flow prediction. The deep learning models, CNN, LSTM and Informer, are trained on the SCOOT dataset ourselves, while Lag-Llama and Chronos are pre-trained time series foundation models, which can be applied for zero-shot prediction. We have found that foundation models with longer context lengths and larger model sizes tend to achieve higher prediction accuracy, while deep learning models show limited improvement. This suggests that deep learning models may overfit or struggle to capture long-term dependencies. In contrast, foundation models demonstrate a superior ability to process and utilise long-term dependencies in time-series prediction. However, there is a trade-off between model performance and inference time, as increasing context length and model size require more substantial computational resources. Moreover, although foundation models are more sensitive to traffic patterns, they outperform deep learning models in both usual and unusual traffic conditions, with Chronos demonstrating particularly strong performance.

In our experiments, we evaluate training and inference times separately to provide a clear understanding of the computational demands of each stage. However, for a comprehensive comparison between deep learning models and foundation models, we calculate the total running time by combining the training and inference durations for the deep learning models. For foundation models that are not trained in this study, the running time is limited to inference time only. Specifically, we calculate the cumulative running time over 100 training epochs for deep learning models and one prediction epoch for all models. As shown in Fig. 7, Chronos’s running times are significantly lower than those of other models, particularly on the small dataset. While the running time of CNN is shorter than that of Lag-Llama in this case, it overlooks hyperparameter tuning, which is essential in the deep learning training process for each prediction task. The tuning process involves testing dozens of hyperparameter combinations^104,105, with each test requiring a time equivalent to the running time observed here (since the inference times of deep learning models are too short to be considered). This leads to a practical running time multiple times greater than the running time here. As a result, the runtime of foundation models is shorter than that of deep learning models. Besides, deep learning models require separate training for each prediction task. In contrast, foundation models can be directly applied to different prediction tasks across various datasets with varying context lengths. This significantly simplifies model deployment and streamlines forecasting pipelines, eliminating the need for task-specific training.

Fig. 7.

Comparison of the cumulative running time of models, averaged across all context lengths (measured on NVIDIA A100 GPU).

A key limitation of foundation models is the performance gap between different models. Our findings indicate that Lag-Llama shows only limited improvements in prediction accuracy compared to deep learning models, while Chronos consistently demonstrates strong performance across various context lengths. Lag-Llama also shows lower stability than Chronos, with a sharper decrease in accuracy at longer prediction steps. To further illustrate this performance gap, we used another publicly available, widely used traffic flow dataset collected by the Caltrans Performance Measurement System (PeMS) in California, USA, from 1 January to 31 December, 2018. As shown in Fig. 8, the results are consistent with those from the SCOOT dataset in our research, with Chronos significantly outperforming Lag-Llama across all context lengths. Since both Chronos and Lag-Llama are pre-trained on diverse publicly available datasets, the observed performance difference may stem from their respective training data. Comparing their datasets, we find that Chronos is trained on seven datasets, while Lag-Llama uses only three. Additionally, Chronos incorporates synthetic data generated by Gaussian processes to enhance its training. The training data for Chronos spans 2009–2022, including the 2020–2021 pandemic period, enabling the model to learn mobility patterns amid pandemic-related disruptions. Lag-Llama’s training data is limited to 2009 and 2014–2016 and therefore does not capture pandemic-related mobility changes. This suggests that training foundation models on a larger, more temporally diverse corpus of time-series data improves zero-shot performance. Moreover, model size plays a crucial role in performance. Chronos offers models ranging from 8 M (Tiny) to 710 M (Large) parameters, which are significantly larger than Lag-Llama’s 2.45 M parameters and are likely contributing to its superior predictive accuracy. However, it is worth noting that while Chronos shows consistently better performance with increasing model size, the improvements from the Small to the Large variants are modest, suggesting a degree of model saturation for this prediction task. The univariate nature of the input and the limited complexity of short-horizon traffic flow prediction may constrain the benefit of adding substantially more parameters. Future work could explore whether multivariate prediction tasks enable larger foundation model variants to realise greater performance increases.

Fig. 8.

Comparison of model performance across different context lengths.

In summary, this research highlights that foundation models can achieve excellent zero-shot performance in traffic flow prediction under both normal conditions and disruptive events. Unlike deep learning models, which require extensive task-specific training and domain expertise, foundation models can be directly applied to datasets with different data sizes, traffic dynamics, and context lengths. Besides, while deep learning models require large-scale historical data for training and validation, foundation models can make accurate zero-shot predictions with only a small subset of contextual data. Those advantages can address critical limitations of deep learning methods, such as time-consuming training processes, overfitting due to task-specific model training, and limited generalisation capabilities. Additionally, they can support the practical deployment of traffic prediction models across diverse, dynamically changing urban environments. However, choosing an appropriate foundation model is crucial, as performance depends on factors such as the diversity of the training data, the time coverage, and the model size. A well-trained foundation model with a comprehensive dataset is more likely to achieve superior zero-shot performance, making it a practical and efficient choice for real-world traffic prediction applications.

Using foundation models for traffic flow prediction and other time series analysis tasks in urban settings presents both opportunities and challenges. The development of foundation models has been one of the fastest-growing areas over the past two and a half years, since OpenAI released the first version of ChatGPT 3.5 in November 2022. As foundation models continue to evolve in both capability and efficiency, the prediction accuracy of time-series foundation models is expected to improve accordingly. Accordingly, the results presented in this study represent a baseline, reflecting the performance of the foundation models available at the time of experimentation in a zero-shot setting. This study focuses on non-spatial time-series models in a single-sensor forecasting setting because current foundation models do not natively incorporate spatial information; therefore, our findings should be interpreted within this specific experimental scope rather than as a replacement for spatial–temporal models, which remain essential for capturing network-level traffic dynamics. Future research may extend these experiments with more recent models and adaptation strategies such as fine-tuning or few-shot learning, and explore spatial–temporal architectures to further improve prediction accuracy and better capture the underlying dynamics of urban traffic systems. As we observed in this research, although foundation models exhibit superior performance in this research, with faster inference and higher accuracy, they still have several limitations. Firstly, the training process of foundation models is typically very expensive and time-consuming, making it less accessible for academic institutions or small research groups to extend or revise pre-trained models.

Furthermore, the efficiency of foundation models heavily relies on the underlying pre-trained architectures, and the most advanced foundation models are often closed-source and developed by leading generative AI companies. As a result, most researchers can only rely on “less advanced” or “older generation” publicly available foundation models to design fine-tuned models for time series prediction. In the future, a better collaboration between AI companies and academia is necessary to enable further customised model development. Lastly, the limited scope and diversity of training data²³ for time-series foundation models lead to performance disparities and prediction biases across foundation models. Future work can focus on building and maintaining large-scale, diverse traffic datasets to improve model training and predictive accuracy across various scenarios. The model can also be further evaluated in regions with limited data, such as those in the Global South, to assess the effectiveness of using foundation models trained on datasets from Global North countries in different geographical contexts. Governments can encourage collaborative data-sharing initiatives between public and private sectors to expand the availability of high-quality traffic data for model development.

Appendix 1: Hyperparameter tuning

Tables 1 and 2 present the optimal hyperparameters for the CNN and LSTM, respectively, selected based on the lowest training loss observed during training. These hyperparameters are fine-tuned to achieve optimal model performance across different context lengths and datasets.

Variance analysis of evaluation metrics

Table 3 presents the variance of evaluation metrics (MAE, RMSE, and MAPE) for all models. For deep learning models (CNN, LSTM, and Informer), the variance is computed across 10 independent inference runs to assess the stability and reproducibility of model predictions. For foundation models (Chronos and Lag-Llama), which produce deterministic outputs for a given input, variance across runs is not applicable. Instead, for all models, we report the variance of evaluation metrics across the six forecast steps, which reflects how prediction performance varies with increasing prediction horizons. Lower variance indicates more consistent performance, while higher variance suggests that model accuracy degrades or fluctuates. In particular, Lag-Llama exhibits relatively high variance, indicating that its prediction accuracy decreases markedly as the forecast horizon increases, while other models show relatively low and similar variance, with more stable performance across different prediction steps.

Author contributions

Yue Li: Conceptualization; Data curation; Formal analysis; Methodology; Visualization; Writing—original draft; Writing—review and editing. Qunshan Zhao: Conceptualization; Writing—review & editing; Supervision; Resources; Project administration; Funding acquisition. Mingshu Wang: Conceptualization; Writing—review & editing; Supervision.

Funding

The first author is funded by the China Scholarship Council (CSC) from the Ministry of Education of P.R. China. Dr Qunshan Zhao has received the ESRC’s ongoing support for the Urban Big Data Centre (UBDC) [ES/L011921/1 and ES/S007105/1], and Royal Society International Exchange Scheme [IEC\NSFC\223042]. The authors want to thank the anonymous reviewers for their insightful comments and suggestions on an earlier version of this manuscript.

Data availability

The data used in this paper is publicly available. Full details about the data acquisition can be found in the documentation available at the GitHub repository: (https://github.com/YueLi-0816/trafficFlowPrediction) .

Declarations

Competing interests

The authors declare no competing interests.