Bayesian estimation of subset threshold autoregressions: short-term forecasting of traffic occupancy

Abstract

Traffic management authorities in metropolitan areas use real-time systems that analyze high-frequency measurements from fixed sensors, to perform short-term forecasting and incident detection for various locations of a road network. Published research over the last 20 years focused primarily on modeling and forecasting of traffic volumes and speeds. Traffic occupancy approximates vehicular density through the percentage of time a sensor detects a vehicle within a pre-specified time interval. It exhibits weekly periodic patterns and heteroskedasticity and has been used as a metric for characterizing traffic regimes (e.g. free flow, congestion). This article presents a Bayesian three-step model building procedure for parsimonious estimation of Threshold-Autoregressive (TAR) models, designed for location- day- and horizon-specific forecasting of traffic occupancy. In the first step, multiple regime TAR models reformulated as high-dimensional linear regressions are estimated using Bayesian horseshoe priors. Next, significant regimes are identified through a forward selection algorithm based on Kullback-Leibler (KL) distances between the posterior predictive distribution of the full reference model and TAR models with fewer regimes. Given the regimes, the forward selection algorithm can be implemented again to select significant autoregressive terms. In addition to forecasting, the proposed specification and model-building scheme, may assist in determining location-specific congestion thresholds and associations between traffic dynamics observed in different regions of a network. Empirical results applied to data from a traffic forecasting competition, illustrate the efficacy of the proposed procedures in obtaining interpretable models and in producing satisfactory point and density forecasts at multiple horizons.

1. Introduction

Rising populations in major cities add stress to traffic management systems tasked with monitoring real-time traffic variables to proactively reduce congestion. Ever since [1] used ARIMA models to forecast measurements from freeways in large US cities – Los Angeles, Minneapolis, and Detroit – significant efforts have been devoted to analyze the massive amount of data collected in transportation networks. The global concern has manifested through independent research in major cities, in countries such as the United Kingdom [22,74], Greece [44,81,82], Italy [3,63], China [42,61,78], and Ethiopia [37]. The state of short-term traffic forecasting 20 years ago has been reviewed in [79], while [87] summarize recent advancements and propose 10 challenges to the research community.

The timely detection of a traffic incident (e.g. recurring or non-recurring congestion) and the accuracy of a corresponding control action (e.g. change of traffic signal plan) rely on efficient modeling and accurate short-term forecasting of important traffic variables. Nowadays, online systems for traffic forecasting can implement multiple forecasting specifications, which are updated on a regular basis (e.g. weekly) and compute forecasts for different horizons using adaptive, or regime-switching combination schemes. Similar systems are used to compute forecasts for wind farm or solar panel energy outputs [76] Three variables have been used to quantify traffic congestion: flow (volume per time), speed and occupancy (the percent of time a detection zone is occupied) [35]. Traffic occupancy is used by traffic control centers as a quality assessment measure, to characterize network links as ‘congested’ [91], if some location-specific thresholds have been exceeded. Hence control actions such as switching of traffic signal plans, depend primarily on the level of occupancy [46]. Previous efforts adopted the same modeling strategies as the ones used for other traffic variables [44], although the dynamics of occupancy, which is bounded on the $[0,1]$ interval, are harder to forecast accurately [51].

Like speed and volume, occupancy exhibits changes in mean and variance as traffic dynamics fluctuate between free flow and congestion in urban environments. The parametric, piecewise linear, threshold autoregressive (TAR) process, first discussed in [83], is a conditional regime-specific autoregressive (AR) model, with regimes that depend on the levels of a transition variable. TAR have been designed to model cyclical and asymmetric phenomena, for which linear models do not offer satisfactory representation. Such piecewise linear specifications, estimated using a frequentist quantile regression approach, have been used to forecast traffic volumes in [43,45]. TAR is appealing to practitioners because it is interpretable: some states are expected to correspond to ‘free-flow’ and others to ‘congestion’ and the regime-specific coefficients provide useful input with regard to controls that should be exercised to improve traffic flow.

Frequentist analysis of the TAR model is typically implemented in two stages, with regime-specific parameters estimated conditional on the transition variable and the threshold parameter. On the other hand, Bayesian methods allow simultaneous inference on all parameters [8,18,30]. Hence uncertainty about the transition variable and the threshold parameter can be incorporated into estimation and inference [19]. Campbell [9] applied reversible jump Markov chain Monte Carlo to select regime-specific AR orders; subset selection of TAR via stochastic search [26] was conducted by [16,80]. In the aforementioned articles, the specifications are restricted to possess at most three regimes. Chan et al. [15] expressed the nonlinear TAR model as a high-dimensional linear regression, which can be effective even when the number of regimes is large. In this case, multi-step procedures based on penalized estimators are used to identify the regimes and estimate regime-specific dynamics [13]. The Bayesian approach of Pan et al. [66] operates similarly by utilizing a sequence of binary inclusion variables to identify change points and select regimes.

This paper combines the regression framework proposed for TAR in [15], with a Bayesian three-step procedure, which automates regime identification and sparse subset AR estimation within regimes. In contrast with [66], the methodology can use existing samplers for high-dimensional regression, for instance the R package bayesreg [77]. Thus the proposed methodology can produce results for t-distributed responses, which is advantageous for applications based on measurements that relate to incidents, outliers, etc. Furthermore, to the best of our knowledge, previous analyses of Bayesian TAR did not investigate important theoretical properties of the samplers, such as geometric ergodicity. Based on [65], hierarchical estimation schemes analogous to the ones adopted here can produce geometrically ergodic Markov chains, which ensure strongly consistent estimation of intractable posterior expectations. Finally, typical applications of penalized estimators such as Lasso, or adaptive LAD Lasso, in problems analogous to ours [45], do not produce confidence or credible intervals for the estimated coefficients and the associated forecasts. In what follows, forecasts are computed by averaging over the posterior samples of the coefficients and scale parameters, allowing estimation uncertainty to be taken into account in the prediction process.

Our methodology is summarized as follows. First, a saturated TAR model is estimated using Bayesian regularization; our analyses were mostly based on a modified horseshoe prior [4,11,12]. Next, the Kullback-Leibler (KL) divergence [48] combined with forward selection, identifies the regimes by comparing posterior predictive distributions of the linear AR to multiple-regime TAR models. This projection-based regime-identification procedure is novel to our knowledge; our numerical experiments suggest that it reduces dramatically the number of columns in the matrix of predictors, and consequently, the computational burden of the real-time implementation. Given a restricted set of regimes, the same procedure is repeated to select the relevant terms within each regime. The result is a parsimonious TAR model with a posterior predictive distribution close in KL distance to the posterior predictive distribution from the full model. In addition to reducing the number of computations in the real-time implementation phase, the adopted strategy provides interpretable models that can help practitioners to identify location-dependent traffic regimes and act, for instance by modifying traffic signal plans, if some location-specific thresholds are expected to be exceeded.

The proposed methods are used to estimate location- day- and horizon-specific models for traffic occupancies. Forecasts for short horizons are typically used from traffic management centers as input in control optimization algorithms, whereas forecasts for longer horizons can be disseminated to the public [91]. The paper provides a detailed evaluation of out-of-sample forecasting performance, using some challenging time series from the 2013 forecasting competition organized by Transportation Research Board's committee on Artificial Intelligence and Advanced Computing Applications. The data is presented in Section 2, while Section 3 discusses TAR models and their high-dimensional linear representation. Section 4 presents a novel Bayesian estimation, regime identification and subset selection strategy. The proposed methods are evaluated in two simulation experiments in Section 5. Section 6 presents forecasting models for occupancies, which are assessed using the mean absolute scaled measure of forecast accuracy (MASFE) proposed in [39] and Section 7 concludes the paper.

2. TRANSportation data FORecasting Competition (TRANSFOR)

The analyzed data were used in the 2013 TRANSportation Data FORecasting Competition (TRANSFOR); results were announced at the 92nd annual meeting of the Transportation Research Board (TRB) in Washington D.C. [43]. The National Technical University of Athens provided traffic occupancy collected every 90 seconds in April 2000 from seven loop detectors, abbreviated $L\in \{A,B,C,D,E,F,G\}$. The detectors are located in Alexandras Avenue, a significant arterial in Athens, the capital of Greece. Figure 1 depicts the network, with arrows representing direction of traffic flow: detectors A, B, C, and D measure traffic occupancy in the westbound, and detectors E, F, and G, in the eastbound direction.

Figure 1.

Locations of loop detectors along Alexandras Ave. in Athens, Greece: arrows indicate direction of traffic flow.

Analysis of raw traffic occupancies measured on 90-second intervals can be problematic due to the large amount of data and the low signal to noise ratio [87]. Temporal aggregation to larger intervals, i.e. 15 min has been practiced over the years as a smoothing technique prior to modeling: this diminishes nonlinear and heteroskedastic dynamics in the underlying signal [86] and makes short-term forecasting irrelevant for short horizons, which are useful to modern Traffic Control Centers. Rather than modeling across different levels of temporal aggregation as in [78], in this work traffic occupancy is aggregated to 3 minute intervals resulting in 480 daily time points per location.

Data during April 2000 covers four complete weeks; the first three are used to fit TAR models and the last week is utilized to evaluate forecasting performance. Similar with numerous previous studies, our analysis ignores weekend data. Time series plots of observed occupancies are depicted in Figure 2; some of their inherent characteristics (e.g. erroneous values and outliers due to incidents) make short-term forecasting challenging.

Figure 2.

Traffic occupancy for all TRANSFOR locations, aggregated on 3-minute intervals: shaded region indicates the forecasting period.

3. Threshold autoregressive model

3.1. Transformed occupancy

Let $O_{L,t}$ represent traffic occupancy for location L at time t, with $o_{L,t}$ denoting an observed value. For each location $L\in \{A,B,C,D,E,F,G\}$, day of the week $D\in \{M,T,W,Th,F\}$, and horizon $h\in \{1,3,5\}$, $(L,D,h)$-specific TAR models are built to forecast ${\hat{O}}_{L,t}=E[O_{L,t}|{\mathcal{I}}_t]$, where ${\mathcal{I}}_t=\{o_{L,k}{\}}_{k=t-h}^{t-h-P+1}$. In the application that follows the chosen levels of h correspond to forecasting 3, 9 and 15 minutes ahead. The purpose of h-specific models is to ensure that possible mis-specification errors of models that produce 1-step ahead forecasts do not propagate through the chain rule; furthermore horizon-specific models provide additional input to practitioners, related to how traffic propagates.

The specifications that follow aim to capture the dynamics of $Y_{L,t}=\mathrm{logit}(O_{L,t})$. The logit transformation, $\mathrm{logit}(x):(0,1)\to \mathbb{R}$ with $\varphi (x)=\log [x/(1-x)]$, has been used for proportional time series at least since [88] and has been included in the extended family of transformations, seeking to obtain approximate linearity in the transformed scale for bounded responses [27]. Occupancy is bounded on the $[0,1]$ interval; recoding 0 with 0.0001 and 1 with 0.9999 was implemented to handle extreme values for which $\mathrm{logit}(.)$ is undefined. Figure 3 displays the transformed series $\{y_{L,t}\}$.

Figure 3.

Logit-transformed traffic occupancy aggregated on 3-minute intervals for all locations: shaded regions indicate the forecasting period.

Essentially, each MCMC cycle produces a set of coefficients that can be combined with the predictors of the regression model in the testing sample, to produce forecasts. Density forecasts are derived by back-transforming the resulting samples, using the inverse-logit transformation. Specifically, evaluation of forecasts takes place on the original scale, using ${\mathrm{logit}}^{-1}(x):\mathbb{R}\to (0,1)$ where ${\mathrm{logit}}^{-1}(x)=\exp (x)/(1+\exp (x))$. Since $\mathrm{logit}(x)$ is a nonlinear transformation, the forecast ${\hat{O}}_{L,t}\ne {\mathrm{logit}}^{-1}({\hat{Y}}_{L,t})$. Unbiased forecasts and quantiles are produced from $\{{\mathrm{logit}}^{-1}({\hat{Y}}_{L,t}^{(s)}){\}}_{s=1}^S$, where $\{{\hat{Y}}_{L,t}^{(s)}{\}}_{s=1}^S$ represents S posterior samples obtained from the posterior predictive distribution $f({\hat{Y}}_{L,t}|{\mathcal{I}}_t^{\ast })$, with ${\mathcal{I}}_t^{\ast }=\{y_{L,k}{\}}_{k=t-h}^{t-h-P+1}$.

3.2. $(L,D,h)$-specific TAR models

$\{Y_t\}$ follows a TAR process of order P with m + 1 regimes if

$$Y_t={\varphi }_0^{(j)}+\sum _{i=1}^P{\varphi }_i^{(j)}{Y}_{t-h-i+1}+\sigma {ϵ}_t,\mathrm{for}{\delta }_{j-1}<Y_{t-h}\le {\delta }_j,$$ (1)

where $\sigma >0$, $j\in \{1,2,\dots ,m+1\}$, and $h\in \mathbb{N}$. The vector of thresholds $\mathit{\delta }=[{\delta }_1,{\delta }_2,\dots ,{\delta }_m{]}^{\mathrm{\prime }}$ with $-\mathrm{\infty }={\delta }_0<{\delta }_1\le {\delta }_2\le \cdots \le {\delta }_m<{\delta }_{m+1}=\mathrm{\infty }$, divides the process into m + 1 regimes. The sequence of errors $\{{ϵ}_t\}$ is assumed to be i.i.d. with zero mean and unit variance. $P\in \mathbb{N}$ represents the maximum short-term lag relevant for forecasting, which should be chosen large enough to cover relevant temporal dependencies across all traffic regimes.

For the illustrative purposes of this work, Equation (1) is more rigid than the classic structure in [18]. Often a delay parameter d is introduced and $Y_{t-d}$ drives regime changes; following [15], d is assumed known, with d = h. Hence, the most recent realization $Y_{t-h}$ determines the current state, which is a reasonable assumption for traffic models and results in ‘self-exciting’ TAR [31]. Furthermore, rather than utilizing regime-specific variance parameters ${\sigma }_j$, homoskedasticity is assumed. It is worth highlighting that linear regression model with heteroskedasticity of unknown form can be specified as a regression model with independent Student-t errors [47]; such specifications will be discussed in the application.

3.3. High-dimensional linear representation

The regression design for estimating (1), which is presented below, follows from the frequentist analysis in [13,15]. Large values of m, the number of possible thresholds in $\mathit{\delta }$, may lead to high-dimensional designs; in such cases parameter estimation is typically based on penalized schemes. Analogous penalized estimators have been applied in change-point analysis for i.i.d. [7] and time series data [14]. Change-point analysis [59] is based on structural break autoregressive processes [28,38], with a time index taking the role of the threshold variable. Giordani et al. [33] proposed a unifying framework for change point and nonlinear time series models and [15] modified the change-point methodology in [14] to estimate threshold models, by constructing a new design matrix and by proving that the convergence rates for the threshold estimators are different relative to change-point estimators.

Specifically, let $\{y_t{\}}_{t=1}^{\mathrm{T}}$ denote the observed time series, with $\mathit{y}=[y_{P+1},\dots ,y_{T-h+1}{]}^{\mathrm{\prime }}$, $\mathit{ϵ}=[{ϵ}_1,\dots ,{ϵ}_{T^{\ast }}{]}^{\mathrm{\prime }}$, and define the $T^{\ast }\times (1+P)$ model matrix $\mathit{X}$, with $T^{\ast }=T-h-P+1$, as:

$$\mathit{X}=\left[\begin{array}{ccccc}1& y_P& y_{P-1}& \dots & y_1\\ 1& y_{P+1}& y_P& \dots & y_2\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 1& y_{T-h}& y_{T-h-1}& \dots & y_{T^{\ast }}\end{array}\right].$$

The second column in $\mathit{X}$ contains the h-specific transition variable. Let $\pi (i):\{1,\dots ,T^{\ast }\}\to \{1,\dots ,T^{\ast }\}$ designate the sorting function, with $\pi (i)$ denoting the time index of the ith smallest element in $[y_P,y_{P+1},\dots ,y_{T-h}{]}^{\mathrm{\prime }}$, ${\mathit{y}}_R=[y_{\pi (1)+h},\dots ,y_{\pi (T^{\ast })+h}{]}^{\mathrm{\prime }}$, ${\mathit{ϵ}}_R=[{ϵ}_{\pi (1)+h},\dots ,{ϵ}_{\pi (T^{\ast })+h}{]}^{\mathrm{\prime }}$ and

$${\mathit{X}}_1=\left[\begin{array}{ccccc}1& y_{\pi (1)}& y_{\pi (1)-1}& \dots & y_{\pi (1)-P+1}\\ 1& y_{\pi (2)}& y_{\pi (2)-1}& \dots & y_{\pi (2)-P+1}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 1& y_{\pi (T^{\ast })}& y_{\pi (T^{\ast })-1}& \dots & y_{\pi (T^{\ast })-P+1}\end{array}\right]=\left[\begin{array}{c}{\mathit{y}}_{\pi (1)}^{\mathrm{\prime }}\\ {\mathit{y}}_{\pi (2)}^{\mathrm{\prime }}\\ ⋮\\ {\mathit{y}}_{\pi (T^{\ast })}^{\mathrm{\prime }}\end{array}\right]$$

are essentially $\mathit{y}$, $\mathit{ϵ}$, and $\mathit{X}$, sorted according to the order statistics of the transition variable. The reordered ${\mathit{ϵ}}_R$ is assumed i.i.d. with mean 0 and variance ${\sigma }^2$.

To reduce the computational burden, it makes sense to limit the TAR model to possess at most m + 1 regimes, requiring the estimation of m thresholds in the range of the transition variable. Let $q(.):[0,1]\to [min\{y_{t-h}:t=P,P+1,\dots ,T\},max\{y_{t-h}:t=P,P+1,\dots ,T\}]$ denote the sample quantile function and consider a sequence $\{p_k{\}}_{k=1}^m$ of m evenly spaced percentiles, such that $p_{min}=p_1<\cdots <p_m=p_{max}$. For a fully saturated TAR model limited to $(m+1)$ regimes, fix a-priori the vector of thresholds $\mathit{\delta }=[q(p_1),q(p_2),\dots ,q(p_m){]}^{\mathrm{\prime }}$; for $j\in \{2,\dots ,m+1\}$, let $k_j$ represent the number of elements in $[y_P,y_{P+1},\dots ,y_{T-h}{]}^{\mathrm{\prime }}$ less than $q(p_{j-1})$ and define:

$${\mathit{X}}_j=\left[\begin{array}{ccccc}0& 0& 0& \dots & 0\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& \dots & 0\\ 1& y_{\pi (k_j+1)}& y_{\pi (k_j+1)-1}& \dots & y_{\pi (k_j+1)-P+1}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 1& y_{\pi (T)}& y_{\pi (T)-1}& \dots & y_{\pi (T)-P+1}\end{array}\right]=\left[\begin{array}{c}0\\ ⋮\\ 0\\ {\mathit{y}}_{\pi (k_j+1)}^{\mathrm{\prime }}\\ ⋮\\ {\mathit{y}}_{\pi (T)}^{\mathrm{\prime }}\end{array}\right].$$

The $(m+1)$-regime TAR process can be expressed as a linear regression:

$${\mathit{y}}_R={\mathit{X}}_R{\mathit{\theta }}_R+{\mathit{ϵ}}_R$$ (2)

where ${\mathit{X}}_R=[{\mathit{X}}_1,{\mathit{X}}_2,\dots ,{\mathit{X}}_{m+1}]$ is a $T^{\ast }\times (P+1)(m+1)$ model matrix and ${\mathit{\theta }}_R=[{\mathit{\theta }}_1^{\mathrm{\prime }},{\mathit{\theta }}_2^{\mathrm{\prime }},\dots ,{\mathit{\theta }}_{m+1}^{\mathrm{\prime }}{]}^{\mathrm{\prime }}$ is a $(P+1)(m+1)\times 1$ vector of coefficients grouped by regime. From Equation (1), set ${\mathit{\varphi }}_j=[{\varphi }_0^{(j)},{\varphi }_1^{(j)},\dots ,{\varphi }_P^{(j)}{]}^{\mathrm{\prime }}$: starting with ${\mathit{\theta }}_1={\mathit{\varphi }}_1$, the state dependent coefficient group ${\mathit{\theta }}_j={\mathit{\varphi }}_j-{\mathit{\varphi }}_{j-1}$ with $j\in \{2,\dots ,m+1\}$ represents the marginal adjustment in dynamics when $y_{t-h}$ crosses the threshold ${\delta }_{j-1}=q(p_{j-1})$.

The high-dimensional regression designs presented in [13,15], correspond to fully saturated TAR models where every realization in the series $\{y_t{\}}_{t=1}^{\mathrm{T}}$ resides in a different regime. On the other hand, the linear form of TAR in Equation (2) arises from restricting the set of possible thresholds $\mathit{\delta }$ to a finite set of quantiles, based on the sampled transition variable. To restrict estimation of the m thresholds to the range of $Y_{t-h}$, slightly informative uniform priors bounded by empirical quantiles $q(p_{min})$ and $q(p_{max})$ ensure that at least $(1-min\{p_{min},1-p_{max}\})\times 100\mathrm{\%}$ of the data belong in the lowest and highest regime [17,18,54,55]; in this work $p_{min}=0.15$ and $p_{max}=0.85$, following [36].

For the $(L,D,h)$-specific models presented in the application, the maximum number of thresholds is fixed to m = 50 and P = 7, corresponding to the last 21 minutes of traffic information. The model matrix ${\mathit{X}}_R$ of the fully saturated 51-regime TAR$(7)$ has dimension $T^{\ast }\times 408$. The fitting period for each model contains T = 1440 discrete time realizations of $\{Y_t\}$ leading to $T^{\ast }=1440-h-P+1>408$. A straightforward extension of the above TAR models would include information from neighboring locations, as in [45], increasing the number of predictors per regime and leading to high-dimensional regressions for a sufficiently large number of neighbors.

The predetermined threshold vector $\mathit{\delta }=[q(0.15),q(p_2),\dots ,q(p_{49}),q(0.85){]}^{\mathrm{\prime }}$ constructed from m evenly spaced percentiles ensures that approximately $\frac{0.85-0.15}{50}=0.014$ of the full time series is represented in each potentially relevant regime. These modifications to the framework presented in [13,15] ensure that the dimensionality of the parameter space is not unnecessarily large. Based on this approach, it is recommended to select m large enough to result in a dense set of quantile-based thresholds, so that a-priori mis-specification of $\mathit{\delta }$ is avoided.

3.4. Baseline seasonal models

In previous studies, seasonal autoregressive integrated moving average models (SARIMA) have been used to forecast traffic data [32,90,94]. As noted in [49], these specifications require large databases of historical measurements to estimate periodicity accurately. Here, similar to [49], a 3 day period is utilized for model training, which is relatively short for traffic applications. For a similar dataset, Kamarianakis et al. [44] applied a smoothing spline with 150 degrees of freedom to characterize daily patterns; such daily profiles typically correspond to static baseline forecasts, which are very ‘cheap’ in their online implementation.

In this work, baseline forecasts are based on harmonic models. Specifically, regressions which estimate daily occupancy profiles from a linear specification over a Fourier basis [53,60] capture weekly and daily periodicities [44]. The harmonic model is restricted to the first H terms of the Fourier series; $\{{ϵ}_t\}$ are again assumed i.i.d. with mean 0 and unit variance:

$$Y_t=\mu +\sum _{j=1}^H[{\alpha }_j\sin \left(\frac{2\pi tj}{480}\right)+{\beta }_j\cos \left(\frac{2\pi tj}{480}\right)]+{\sigma }_F{ϵ}_t.$$ (3)

Equation (3) is a linear regression:

$${\mathit{y}}_F={\mathit{X}}_F{\mathit{\theta }}_F+{\mathit{ϵ}}_F,$$ (4)

where ${\mathit{y}}_F=[y_1,y_2,\dots ,y_T{]}^{\mathrm{\prime }}$, ${\mathit{ϵ}}_F=[{ϵ}_1,{ϵ}_2,\dots ,{ϵ}_T{]}^{\mathrm{\prime }}$, and ${\mathit{\theta }}_F=[\mu ,{\alpha }_1,\dots ,{\alpha }_H,{\beta }_1,\dots ,{\beta }_H{]}^{\mathrm{\prime }}$. The model matrix is formulated as ${\mathit{X}}_F=[\mathbf{1},\mathit{S}\mathit{I}\mathit{N},\mathit{C}\mathit{O}\mathit{S}]$, where $\mathbf{1}$ is a $T\times 1$ vector of 1s and $\mathit{S}\mathit{I}\mathit{N}$ and $\mathit{C}\mathit{O}\mathit{S}$ are $T\times H$ matrices containing the Fourier sine and cosine terms, respectively. As h increases, $(L,D)$-specific harmonic models are expected to dominate over $(L,D,h)$-specific TAR.

For $(L,D)$-specific harmonic models, the number of sine/cosine pairs H must be less than half the period [53]. Preliminary regularized estimation of these models using adaptive LASSO [95] indicated that the largest significant harmonic of the model in Equation (3) across locations and days was for j = 139. Before estimating location- and day-specific profiles under the Bayesian framework, a maximum number of harmonics, H = 150, is chosen. The vector ${\mathit{\theta }}_F$ contains 2H + 1 = 301<1440 = T parameters; to ensure weekly periodic signals are smooth, sparse estimation of ${\mathit{\theta }}_F$ is performed using shrinkage priors, which are discussed in the next Section.

4. Bayesian estimation, regime identification, and subset selection

The purpose of representing the $(m+1)$-regime TAR process as a linear regression is to make Bayesian posterior estimation and model selection computationally feasible for multiple-regime TAR. The fully saturated regression ${\mathit{y}}_R={\mathit{X}}_R{\mathit{\theta }}_R+{\mathit{ϵ}}_R$ nests a finite, but extensive, library of $(m^{\ast }+1)$-regime subset TAR$(P)$ models, where $0\le m^{\ast }\le m$, which includes all linear subset AR$(P)$ models. For simplicity, let $\mathbf{\Theta }=[{\mathit{\theta }}_R^{\mathrm{\prime }},{\sigma }^2{]}^{\mathrm{\prime }}=[{\mathit{\theta }}_1^{\mathrm{\prime }},\dots ,{\mathit{\theta }}_{m+1}^{\mathrm{\prime }},{\sigma }^2{]}^{\mathrm{\prime }}$. A-priori it is expected that $m^{\ast }$ is small, implying that only $m^{\ast }+1$ of the vectors in $\{{\mathit{\theta }}_1^{\mathrm{\prime }},{\mathit{\theta }}_2^{\mathrm{\prime }},\dots ,{\mathit{\theta }}_{m+1}^{\mathrm{\prime }}\}$ are nonzero. Hence it is anticipated that ${\mathit{\theta }}_R$ is sparse.

To simultaneously estimate ${\mathit{\theta }}_R$, choose $m^{\ast }$ and identify the thresholds, Chan et al. [15] recommended the (penalized) group LASSO estimator [93]:

$${\hat{\mathit{\theta }}}_{GL}=\underset{{\mathit{\theta }}_R}{\mathrm{argmin}}=\frac{1}{T}||{\mathit{y}}_R-\mathit{X}{\mathit{\theta }}_R|{|}_2^2+\lambda \sum _{j=1}^{m+1}||{\mathit{\theta }}_j|{|}_1.$$ (5)

The parameter λ controls regularization, $||\cdot |{|}_2$ denotes the ${\ell }_2$-norm, and $||\cdot |{|}_1$ is the ${\ell }_1$-norm. When ${\mathit{X}}_R$ is constructed as in [15], the set of thresholds identified from ${\hat{\mathit{\theta }}}_{GL}$ estimates the true thresholds consistently, if the true $m^{\ast }$ is known a-priori. In practice, $m^{\ast }$ is unknown, and ${\hat{\mathit{\theta }}}_{GL}$ overestimates the number of regimes. Second stage selection of the best subset of the group LASSO identified thresholds via penalized information criteria, i.e. AIC [52], BIC [92], or MDL [21], leads to consistent estimation of the true set of thresholds [15]. The three-step procedure of [13], primarily based on a group orthogonal greedy algorithm (GOGA) and high-dimensional information criteria (HDIC), significantly outperformed the two-step group LASSO approach in [15].

What has a Bayesian approach to offer relative to standard penalized estimators? First, Bayesian estimators can be relatively easily modified to produce results for t-distributed (or Laplace) innovations, which is advantageous for the application we consider, as some measurements may relate to outliers, incidents etc. Furthermore, typical applications of penalized estimators such as Lasso, or adaptive LAD Lasso, in problems analogous to ours (see for instance [45]), do not compute confidence or credible intervals for the estimated coefficients and the associated forecasts.

4.1. Bayesian estimation

4.1.1. Conditional likelihood

Bayesian inference is based on the full posterior distribution $p(\mathbf{\Theta }|{\mathit{y}}_R,{\mathit{X}}_R)$, which is proportional to the product of the likelihood $p({\mathit{y}}_R|{\mathit{X}}_R,\mathbf{\Theta })$ with the prior $p(\mathbf{\Theta })$:

$$p(\mathbf{\Theta }|{\mathit{y}}_R,{\mathit{X}}_R)\propto p({\mathit{y}}_R|{\mathit{X}}_R,\mathbf{\Theta })p(\mathbf{\Theta }).$$ (6)

Given the linear model ${\mathit{y}}_R={\mathit{X}}_R{\mathit{\theta }}_R+{\mathit{ϵ}}_R$, $p({\mathit{y}}_R|{\mathit{X}}_R,\mathbf{\Theta })$ stems from a distributional assumption about the errors $\{{ϵ}_{\pi (t)+h}\}$ in ${\mathit{ϵ}}_R$. In what follows we make the conventional assumption $\{{ϵ}_{\pi (t)+h}\}\sim \mathrm{i}.\mathrm{i}.\mathrm{d}.\mathcal{N}(0,{\sigma }^2)$ which produces the normal regression model:

$${\mathit{y}}_R|{\mathit{X}}_R,\mathbf{\Theta }\sim {\mathcal{N}}_T({\mathit{X}}_R{\mathit{\theta }}_R,{\sigma }^2\mathit{I})$$ (7)

where ${\mathcal{N}}_T$ is a T-dimensional Multivariate normal distribution and $\mathit{I}$ is a $T\times T$ identity matrix. For future reference, we let TAR and SEAS denote the normal regression models from Equations (2) and (4); in both TAR and SEAS, Jeffreys' prior is used for ${\sigma }^2$. Derivation of the inverse-gamma full conditional distribution for ${\sigma }^2$ for both linear models is presented in [77].

4.1.2. Horseshoe+ priors for penalized regression

Mallick and Yi [58] presents an overview of Bayesian model selection in high-dimensional linear models. Estimates based on continuous scale-mixture priors [64,72] have been proposed to approximate the spike-and-slab shape of discrete mixture priors [10,26,40,41,50,56,62]. Specifically, the Bayesian horseshoe (BHS) estimator of [11,12] has been successful in the estimation and detection of sparse signals; furthermore BHS possesses desirable theoretical properties and computational feasibility in high dimensions [20,67,70,73,84]. The BHS prior falls in the class of shrinkage priors with global-local hierarchical representations [72]. A global tuning parameter is used to enforce variable selection by shrinking coefficients toward 0; similar to adaptive LASSO, BHS utilizes additional coefficient-specific tuning parameters to ensure that significant effects are not over-shrunk.

The horseshoe+ estimator (${\mathrm{BHS}}^{+}$) of [4] results from a slightly modified hierarchy with additional tuning on the local level. Specifically, the ${\mathrm{BHS}}^{+}$ hierarchical prior for each parameter ${\theta }_i$ in the full parameter vector ${\mathit{\theta }}_R$ is represented as:

$$\begin{array}{rl}{\theta }_i|{\lambda }_i,\tau ,{\sigma }^2& \sim \mathcal{N}(0,{\lambda }_i^2{\tau }^2{\sigma }^2)\\ {\lambda }_i& \sim {\mathcal{C}}^{+}(0,{\eta }_i)\\ {\eta }_i& \sim {\mathcal{C}}^{+}(0,1)\\ \tau & \sim {\mathcal{C}}^{+}(0,1)\end{array}$$ (8)

where ${\mathcal{C}}^{+}$ denotes the half-Cauchy distribution. Theoretical and empirical advantages of ${\mathrm{BHS}}^{+}$ relative to BHS in the high-dimensional regression setting, are presented in [5]. ${\mathrm{BHS}}^{+}$ is adopted in the TAR and SEAS models that follow, since it provides better detection of ultra-sparse signals than the original BHS [4]. Thus ${\mathrm{BHS}}^{+}$ is appropriate in the general setting in which location-specific models include information from various locations in a network, resulting in high-dimensional regressions in the first modeling stage, before projection-based regime identification.

The original hierarchy in Equation (8) makes posterior sampling difficult since full conditional distributions are not obtainable. By exploiting the scale-mixture decomposition of the half-Cauchy using inverse gamma distributions, abbreviated $\mathcal{I}\mathcal{G}$, Wand et al. [89] and Makalic and Schmidt [57] derived full conditional distributions for all parameters in $\mathbf{\Theta }$. Thus Gibbs sampling [24,25] can be utilized to sample from the full posterior distribution $p(\mathbf{\Theta }|{\mathit{y}}_R,{\mathit{X}}_R)$. Equation (9) reflects the changes to the ${\mathrm{BHS}}^{+}$ hierarchy for each parameter ${\theta }_i$ in ${\mathit{\theta }}_R$:

$$\begin{array}{rl}{\theta }_i|{\lambda }_i^2,{\tau }^2,{\sigma }^2& \sim \mathcal{N}(0,{\lambda }_i^2{\tau }^2{\sigma }^2)\\ {\lambda }_i^2|{\nu }_i& \sim \mathcal{I}\mathcal{G}(1/2,1/{\nu }_i)\\ {\nu }_i|{\eta }_i& \sim \mathcal{I}\mathcal{G}(1/2,1/{\eta }_i^2)\\ {\eta }_i^2|{\zeta }_i& \sim \mathcal{I}\mathcal{G}(1/2,1/{\zeta }_i)\\ {\zeta }_i& \sim \mathcal{I}\mathcal{G}(1/2,1)\\ {\tau }^2|\xi & \sim \mathcal{I}\mathcal{G}(1/2,1/\xi )\\ \xi & \sim \mathcal{I}\mathcal{G}(1/2,1).\end{array}$$ (9)

4.1.3. Posterior sampling

Even though ${\mathrm{BHS}}^{+}$ shrinkage priors are applied for both TAR and SEAS, posterior means of irrelevant effects in ${\mathit{\theta }}_R$ and ${\mathit{\theta }}_S$ will never equal 0. Let ${\mathcal{M}}_R$ represent the fully saturated TAR model fitted using ${\mathrm{BHS}}^{+}$. The primary goal of the next two steps is to search for the best sub-model ${\mathcal{M}}_{\ast }$ from the $2^{(m+1)(P+1)}$ different possible sub-models ${\mathcal{M}}_{\perp }$; when m = 50 and P = 7, there are $6.61\times {10}^{122}$ subset TAR$(P)$ models. Hence in order to reduce computations in the real-time implementation of the forecasting models, it is beneficial to adopt a strategy, similar to the one outlined in [68]: first identify a model that predicts well, then find a minimal subset of terms that provide similar predictions. This strategy has shown empirically very good performance in comparison to other methods with good tradeoff between sparsity and predictive accuracy [71].

4.2. Regime identification

The samples $\{{\mathit{\theta }}_R^{(s)}{\}}_{s=1}^S$ and $\{{\sigma }^{(s)}{\}}_{s=1}^S$ from the joint posterior distribution

$$p({\mathit{\theta }}_R,{\sigma }^2|{\mathcal{M}}_R,{\mathit{y}}_R,{\mathit{X}}_R)$$

provide a starting point for forecasting since ${\mathrm{BHS}}^{+}$ priors were used to enforce sparsity. Under ${\mathcal{M}}_R$, density forecasts at time T + 1 can be obtained from the posterior predictive distribution:

$$p(y_{T+1}|{\mathcal{M}}_R,{\mathit{y}}_R,{\mathit{X}}_R).$$

Assuming ${\mathcal{M}}_R$ produces reasonable forecasts, it can serve as a valid reference model. Given a simpler submodel ${\mathcal{M}}_{\perp }$, the Kullback-Leibler (KL) divergence [48] measures the distance between their posterior predictive distributions [23,34]; if the discrepancy between ${\mathcal{M}}_R$ and ${\mathcal{M}}_{\perp }$ is not significant, the more parsimonious model is favored.

Samples $\{{\mathit{\theta }}_{\perp }^{(s)}{\}}_{s=1}^S$ and $\{{\sigma }_{\perp }^{(s)}{\}}_{s=1}^S$ from the joint posterior distribution

$$p({\mathit{\theta }}_{\perp },{\sigma }_{\perp }^2|{\mathcal{M}}_{\perp },{\mathit{y}}_{\perp },{\mathit{X}}_{\perp })$$

are obtained via projection, hence repeated Gibbs sampling is not necessary. In this representation, ${\mathit{y}}_{\perp }={\mathit{y}}_R$ and ${\mathit{X}}_{\perp }$ contains the columns of ${\mathit{X}}_R$ associated with the sub-model ${\mathcal{M}}_{\perp }$. Piironen and Vehtari [71] derived analytical solutions to acquire the projected samples and measure the KL divergence from forecasts under $[{\mathit{\theta }}_R^{{}^{\mathrm{\prime }}(s)},{\sigma }_R^{(s)}{]}^{\mathrm{\prime }}$ versus $[{\mathit{\theta }}_{\perp }^{{}^{\mathrm{\prime }}(s)},{\sigma }_{\perp }^{(s)}{]}^{\mathrm{\prime }}$ for linear Gaussian models. The sth posterior sample $[{\mathit{\theta }}_{\perp }^{{}^{\mathrm{\prime }}(s)},{\sigma }_{\perp }^{(s)}{]}^{\mathrm{\prime }}$ is given by:

$$\begin{array}{rl}{\mathit{\theta }}_{\perp }^{(s)}& =({\mathit{X}}_{\perp }^{\mathrm{\prime }}{\mathit{X}}_{\perp }{)}^{-1}{\mathit{X}}_{\perp }^{\mathrm{\prime }}{\mathit{X}}_R{\mathit{\theta }}_R^{(s)}\\ {\sigma }_{\perp }^{(s)}& =\sqrt{({\sigma }_R^{(s)}{)}^2+\frac{({\mathit{X}}_R{\mathit{\theta }}_R^{(s)}-{\mathit{X}}_{\perp }{\mathit{\theta }}_{\perp }^{(s)}{)}^{\mathrm{\prime }}({\mathit{X}}_R{\mathit{\theta }}_R^{(s)}-{\mathit{X}}_{\perp }{\mathit{\theta }}_{\perp }^{(s)})}{T}.}\end{array}$$ (10)

For each posterior draw from $p({\mathit{\theta }}_R,{\sigma }_R^2|{\mathcal{M}}_R,{\mathit{y}}_R,{\mathit{X}}_R)$, the associated KL divergence $d_{\perp }$ is formulated as:

$$d_{\perp }^{(s)}({\mathit{\theta }}_R^{(s)},{\sigma }_R^{(s)})=\frac{1}{2}\log {\left(\frac{{\sigma }_{\perp }^{(s)}}{{\sigma }_R^{(s)}}\right)}^2.$$ (11)

Averaging the KL divergences across all posterior samples estimates the overall discrepancy $D({\mathcal{M}}_R||{\mathcal{M}}_{\perp })$ between posterior predictive distributions of ${\mathcal{M}}_R$ and ${\mathcal{M}}_{\perp }$:

$$D({\mathcal{M}}_R||{\mathcal{M}}_{\perp })=\frac{1}{S}\sum _{s=1}^S{d}_{\perp }^{(s)}({\mathit{\theta }}_R^{(s)},{\sigma }_R^{(s)}).$$ (12)

Using these concepts, surveying the entire model space is avoided by employing a forward stepwise selection algorithm similar to [69]. Starting with the linear AR$(P)$ model, denoted ${\mathcal{M}}_{\perp }^{(1)}$ where ${\mathit{\theta }}_j=0$ if j>1 and ${\mathit{\theta }}_{\perp }^{(1)}=[{\mathit{\theta }}_1^{\mathrm{\prime }},{\mathbf{0}}^{\mathrm{\prime }},{\mathbf{0}}^{\mathrm{\prime }},\dots ,{\mathbf{0}}^{\mathrm{\prime }}{]}^{\mathrm{\prime }}$, the initial discrepancy $D({\mathcal{M}}_R||{\mathcal{M}}_{\perp }^{(1)})$ represents the maximum divergence between the fully saturated ${\mathcal{M}}_R$ and all nested TAR$(P)$ models with less than $(m+1)$ regimes. For each $j\in \{2,\dots ,m+1\}$, ${\mathit{\theta }}_j$ is added to ${\mathit{\theta }}_{\perp }^{(1)}$ and the best 2-regime TAR$(P)$ model ${\mathcal{M}}_{\perp }^{(2)}$ that minimizes the discrepancy in Equation (12) is selected. Likewise, this procedure is continued to identify the best 3-regime TAR$(P)$, 4-regime TAR$(P)$, and j-regime models, denoted ${\mathit{\theta }}_{\perp }^{(3)}$, ${\mathit{\theta }}_{\perp }^{(4)}$, and ${\mathit{\theta }}_{\perp }^{(j)}$, respectively. Although this process can be continued up to j = m + 1, where $D({\mathcal{M}}_R||{\mathcal{M}}_{\perp }^{(m+1)})=0$, a stopping rule is enforced based on relative explanatory power (RelE) [23]:

$$RelE({\mathcal{M}}_{\perp })=1-\frac{D(\mathcal{M}||{\mathcal{M}}_{\perp })}{D(\mathcal{M}||{\mathcal{M}}_{\perp }^{(1)})}.$$ (13)

Based on the additive properties of KL, RelE strictly increases from 0 to 1. In the traffic forecasting application, regime-specific AR$(P)$ parameter groups are added until RelE exceeds 0.95. It was observed that this choice, which was adopted in previous works [68] and was implemented by modifying the codes provided in [69], leads satisfactory forecasting accuracy relative to the full models. Moreover, it produces parsimonious, interpretable specifications in most cases, which are far less demanding in terms of computations in the online stage relative to the full model. In addition, this strategy can help practitioners identify location-dependent traffic regimes that correspond to congestion, free-flow, etc., and act, for instance by modifying traffic signal plans if some thresholds have been or are expected to be exceeded. This is an important secondary application of the adopted specifications; an alternative method to identify traffic regimes was based in quantile regressions [91].

4.3. Subset variable selection

Let $\mathcal{J}=\{j:{\mathit{\theta }}_j\ne 0\}$ indicate the AR$(P)$ parameter groups in ${\mathit{\theta }}_R$ selected via the algorithm outlined in Section 4.2. The final $|\mathcal{J}|$-regime TAR$(P)$, designated as ${\mathcal{M}}_{\perp }^{(|\mathcal{J}|)}$, is likely to include irrelevant parameter groups ($|\cdot |$ denotes the cardinality of a set). When some subset of the linear AR$(P)$ model is optimal, the initial discrepancy $D({\mathcal{M}}_R||{\mathcal{M}}_{\perp }^{(1)})$ is small. As higher regime models are considered, the reduction in the discrepancy may decrease at a slow rate.

Let ${\theta }_{i,j}$ represent the ith parameter in the jth vector ${\mathit{\theta }}_j$ for $i\in \{1,2,\dots ,P+1\}$ and $j\in \{1,2,\dots ,m+1\}$. Following regime identification, the set $\mathcal{I}=\{{\theta }_{i,j}:i=1,\dots ,P+1\mathrm{and}j\in \mathcal{J}\}$ contains potentially relevant parameters in ${\mathit{\theta }}_R$. Because $\mathcal{J}$ may still contain irrelevant AR$(P)$ parameter groups, the regime identification stage can be considered a filtering step leading to a restricted model space with $2^{|\mathcal{I}|}=2^{(P+1)|\mathcal{J}|}$ different $j^{\ast }$-regime subset TAR$(P)$ where $j^{\ast }\le |\mathcal{J}|$.

After fixing all ${\theta }_{i,j}\notin \mathcal{I}$ to 0, the forward selection algorithm is repeated to search for the best and final subset TAR$(P)$ model ${\mathcal{M}}_{\ast }$, resulting in sparse estimation of Equation (2). The intercept-only model, where ${\theta }_{i,j}=0$ unless i = j = 1, is the starting point and identified as ${\mathcal{M}}_{\perp }^{(1)}$. Parameters from $\mathcal{I}$ are added one-at-a-time to the intercept-only model to obtain a chain of optimal subset models ${\mathit{\theta }}_{\perp }^{(2)},{\mathit{\theta }}_{\perp }^{(3)},{\mathit{\theta }}_{\perp }^{(4)},\dots$. The superscript of these models does not indicate the number of regimes, but rather the number of nonzero parameters in ${\mathit{\theta }}_R$. The final model ${\mathcal{M}}_{\ast }$ is identified using the same stopping rule seen in Section 4.2. Based on the ordering of ${\mathit{\theta }}_R$ in Equation (2), the subset of parameters in $\mathcal{I}$ selected in ${\mathcal{M}}_{\ast }$ imply the optimal number of regimes $(m^{\ast }+1)\le (m+1)$, the optimal choice of the $m^{\ast }$ thresholds in δ, and the relevant parameters within each of the regimes.

This additional model-building step reduces further computations in the online implementation. For extended specifications that contain information from other locations of the network (in the presented application this would not make much sense as the TRANSFOR competition focused on a small part of the Athens network), the derived models may provide useful information to practitioners as they may reveal regime-specific associations of traffic dynamics observed in different locations of a network.

5. Monte Carlo experiments

Before applying the methodology presented in Section 4 to traffic data, we illustrate its performance using two Monte Carlo experiments, similar to the ones presented in [66]. Both DGPs are specified according to the classic TAR representation in Equation (1). In the linear specification ${\mathit{y}}_R={\mathit{X}}_R{\mathit{\theta }}_R+{\mathit{ϵ}}_R$, we consider m = 50 thresholds and fix P = 3. After Bayesian regularization, regime identification, and subset selection, the estimate ${\hat{\mathit{\theta }}}_R$ is obtained for final model ${\mathcal{M}}_{\ast }$. Regime-specific coefficient groups ${\mathit{\varphi }}_j$, under Equation (1), are sequentially obtained given ${\hat{\mathit{\varphi }}}_1={\hat{\mathit{\theta }}}_1$ and ${\hat{\mathit{\varphi }}}_j={\hat{\mathit{\theta }}}_j+{\hat{\mathit{\varphi }}}_{j-1}$ for $j\in \{2,\dots ,m+1\}$.

In both DGPs, 100 replications each of length 1000, with i.i.d. standard normal errors, are simulated. The delay d = h = 1 in all DGPs is assumed known. For the high-dimensional penalized regression, 5000 posterior samples are computed after a burn-in period of 5000, with a thinning of 20. For regime identification and subset variable selection, we use 0.95 as a stopping rule for RelE. We report the percentage of replications that correctly identified the number of regimes along with the percentage of replications that overestimated and underestimated the true m + 1. For replications that accurately identified the true number of regimes, we report the mean and standard deviation of posterior means for non-zero parameters including the thresholds when m + 1>1. The mean bias (MB) and root mean squared error (RMSE) are also reported to evaluate the overall estimation error across these replications.

5.1. Simulation 1

The first data generating process is AR(1):

$$y_t=0.5y_{t-1}+2{ϵ}_t.$$ (14)

Using the high-dimensional approach, we estimate the marginal change in autoregressive dynamics as thresholds are crossed. Autoregressive parameters correspond to one of 51 distinct regimes defined by a specific threshold; for each of regime, we evaluate the current state of the autoregressive parameters. Figure 4 traces the autoregressive estimates for all 100 replications as thresholds are passed. Solid black lines show the true paths of these estimates under Equation (14). The intercept is often estimated to be nonzero, but this is a parameter forced into every considered submodel. A numerical evaluation for Simulation 1 is provided in Table 1: our methodology correctly identifies the number of regimes 100% of the time and consistently estimates the non-zero parameters ${\varphi }_1^{(1)}$ and σ accurately.

Figure 4.

Regime coefficient progression across possible thresholds for Simulation 1.

Table 1. Evaluation of Simulation 1.

Parameter	Actual	Mean	SD	MB	RMSE	Correct(%)	Over(%)	Under(%)
${\varphi }_1^{(1)}$	0.5	0.49	0.03	−0.01	0.00	–	–	–
σ	2	2.00	0.05	−0.00	0.00	–	–	–
m + 1	1	–	–	–	–	100	0	0

5.2. Simulation 2

Next, we consider the 2-regime TAR, which is similar to the one investigated in [66]:

$$y_t=\{\begin{array}{cc}0.5y_{t-1}+2{ϵ}_t,& \mathrm{if}{y}_{t-1}\le -0.4\\ -0.5y_{t-1}+2{ϵ}_t,& \mathrm{if}{y}_{t-1}>-\mathrm{0.4.}\end{array}$$ (15)

The dynamics of this DGP is difficult to estimate as the two regimes possess similar structure with no intercept [13]. Using 0.95 as a stopping threshold for RelE in both the regime identification and variable selection, 79% of the replications identified correctly the number of regimes. All other replications led to final models with more than 2 regimes. Lowering the stopping rule for RelE to 0.9, thus favoring more parsimonious TAR models, results in improved identification of the number of regimes (84% accuracy). Figures 5 and 6 show the posterior means of autoregressive parameters for potential thresholds under both stopping rules. Likewise, Table 2 summarizes empirical performance, results corresponding to 0.9 stopping threshold given in parentheses. Estimation of the regime-specific parameters is excellent in both cases. Figures 5 and 6 and Table 2, suggest a difficulty in the estimation of the threshold ${\delta }_1=-0.4$; the same problem has been observed in [66] where the true parameter was outside the 95% credible region.

Figure 5.

Coefficient progression across possible thresholds for Simulation 2 using 0.95 as a stopping threshold.

Figure 6.

Coefficient progression across possible thresholds for Simulation 2 using 0.90 as a stopping threshold.

Table 2. Evaluation of Simulation 2 (Results under 0.9 Stopping Rule Used are Given in Parentheses).

Parameter	Actual	Mean	SD	MB	RMSE	Correct(%)	Over(%)	Under(%)
${\varphi }_1^{(1)}$	0.5	0.49 (0.49)	0.03 (0.04)	−0.01 (−0.01)	0.03 (0.04)	–	–	–
${\varphi }_1^{(2)}$	−0.5	−0.50 (−0.50)	0.07 (0.07)	−0.00 (−0.00)	0.07 (0.07)	–	–	–
σ	2	2.00 (2.00)	0.05 (0.05)	−0.00 (−0.00)	0.05 (0.05)	–	–	–
${\delta }_1$	−0.4	−0.20 (−0.17)	0.30 (0.35)	0.20 (0.23)	0.36 (0.41)	–	–	–
m + 1	2	–	–	–	–	79 (84)	21 (16)	0 (0)

6. Forecasting performance

6.1. Full models

Continuous shrinkage priors are implemented using R package bayesreg [77], which can handle Gaussian, Laplace and Student-t distributed ${\mathit{ϵ}}_R$ and ${\mathit{ϵ}}_F$. The TAR and SEAS models contain large ${\mathit{X}}_R$ and ${\mathit{X}}_F$ design matrices; the full conditional distributions of ${\mathit{\theta }}_R$ and ${\mathit{\theta }}_S$ are based on $408\times 408$ and $301\times 301$ matrices, respectively. To obtain S posterior samples $\{{\mathit{\theta }}_R^{(s)}{\}}_{s=1}^S$ and $\{{\mathit{\theta }}_F^{(s)}{\}}_{s=1}^S$, inversion of ${\mathit{X}}_R^{\mathrm{\prime }}{\mathit{X}}_R$ and ${\mathit{X}}_S^{\mathrm{\prime }}{\mathit{X}}_S$ is required. The algorithm of [75] performs fast Gibbs sampling in scenarios similar to our application, but for larger choices of m, and P, leading to high-dimensional designs, the algorithm of [6] is an efficient alternative. For both TAR and SEAS models, S = 5000 posterior samples after a burn-in period of 5000 and a thinning of 10 was large enough to ensure that the minimum effective sample size of all parameters was larger than 150 and that the multivariate effective sample size [85] was larger than 10000 in all examined cases. Figure 7 depicts the Geweke diagnostic [29] for the basic specifications that are based on Gaussian errors and estimate 408 coefficients per location and per day for h = 1. The observed Z-scores do not suggest large differences in means between the first (10%) and last (50%) parts of Markov chains.

Figure 7.

Distribution of calculated Geweke diagnostics for all coefficients and all models that correspond to h = 1.

How sensitive are the posterior distributions of the unknown coefficients and the forecasts to different choices of priors and alternative likelihood specifications? Figure 8 shows the estimated posterior means for all coefficients, of all models that correspond to h = 1, for the first 2 days of our study. It is evident that the choice of a likelihood which ‘robustifies’ models against outliers (Laplace) and/or heteroscedasticity (Student-t), is far more important than the choice of shrinkage priors in this evaluation.

Figure 8.

Posterior means of estimated coefficients for h = 1 for the first 2 days of the study. Horseshoe plus priors versus Horseshoe and double Exponential (Lasso) priors (top). Normal versus Laplace and Student-t errors combined with Horseshoe plus priors (bottom).

Forecasts from the 5 alternative full specifications for h = 1 are evaluated using the mean absolute scaled forecast error (MASFE) metric proposed in [39]:

$$\mathrm{MASFE}(h)=\frac{1}{T_h}\sum _{t=P+h}^{480}\left|\frac{O_{L,t}-{\hat{O}}_{L,t}}{{\mathrm{MAE}}_{RW}(h)}\right|$$ (16)

${\mathrm{MAE}}_{RW}(h)$ represents the mean absolute error from a h-specific naive random walk (RW) model over the fitting period where ${\hat{O}}_{L,t}$ equals the observed $O_{L,t-h}$. Whenever MASFE$(h)<1$, the model produces absolute forecast errors that are, on average, less than the forecast errors from a RW. Table 3 suggests that although naive, the RW should be included in the 'model-library' of an online system, since quite frequently it outperforms more complex specifications for h = 1. The choice of shrinkage priors does not influence results dramatically; double exponential priors perform slightly worse than BHS and ${\mathrm{BHS}}^{+}$, in accordance with what is reported in [5]. Interestingly ‘robustified-models’ based on Student-t(5) likelihoods achieve the best performance and should also be considered in the 'model-library' of an online system. The next section presents the model building steps for models with Gaussian errors and ${\mathrm{BHS}}^{+}$.

Table 3. 1-Step ahead MASFE forecast comparison for alternative likelihood-prior combinations.

		Location
Full Model	Day	A	B	C	D	E	F	G
Normal ${\mathrm{BHS}}^{+}$	M	0.99	1.07	0.88	1.04	1.77	0.81	1.44
	T	0.92	1.07	1.03	0.96	1.38	1.01	1.88
Normal BHS	M	0.99	1.09	0.88	1.04	1.76	0.82	1.45
	T	0.92	1.07	1.03	0.96	1.38	1.01	1.90
Normal BLasso	M	1.08	1.09	0.88	1.06	1.72	0.80	1.49
	T	1.04	1.08	1.01	0.94	1.40	1.05	2.04
Laplace ${\mathrm{BHS}}^{+}$	M	1.01	1.08	0.85	1.06	1.56	0.63	1.27
	T	0.93	1.04	0.96	0.94	1.10	0.88	1.37
Student ${\mathrm{BHS}}^{+}$	M	0.90	1.07	0.85	1.02	1.46	0.56	1.21
	T	0.9	1.05	0.96	0.94	0.99	0.76	1.43

6.2. Regime identification

The model building procedure outlined in Section 4 resulted in h-specific TAR models for the TRANSFOR data, which vary in complexity across locations and days. Final models range from linear, 1-regime AR(P), to nonlinear 9-regime TAR(P), when a 0.95 stopping threshold is used for RelE. Figure 9 summarizes models according to the number of regimes identified: more than 50% of all final models are either 2-regime or 3-regime TAR(P), which is in accordance with prior expectations for this application. It is worth highlighting that this implies a dramatic reduction in the number of coefficients that will be used in real-time, hence significant computational savings.

Figure 9.

Percentage of final models based on number of regimes.

Models with many regimes can be hard to explain; for the few scenarios where m + 1>5, we repeat our three-step algorithm lowering the stopping rule for RelE, to 0.9. Tables 4–6 display the estimated number of regimes for all $(L,D,h)$-specific models. The final $(L,D,h)$-specific TAR models for traffic occupancies vary in the number of selected regimes by location, day, and horizon. Figures 10–12 depict on the original [0-1]-interval, the m thresholds in occupancies that characterize traffic regimes. Horizontal lines represent scenarios where regime-switching behavior is absent.

Table 4. Number of Regimes Identified for 1-Step Ahead Models. Results corresponding to a reduced stopping threshold are given in parentheses.

	Location
Day	A	B	C	D	E	F	G
M	1	2	1	2	3	2	4
T	2	5	4	1	2	3	3
W	4	5	3	1	3	4	3
Th	5	9 (4)	4	3	7 (4)	3	3
F	5	6 (2)	3	5	2	3	2

Table 5. Number of Regimes Identified for 3-Step Ahead Models. Results corresponding to a reduced stopping threshold are given in parentheses.

	Location
Day	A	B	C	D	E	F	G
M	3	2	2	2	4	2	2
T	2	6 (3)	4	2	2	2	2
W	6 (4)	4	3	2	3	3	5
Th	3	5	2	3	2	3	5
F	9 (5)	5	4	6 (3)	3	2	2

Table 6. Number of Regimes Identified for 5-Step Ahead Models. Results corresponding to a reduced stopping threshold are given in parentheses.

	Location
Day	A	B	C	D	E	F	G
M	3	3	5	2	4	2	3
T	2	7 (3)	6 (4)	4	2	2	5
W	8 (3)	6 (4)	6 (3)	3	2	3	6 (3)
Th	5	7 (4)	2	2	2	2	2
F	4	4	3	4	4	2	2

Figure 10.

Thresholds identified for 1-step ahead models using 0.95 as a stopping threshold in model building.

Figure 11.

Thresholds identified for 3-step ahead models using 0.95 as a stopping threshold in model building.

Figure 12.

Thresholds identified for 5-step ahead models using 0.95 as a stopping threshold in model building.

To illustrate the different models, we start with the $(C,M,1)$ model:

$${\hat{Y}}_t=-0.12+0.52Y_{t-1}+0.10Y_{t-2}+0.10Y_{t-4}+0.18Y_{t-7}$$ (17)

which is linear and outperforms in terms of forecasting accuracy both the harmonic model and the naive random walk. Only estimates for non-zero autoregressive terms are displayed; furthermore, these estimates in ${\mathit{\theta }}_R$ are sequentially modified in Equation (1) using ${\mathit{\varphi }}_j={\mathit{\theta }}_{\mathit{j}}+{\mathit{\varphi }}_{j-1}$. In Equations (18) and (19), occupancy thresholds are depicted on their original scale in parentheses. For the $(G,F,3)$-specific TAR model, 2 regimes lead to optimal forecasting performance:

$$\begin{array}{rl}{\hat{Y}}_t& ={\hat{\varphi }}_0^{(j)}+\sum _{i=1}^2{\hat{\varphi }}_i^{(j)}{Y}_{t-3-i+1}\text{for }{\delta }_{j-1}<Y_{t-3}\le {\delta }_j\text{ where}\\ {\hat{\varphi }}_0^{(1)}& =-0.69,{\hat{\varphi }}_1^{(1)}=0.45,{\hat{\varphi }}_2^{(1)}=0.31,{\hat{\varphi }}_0^{(2)}=-0.24,\text{and}{\hat{\delta }}_1=\underset{(0.07)}{-2.59}.\end{array}$$ (18)

Finally, an example where a 3 regime model dominates is given by the $(A,M,5)$-specific TAR:

$$\begin{array}{rl}{\hat{Y}}_t& ={\hat{\varphi }}_0^{(j)}+\sum _{i=1}^7{\hat{\varphi }}_i^{(j)}{Y}_{t-5-i+1}\mathrm{for}\\ delt{a}_{j-1}<Y_{t-5}\le {\delta }_j\mathrm{where}\\ {\hat{\varphi }}_0^{(1)}& =0.05,{\hat{\varphi }}_1^{(1)}=0.59,{\hat{\varphi }}_2^{(1)}=0.18,{\hat{\varphi }}_7^{(1)}=0.19,\\ {\hat{\varphi }}_5^{(2)}& =0.15,{\hat{\varphi }}_1^{(3)}=0.22,{\hat{\delta }}_1=\underset{(0.48)}{-0.08},\mathrm{and}{\hat{\delta }}_2=\underset{(0.73)}{0.99}.\end{array}$$ (19)

Interestingly, in this case, the third regime corresponds to levels of occupancy greater than 0.73, which conforms with heavily congested traffic conditions.

Traffic occupancy forecasts are evaluated over the final week of April on the original $[0,1]$-interval, using a rolling-window, without re-estimation. For each $(L,D,h)$-specific TAR model, there are $T_h=480-P-h+1$ time points requiring forecasts. Using the $(L,D)$-specific harmonic models estimated from the first three weeks of April, forecasts ${\hat{O}}_{L,t}$ can be quickly calculated for all horizons. Figure 13 displays the SEAS models fitted to the last week of April. The 3-min ahead forecasts are depicted in Figure 14; figures corresponding to longer horizons are provided as supplemental files. In the TAR plots, $(1-2\alpha )\times 100\mathrm{\%}$ credible regions are displayed instead of point forecasts for $\alpha \in \{0.4,0.25,0.2,0.15,0.1,0.05\}$. The last Friday of April, which falls on a Greek holiday weekend (Labor Day), had unusually low levels of occupancy relative to the first three Fridays of April. Although forecasting accuracy is less critical for low congested states, this day illustrates the significance of short-term dependencies in modeling traffic dynamics.

Figure 13.

Forecasts based on harmonic models; observed occupancies are depicted in gray.

Figure 14.

1-Step Ahead TAR Density Forecasts.

By scaling errors using ${\mathrm{MAE}}_{RW}(h)$, TAR can be compared to SEAS and both can be compared to naive RW. Tables 7–9 compare final TAR and SEAS models for all days and locations at horizons $h\in \{1,3,5\}$, respectively. Results in parentheses display the MASFE under final models developed using a 0.9 stopping rule for RelE. The purpose of lowering the stopping rule was to force simpler, more intuitive models that would hopefully improve forecasting performance. For most of the cases when $(m+1)>5$ under 0.95, the MASFE slightly changed when the stopping rule decreased; however, there are 2 cases where the search of a simpler model led to a significant decline in forecasting performance. Specifically, the MASFE increased for the $(E,Th,1)$-specific TAR model from 1.48 to 2.55 and for the $(A,F,3)$-specific TAR model from 3.04 to 3.21 when the stopping rule decreased. On the other hand forecasting performance did not get worse for models derived with the 0.95 rule, relative to the full specifications (3); on the contrary slight improvements can be observed due to removing a large number of weak predictors.

Table 7. 1-Step ahead MASFE forecast comparison; results with the 0.9 stopping threshold are given in parentheses.

		Location
Day	Model	A	B	C	D	E	F	G
M	TAR	1.02	1.10	0.88	1.07	1.87	0.81	1.48
	SEAS	1.80	1.47	1.15	1.43	4.02	1.57	3.66
T	TAR	0.90	1.05	1.04	0.98	1.36	1.03	1.95
	SEAS	1.35	1.36	1.22	1.46	3.36	1.65	3.29
W	TAR	1.04	1.11	0.91	0.97	2.27	1.86	1.55
	SEAS	1.39	2.01	2.18	1.61	4.65	2.90	2.80
Th	TAR	0.93	0.89 (0.94)	0.82	0.92	1.48 (2.55)	1.52	1.83
	SEAS	1.44	1.43	1.51	1.42	3.98	2.74	4.07
F	TAR	1.80	1.08 (1.02)	1.01	0.85	1.45	2.40	1.16
	SEAS	4.77	2.23	1.98	1.83	4.37	6.24	3.78

Table 8. 3-Step ahead MASFE forecast comparison; results with the 0.9 stopping threshold are given in parentheses.

		Location
Day	Model	A	B	C	D	E	F	G
M	TAR	0.94	1.06	0.88	1.13	1.85	0.88	1.50
	SEAS	1.36	1.17	0.93	1.14	2.91	1.19	2.57
T	TAR	0.87	1.04	0.96	1.03	1.46	1.15	1.21
	SEAS	1.04	1.06	0.91	1.10	2.24	1.22	2.15
W	TAR	1.09 (1.08)	1.15	1.10	0.99	1.75	2.00	1.26
	SEAS	1.15	1.61	1.69	1.21	2.94	2.24	1.71
Th	TAR	0.90	0.96	0.82	0.93	1.88	1.47	1.14
	SEAS	1.15	1.09	1.14	1.03	2.69	1.96	2.68
F	TAR	3.04 (3.21)	1.09	1.00	0.66 (0.68)	1.14	2.47	0.92
	SEAS	3.53	1.57	1.42	1.33	3.12	4.35	2.42

Table 9. 5-Step ahead MASFE forecast comparison; results with the 0.9 stopping threshold are given in parentheses.

		Location
Day	Model	A	B	C	D	E	F	G
M	TAR	0.94	0.99	0.89	1.12	1.97	0.86	1.68
	SEAS	1.24	1.06	0.88	1.06	2.46	1.08	2.17
T	TAR	0.81	1.05 (1.02)	0.95 (0.95)	1.00	1.47	1.13	1.15
	SEAS	0.95	0.95	0.85	0.99	1.85	1.07	1.77
W	TAR	1.02 (1.02)	1.12 (1.15)	1.04 (1.00)	0.99	1.67	1.94	1.26 (1.19)
	SEAS	1.01	1.44	1.56	1.12	2.30	1.95	1.44
Th	TAR	0.84	0.98 (0.99)	0.82	0.87	1.51	1.48	1.22
	SEAS	1.05	1.03	1.07	0.94	2.13	1.73	2.24
F	TAR	2.85	1.20	0.88	0.70	1.26	2.52	1.00
	SEAS	3.07	1.48	1.33	1.19	2.59	3.82	2.01

In accordance with prior expectations, multiple-regime TAR models consistently outperform harmonic profiles at all locations and days when forecasting 1-step ahead. For 3-step ahead forecasts, the SEAS model for Tuesday at location C has a smaller MASFE, by a negligible amount. For 5-step ahead forecasts, more occurrences of SEAS producing forecasts, as good or better, than TAR are observed. The opposite pattern is exhibited for h-specific RW. For 1-step ahead forecasts, $\mathrm{MASFE}>1$ for many models; as h increases, a clear advantage of using more complicated models (TAR and SEAS) to capture nonlinear and/or seasonal dynamics can be noticed. This is generally true except for locations E, F, and G where traffic occupancies are considerably more difficult to forecast accurately; this is indicated by the wide credible regions in Figure 14, which are computed conditional on the model selection procedure.

7. Conclusion

Algorithms for real-time short-term forecasting of traffic occupancies constitute an important module of Intelligent Transportation Systems. Quite frequently the dynamics of occupancy are hard to estimate accurately and the resulting forecasts are unsatisfactory. This article presented interpretable threshold autoregressions that capture non-linearities in traffic dynamics using piecewise linear processes. In practice, specifications similar to the ones presented here can be used as components of a ‘library-of-models’, with final forecasts computed using regime-switching combination schemes [2].

In the vast majority of published works, researchers focused on TAR models with at most three regimes, due to computational challenges. In this application a high-dimensional linear model which nests a large collection of TAR specifications is constructed. Sparse estimation of the coefficients not only quantifies the optimal number of regimes, but also identifies the thresholds. After fixing the maximum autoregressive order and the maximum number of regimes, we present a three-step model building procedure that automates subset TAR$(P)$ selection. In contrast with previous Bayesian investigations of TAR, the hierarchical scheme presented here (a) can be implemented using existing samplers; (b) results in geometrically ergodic Markov chains, which ensures strongly consistent estimation for selected shrinkage priors; and (c) can model heteroscedastic data with outliers by adopting non-Gaussian likelihood specifications.

The proposed methodology can be improved in a number of ways. The posterior prediction projective method for subset selection of TAR uses the assumption of normal and homoscedastic errors. Although we analyze logit-transformed data and capture some of the heteroskedasticity when converting back to the original scale, our experiments suggest that it may be advantageous to modify the hierarchical Bayesian scheme using Student-t-distributed errors. In addition, regime-specific autoregressive dynamics can be augmented to include information from neighboring detectors, leading to space-time models. These topics are left for future research.

Disclosure statement

No potential conflict of interest was reported by the author(s).