Tourism forecasting competition in the time of COVID-19: An assessment of ex ante forecasts

Background

The COVID-19 pandemic and corresponding border control policies of various destinations have had a profound impact on the tourism industry worldwide. Although a few destinations adopted a ‘co-existence’ policy by partially re-opening their borders, changes in the COVID-19 situation, particularly with respect to the spread of new variants, have caused major uncertainties for the tourism industry.

Against this background, with the goal of advancing tourism forecasting methodologies and informing industrial practitioners about good practices in tourism forecasting and the predicted impact of COVID-19, in July 2020 the Curated Collection of Annals of Tourism Research on Tourism Forecasting announced a forecasting competition. Three competing teams, namely the Asia and Pacific team (Qiu et al., 2021), the Europe team (Liu et al., 2021) and the Africa team (Kourentzes et al., 2021), implemented two stages of tourism demand forecasting (ex post forecasts for 2019Q1–2019Q4 and ex ante forecasts for 2020Q1–2021Q4), represented by inbound visitor arrivals or hotel nights, for 20 countries/regions.

The competition rules for both stages and the results of the accuracy evaluation of the first-stage forecasting (i.e., ex post forecasting for the period 2019Q1–2019Q4) were described in Song and Li (2021). The Asia and Pacific team won Stage 1 of the competition by stacking five time-series models, which outperformed the benchmark seasonal naïve model by 22 % in terms of accuracy, evaluated by the relative mean absolute scaled error (MASE) against seasonal naïve model. The forecasting evaluation results for Stage 2 and the overall competition results of the three teams are set forth in this commentary and will be presented at the 8th Conference of the International Association for Tourism Economics in Perpignan, France in June 2022.

Overview of competing teams' modelling strategies and outcomes

Various advanced forecasting methods, such as artificial intelligence models, hybrid models and combination forecasting techniques, were adopted together with traditional time-series and econometric models by the three teams. The stacking of five time series models was selected by the Asia and Pacific team to produce Stage 2 baseline forecasts because it had the lowest MASE overall. The Europe team utilised forecasting methods such as time series models, artificial intelligence models, a hybrid of time series and artificial intelligence approaches and combined models, in the first stage, to perform ex post forecasting. Because the lowest MASE was detected from the forecasting result of the combination of four time-series models, that combination was selected to produce baseline forecasts for Stage 2. The Africa team used univariate models, including traditional time-series models, those with temporal and cross-sectional hierarchy, artificial intelligence models and multivariate models incorporating explanatory variables, in its Stage 1 forecasting. The hierarchical exponential smoothing model with cross-sectional and temporal aggregation generated the most accurate forecast over the validation sample (2017Q3–2019Q4) and was utilised to produce baseline ex ante forecasts.

In Stage 2, to compensate for the shortage of data amid the COVID-19 pandemic and associated travel restrictions, each team proposed creative judgemental adjustments and developed three COVID-19 scenarios (mild, medium and severe) to produce ex ante forecasts for 2020Q1–2021Q4 on top of the baseline models that performed best in Stage 1. The forecasting strategies adopted by each team, together with the lowest, highest and average predicted recovery rates in the medium scenario, are listed in Table 1 .

Table 1.

Ex ante forecasting strategies and predicted recovery rates in the medium scenario.

Team	Model used to generate baseline forecasts	Stage 2 judgmental adjustment strategies	Lowest recovery rate (destination)	Highest recovery rate (destination)	Average recovery rate
Asia and Pacific	Stacking of 5 time-series models	Assume important dates for each scenario, determine number of arrivals for each period, apply seasonal adjustments	29 % (Chile)	45 % (Mauritius)	36 %
Europe	Combination of 4 time-series models	Develop CORE index and country-specific adjustment coefficients	36 % (Mauritius)	77 % (Singapore)	51 %
Africa	Hierarchical exponential smoothing with cross-sectional and temporal aggregation	Impose quantified judgement of travel disruptions and set the steepness of adjustment to generate scenarios	39 % (Chile)	72 % (Japan)	58 %

The Asia and Pacific team defined the three scenarios according to the assumed recovery patterns with nation specialties. For each recovery pattern, important dates, namely the start of COVID-19, the duration of the pandemic, and the starting point and duration of recovery, were determined together with the number of visitor arrivals for each time period. Moreover, seasonal multipliers were constructed to seasonally adjust the trend line using 2018–2019 data.

Based on the team members' views, collected via the Delphi approach, the Europe team developed a COVID-19 exposure index that integrated two sub-indexes measuring country-specific accessibility risk and countries' self-protecting responses. Based on the developed COVID-19 exposure index, country-specific adjustment coefficients were then produced and used to predict tourist arrivals for the three COVID-19 scenarios.

The Africa team first grouped the countries according to geographical and climatic proximity and attached analysts' quantified expectations for travel restrictions between each pair of groups. The judgmental expectations and scenario indicators (α = 1, 10, 0.1 for medium, severe and mild scenarios, respectively) were used to adjust the forecasts in Stage 2. In addition, the Africa team consulted regional and industry experts to confirm the scenario settings.

Comments on ex ante forecasting results

To visualise the forecasting results, we selected five destinations – South Africa, Indonesia, Canada, Finland and Bulgaria – that commonly appeared in each team's top 10 destinations with the most accurate ex ante forecasts. For each selected destination, actual total tourist arrivals and each team's predicted total tourist arrivals in the severe scenario for 2021Q1-2021Q4 are presented in Fig. 1 . In most cases, the Africa team's forecasts of total tourist arrivals to each destination were the most optimistic. The Asia and Pacific team produced the most pessimistic forecasts, which were also the closest to the actual tourist arrivals in South Africa, Indonesia and Canada.

Fig. 1.

Actual and predicted total tourist arrivals in the severe scenario (2021Q1–2021Q4).

For Finland and Bulgaria (see Fig. 1), the forecasts of total tourist arrivals produced by the Asia and Pacific team seemed too pessimistic, and their predictions were much smaller than the actual values. The Africa and Europe teams captured larger proportions of total tourist arrivals in these two cases, with relatively optimistic expectations over the forecasting period.

Although the destinations most accurately forecast by the teams varied, the greatest prediction errors appeared in seven common destinations, namely Singapore, Australia, New Zealand, Japan, Thailand, Malaysia and Mauritius. For each of these destinations, the prediction errors were significant, with the smallest errors generated by the Asia and Pacific team. This reflects the unprecedented nature of the COVID-19 pandemic, which is full of uncertainties and thus creates great challenges to accurate forecasting. Continuous efforts are necessary to develop better methods of tourism demand forecasting in a crisis situation.

As stated in the competition rules, the MASE was employed to measure the forecasting accuracy of the Stage 2 results based on the most accurate set of scenario forecasts over the forecasting period of 2021Q1–2021Q4. Moreover, to aggregate the performance of forecasts in both stages, a weighting scheme of 40 % and 60 % was attached to the forecasting outcomes of Stage 1 and Stage 2, respectively. Table 2 exhibits the overall MASEs for both stages and the general performance by aggregating the ex post and ex ante forecasting accuracies.

Table 2.

Ex ante (2021Q1–2021Q4) and overall forecasting performance of each team.

Team	Stage 1 overall MASE of best-performing model	Stage 2 overall MASE			General performance (Stages 1 & 2)
		Mild scenario	Medium scenario	Severe scenario
Asia and Pacific	1.00	7.47	4.41	2.51	1.91
Europe	1.18	9.10	6.09	3.35	2.48
Africa	1.16	9.68	6.93	4.29	3.04

Notes: (i) To date, actual source-region-specific tourist arrivals data for the United Kingdom and Tunisia in 2021, Malaysia in the fourth quarter of 2021, the United States from Mexico and Canada in the fourth quarter and total arrivals are not available; we thus excluded these data when calculating prediction errors. (ii) The MASE of the Africa team in Stage 1 is based on the best performing model for the required test period (2019Q1–2019Q4), namely the ARIMA model with temporal hierarchy. (iii) The bold values indicate the most accurate forecasts among the three scenarios.

Table 2 yields three key findings, as follows. First, the most accurate scenario forecasts were generated under the severe scenario by all of the teams, as indicated by the smallest MASE values in Table 2. That is, under the circumstance of a global pandemic and partially reopened borders, the real tourist recovery in 2021Q1–2021Q4 was relatively modest; further stimulation is required. Second, the Asia and Pacific team obtained the smallest Stage 2 MASE among all candidate teams' MASEs under the severe scenario. That is, the Asia and Pacific team generated the most accurate ex ante forecasts in Stage 2. Third, general performance, which was the weighted average of both stages' MASEs, indicated that the Asia and Pacific team won the overall competition.

The results of the forecasting competition during the COVID-19 pandemic provide the tourism forecasting community with future research directions. First, although the three teams have made valuable attempts in this forecasting competition, how to properly take into account the uncertainties associated with a crisis and crisis-related policy responses to further improve forecasting accuracy still remains a challenge as well as an opportunity for future tourism forecasting research. Specifically, more efforts should be made in the research on how to set up appropriate scenarios in forecasting tourism demand during a major crisis. Second, this forecasting competition was focused mainly on point forecasts though scenarios associated with these point forecasts were developed. In the future, interval forecasts should also provide valuable information for decision makers, as they consider the uncertainties caused by crises in the forecasts.

CRediT authorship contribution statement

Haiyan Song: Project administration, Supervision, Resources, Writing – review & editing. Gang Li: Investigation, Methodology, Writing – review & editing. Yanting Cai: Investigation, Validation, Writing – original draft.

Declaration of competing interest

None.

Biographies

Haiyan Song serves as Mr and Mrs Chan Chak Fu Professor in International Tourism at The Hong Kong Polytechnic University. His research mainly focuses on tourism demand modelling and forecasting.

Gang Li is a professor of tourism economics at the University of Surrey, UK.

Yanting Cai is a Research Assistant at The Hong Kong Polytechnic University, and a PhD candidate in the School of Economics, University of Nottingham Ningbo China.

Associate editor: Scott McCabe

Data availability

No data was used for the research described in the article.