Post-script—Retail forecasting: Research and practice

Abstract

This note updates the 2019 review article “Retail forecasting: Research and practice” in the context of the COVID-19 pandemic and the substantial new research on machine-learning algorithms, when applied to retail. It offers new conclusions and challenges for both research and practice in retail demand forecasting.

“The retail sector is experiencing seismic change” — this is the opening sentence of the conclusions of the retail review article, available online in 2019 but printed for the first time in this issue (Fildes, Ma, & Kolassa, 2022 hereafter “the Review”). As predictions go, this has withstood the test of time. All major countries in Europe have seen dramatic increases in online activity, for example, the UK bricks-and-mortar retailers have continued to lose substantial ground, whereas online sales share in 2020 increased 6.8%: even Italy (with an estimated penetration in 2020 of only 6%) has seen a doubling of activity.1 The latest US estimates of online retail sales’ share increased to 14% in 2020.2 Hand-in-hand with this expansion in both absolute and relative terms, we have seen the dramatic decline in shop purchases (apart from grocery supermarkets) and closure of shops in the wake of the COVID-19 pandemic and attendant lockdowns. In China, online retail sales were up 7.3%, whereas total retail sales of consumer goods were down 9.9% in the first half of 2020 with respect to the same period in 2019.3

2021 has seen an equally dramatic resurgence of retail sales with consumers returning to shopping both on and off-line in countries such as the US and the UK. These changes raise fundamental questions as to how retail is to be reconfigured. From our forecasters’ perspective, they underline two forecasting issues we identified in our Review: the need to consider store closures in location decisions, and second, the complementarity and competition of online with other retail forms and the emergence of omni-retailing.

In addition, future analyses will need to account for the impact of the COVID-19 pandemic. Any training data used for retail forecasting for the next years will require adjustment for the still developing epidemic. And, critically, we should learn from the pandemic to prepare for any future similar events. Although the undermining of established retail patterns was already well underway when the Review was published online, the COVID-19 pandemic surging forward in February 2020 and, at the time of writing (September 2021), still raising chaotic waves of retail changes, was not foreseen (epidemiologists excepted). The pandemic has forced many changes on the role of the demand planner and the forecasting systems they use. In this postscript, we will consider what new has been written and offer our conclusions as to the research needing urgent attention.

Although consumers’ acceptance of the developing importance of omni-retailing has been rapid, research in the area, inevitably lagging, has been limited. Online purchasing offers the researcher many advantages despite the problems of scale, in particular the vast quantities of data across product groups. For online retailers, in addition to standard demand forecasting problems, the effects of web-site design and recommender systems are operationally important and could (but have yet to) provide a rich data base for researchers. The growing importance of the web in the product purchasing decision has raised the question of whether various online data sources from online reviews to influencers and their followers affect retail sales. Research has covered these new sales drivers, associated measurement issues and their value in forecasting demand (Schaer, Kourentzes, & Fildes, 2019). New models might also be needed to capture the relationship between a social media event and any resulting sales.

If the developing changes in the retail markets were proving challenging, the COVID-19 crisis was an event without precedent (often described as a black swan); it has inevitably amplified the role of the demand planner’s judgment at the expense of the established model-based forecasts. In Section 1 we comment on the limited methods that have been proposed to deal with such instability and suggest research routes that might be productive.

The 2019 Review highlighted various methodological issues such as the ‘increased use of AI/machine-learning methods and the lack of validation in the research literature’. Here we have seen rapid progress with the results of the M5 Kaggle competition on Walmart data (Makridakis, Spiliotis, & Assimakopoulos, 2022: this issue) being particularly important. In Section 2 we will review the methodological advances to ensure the challenges we set out two years ago remain important to researchers and retail practitioners alike. The final section revisits the Review’s conclusions, offering some revised research priorities.

1. Retail demand forecasting in a period of exceptional change

1.1. The growth of online and omni-channel retail

Demand forecasting for online retailers presents a different set of challenges, compared with offline retailers. Online retailers can cover wider geographical locations, have the ability to adopt personalised pricing and promotions, and potentially compete in far more product categories, selling many more SKUs including the rapid introduction of many new products. There is also a potential interaction for an omni-retailer with complementary effects, leading to changes in emphasis on store format and location and the need to incorporate e-commerce into location models (Beckers et al., 2021; see also the discussion in Caro, Gürhan Kök, & Martínez-De-Albéniz, 2020). In addition, the integration of the two channels provides a means for operational improvements (Pereira & Frazzon, 2021). A key decision for many established retailers has been whether to develop an online presence and how to manage and integrate the different customer segments split between store, online and joint use (Frasquet, Ieva, & Ziliani, 2021).

Online retailers have a data advantage over traditional offline retailers, which creates the opportunity for more accurate demand predictions. Online retailers have the ability to track consumers’ purchasing behavior, but also their browsing behavior, reviews and much more. We discussed these unique “big data” resources online retailers command in Section 4.3.7 of the Review. The effective use of these data sources has already seen a growth in research in the three years since the Review, and we expect they will play a key role in improving the accuracy of online demand forecasting, and bring new opportunities for researchers. One example is Martínez-de-Albéniz, Planas, and Nasini (2020), who used early clickstream information to build demand models and study the pricing problem for a flash sale retailer. Their model enables flash sales retailers to learn about the performance of new products in a few hours and to update prices so as to better match supply and demand forecasts, the result being improved profits. However, not all online information has proved valuable, and Schaer et al.’s (2019) critical review of the value of online search data has not yet seen subsequent convincing counter-evidence.

The last years’ growth in online sales, which has been accelerated by COVID-19, presumably does not represent net new demand, but mainly a shift in the channel mix: products that used to be bought in a store are now ordered online (and delivered from a distribution centre, or from a store, or picked up in a store). Thus, demands online and offline are linked, and accounting for this should have the potential to improve forecasting. We have not seen any research in this direction, possibly because linking the data is another source of complexity, but this may well represent the next important topic in retail forecasting.

In summary, we call for more researchers to take online retailers’ unique data advantages to build creative methods to improve retailers’ demand forecasting and operational decisions, including the interactive cross-channel effects for omni-retailers.

1.2. The effects of the COVID-19 pandemic on retail and retail forecasting

The worldwide COVID-19 pandemic has had major and ongoing impacts on the retail sector, with corresponding repercussions for forecasting. The most immediate and obvious consequence was a series of successive lockdowns and re-openings, which completely closed down stores for weeks at a time and impacted order-inventory decisions. However, even stores that remained open, perhaps with more limited hours (mostly grocery and other “essential” stores), saw major disruptions: customers minimised their trips to stores and bought larger amounts in each trip, leading to an increase in demand lumpiness and variance; they also attempted to visit fewer stores in each shopping trip, which was advantageous to larger warehouse-type retailers and disadvantageous to specialty retailers and discounters. To make matters worse, panic purchases completely depleted stocks for certain products (Boone & Ganeshan, 2021; cf. Figure 1) or led to enormous impacts of, e.g., promotions on staple products during the early phases of the pandemic. Finally, specific products like facemasks or hand sanitiser of course saw unprecedented breaks in demand patterns – which also occurred in seemingly unrelated categories like jigsaw puzzles or cat litter boxes.

Next, there are what might be called “secondary” effects on retail (Stephens, 2021). The sudden shift to work and education from home abruptly cut off large parts of daily commuter flows, shutting off customer traffic from the entire retail ecosystem that has grown up to service this reliable demand, from convenience stores near office locations to gas stations near commuter routes and railway station infrastructure. Simultaneously, at least part of this demand shifted online, or to other retail locations.

In terms of forecasting, the challenges were threefold:

• (i)Retailers have needed to identify periods during which recorded sales (or non-sales) were not informative for future forecasting. That is, they needed to mark zero sales during complete store closures as invalid, but also zero sales during stock-outs as in Fig. 1, and, in addition, panic purchases in the run-up to such stock-outs.
• (ii)“Informative” sales during the COVID-19 pandemic fall into two distinct types, which have needed to be treated separately. On the one hand, sales patterns may not be repeated after a return to normalcy. For instance, we have seen enormous uplifts for promotions on staples like olive oil during the early phases of the pandemic. People likely hoarded these in fear of supply disruptions, but such strong promotional uplifts are unlikely to recur in more ‘normal’ times.
• (iii)Alternatively, demand patterns have developed that may indicate a “new normal”. Face masks and hand sanitisers have seen structural breaks, both in baseline sales and in promotional response. Although demand for these products may decrease after the world has been vaccinated, it appears likely that a much higher appreciation for these products will continue and drive demand for the next years. Such changes can be modelled using Boolean indicators that recur in the future. (Alternatively, one can set the predictor to 1 for past sales pre-COVID-19, so it can be dropped out of the model as this time recedes out of the training sample.) Using multiple different predictors allows modelling the differential impacts of the different phases of the pandemic and attendant insecurity and lockdowns. A more ambitious approach which is supported by company evidence is to use indicators such as the number of COVID-positive cases to influence the model forecasts. Including interaction terms between these predictors and promotional predictors will model changes in promotional responsiveness. Of course, forecasters need to be careful about the resulting explosion of predictors (cf. Section 4.5 of the Review). Formal methods for incorporating any structural change in the model’s features (such as promotional effects or the level change) may have potential through moving windows estimation or intercept correction (Huang, Fildes, & Soopramanien, 2019): both approaches weigh more recent data more heavily.

Fig. 1.

Toilet paper shelves in a German supermarket, December 2020.

Judgmental forecasts have inevitably substituted for model-based forecasts for many products in the initial phase of the epidemic. As a limited stability has developed with some components such as seasonality potentially remaining consistent, whereas others – such as the level of sales – changed, the question of how to integrate these two sources of forecast information has become acute: estimation has necessarily mostly relied on demand planners’ marketing intuitions. New methods of combining judgment with model-based information may well prove valuable beyond standard forecast combination (but none have yet surfaced in the research literature). Premature abandonment of all model-based forecasting is surely dangerous.

The key question posed by (ii) is of course how to distinguish the ephemeral from the persistent. Which of the effects we have seen in the last years (2020–2022) will not re-occur, and which ones represent a new pattern of customer behaviour? Will office workers mostly return to the office, or will there be a large amount of home office work going forward, with consequences for commuter flows? Even if “only” a small percentage of workers continue to work from home in the future, this may be enough to push struggling retail stores over the edge into unprofitability (Stephens, 2021).

Somewhat surprisingly, we have not been able to find any published research on retail forecasting that focuses on COVID-19 effects at a retailer level. Alicke and Hoberg (2020) expand on the issues discussed above, whereas Nikolopoulos, Punia, Schäfers, Tsinopoulos, and Vasilakis (2021) discuss aggregate effects and Alexandrov, Brooks, Lee, and Monokroussos (2021) have provided a case study where dynamic factor models have been successfully combined with executive judgments. We hypothesise that, on the one hand, retailer forecasters have been too busy to collaborate with academic forecasters during this period. In contrast, lockdowns, vaccinations and other developments have been extremely dynamic, which does not lend itself to typical academic publishing time cycles. In any case, how the above effects should be included will be a question that every publication in this research space will need to address for the foreseeable future.

2. Methodological developments

Recent progress in retail forecasting has mainly come from the development and application of machine-learning (ML) methods in large-scale retail demand forecasting. These methods and their evaluation have provided the largest number of new references to supplement the full review. Three years ago, we concluded that the evidence for ML methods generally leading to better forecasting accuracy was weak. However, now it is becoming clear that ML methods have superior forecasting power than traditional time series or linear regression methods when there are many related sales series to be forecasted and they should be trained in a global learning manner, i.e., where the model parameters are estimated jointly from a group of sales time series (Januschowski, Gasthaus, Wang, Rangapuram, & Callot, 2018). For example, Huber and Stuckenschmidt (2020) provided evidence that global ML models (feed-forward and recurrent neural networks, gradient boosted regression trees) could achieve a higher forecast accuracy compared to time series models with adjustments or a regularised linear regression model, and the ranking of the evaluated methods remained constant over different horizons. Kharfan, Chan, and Efendigil (2020) applied an ML pipeline comprising clustering, classification and prediction to forecast newly launched fashion products. Punia, Singh, and Madaan (2020) calculated coherent hierarchical forecasts in a multidimensional hierarchy comprising location, product and online vs. offline channels, again using an LSTM.

However, automatic application of ML methods has not proved to be inevitably successful. Ma and Fildes (2021) showed that ML models, including support vector regression and extreme learning machines, using local training, performed worse than simple autoregressive distributed lag (ADL) regression. Using global training, however, ML methods on average perform better than regressions and time series models, especially when using more sophisticated ML methods including gradient boosting trees and random forests. The results from the M5 competition have provided further evidence on the superior performance of global ML methods when applied to retail data (Makridakis et al., 2022; see also the discussion papers in this same issue). Nearly all the top-ranking solutions in M5 used global ML methods. Montero-Manso and Hyndman (2021) provided a theoretical explanation for the performance of global methods: global models can be more complex than individual models in a local algorithm and still benefit from better generalisation.

Though ML methods on average can provide more accurate forecasts on large-scale demand forecasting tasks such as that required for retail chains, it does not mean that traditional methods, e.g. time series models, have become useless in this area. Wolters and Huchzermeier (2021) investigated promotional forecasting for strongly seasonal products by first extracting seasonal patterns using harmonic regression, then fitting multiplicative promotional uplifts in a standard log–log model, including post-promotional effects and promotion $\times$ seasonality interactions. They counteracted the effects of overfitting because of the large parameter space through regularisation by ridge regression. Ulrich, Jahnke, Langrock, Pesch, and Senge (2021) applied Generalised Additive Models for Location, Shape and Scale (GAMLSS) to e-grocery demands to forecast very high quantiles as inputs to a newsvendor inventory model. Ma and Fildes (2021) found that ETS, though performing the worst among all their benchmarks on average, outperformed its competitor forecasters (including global ML models) most often. For the M5 data, we also find that ETS proved equally effective – the top contender YJ_STU was better than ES_bu (exponential smoothing with bottom-up aggregation) only in 58.5% of series on store $\times$ SKU granularity. This finding is consistent with the no-free-lunch theorem of Wolpert and Macready (1997): there is no guarantee that any method, however complex it may be, performs better for a different set of series than another method; this implies that it is unlikely that one single method will dominate others for all products and all future time periods. Ma and Fildes (2021) therefore proposed a meta-learning framework based on newly developed deep convolutional neural networks, which can first learn a feature representation from raw sales time series data automatically, and then link the learnt features with a set of weights which are used to combine a pool of base-forecasting methods. Their experiments, based on IRI weekly data, which contains more promotional events than the Walmart data, showed that their proposed meta-learner provided superior forecasting performance compared with a number of state-of-the-art benchmarks.

ML forecasting methods are often criticised as lacking interpretability, working as a black box, which limits wider adoption of ML in forecasting applications (Gür Ali & Gürlek, 2020). But interpretability has been a key area in the new wave of ML research and much progress has been made in recent years. Antipov and Pokryshevskaya (2020) applied a recently developed unified approach to interpreting model predictions called Shapley Additive Explanations. They showed that the Shapley additive explanation of boosting tree predictions are very insightful, uncovering the effects of prices and promotions, as well as ideas related to stockpiling and cross-product effects. As the interpretability of ML in retail forecasting is still limited, we believe that it is a promising direction for future studies. For retail chains where promotional events are key, the interaction between the ML algorithm, the data scientist and the demand planner, and whether expert judgment still has an important role also remains important. As far as we are concerned the claims by software providers that their software removes the need for judgmental adjustments with consequential effects on demanding planning are unfounded. Most obviously, when an event as dramatic as the pandemic occurs there are no automatic routes available and the forecasting systems are not established which can formally incorporate and weigh the expert judgments of the demand planners facing volatile human behaviour and ML methods that are ill-understood. What research there is suggests the weights given to these two pieces of evidence are likely to be far from optimal.

3. Conclusions

Research in retail forecasting has seen more and more interest recently: a search on ScienceDirect turns up no less than 13600 articles since 2017, whereas Web of Science yields “only” 2000 articles (accessed 25 September 2021, with search terms: retail* AND (forecast* OR predict*) AND (sales OR demand)). Even taking into account the shot-gun characterisation of the search, our aim of giving a comprehensive overview of the state of the art likely is not realistic any more, and the field inevitably will see further fragmentation into subfields.

Two areas highlighted in the 2019 Review (Fildes et al., 2022) have seen much additional research activity: the development of online and omni-channel retailing, and the use and benefits of ML methods. Often the two have been linked, where brute computational force has been brought to bear and proved its effectiveness. We also see evidence of ML methods’ applicability and benefits in new product forecasting (van Steenbergen & Mes, 2020) and fashion (Ren, Chan, & Siqin, 2020). The implementation issues in a world of demand planners with specialist information have not yet been considered: although we know the demand for ML software is widespread, we are less sure that many organisations have the skilled and computational resources to adopt these methods successfully. Some retailers (e.g., Amazon, Walmart and some others) have invested heavily in data science capabilities and can handle the most recent advances in ML forecasting — but others lag and are restricted to either using less sophisticated methods, or to being at the mercy of software vendors and consultants.

Both of the major trends in retail forecasting discussed above – the influence of COVID-19 and the shift to ML methods – have impacts on the future role of forecasters, who will need to spend more and more time and effort in cleansing and pre-processing data from disparate sources and with varying quality (Alicke, Hoberg, & Rachor, 2019). Complete store closures may be easier to account for in models than reduced hours, limits on numbers of customers allowed in stores or mask mandates. Similarly, although the shift to online retail was strongly exacerbated by the COVID pandemic, the behavioural changes in shopping habits may well persist with retailers with little or no online presence being disadvantaged, and the impact may differ depending on categories. Again, the models needed require a changing parameters framework and potentially judgmental interventions if they are to succeed in using information both pre- and post-COVID, and forecasters will need to drive this issue. These model and data modifications can only be automated to some degree, and the modeling pipelines will need to be periodically revisited by forecasters (Seaman & Bowman, 2022, this issue; Kolassa, 2022, this issue). Ideally, forecasters’ skills will need to cover all of statistics, programming, communication (in order to actually get the relevant information and predictors from other stakeholders) and a deep business understanding (Kolassa, 2016). Realistically, companies will focus on the skill set that ‘fits’ with their self-image and perceived competencies. The changing role of forecasters and organisational aspects of the post-COVID ML forecasting world will make for interesting avenues for future research.

As the evidence we have summarised above shows, ML methods by now consistently yield better forecasts, but the differences compared to more classical approaches are not always substantial, and together with the loss in explainability, it is by no means clear that an investment in data science resources is always a worthwhile investment – in particular as better forecasts do not automatically translate into better operations (Kolassa, 2022, this issue). Seaman and Bowman (2022, this issue) explore some of these issues as they apply to Walmart. The organisational challenges provide an important test-bed to establish effective routes to implementation (Fildes & Goodwin, 2021).

The dramatic changes both in the retail world and the world outside highlight a key question for retailers (and academics): in what circumstances do improvements in forecasting accuracy, our focus, add organisational value? Recently, there has been an increased emphasis on looking at the financial and service consequences of improved accuracy: in promotional planning, Ma and Fildes (2017) have linked improved forecasting and modelling with increased revenue, in logistics and warehouse planning (Steinker, Hoberg, & Thonemann, 2017). Kolassa (2022, this issue) notes the impact of logistical replenishment pack sizes in the translation of forecast accuracy to better stock positions: it is not immediately obvious that improved forecasts will yield better inventories. In addition, low volume items often have their supply chain performance driven primarily by the case pack quantity (e.g., 12 or 24 units per box); it hardly matters whether the average forecast error is 150% or 350% if the true mean demand is 0.2 units over the lead-time ＋ review period and the order quantities are fixed at 24.

Events in the world of retailing have led the practice of retail demand forecasting to be turned upside down. Where retailers have chosen to ignore the upheaval, they have incurred heavy opportunity losses. But as this postscript has shown, in dealing with the COVID pandemic there have been no published proposals that suggest research-based routes to overcome the hazards we have described (we three are developing methods we hope will prove helpful). There are plenty of important and new research questions that merit urgent study.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.