Skip to main content
. Author manuscript; available in PMC: 2022 Mar 1.
Published in final edited form as: Wiley Interdiscip Rev Comput Stat. 2020 Jun 16;13(2):e1514. doi: 10.1002/wics.1514

Table 4:

List of close-ended questions asked of each full-text article. Questions focus on the forecasting target, model, analysis data, and experimental design.

Question Possible answers

Forecasting target
Identify the primary predictive target? predictive target
The primary target was categorical Y/N
The primary target was continuous Y/N
The primary target was from a time series Y/N
Experts were given data related to the forecasting target? Y/N
Terminology
List terms specific to aggregating crowdsourced data and quoted definition term,def;term,def
Model
What models were used in forecasting? model1, model2, …, modeln
Please list covariates included in any model cov1, cov2, …, covn
A novel model/method was developed Y/N
Did the authors implement a Bayesian technique? Y/N
Did the authors use a Frequentist technique? Y/N
Did the model account for correlation among experts? Y/N
The model combined point estimates Y/N
The model combined probabilistic distributions Y/N
The model was parametric Y/N
The model was nonparametric Y/N
Analysis data
Experts depended on data that could be updated, revised, or rapidly change? Y/N
Experimental design
A comparative experiment was conducted Y/N
How many expert forecasters were included? integer
How many total forecasts were made? integer
What evaluation metrics were used? metric1, metric2, …, metricn