Evaluation of Google search trends for COVID-19 vaccination administration efforts within an urban department of public health

Abstract

Objective

To evaluate the utility of the Google COVID-19 Vaccination Search Insights (VSI) dataset for predicting and forecasting COVID-19 vaccine doses.

Methods

This retrospective, ecological time-series analysis used Google VSI data and Chicago Data Portal COVID-19 Daily Vaccinations dataset to predict and forecast weekly COVID-19 vaccination doses administered in Chicago between January 4, 2021 and December 20, 2021. Model performance was evaluated using root mean squared error (RMSE) and mean percent differences between the predicted and forecasted vaccination values and actual doses administered.

Results

SARIMAX models incorporating Google COVID-19 vaccination search frequencies, lagged by an average of 1.67 weeks, improved in-sample predictive accuracy relative to univariate SARIMA models, with gains most pronounced in high-vulnerability communities (High CCVI ZIP codes; mean RMSE reduction 84.73% across cross-validation iterations). Improvements in out-of-sample forecasting were similarly concentrated in High CCVI ZIP codes; in the remaining ZIP codes, SARIMAX forecasts were less accurate than SARIMA. SARIMAX forecasts produced narrower 95% confidence intervals across all strata, indicating greater model confidence.

Conclusions

These findings demonstrate the potential utility of the Google COVID-19 VSI dataset as a supplementary tool for informing COVID-19 vaccination distribution planning and resource allocation, particularly in socially vulnerable communities.

Introduction

As of January 26, 2022, there have been over 72.5 million COVID-19 cases and 875,000 deaths in the US. ¹ COVID-19 vaccines were developed and approved at unprecedented rates,^2,3 but vaccination rates remained inequitable, with disproportionately lower uptake in communities with higher COVID-19 burden, lower incomes, and greater structural barriers to vaccinations.^4,5 Given these inequalities, timely, actionable data on vaccination demand are critical for public health departments to allocate resources equitably and respond to fluctuations in real-time. Although the height of the pandemic has subsided and vaccination has transitioned toward routine, seasonal administration, the methodological question this study addresses remains relevant: whether population-level search behavior can supplement conventional data to anticipate localized shifts in vaccination demand, with implications for future public health emergencies that require rapid, equitable vaccination efforts.

During the pandemic, vaccination uptake was influenced not only by vaccine availability but also by individual- and community-level preferences related to vaccine safety, access, concerns about side effects, trust in public health institutions, and perceived effectiveness and risk, which varied across populations and over time.^6–10 Information-seeking behavior, including online searches related to vaccine eligibility, availability, and safety, reflected these changing concerns and preferences and has been associated with subsequent vaccination decisions.^10,11

In preparation for COVID-19 vaccination distribution, the Chicago Department of Public Health (CDPH) created the COVID-19 Community Vulnerability Index (CCVI) in January 2021. Community vulnerability for CCVI is defined as a combination of cumulative COVID-19 burden and sociodemographic, epidemiological, and occupational factors. ¹² The City of Chicago utilizes the CCVI for its Protect Chicago Plus program, which prioritizes communities with high CCVI scores (i.e., those communities most at-risk for COVID-19 and with more structural barriers to receiving COVID-19 vaccinations) in the COVID-19 vaccine distribution plan for the city. ¹³

In addition to the Chicago CCVI, CDPH utilizes conventional vaccination data sources, such as the number of doses administered and the demographics of those receiving the doses, to develop its vaccination distribution strategies. One such data source is the Google COVID-19 Vaccination Search Insights (VSI) dataset, which CDPH and other public health departments could use to better target their vaccination dissemination strategies. This publicly available dataset provides aggregated, anonymized data that capture population-level information-seeking behavior through Google searches related to COVID-19 vaccinations at the national, state, county, and ZIP code level. ¹⁴ Although Google search data have been used in prior infodemiology studies, most of this work has correlated general search activity with COVID-19 case counts rather than using a purpose-built search dataset to forecast an operational outcome, such as vaccine doses administered.

A potential benefit of the VSI dataset is giving public health departments access to timely, and otherwise unavailable data on community members’ vaccination-related searches. Insights gained from the dataset could inform public health departments about the level of interest in and/or intent to become vaccinated, which could allow departments to better allocate vaccination resources to communities that demonstrate high interest but have low vaccination rates. However, before incorporating an unevaluated data source into CDPH’s workflow, it is important to evaluate its utility.

The primary objective of this study was to evaluate the utility of the Google VSI dataset in predicting and forecasting COVID-19 vaccine doses administered in Chicago. The ability to make this type of forecast could allow CDPH and other public health departments to better allocate and prepare for fluctuations in COVID-19 vaccination demand at the ZIP code level.

Methods

Nature of study

This study is a retrospective, ecological time-series analysis evaluating the utility of Google COVID-19 Vaccination Search Insights (VSI) data as an exogenous signal for predicting and forecasting weekly COVID-19 vaccine doses administered in Chicago. Analyses used publicly available, anonymized, ZIP code level data and did not involve individual-level information. The study setting was Chicago, Illinois, with weekly data collected between January 4, 2021 and December 20, 2021.

Data sources

Google COVID-19 Vaccination Search Insights (VSI) dataset

The VSI dataset provides aggregated, anonymized Google relative search trend frequencies related to COVID-19 vaccinations and was made publicly available by Google. ¹⁴ Data were normalized by total search activity within each ZIP code, producing weekly time-series data for three search categories: 1) COVID-19 Vaccination (all searches related to COVID-19 vaccinations; overall search), 2) Vaccination Intent (searches related to eligibility, availability, and accessibility of vaccines; intent search), and 3) Safety and Side Effects (searches related to the safety and side effects of the vaccines; safety/side effects search). ¹⁵ The overall search category is a superset containing the other two VSI search categories, as well as other COVID-19 vaccine-related searches. For this study, relative search frequencies were extracted for Chicago ZIP codes between January 4, 2021 and December 20, 2021 (Appendix A).

Chicago Vaccination administration dataset

Daily COVID-19 vaccine doses administered to Chicago residents were obtained from the Chicago Data Portal (vaccination administration dataset). ¹⁶ Data were available at the ZIP code level and limited to the same 51-week period as the VSI data. Vaccination counts were aggregated weekly and divided by ZIP code population estimates from the US Census Bureau American Community Survey 2015-2019, the most recent five-year estimates available at the time of analysis, yielding a weekly proportion of doses administered per resident. ¹⁷ This normalization accounted for differences in population size and enabled comparisons of vaccination rates across ZIP codes.

Data collation

Given that the vaccination administration data were updated daily while the VSI data were updated weekly, the vaccination administration data were aggregated into weekly sums and matched to the VSI data by ZIP code and week. Each ZIP code (n=51) was classified using the Chicago CCVI, which categorizes communities as “High,” “Medium,” and “Low” based on cumulative COVID-19 burden and structural barriers to vaccine uptake, with “High” indicating greater vulnerability. ¹² For this study, given CDPH’s mission of ensuring equitable vaccination uptake and its prioritization of High CCVI communities, each Chicago ZIP code was categorized as either “High CCVI” (n=13), or “Low/Medium CCVI” (n=38) (Appendix A).

As this was an ecological study using secondary, aggregated data, the unit of analysis was the ZIP code. Chicago ZIP codes were eligible for inclusion if both Google VSI and vaccination administration data were available for the study period. ZIP codes with missing data from either source were excluded; no imputation was performed. No formal sample size or power calculation was conducted, because the analytic sample size was determined by data availability.

Statistical analysis

The primary outcome was the weekly number of COVID-19 vaccine doses administered per ZIP code, normalized by population size. The primary exposures were weekly VSI relative search frequencies, including overall vaccination, vaccination intent search, and safety and side effects search categories. ZIP codes were stratified by CCVI category (High vs Low/Medium CCVI). Welch’s t-tests were conducted to compare VSI search frequencies and population-normalized vaccination doses between CCVI strata.

Predictive modeling rationale

A time-series modeling framework was selected for this study because the primary objective was to predict and forecast weekly COVID-19 vaccination doses over time rather than estimate cross-sectional associations. Vaccination data exhibit temporal dependence, trends, and seasonal patterns driven by factors, such as eligibility changes and public health campaigns, making time-series approaches appropriate. Seasonal AutoRegressive Integrated Moving Average (SARIMA) models were chosen as a baseline, because they explicitly account for autocorrelation and seasonality in longitudinal public health data. SARIMAX models were then utilized to evaluate whether incorporating lagged Google VSI data improved predictive accuracy and forecast performance beyond historical vaccination trends. While alternative approaches, such as Prophet or Long Short-Term Memory (LSTM) networks, may offer flexibility for larger or more complex datasets, SARIMA(X) models were chosen because of the relatively short 51-week time-series, their interpretability, and their capacity to explicitly evaluate the lagged effects of exogenous variables on vaccination outcomes.

Predictive modeling approach

SARIMA models account for seasonality in time-series data and are defined by autoregressive, differencing, and moving-average parameters for non-seasonal and seasonal components. To incorporate external predictors, SARIMAX models extend SARIMA by including exogenous variables. This framework allows evaluation of the lagged effects of external signals on vaccination predictions.^18–20

All analyses were performed on three datasets: 1) the full dataset (51 ZIP codes), 2) High CCVI dataset (13 ZIP codes), and 3) Low/Medium CCVI dataset (38 ZIP codes). For each dataset, a 5x2 cross-validation (CV) procedure was used. In each iteration, ZIP codes were randomly partitioned into training and test sets while preserving the full temporal structure of each ZIP code time series. This ZIP code level partitioning was chosen to evaluate generalization across Chicago ZIP codes while preserving each ZIP code’s temporal structure. Future-week performance was assessed separately through out-of-sample forecasts generated from the fitted models. A time-based cross-validation scheme would instead evaluate temporal extrapolation directly within the resampling procedure. Training data were aggregated weekly to create a single time-series by summing vaccination doses and population counts and normalizing doses to population size, whereas test ZIP codes were retained as individual evaluation units. Model performance was assessed using root mean squared error (RMSE), with lower values indicating better predictive accuracy.

For each dataset, a hyperparameter grid search identified the SARIMA model with the lowest Akaike Information Criterion (AIC), and the resulting hyperparameters were used for all SARIMAX modeling. Prior to model fitting, autocorrelation function (ACF) and partial autocorrelation function (PACF) plots were examined to confirm the presence of temporal autocorrelation and inform the autoregressive and moving average orders of the SARIMA models. A baseline univariate SARIMA model was trained using population-normalized COVID-19 vaccine doses. SARIMAX models were trained with the addition of the three VSI categories to determine whether VSI data provide a meaningful signal for predicting and forecasting vaccine doses and, if so, which combination of VSI categories results in the greatest improvement in model accuracy. Specifically, four SARIMAX model experiments were performed with the following exogenous variables: 1) overall search, 2) intent search, 3) safety/side effects search, and 4) all three search categories. To evaluate the effect of the exogenous variable(s) having a lagged influence on predictions, each SARIMAX model was trained and tested with the exogenous variable(s) lagged by integer increments of one to five weeks to precede the vaccine doses, with the lag minimizing RMSE selected for each model. To ensure reproducibility and a fair comparison between the four SARIMAX model experiments, the same random seed was used for the 5x2 CV splitting procedure.

After training each SARIMA and SARIMAX model, predictions and five-week out-of-sample forecasts (weeks inclusive of December 27, 2021 through January 24, 2022) for total weekly vaccine doses were made for each ZIP code in the test set. The RMSE for the entire test set was calculated by dividing the total squared error by the total number of predictions and taking the square root. For the purposes of this study, and to be conservative in model evaluation, the SARIMA and SARIMAX models reported for prediction and forecasting analyses were those associated with the cross-validation iteration exhibiting the median RMSE percent difference between the SARIMA and SARIMAX models. To characterize performance variability, the distribution of RMSE and RMSE percent reduction across all 10 (5x2) cross-validation iterations was summarized for each ZIP code stratum as the across-iteration mean ± standard deviation (SD) and interquartile range (IQR). These distributions are reported alongside the median-iteration point estimates, which served as the primary reported values.

The measures used to evaluate the predictive and forecasting performances of these SARIMA and SARIMAX models were: 1) mean percent difference between the predicted and actual vaccine doses, 2) mean percent difference between the forecasted and actual vaccine doses, and 3) mean percent change in the 95% confidence interval (CI) ranges between the SARIMA and SARIMAX forecasted values (with narrower CIs indicating greater model confidence rather than greater forecast accuracy). RMSE, reported in absolute units, served as the primary performance metric because, unlike the mean percent difference, it is not inflated by the small denominators that arise when actual vaccination proportions approach zero. The mean percent difference is reported as a complementary and interpretable measure of relative error. All analyses were conducted using the statsmodels-v0.13.1 and scikit-learn-v1.0.2 packages for the Python programming language.^21,22

Bias

Several steps were taken to address potential sources of bias. To minimize ecological bias, all analyses were conducted and interpreted at the ZIP code level, and no individual-level inferences were made. Measurement bias was mitigated by using standardized, publicly available data sources with consistent definitions across all ZIP codes. To reduce temporal ambiguity and modeling bias, exogenous search frequency variables were lagged relative to vaccination outcomes, and model performance was evaluated using repeated cross-validation. Additionally, the same modeling framework and evaluation metrics were applied across all ZIP code strata to ensure comparability of results.

Reporting guideline

The reporting of this study conforms to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, supplemented by the REporting of studies Conducted using Observational Routinely-collected Data (RECORD) extension, which addresses reporting considerations specific to studies using routinely collected data.^23,24 The completed STROBE-RECORD checklist is provided as a supplementary file (Supplementary File 1).

Results

During the 51-week study period, limiting both VSI and vaccination datasets to Chicago ZIP codes yielded 2,652 weekly VSI records and 3,248 weekly vaccination records. Matching the datasets by week (N=51) and ZIP code (N=51) resulted in 2,601 weekly observations of VSI frequencies and vaccine dose counts included in the study (Figure 1). Of these observations, 75% (n=1,938) occurred in Low/Medium CCVI ZIP codes (n=38) and 25% (n=663) occurred in High CCVI ZIP codes (n=13). Welch’s t-tests showed that mean relative search frequencies across all three VSI categories and mean population-normalized weekly vaccination doses were significantly higher in the Low/Medium CCVI ZIP code category than in the High CCVI ZIP code category (p < 0.001; Table 1).

Figure 1.

Weekly time-series graph for COVID-19 vaccinations administered and the three Google VSI search frequencies in Chicago, IL for the weeks between January 4, 2021 and January 24, 2022.

^a Data sourced from the Google COVID-19 Vaccination Search Insights (VSI) dataset and the Chicago Data Portal vaccination administration dataset. The figure displays weekly data from January 4, 2021 through January 24, 2022 (N=51 Chicago ZIP codes).

Table 1.

Mean COVID-19 vaccination search frequencies and population-normalized weekly vaccination doses, stratified by Chicago Community Vulnerability Index (CCVI) category.

​	Low/Medium CCVI a (n = 1,938 weekly observations) mean (SD)	High CCVI a (n = 663 weekly observations) mean (SD)	p-value
Google VSI b Search Category
Overall Search	59.66 (38.80)	29.79 (14.95)	<0.001
Intent Search	18.58 (21.42)	8.20 (7.68)	<0.001
Safety/Side Effects Search	9.80 (8.29)	5.41 (3.69)	<0.001
Total Normalized Weekly Doses c	0.03 (0.02)	0.02 (0.01)	<0.001

^aCOVID-19 Community Vulnerability Index (CCVI) is a composite measure of community vulnerability. Chicago ZIP codes were stratified into High CCVI and Low/Medium CCVI categories based on overall CCVI scores, with higher scores indicating greater community vulnerability.

^bGoogle COVID-19 Vaccination Search Insights (VSI) relative search frequency, a normalized measure of COVID-19 vaccination-related Google search activity.

^cCount of weekly doses administered for each ZIP code was normalized to the population counts for each ZIP code, using the U.S. Census Bureau American Community Survey 2015-2019 population estimates.

SARIMAX modeling results

SARIMAX hyperparameters

A hyperparameter grid search for the lowest AIC identified the optimal SARIMA model specifications for each ZIP code stratification: 1) all Chicago ZIP Codes: SARIMA (0,1,0)(2, 1, 2, 7) [AIC: -198.32], 2) High CCVI ZIP Codes: SARIMA (0,1,1)(2, 1, 2, 7) [AIC: -108.08], and 3) Low/Medium CCVI ZIP Codes: SARIMA (0,1,0)(2, 1, 2, 7) [AIC: -189.09]. Unless otherwise specified, the RMSE and percent-reduction values reported below are point estimates from the cross-validation iteration with the median RMSE percent difference between the SARIMA and SARIMAX models.

RMSE values for SARIMAX models

Across all ZIP code stratifications and exogenous variables, SARIMAX models consistently produced lower RMSE values than baseline SARIMA models. Each SARIMAX model was fit with an integer lag of one to five weeks selected by grid search; the optimal lag was two weeks for eight models and one week for four models (Table 2). Given this, the reported 1.67-week value is the mean of the twelve optimal integer lags, not a fractional lag used in model fitting. Averaged across the four search category models within each stratum, RMSE reductions were 85.96% in High CCVI ZIP codes, 8.12% in Low/Medium CCVI ZIP codes, and 6.36% across all Chicago ZIP codes, yielding an unweighted mean reduction of 33.48%. As an unweighted average, this 33.48% figure is likely influenced by the large High CCVI reduction; the stratum-specific values may provide a more representative summary.

Table 2.

Root mean squared error (RMSE) values for SARIMA and SARIMAX models, stratified by Chicago CCVI category and Google VSI search category.

ZIP code category | Search category included in SARIMAX model	Baseline SARIMA model	SARIMAX model	Time lag (Week) a	% difference between SARIMA and SARIMAX RMSE values
All Chicago ZIP Codes
Overall Search	0.0108	0.0100	2	-6.87%
Intent Search	0.0108	0.0101	2	-5.91%
Safety/Side Effects Search	0.0108	0.0102	1	-5.32%
All Three Searches	0.0108	0.0098	2	-7.34%
Low/Medium CCVI ZIP Codes
Overall Search	0.0120	0.0109	2	-9.35%
Intent Search	0.0125	0.0117	2	-6.90%
Safety/Side Effects Search	0.0125	0.0117	1	-6.39%
All Three Searches	0.0120	0.0109	2	-9.83%
High CCVI ZIP Codes
Overall Search	0.0657	0.0093	1	-85.88%
Intent Search	0.0657	0.0092	1	-85.92%
Safety/Side Effects Search	0.0657	0.0089	2	-86.47%
All Three Searches	0.0479	0.0069	2	-85.58%

^aTime lag denotes the number of week(s) by which Google VSI search frequencies preceded the vaccination doses in the SARIMAX models.

To evaluate the consistency of the observed improvements, RMSE percent reductions were summarized across all 10 cross-validation iterations for the best-performing model in each ZIP code stratum. Mean RMSE percent reductions were 7.31% (SD 1.94%; IQR 6.1-8.7%) for all Chicago ZIP codes, 8.99% (SD 2.81%; IQR 8.0-10.9%) for Low/Medium CCVI ZIP codes, and 84.73% (SD 5.52%; IQR 79.7-88.9%) for High CCVI ZIP codes. These results indicate that the improvements were generally consistent across cross-validation iterations in each stratum.

The largest improvement was observed in High CCVI ZIP codes. All 10 cross-validation iterations favored SARIMAX over SARIMA, with RMSE reductions ranging from 76.8% to 91.7%. This stability reflects the SARIMAX RMSE remaining low and tightly distributed across iterations (mean 0.0077, SD 0.0009), whereas the univariate SARIMA baseline error varied widely (mean 0.0583, SD 0.0254). The wider SARIMA variability in this stratum may reflect the relatively small number of High CCVI ZIP codes (n=13) available for partitioning during cross-validation.

Model performance also varied by search category. In the all Chicago and Low/Medium CCVI strata, SARIMAX models incorporating all three VSI categories yielded the greatest RMSE reduction, whereas in High CCVI ZIP codes, models including the safety and side effects category performed best (Table 2).

Predictive performance

For all Chicago ZIP codes, SARIMAX models that included the intent search, safety/side effects search, or all three VSI categories resulted in predictions closer to the observed vaccination counts than the baseline SARIMA model, as indicated by smaller mean percent differences. In contrast, SARIMAX models including only the overall search category performed slightly worse than the SARIMA baseline.

For the Low/Medium CCVI ZIP codes, SARIMAX models that included intent or safety/side effects search categories outperformed the SARIMA baseline, whereas models including the overall search category or all three VSI categories showed poorer predictive performance. For the High CCVI ZIP codes, all SARIMAX model variants produced predictions closer to the observed vaccination counts than the baseline SARIMA model (Table 3).

Table 3.

Predictive and forecasting performance of SARIMA and SARIMAX models for weekly COVID-19 vaccination doses administered in Chicago, stratified by CCVI category and Google VSI search category.

ZIP code category | search category included in SARIMAX model	Predictive performance: Mean % difference between SARIMA(X) predicted and actual doses a		Forecasting performance: Mean % difference between SARIMA(X) forecasted and actual doses b		% change in 95% CIs between SARIMA and SARIMAX forecasts c
All Chicago ZIP Codes	SARIMA Model	SARIMAX Model	SARIMA Model	SARIMAX Model	​
Overall Search	26.74%	27.56%	31.85%	50.33%	-20.24%
Intent Search	26.74%	26.49%	31.85%	49.38%	-20.43%
Safety/Side Effects Search	26.74%	18.83%	31.85%	32.58%	-21.82%
All Three Searches	26.74%	26.12%	31.85%	50.75%	-26.36%
Low/Medium CCVI ZIP Codes
Overall Search	31.06%	31.32%	38.72%	55.57%	-18.68%
Intent Search	31.24%	28.84%	37.74%	55.24%	-19.28%
Safety/Side Effects Search	31.11%	22.57%	37.74%	40.31%	-22.78%
All Three Searches	30.93%	39.27%	38.72%	58.30%	-26.21%
High CCVI ZIP Codes
Overall Search	160.97%	11.84%	251.81%	20.58%	-85.52%
Intent Search	163.45%	21.21%	251.81%	20.99%	-85.02%
Safety/Side Effects Search	160.97%	31.56%	251.81%	15.16%	-85.99%
All Three Searches	158.77%	32.72%	243.10%	17.62%	-89.73%

^aMean percent difference between the model’s predicted values and actual doses administered for the 51 weeks used for predictions. The lower the percent difference, the closer the model’s predictions were to the actual doses administered.

^bMean percent difference between the model’s forecasted values and actual doses administered for the 5 weeks used for forecasting. The lower the percent difference, the closer the model’s forecasts were to the actual doses administered.

^cPercent change in 95% CI ranges between the SARIMA and SARIMAX forecasts allows for the evaluation of whether the 95% CI range grows or shrinks with the inclusion of the Google VSI categories.

Forecasting performance

For the five-week-ahead (out-of-sample) forecasts, incorporating VSI data did not improve forecasting accuracy in the all Chicago and Low/Medium CCVI ZIP code strata. In these strata, baseline SARIMA models produced vaccination dose estimates closer to the observed values than SARIMAX models. In contrast, for High CCVI ZIP codes, all four SARIMAX model variants outperformed the SARIMA baseline, with mean percent differences falling from 251.81% to 15.16-20.99% (Table 3).

Across all ZIP code stratifications and exogenous variable configurations, SARIMAX forecasts produced narrower 95% confidence intervals than SARIMA forecasts. The greatest reduction in confidence interval width occurred in High CCVI ZIP codes, with an average reduction of 86.56%. Average reductions for all Chicago and Low/Medium CCVI ZIP codes were 22.21% and 21.74%, respectively (Table 3). These narrower intervals reflect greater model certainty rather than improved forecast accuracy, particularly in the all Chicago and Low/Medium CCVI strata, where SARIMAX point forecasts were less accurate than the SARIMA baseline.

Discussion

Principal findings

The aim of this study was to evaluate the utility of Google’s COVID-19 VSI dataset for predicting and forecasting COVID-19 vaccine doses within CDPH. When stratified by Chicago CCVI categories, the relative search frequencies for all three VSI categories, as well as the population-normalized weekly vaccine doses, were significantly higher in the Low/Medium CCVI ZIP codes compared with the High CCVI ZIP codes. These findings suggest that residents of Low/Medium CCVI ZIP codes (i.e., communities that have fewer barriers to COVID-19 vaccines) are both engaging in COVID-19 vaccination-related searches and receiving COVID-19 vaccinations at higher per-capita rates than residents of High CCVI ZIP codes.

The results of the SARIMA and SARIMAX modeling indicate that including VSI categories as exogenous variables, lagged by an average of 1.67 weeks, produced lower RMSE values than the baseline SARIMA model. Predictive improvements were most pronounced in High CCVI ZIP codes, where all four SARIMAX variants outperformed the baseline model. However, improvements in out-of-sample forecasting were limited to High CCVI ZIP codes, while SARIMA models remained more accurate in the all Chicago and Low/Medium CCVI strata. Although SARIMAX forecasts produced narrower 95% confidence intervals across all strata, these narrower intervals reflect greater model confidence rather than greater forecast accuracy.

Taken together, these findings suggest that Google VSI data may serve as a useful supplementary signal alongside existing vaccination surveillance efforts. The observed average lag of 1.67 weeks between population-level VSI search activity and vaccine dose administration suggests that changes in vaccination-related information-seeking behavior may provide advance indication of shifts in vaccination demand. As an ecological association, this cannot establish that information-seeking preceded vaccination at the individual level, but the population-level lag is consistent with prior evidence that information-seeking behavior precedes vaccination decisions.^10,11 Given that SARIMAX models improved out-of-sample forecasting only in High CCVI ZIP codes, the operational utility of VSI-augmented forecasts may be greatest in high-vulnerability communities rather than uniformly across the city.

Limitations

This study has several limitations. First, given its exploratory nature, we did not account for demographic characteristics (e.g., race, ethnicity, gender, sex, income, education) that likely contribute to vaccination uptake. However, the Chicago CCVI categories capture community-level vulnerability and barriers to vaccination that may partially reflect many of these characteristics. Differences in digital literacy, internet access, and cultural and linguistic factors may also have influenced observed search behavior and the representativeness of the VSI data, particularly in High CCVI ZIP codes, where limited internet access may underrepresent vaccination-related search activity relative to actual community interest or need.

In addition, ZIP code population normalization was conducted using the ACS 2015-2019 estimates, which were the most recent five-year estimates available at the time of analysis. These estimates predate the study period and may not reflect pandemic-era population shifts, particularly in highly mobile ZIP codes, though the extent and any differential impact across CCVI strata are unknown.

Second, the direct effects of vaccination-related communication and incentive efforts on the search frequencies and doses administered were not considered; although, these effects may be indirectly captured since campaigns have been shown to increase related searches.^25–27 More broadly, other behavioral or media events, such as policy announcements, news coverage of safety concerns, or eligibility changes, may have independently influenced search frequency in ways unrelated to vaccination intent, affecting the interpretation of VSI data as a demand signal.

Third, the analyses were limited to the 51-week period and do not capture the effects of subsequent approvals, such as booster doses or vaccinations of children under five years old. No formal sample size or power calculation was performed, because the study relied on secondary, population-level data with a fixed number of ZIP codes and time points, which may limit the precision of some subgroup comparisons. A further consideration is the ongoing availability of the Google VSI dataset. For this study, the dataset was accessed in February 2022, but according to the data provider, it is no longer updated as of February 5, 2024; all historical data remain publicly available. ¹⁴ Given this, the dataset is best regarded as a historical resource, and the operational applications discussed below (e.g., real-time monitoring dashboard) would require either renewed maintenance of the VSI dataset or an equivalent, currently maintained source of aggregated, privacy-preserving search data.

Finally, cross-validation was performed by randomly splitting ZIP codes rather than time points to preserve within-ZIP code temporal dependence; temporal-split strategies may yield different predictive performance estimates. Relatedly, the large RMSE reduction observed in High CCVI ZIP codes partly reflects the comparatively high and variable univariate SARIMA baseline in this stratum, which is attributable to its smaller number of ZIP codes and greater variability in per-capita vaccination patterns. Although the consistency of the SARIMAX performance across cross-validation iterations argues against an arbitrary covariate effect, a formal placebo analysis was not performed. Additionally, the mean percent difference is sensitive to small denominators. Given that the population-normalized vaccination proportions were often less than 0.05, relatively small absolute prediction errors could lead to large percentage differences. Therefore, RMSE, an absolute error metric expressed in the same units as the outcome, was used as the primary metric for model comparison.

Future studies should: 1) examine correlations between communication efforts and Google search frequencies and/or vaccine doses administered in Chicago, 2) evaluate whether these findings are found after subsequent COVID-19 vaccination approvals, 3) explore whether other variables, such as vaccine availability and convenience, can serve as additional predictive indicators, and 4) replicate these analyses in other cities, states, or countries.

Prior work

The use of Google Search Trends to monitor and predict disease outbreaks is not a novel concept. In a systematic review, Mavragani, Ochoa and Tsagarakis, ²⁸ identified over 100 peer-reviewed infodemiology studies that evaluated the use of Google Search Trends in health-related topics. However, in this systematic review, only 8.7% of the studies used Google Search Trends data for health-related predictions, which signifies a gap that currently exists in forecasting using Google Search Trends data. ²⁸ Specific to COVID-19, a few peer-reviewed studies have used Google Search Trends data, but most focused on correlating these data with COVID-19 cases.^29–32 Unlike prior studies that leveraged general Google Search Trends data to correlate search activity with COVID-19 case counts or hospitalization, this study applies the purpose-built Google COVID-19 VSI dataset within a SARIMAX time-series forecasting framework to predict and forecast COVID-19 vaccine doses administered. This approach represents a novel operational application of infodemiology tools within a local public health department context.

Public health implications

The results of this study indicate that there is potential utility to incorporating Google’s COVID-19 VSI dataset within a large city’s public health department to supplement existing vaccination distribution and administration efforts. Specifically, the lag pattern and forecasting results suggest that vaccination-related search behavior may offer CDPH an additional signal to inform resource allocation and planning for possible influxes in vaccination demand. In practice, provided that an up-to-date source of aggregated vaccination-related search data is available, CDPH or similar agencies could implement a real-time monitoring dashboard that integrates weekly VSI search frequencies with vaccination administration data, allowing public health staff to identify emerging shifts in community interest or demand approximately 1-2 weeks in advance. Such a system could serve as an early-warning tool to guide targeted outreach, clinic scheduling, and vaccine supply allocation, particularly in high-vulnerability communities.

Collaboration with organizations such as the Centers for Disease Control and Prevention (CDC), Google, or other public health partners could facilitate development of standardized open-source toolkits. Such toolkits could include pre-built data pipelines, model templates, and visualization tools tailored to varying levels of technical capacity, lowering the barrier for local and state health departments to incorporate search-based signals into their routine surveillance and planning activities. Although this analysis focused on Chicago, the analytic framework and use of aggregated, privacy-preserving search data may be applicable to other public health departments with comparable vaccination reporting infrastructure.

More broadly, the SARIMAX framework integrating publicly available search data with vaccination outcomes may extend to other vaccine-preventable diseases, but this would require empirical evaluation. Future work could examine whether this approach applies to seasonal influenza vaccination campaigns. However, an important caveat is that the framework assumes population-level information-seeking precedes health action, which is an assumption most plausible for novel or emergent health events, where unfamiliarity drives anticipatory searching. On the other hand, for routine, recurring vaccination contexts, such as annual influenza vaccination, the predictive value of search-based signals is uncertain, because familiarity with the vaccine may reduce anticipatory information-seeking. Beyond vaccination campaigns, the integration of real-time search data into time-series forecasting frameworks may offer value during novel infectious disease outbreaks or public health emergencies, where population-level information-seeking is most likely to precede health action. This approach could provide public health departments with a scalable and privacy-preserving tool for anticipating shifts in resource demand.

Conclusion

This study demonstrates that Google COVID-19 Vaccination Search Insights data can serve as a useful supplementary data source for predicting and forecasting COVID-19 vaccination demand at the local level. Incorporating lagged vaccination-related search frequencies into SARIMAX models improved in-sample predictive performance, particularly in communities with higher vulnerability to COVID-19 vaccination barriers. However, improvements in out-of-sample forecasting accuracy were concentrated in High CCVI ZIP codes; in the remaining strata, the univariate SARIMA models forecast as well as or better than the SARIMAX models. These findings suggest that population-level search behavior may serve as an early indicator of shifts in vaccination demand, particularly in high-vulnerability communities. Public health departments may leverage such data to enhance short-term planning, resource allocation, and preparedness for fluctuations in vaccination uptake. More broadly, this work highlights the utility of integrating non-traditional, privacy-preserving data sources into public health surveillance and decision-making workflows.

Appendix.

Appendix A.

Appendix A.

Chicago ZIP codes, by CCVI category

High CCVI ZIP codes (N = 13)	Low and medium CCVI ZIP codes (N = 38)
60609, 60620, 60621, 60623, 60624, 60628, 60629, 60632, 60636, 60639, 60644, 60651, 60707	60605, 60607, 60608, 60610, 60611, 60612, 60613, 60614, 60615, 60616, 60617, 60618, 60619, 60622, 60625, 60626, 60630, 60631, 60633, 60634, 60637, 60638, 60640, 60641, 60642, 60643, 60645, 60646, 60647, 60649, 60652, 60653, 60655, 60656, 60657, 60659, 60660, 60827

ORCID iD

Nicholas D. Soulakis https://orcid.org/0000-0002-7666-8977

Ethical considerations

This study was reviewed by the Loyola University Chicago Institutional Review Board and determined to be exempt (approval #218678; September 4, 2024).

Data Availability Statement

The Google COVID-19 Vaccination Search Insights (VSI) dataset used in this study is publicly available at https://google-research.github.io/vaccination-search-insights/ (accessed February 9, 2022). The Chicago COVID-19 vaccination administration dataset is publicly available through the Chicago Data Portal at https://data.cityofchicago.org/Health-Human-Services/COVID-19-Vaccinations-by-ZIP-Code-Historical/553k-3xzc/about_data (accessed February 9, 2022). Use of the Google COVID-19 VSI dataset is subject to the Google Terms of Service (https://policies.google.com/terms). The Chicago Data Portal vaccination administration dataset is made publicly available subject to the City of Chicago’s Data Terms of Use (https://www.chicago.gov/city/en/narr/foia/data_disclaimer.html). Both datasets were used in accordance with their respective terms of use.