Analysing COVID-19 outcomes in the context of the 2019 Global Health Security (GHS) Index

Abstract

Introduction

The Global Health Security Index benchmarks countries’ capacities to carry out the functions necessary to prevent, detect and respond to biological threats. The COVID-19 pandemic served as an opportunity to evaluate whether the Index contained the correct array of variables that influence countries’ abilities to respond to these threats; assess additional variables that may influence preparedness; and examine how the impact of preparedness components change during public health crises.

Methods

Linear regression models were examined to determine the relationship between excess mortality per capita for the first 500 days of countries’ COVID-19 pandemic and internal Index variables, as well as external variables including social cohesion; island status; perceived corruption; elderly population size; previous epidemic experience; stringency of non-pharmaceutical interventions; and social and political polarisation.

Results

COVID-19 outcomes were significantly associated with sociodemographic, political and governance variables external to the 2019 Index: social cohesion, reduction in social polarisation and reduced perceptions of corruption were consistently correlated with reduced excess mortality throughout the pandemic. The association of other variables assessed by the Index, like epidemiological workforce robustness, changed over time. Fixed country features, including geographic connectedness, larger elderly population and lack of prior coronavirus outbreak experience were detrimental to COVID-19 outcomes. Finally, there was evidence that countries that lacked certain capacities were able to develop these over the course of the pandemic.

Conclusions

Additional sociodemographic, political and governance variables should be included in future indices to improve their ability to characterise preparedness. Fixed characteristics, while not directly addressable, are useful for establishing countries’ inherent risk profile and can motivate those at greater risk to invest in preparedness. Particular components of preparedness vary in their impact on outcomes over the course of the pandemic, which may inform resource direction during ongoing crises. Future research should seek to further characterise time-dependent impacts as additional COVID-19 outcome data become available.

Key questions.

What is already known?

• The Global Health Security (GHS) Index was published in 2019 to evaluate countries’ capacities to prevent, detect and respond to infectious disease threats and concluded that the world was unprepared.

• Early analyses found that preparedness assessment tools such as the GHS Index and WHO’s Joint External Evaluation do not correlate with crude COVID-19 outcome measures, leading some to question the value of such tools.

• The ability to evaluate countries’ response to the ongoing COVID-19 pandemic is still limited but is rapidly evolving as higher quantities and quality of outcome information become available.

What are the new findings?

• Sociopolitical and governance variables characterising social polarisation, social cohesion and perception of corruption significantly influenced excess mortality throughout the COVID-19 pandemic.

• The impact of different preparedness components, like the robustness of the epidemiological workforce, vary throughout ongoing public health emergencies.

• Fixed characteristics of countries, like increased geographic connectedness, an ageing population or lack of experience responding to biological events, significantly increase their risk profile and can be used to identify particularly vulnerable countries.

• Countries were able to develop preparedness capacity on the fly during the COVID-19 pandemic, despite an initial lack of capacity.

Key questions.

What do the new findings imply?

• Sociodemographic, political and governance variables external to the inaugural GHS Index may aid future iterations in characterising countries’ ability to prepare for, detect and respond to significant biological events.

• Strategies for responding to ongoing public health emergencies should consider the relative impact of investing in different preparedness areas.

• Fixed characteristics, while not directly addressable, highlight particularly vulnerable countries that have a greater need for preparedness investments.

• Countries may be able to leverage their experience against COVID-19 to improve aspects of their preparedness capacity for future events.

Introduction

The Global Health Security (GHS) Index was the first comprehensive health security capacity assessment of the 195 countries subscribed to the 2005 International Health Regulations. Its primary objective is to provide an independent, broad, universal metric to evaluate countries’ ability or potential to carry out the functions necessary to prevent, detect and respond to acute infectious disease threats, be they naturally occurring, accidental or deliberate.1 In doing so, it serves as a useful tool for measuring and highlighting gaps in health security at national and regional levels to assist decision makers in directing attention and resources.

Countries are scored by Economist Impact, under a framework developed in consultation with Johns Hopkins Center for Health Security and the Nuclear Threat Initiative and vetted by an international advisory panel of experts. The framework evaluates countries using publicly accessible information pertaining to six categories: prevention; detection and reporting; rapid response; health systems; compliance with international norms; and risk environment. These categories are further broken down into 34 indicators and 85 subindicators that are evaluated using 140 questions. These are intended to build on existing external evaluations to more broadly encompass the technical and societal measures that impact preparedness. The inaugural iteration of the GHS Index was published in October 2019 and ultimately concluded that no country was sufficiently prepared for epidemic or pandemic events.2

Early analyses found preparedness assessment tools, like the GHS Index and the WHO’s Joint External Evaluation (JEE), did not correlate with crude COVID-19 outcome measures, such as total cases or deaths, despite evidence of moderate external validity of GHS scores with reduced deaths from communicable disease.3–5 This raised important questions about the role of health system capacities in preventing adverse outcomes and the value of existing preparedness assessments.6–10 However, these questions often misinterpret what benchmarking tools are meant to measure and convey and oversimplify the complexity of predicting performance of events like pandemics. Unlike models built to forecast outcomes, benchmarking tools serve as inventories of capacities and capabilities that countries can access during an emergency. These tools help inform countries’ preparedness efforts, advocate for greater investment in building capacity and highlight gaps for targeted investment. Neither the JEE or the GHS Index can predict whether decision makers will choose to use the capacities and capabilities available to them during an ongoing crisis. Furthermore, the GHS Index is a snapshot of preparedness at the close of July 2019, not a dynamic tracker. As such, it cannot capture whether countries bolstered or dismantled their capacities throughout the pandemic. Media reports suggest that some countries, like New Zealand, designed a COVID-19 strategy that avoided potential exacerbation of the vulnerabilities in their capacity identified by the 2019 GHS Index.11 12

An analogy for the appropriate use of benchmarking tools is to consider how one would measure jurisdictions’ readiness for fires. A benchmarking tool would likely measure the existence of safety capacities, such as efforts to test fire alarm functionality or exercise building evacuation plans. It may also include measurement of a jurisdiction’s baseline risk for fires, such as the existence of environmental hazards that make fires more likely to occur and spread. However, the sum of these is still not entirely reflective of how a jurisdiction chooses to respond to a fire. That response is also driven by other unmeasurable variables like whether building occupants heed the warning fire alarms provide or whether inhabitants were encouraged to remain in place rather than evacuate a burning building. These variables can be difficult to measure in advance of an event and while they impact outcomes do not negate the inherent value of protective capacities like fire alarms.

The COVID-19 pandemic serves as an opportunity to evaluate whether the Index contained the correct array of factors that enable or hinder countries abilities to respond to an infectious disease emergency. In analysing data on the relationship between COVID-19 outcomes and GHS Index scores, our objectives were threefold: (1) to understand if and how features of the GHS Index related to countries’ COVID-19 outcome measures; (2) to identify additional variables that could inform future GHS Index iterations and (3) to understand how these relationships changed over the course of the pandemic. Specifically, we evaluated the correlation between a mixture of GHS Index indicators and relevant external variables and the excess mortality per 100 000 inhabitants in each country 100, 300, 400 and 500 days after their first confirmed case of COVID-19.

Methods

Study population

Participating countries for the analyses assessing excess mortality at different stages of the COVID-19 pandemic included the 195 countries evaluated in the 2019 GHS Index, which were assessed for inclusion based on data availability for the explanatory variables outlined below. One hundred and twenty-eight countries were included in the analysis conducted at 100 and 300 days. Twenty-eight of those countries (21.9%) were categorised as least prepared, with an overall GHS Index score of <33.3. Eighty-nine countries (69.5%) were categorised as more prepared (33.4–66.6) and 11 countries (8.60%) were categorised as most prepared (>66.7). Of the 112 countries evaluated at 400 days, 18 were categorised as least prepared (16.1%), 85 as more prepared (75.9%) and 9 as most prepared (8.04%). Of the 117 countries included in the analysis evaluating excess mortality at 500 days, 21 (18.0%) were categorised as least prepared, 85 (72.6%) as more prepared and 11 countries (9.40%) as most prepared.

Outcome variable selection

The outcome measured was countries’ total excess mortality per 100 000 individuals in the population 100, 300, 400, 500 days after the country’s first detected case of COVID-19. We identified the date of the first detected COVID-19 case using data from the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University and aligned countries by date of COVID-19 introduction, to account for variation in epidemic timelines between countries (see online supplemental table 1).13

Excess mortality is defined as deaths from all causes during a period, after accounting for expected deaths.14 It estimates how many more people died during the COVID-19 pandemic than would be expected under usual conditions. Excess mortality is a more comprehensive assessment of the pandemic toll than confirmed COVID-19 deaths, as it also captures incorrectly diagnosed or reported deaths and indirect mortality resulting from overburdened healthcare systems or exacerbated poverty. Greater values of excess mortality per 100 000 individuals indicates poor pandemic response.

Alternative outcomes, like hospitalisations or test positivity rate, were considered but ultimately discarded due to a lack of globally comparable data. While many countries in the dataset have publicly available COVID-19 case and deaths data, the ability to measure these is a feature of a country’s capacity itself. This leads to widespread under-reporting of these metrics, in conjunction with significant variation in how they are reported. As such, numbers of reported cases or deaths were deemed unsuitable outcome measures.

Data collection

We used country-level data on preparedness against biological threats from the 2019 GHS Index.2 Excess mortality data were obtained from The Economist’s data repository and incorporated as available, up to and including 15 September 2021.15 Data pertaining to case numbers, detection, stringency of government interventions, vaccinations and testing were obtained from Our World In Data (a collaborative project between the University of Oxford and non-profit Global Change Data Lab)16 and CSSEGIS (a COVID-19 data repository maintained by CSSE at Johns Hopkins University)17 for the dates of interest. Data characterising political and social polarisation were obtained from the Varieties of Democracy Equal Distribution of Resources Index.18 Data pertaining to governance,19 income status20 and population age structure21 were obtained from the World Bank. Data on corruption perception were obtained from the 2020 Corruptions Perceptions Index.22 Data assessing social cohesion were obtained from the 2019 Democracy Index.23 Finally, island grouping and previous epidemic experience with severe acute respiratory syndrome (SARS) or Middle East respiratory syndrome (MERS) datasets were generated by Economist Impact and are available in the online supplementals table 2 and 3.

Statistical analysis

Multiple linear regression was performed to evaluate the association between GHS Index indicator and external variables and excess mortality per 100 000 individuals at 100, 300, 400 and 500 days following the first reported COVID-19 case. GHS Index indicator variables were selected using backward stepwise regression by elimination on all GHS Index subindicators (at the X.X level), removing the two subindicators with the highest p values until approximately two-thirds of the coefficients remained significant. The final model selection encompasses the 12 GHS Index indicator variables described in table 1. These analyses would need to be reperformed when evaluating other health security events to determine which GHS Index indicators are most salient within the context of those events.

Table 1.

GHS Index indicator variables selected for model inclusion based on their relationship with excess mortality per capita over the course of the COVID-19 pandemic

Category	Indicator
(1) Prevention	(1.6) Immunisation
(2) Detection and reporting	(2.3) Epidemiology workforce
	(2.4) OneHealth data integration
(3) Rapid response	(3.2) Exercising response plans
	(3.7) Trade and travel restrictions
(4) Health system	(4.3) Healthcare access
(5) Compliance with international norms	(5.1) IHR reporting compliance and disaster risk reduction
	(5.3) International commitments
	(5.4) WHO’s JEE and PVS
	(5.6) Commitment to sharing of genetic and biological data and specimens
(6) Risk environment	(6.1) Political and security risk
	(6.4) Environmental risks

These analyses would need to be reperformed for other health security events to determine which GHS Index indicators are most salient within the context of those events.

GHS, Global Health Security; IHR, International Health Regulations; JEE, Joint External Evaluation; PVS, Performance of Veterinary Services.

To control for potential confounders, project team members with subject matter expertise selected variables external to the Index based on a review of recent literature and consultation with an expert panel. These external variables described: social cohesion (as measured by the Democracy Index); island nation status; social perception of corruption (measured by the 2020 Corruption Perception Index); the share of the population over age 65 years; previous coronavirus epidemic experience (defined as having reported at least one SARS or MERS case prior to 2019); the stringency of non-pharmaceutical government interventions 15 days prior to the time point assessed; and social and political polarisation (as measured by the Equal Distribution of Resources Index). The number of fully vaccinated individuals was included in evaluations of excess mortality at 400 and 500 days.

Multicollinearity was assessed by evaluating the variance inflation factor of each variable. We also accessed the heteroscedasticity of the model using the Cook-Weisberg test. A lower number of excess deaths per 100 000 individuals reflects a better COVID-19 pandemic response. Negative coefficients indicate a variable is negatively correlated with excess mortality: that is, that the presence of (binary) or increase in (continuous) that variable is correlated with a decrease in excess mortality, or better performance.

All statistical analyses were performed using STATA V.16.24

Results

Correlations between GHS Index indicators, external variables and excess mortality at 100, 300, 400 and 500 days following the first confirmed case of COVID-19 within a country are displayed in table 2. The explanatory variables with the greatest effects on excess mortality remained consistent throughout the COVID-19 pandemic. External variables measuring societal perception of corruption, island status and social cohesion had the strongest protective effect on excess mortality at 100, 300, 400 and 500 days. One hundred days after a confirmed case of COVID-19, a one-point increase in a country’s Corruption Perception Index (CPI) score (where 0=highly corrupt and 100=very clean), correlated to 1.9 fewer deaths per 100 000 individuals (p<0.001). By 500 days, this increased to 3.74 fewer deaths per 100 000 individuals (p=0.002).

Table 2.

Linear regression output for excess mortality per 100 000 individuals evaluated at 100 days, 300 days, 400 days and 500 days following the first confirmed case of COVID-19 within a country

	Excess mortality per 100 000 individuals at 100 days				Excess mortality per 100 000 individuals at 300 days				Excess mortality per 100 000 individuals at 400 days after adjusting for vaccination				Excess mortality per 100 000 individuals at 500 days after adjusting for vaccination
Explanatory variable	Coefficient	Robust SE	P value	Std. β	Coefficient	Robust SE	P value	Std. β	Coefficient	Robust SE	P value	Std. β	Coefficient	Robust SE	P value	Std. β
Sociodemographic
Island status	−50.967	−11.722	0.000	−0.281	−49.172	11.621	0.000	−0.271	−76.398	22.161	0.001	−0.277	−120.641	24.526	0.000	−0.341
Share of pop. >65 years	335.504	−120.189	0.006	0.342	339.927	121.548	0.006	0.347	447.300	209.383	0.035	0.306	518.959	278.995	0.066	0.268
Governance and politics
Corruption Perceptions Index	−1.909	−0.517	0.000	−0.536	−1.795	0.505	0.001	−0.504	−2.946	0.980	0.003	−0.547	−3.735	1.152	0.002	−0.528
Social cohesion	−0.709	−0.161	0.000	−0.349	−0.728	0.152	0.000	−0.358	−0.769	0.299	0.012	−0.248	−1.479	0.442	0.001	−0.370
(6.1) Political and security risk	1.088	−0.314	0.001	0.357	0.965	0.313	0.003	0.316	1.353	0.636	0.036	0.275	2.078	0.765	0.008	0.322
Political polarisation*	−6.928	−4.495	0.126	−0.142	−7.141	4.232	0.094	−0.146	−8.624	7.378	0.246	−0.118	−10.594	8.271	0.203	−0.108
Public health capacity
(2.4) Data integration between human, animal and environmental health sectors	0.175	−0.108	0.109	0.125	0.169	0.107	0.115	0.122	0.124	0.166	0.458	0.060	0.406	0.230	0.081	0.148
(1.6) Immunisation	0.383	−0.397	0.337	0.066	0.261	0.401	0.517	0.045	0.535	0.841	0.526	0.054	1.792	1.010	0.079	0.160
(4.3) Healthcare access	−0.281	−0.335	0.404	−0.058	−0.261	0.339	0.443	−0.054	0.015	0.521	0.976	0.002	−0.571	0.676	0.401	−0.055
Response
(3.7) Trade and travel restrictions	0.77	−0.272	0.006	0.154	0.725	0.259	0.006	0.145	1.095	0.431	0.013	0.139	0.912	0.603	0.134	0.091
Previous experience with SARS or MERS	−24.241	−10.387	0.021	−0.167	−24.901	10.542	0.020	−0.172	−29.250	20.168	0.150	−0.134	−14.213	24.254	0.559	−0.050
(3.2) Exercising response plans	0.32	−0.143	0.028	0.144	0.309	0.141	0.030	0.140	0.364	0.244	0.140	0.111	0.295	0.306	0.337	0.068
Stringency of interventions at day X	0.219	−0.231	0.346	0.060	0.523	0.269	0.055	0.126	0.055	0.557	0.922	0.009	0.572	0.567	0.315	0.074
Other
(6.4) Environmental risks	−1.529	−0.551	0.007	−0.227	−1.474	0.543	0.008	−0.218	−2.147	1.025	0.039	−0.209	−4.958	1.108	0.000	−0.374
Polarisation of society*	−10.66	−4.542	0.021	−0.217	−9.840	4.627	0.036	−0.200	−14.360	8.214	0.084	−0.192	−11.491	11.421	0.317	−0.117
(5.3) International commitments	0.345	−0.187	0.067	0.167	0.296	0.187	0.115	0.143	0.436	0.338	0.201	0.141	0.823	0.453	0.072	0.199
(5.4) JEE and PVS	−0.345	−0.237	0.148	−0.106	−0.296	0.235	0.211	−0.091	−0.399	0.424	0.349	−0.082	−0.170	0.536	0.752	−0.028
(5.6) Commitment to sharing of genetic and biological data and specimens	−0.65	−0.611	0.290	−0.088	0.557	0.584	0.343	−0.075	−0.372	0.922	0.687	−0.036	−0.762	1.086	0.484	−0.054
(5.1) IHR reporting compliance and disaster risk reduction	0.193	−0.196	0.326	−0.142	0.190	0.194	0.328	0.066	0.316	0.348	0.367	0.073	−0.064	0.468	0.892	−0.011
(2.3) Epidemiology workforce	−0.14	−0.181	0.439	−0.060	−0.115	0.184	0.533	−0.049	−0.396	0.318	0.216	−0.107	−0.561	0.380	0.143	−0.117
Fully vaccinated people per 100 individuals	–	–	–		–	–	–		−0.378	0.973	0.699	−0.023	0.143	0.799	0.858	0.019
Constant	102.88	65.894	0.121		98.572	65.871	0.137		147.662	109.237	0.180		271.6561	142.2674	0.059
Mean variance inflation factor	1.850				1.860					1.800				1.900
Observations	128				128					112				117
R-squared	0.588				0.597					0.534				0.566
Indicates statistical significance to p<0.05 level.
Indicates statistical significance to p<0.1 level.

The standardised beta compares the strength of the effect of each individual independent variable to the dependent variable. A negative beta value (represented in green) indicates fewer excess deaths and therefore a better response. A positive beta value (represented in orange) suggests that the variable had a detrimental effect on the quality of the response, correlating with a greater number of deaths. Yellow shading represents a result that is statistically significant to the p<0.05 level. Blue shading represents a result that is statistically significant to the p<0.10 level.

*Variables measuring political and societal polarisation were scored in a manner where a lower score indicates significant polarisation, and an increase in score represents a decrease in polarisation.

JEE, Joint External Evaluation; MERS, Middle East respiratory syndrome; PVS, Performance of Veterinary Services; SARS, Severe acute respiratory syndrome.

A one-point increase in countries’ social cohesion—per the Democracy Index (scored in intervals from 0 to 1, where 0=insufficient cohesion to maintain a functioning democracy)—correlated to a decrease in excess mortality of 0.71 deaths per 100 000 individuals (p<0.001) at 100 days. At 500 days, a one-point increase correlated to a decrease of 1.48 excess deaths per 100 000 individuals (p=0.001).

When evaluated at 100 days, island countries saw a reduction of 51.0 excess deaths per 100 000 individuals compared with non-islands (p<0.0001). By 500 days, the reduction associated with island status was 120.64 fewer excess deaths per 100 000 individuals (p<0.0001).

Reduced societal polarisation and environmental risk were also consistently associated with fewer excess deaths. Societal polarisation was defined as the extent to which society differs in opinion on major political issues leading to major clashes of views. At 100 days, a one-point increase in a countries’ societal polarisation score (where a score of 0=serious polarisation and an increasing score indicates limited polarisation) was associated with 10.66 fewer excess deaths per 100 000 individuals (p=0.021, β=−0.217). This relationship persisted across time: at 500 days the same increase corresponded to a reduction of 11.49 excess deaths per 100 000 individuals (p=0.317, β=−0.117). With respect to environmental risk, a one-point increase on GHS Index subindicator 6.4 corresponded to 1.529 fewer excess deaths per 100 000 individuals (p=0.007, β=−0.227) at 100 days. By 500 days, this increase correlated with a reduction of 4.96 deaths per 100 000 individuals (p<0.0001, β=−0.374).

The GHS Index indicator assessing political and security risk (6.1) and the proportion of population greater than 65 years old had the strongest detrimental effect on excess deaths across all time points. At 100 days, a one-point increase in political and security risk score (where a higher score reflects a more favourable, less risky environment) was correlated with an increase in excess mortality of 1.1 deaths per 100 000 individuals (p=0.001). At 500 days, this had increased to an additional 2.08 deaths per 100 000 individuals with each one-point increase (p=0.008). For each percentage point increase in the proportion of the population over age 65 years, excess mortality at 100 days increased by 3.35 deaths per 100 000 individuals (p=0.006). By 500 days, this percentage point increase correlated to 5.19 excess deaths per 100 000 individuals (p=0.066).

Although variables with the greatest effect on excess mortality remained consistent, the effect of other explanatory variables varied over time. Up until 300 days, previous experience dealing with SARS or MERS was correlated with a significant decrease in excess mortality (p<0.02, β=−0.172). By 500 days, the strength of this effect had decreased, though it remained protective (p=0.559, β=−0.005).

While the robustness of countries’ epidemiological workforce had little effect on excess mortality at 100 and 300 days, this evolved into a negative relationship by 400 days. At day 400, a one-point increase in a country’s epidemiological workforce indicator was correlated with 0.4 fewer deaths per 100 000 individuals (p=0.216, β=−0.107). A similar relationship was observed at 500 days. The completion of preparedness assessments was initially shown to have a strong protective effect on excess mortality at 100 days, though there appeared to be little effect beyond this time point.

Several GHS variables were positively correlated with excess mortality over the course of the pandemic. For the indicator assessing international health security commitments, a one-point score increase correlated to 0.345 more deaths per 100 000 individuals at 100 days (p=0.067, β=0.167). By 500 days, the same increase correlated with an increase of 0.823 deaths per 100 000 individuals (p=0.072, β=0.199). Exercising response plans was also consistently associated with increased excess mortality: a one-point increase correlated to 0.320 additional deaths per 100 000 individuals (p=0.028, β=0.144) at 100 days and 0.364 deaths per 100 000 individuals (p=0.140, β=0.111) at 400 days. Finally, a one-point increase in the indicator assessing trade and travel restrictions was associated with 0.770 more deaths per 100 000 individuals on day 100 (p=0.006, β=0.154) and 0.912 more deaths per 100 000 individuals by day 500 (p=0.134, β=0.091).

For the regression examining excess mortality at 100 days, the Cook-Weisberg test for heteroscedasticity resulted in a χ² of 4.26, corresponding to Prob>χ²=0.039, indicating heteroscedasticity may be present. The regression examining excess mortality at 300 and 400 days had a χ² of 4.14, corresponding to Prob>χ²=0.0419 and a 2 of 4.95, corresponding to Prob>χ²=0.026, respectively. When conducted for the regression examining excess mortality at 500 days, the Cook-Weisberg test resulted in a χ² of 3.37, corresponding to Prob>χ²=0.067, indicating no heteroscedasticity was present. To account for potential heteroscedasticity across all analyses, robust variance estimates were used.

Finally, correlation analyses were performed to quantify the relationship between countries’ surveillance and reporting capacities and other components of capacity measured by the Index. Increasing urbanisation was found to be correlated with greater surveillance and reporting capacity (figure 1). Due to normalised scoring of the urbanisation subindicator (where 100=a lack of urbanisation), this relationship is presented as a negative correlation (Pearson coefficient=−0.370). The subindicators assessing international commitments (Pearson coefficient=0.669) and political and security risk (Pearson coefficient=0.299) had strong and moderate positive correlations with surveillance and reporting capacity, respectively (figure 2).

Figure 1.

Correlation analysis of real time surveillance and reporting (2.2) and urbanisation (6.4.1.a) indicators. Indicators are normalised on a scale of 0–100, where 100=most favourable for preparedness.

Figure 2.

(A) Correlation analyses of GHS index real time surveillance and reporting (2.2) and international commitment (5.3) indicators; (B) correlation analyses of GHS index real time surveillance and reporting (2.2) and political and security risk (6.1) indicators. GHS, Global Health Security.

Discussion

Our findings indicate a significant, consistent correlation between excess mortality and several variables contained in and external to the GHS Index, highlighting the importance of investing in their improvement and their potential to help characterise capacity in future Index iterations. They also reveal time-dependent impacts of other explanatory variables on excess mortality, which may aid future analyses of COVID-19 outcomes and inform responses to ongoing public health events.

Characterising COVID-19 outcomes using the GHS Index

Our results suggest that GHS Index indicators had a time-dependent impact on excess mortality. The indicator inventorying countries’ epidemiology workforce (2.3) initially had a weak correlation with excess mortality that translated to a stronger protective effect by 400 days. This could indicate the importance of a robust epidemiology workforce in sustaining successful strategies over extended periods and suggests continued investment in building this workforce would be valuable during ongoing emergencies. Other changes over time offer evidence that countries improved their ability to combat pandemic, despite an initial lack of capacity. The GHS Index indicator inventorying JEE and PVS completion was associated with decreased excess mortality 100 days into the pandemic, but this effect was less pronounced by 300 days and beyond. This suggests the relative advantage conferred by the completion of benchmarking tools decreases as countries learn to recognise and address their shortcomings on the fly.

The GHS Index indicator variable assessing environmental risk (6.4) was associated with reduced excess mortality throughout. Countries with fewer urban centres, limited deforestation or lower risk of disruption following natural disasters, experienced less excess mortality than countries with greater environmental risks. This may be because countries with fewer densely populated urban centres—which are responsible for 90% of reported cases25—experience less COVID-19 transmission. It could also be influenced by improved surveillance or reporting capacities in predominantly urbanised countries, which is supported by the correlation between these subindicators (figure 1). Future analyses should explore this relationship further.

We found that other GHS Index indicators were unexpectedly correlated with increased excess mortality, which we attribute to the inherent dependency of COVID-19 outcomes on countries’ capacity. Specifically, countries with demonstrated international health security commitments (5.3) or lower political and security risk environments (6.1) experienced greater excess mortality. We hypothesise this is because countries who make these commitments or have more effective governments are more likely to have robust surveillance systems and thus may be more likely to capture COVID-19 cases and related deaths, as well as vital statistics such as all-cause deaths. These vital statistics are required to produce accurate excess mortality estimates, which is supported by the average surveillance and reporting score of countries with excess mortality estimates available versus those without (57.9/100 compared with 22.5/100). Our hypothesis was supported by the correlations between these subindicators and real-time surveillance and reporting indicators (figure 2).

GHS Index indicators assessing trade and travel restrictions (3.7) and exercising of response plans (3.2) were also positively correlated with excess mortality. This may be attributed to the lack of score granularity within these subindicators. The subindicator assessing trade and travel restrictions—which penalises countries for enacting restrictions due to an infectious disease outbreak without international or bilateral support prior to 2019—shows little discrimination between countries, with only 10 countries receiving less than 100 (all of which scored 50/100). Subindicator 3.2 had a similarly limited spread, with countries receiving only a 0, 50 or 100, the majority of which received a zero (72.9%). Nonetheless, these findings reinforce the need for future analyses to account for the inherent reliance of outcome data on reporting and surveillance capacities.

The contribution of external variables to preparedness tools

We found that political and governance-related features of a country not captured by the 2019 Index, like social cohesion, lesser perceived corruption and less societal polarisation, were consistently negatively correlated with excess mortality. This suggests these population features have a sustained positive impact on a countries’ ability to prepare for, detect and respond to significant biological events and should therefore be considered for inclusion in future indices.

We hypothesised that social cohesion—the willingness of societies’ members to cooperate for survival and prosperity—may result in increased compliance with government guidelines and safer social behaviours, leading to fewer deaths. We believe a similar effect underlies the reduced excess mortality associated with quality of and trust in governance measured by the CPI. That is, populations who perceive their government as trustworthy are more likely to comply and respond well to their government’s recommendations and strategies. This may partially characterise the performance of countries like New Zealand, who are perceived as the least corrupt government in the world by their population. (As reflected in their #1 ranking in the 2020 CPI.) The New Zealand population were largely compliant with government guidelines and reported little public dissatisfaction towards their government’s COVID-19 approach.26–28 Hypotheses relating to greater social trust and improved COVID-19 outcomes have been explored and supported elsewhere in recent literature.29–31

In addition to assessing perception, we looked at objective measures of social and political polarisation. Our findings demonstrate consistent correlation between lower social polarisation and reduced excess mortality. Countries that are polarised politically or socially face issues in uniting behind common goals, which could result in a reduction in support of, or compliance with, government recommendations related to COVID-19.32 33 Though political polarisation—which reflects a reluctance to engage with opposing political camps—was not found to be significantly associated with excess mortality in this study, it can also beget social polarisation.34 This may have contributed to the performance of countries like the USA and UK: both populations have experienced increasing political and societal polarisation in recent years and faced significant public pushback against, and non-compliance with, their COVID-19 guidance and control measures.35–38

These findings suggest that improved social cohesion and reduced sociopolitical polarisation are key contributing factors to countries’ ability to leverage their preparedness measures and should be areas of concern when considering future biological threats. This, in addition to the sizeable ‘peacetime’ benefits conferred by these population features—which could also extend to reduce ‘war time’ risks outside of pandemics—suggest investment in strategies to foster societal cohesion and mitigate polarisation would be extremely valuable.

The time-dependent impact of other external variables offers further evidence that countries developed capacity over the course of the pandemic. Previous experience dealing with SARS or MERS was significantly correlated with reduced excess mortality at 100 and 300 days. We hypothesise this was due to the applicability of prior strategies or the opportunity for previous shortcomings to inform strategies. This is true of countries like South Korea, who reformed their capacity following a 2015 MERS outbreak and were one of few countries that experienced no excess mortality due to COVID-1939–41 However, this effect was less pronounced by 500 days, offering evidence that the relative advantage of this experience decreased as other countries adapted their capacities and responses on the fly.

Correlations between external variables and excess mortality during the COVID-19 pandemic highlight the potential utility of these variables within future Index iterations. However, caution should be applied when extrapolating findings from COVID-19 to pandemic preparedness against other biological events, like other pathogen types or threats deliberate in nature. Additional variables should remain consistent with the Index’s aim to benchmark countries’ capacities against all categories of potential biological threats.

Learning from COVID-19

The COVID-19 pandemic exposed many of the gaps in capacity also identified in the 2019 GHS Index, further emphasising its finding that no countries were adequately prepared for a major biological event. However, in putting capacities to the test, it also served as an opportunity for countries to identify and remedy their shortcomings. Our findings offer some evidence that countries improved components of their capacity as the pandemic progressed. However, sustaining these improvements and translating them into enduring health systems—thereby strengthening preparedness against future biological threats—will require significant investment and political will.

Fixed features versus controllable factors

In considering how our findings should inform future preparedness efforts, it is crucial to distinguish between fixed variables and controllable factors that are reflective of a country’s preparation or capacity. While fixed variables are outside a country’s control (island status) or difficult to address in the short term (corruption), being able to understand which countries are likely to experience the greatest challenges during an unfolding event is useful for global risk assessment efforts.

While characterising countries’ vulnerability is useful, attention and resources must be invested into addressing controllable factors. In analysing excess mortality across the first 500 days of the COVID-19 pandemic, we were able to further distinguish between controllable factors that are associated with reduced excess mortality throughout and those that are impactful at particular stages. Our findings could be used, in conjunction with countries’ risk profiles, to inform national priorities during non-crisis periods and provide suggestions for how these might change during an ongoing crisis.

We identify two fixed sociodemographic variables that help characterise countries’ risk profile: island status and the population share over 65 years. Our results indicate that island countries have a consistently reduced risk profile in the face of biological events. This protective effect is likely attributed to the increased control over the inflow of potentially infected individuals. Being an island may also be correlated with historically increased consideration of—and investment in—biosecurity relative to non-island countries, given the inherent fragility of island ecosystems.42

Countries with more elderly individuals tended to have greater excess mortality, at all time points assessed, consistent with previous coronavirus epidemics.43 While governments cannot intervene to address these features, our findings serve as further motivation for countries who are particularly vulnerable due to their geographic connectivity or older populations to compensate by improving the other features of their capacity.

Finally, it is important to note that an inherently lower risk profile does not justify less investment in health security capacity. Biological threats, whether natural or man made in nature, do not respect national borders. Increasing global connectedness means that any weaknesses in global health security pose a threat to countries worldwide, underscoring the need for all countries to be prepared.

Limitations

Our study is subject to several limitations. First, any quantification of countries’ COVID-19 performance is subject to limitations given the ongoing nature of the pandemic and the lack of standardisation of COVID-19 outcome data. Though using excess mortality as our outcome somewhat accounts for reporting discrepancies, countries still vary in their maintenance of vital statistics and ability to enumerate all-cause deaths. The outcomes and relationships presented in this analysis are therefore subject to change as more information becomes available.

Similarly, the outcome data used are best estimates of countries’ excess mortality during COVID-19 given available data and are limited by the lack of information needed to make accurate assessments in the lowest income settings. It could be the case that countries lacking robust surveillance capabilities are unable to adequately track and report all-cause deaths, thus obscuring the extent of excess mortality occurring because of COVID-19. This is supported by the mean surveillance and reporting subindicator score of 7.75/100 for the 17 countries lacking excess mortality data at 500 days, nine of which had scores of zero. This may have contributed to conservative excess mortality estimates for lesser prepared countries that artificially improved the quality of their COVID-19 response. Our study is also subject to the limitations of the GHS Index itself, including the exclusive reliance of the Index on publicly accessible material to quantify countries’ capacities.

Like all ecological studies, our study is subject to the ecological fallacy given the country-level nature of our data. Country-level analysis may also obscure variation within countries: a single stringency value for the USA cannot capture granular county-level differences, which may also partially account for inconsistencies in outcomes compared with national preparedness assessments. We are also restricted to presenting correlation and cannot demonstrate causation.

While external variables like social cohesion account for some variation in excess mortality, these features may be insufficiently captured by a single metric. Finally, additional factors influencing excess mortality were not incorporated into these analyses. For instance, we lack global data on the presence and structure of aged-care facilities, which have been linked to increased mortality from COVID-19.44–47 While, ideally, COVID-19 outcome data would be age adjusted, global age-adjusted mortality data are lacking, although we attempted to account for this using the share of the population over 65 as an imperfect proxy. We also did not include a variable to adjust for the severity of the pandemic within countries, as data reflecting this were not available for many countries at the time points of interest. Failure to adjust for these factors may have resulted in the presence of residual confounding.

Conclusion

Our results suggest future Index iterations and other preparedness tools would benefit from the inclusion of additional political and governance variables. Indicators assessing sociopolitical polarisation, social cohesion and perception of corruption had strong associations with countries’ excess mortality throughout the first 500 days of the COVID-19 pandemic. This suggests they could help characterise countries’ ability to prepare for, detect and respond to significant biological events and highlights them as potential areas for future preparedness investments. We identified other, fixed characteristics like geographic connectedness and elderly population size that increase countries’ risk profile and should be used to identify particularly vulnerable countries with a greater need for preparedness investments.

Our findings also offer insight into the relative impact of strengthening different preparedness areas during ongoing public health emergencies. Data suggest investing in the epidemiological workforce during a biological event may positively impact the outcome of such events, whereas the completion of preparedness assessments has greater impact prior to, or at the immediate outset of, an event. Further exploration of these relationships and other real-time impact assessments is desirable.

COVID-19 is unlikely to be the last biological catastrophe we face. The risk of biological threats, be they naturally occurring, accidental or deliberate, continues to rise, and it is clear we are globally insufficiently prepared to handle them. The COVID-19 pandemic provided an opportunity to assess the inaugural GHS Index’s array of factors hypothesised to influence countries abilities to respond to such events and identify factors for inclusion in future iterations. These additions enhance the usefulness of the Index as a tool for characterising individual countries’ risk profiles and identifying gaps in their health security, allowing governments to effectively allocate resources and develop policies to address shortcomings. In identifying risks and gaps in the capacity of countries worldwide, the GHS Index remains an important tool for helping decision makers at the regional and international level monitor progress in, advocate for and take action towards strengthening global health security.

Data availability statement

Data are available in a public, open access repository. All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study does not involve human participants.