Lessons from a modern review of the smallpox eradication files

Background

Smallpox was declared eradicated in 1980.¹ Nevertheless, the virus has not lost importance with contemporary biosecurity concepts contributing to the revival of smallpox research and new investigations in smallpox vaccines.^2,3 In addition, eradication of smallpox is a frequently used argument in public debates about vaccine success. The statement “the vaccine has eradicated smallpox” often serves as an historical proof-of-concept for the high potential impact of vaccines for eliminating other pathogens, e.g. polio, measles and malaria.

In the hot phase of the campaign, a series of studies were carried out to determine the transmission conditions of the virus. Data from these studies are cited as evidence that the protective efficacy of the vaccine was close to 100%. For example, data from five studies carried out in the end of the 1960s and early 1970s in two major Indian cities, and in rural communities of West Pakistan,^4–8 are listed in the definitive book about smallpox eradication published by Fenner et al.¹ on behalf of the World Health Organization (WHO).

The five studies evaluated smallpox attacks in contact persons of index cases retrospectively. When an outbreak was reported, a team of WHO collaborators visited the families and traced the path of intrafamilial transmission. The vaccination status of family members was among the parameters gathered in the surveys. However, none of the case-control studies had the adequate design to determine vaccine efficacy.

Nonetheless, the formula for calculating vaccine efficacy was applied: vaccine efficacy = (1 − R) × 100, where R is the relative risk, i.e. the percentage of vaccinated persons with smallpox divided by the percentage of unvaccinated persons with smallpox. In the five publications, vaccine efficacy ranged between 91% and 97%. These data were reproduced in a single table on vaccine efficacy in Fenner et al.¹

The smallpox fact sheet of the Centers of Disease Control and Prevention (CDC) mentioned at this point: ‘It is important to note, however, that at the time when the smallpox vaccine was used to eradicate the disease, testing was not as advanced or precise as it is today, so there may still be things to learn about the vaccine and its effectiveness …’⁹

Methods

Careful reading of the five studies.^4–8

Results

Madras:⁴ The families of 254 index cases were visited in Madras, India, and 1249 contact persons were registered. Of these, 52 individuals acquired smallpox and vaccine efficacy was calculated as 97% (14/1146 vaccinated vs. 38/103 unvaccinated). Secondary cases were concentrated only in a small portion of the families (36/254, 14%) and the highest transmission was found under overcrowded conditions (6–8.9 persons per room). Although the risk of infection in families with less than 2.9 persons per room was assessed to be low and transmission appeared to take place focally, the total number of contacts was taken as the at risk population. Importantly, vaccine efficacy would be almost 10% lower, if only families with secondary cases were included as the population at risk (Table 1). Moreover, a downward trend of vaccine efficacy was observed with decreasing income of the families. In poor families with low vaccination rates, the risk of infection increased for those vaccinated. As a consequence, vaccine efficacy dropped to 59% in families with more than two unvaccinated members.

Table 1.

Raw data of the five case-control studies listed in Fenner et al.¹

Site	Secondary cases from	Confounder	VC	VA	UC	UA	Vaccine efficacy (%)
Madras4	All families with index cases		14	1146	38	103	97
	Only families with secondary cases Allowing for confounders	ABCD	14	164	38	54	88 (P)
Punjab5	Study participants plus ex post included		6	190	33	45	96
	Original study design Allowing for confounders	ABCD	(P)	143	0	3	(P) (P)
Punjab6	Data of unclear origin		3	238	10	22	97
	Data of publication* Allowing for confounders	ABCD	16	331	54	112	90 (P)
Sheikhupura7	51/956 secondary cases		13	180	38	43	92
	All 956 secondary cases Allowing for confounders	ABCD	(P)	(P)	(P)	(P)	(P)
Calcutta8	All compounds with index cases		47	661	61	80	91
	Only compounds with secondary cases Allowing for confounders	ABCD	(P)	(P)	(P)	(P)	(P)

VC: vaccinated smallpox cases; VA: all vaccinated individuals (population at risk); UC: unvaccinated smallpox cases; UA: all unvaccinated individuals (population at risk); (P): pending; A: age; B: behaviour; C: clustering; D: dwelling density.

First line: Figures from publications^4–8 cited in Fenner et al.¹

Second line: Figures from publications^4–8 that result in vaccine efficacy different from Fenner et al., or are pending.

Third line: Factors discussed in the publications^4–8 for their impact on transmission, but not integrated as possible confounders in vaccine efficacy calculations.

^*Including all cases intended to investigate. Excluding cases with unclear vaccination status, e.g. deceased individuals whose vaccination status could not be determined.

Punjab:⁵ Blood samples were taken from contacts of index cases in Punjab villages near Lahore, Pakistan, in which 143 of 146 contacts (98%) were vaccinated. No smallpox lesions were seen in the three unvaccinated contacts. It was neither confirmed nor disclaimed if there were smallpox attacks in the 143 vaccinated contacts. Through the ex post capture of additional family members, vaccine efficacy was calculated to be 96% (6/190 vs. 33/45) (Table 1). Among the additionally included individuals were children, who initially had been excluded from the study. It was noted that the vaccinated group mainly comprised adults, whereas the unvaccinated group was largely children. Therefore, the study groups were not comparable with respect to exposure and immunity maturation. Interestingly, close to half of the vaccinated individuals showed antibody titres ‘suggestive of inapparent infection’.

Punjab:⁶ 47 families with 464 contacts of index cases were visited and 91 contacts were reported to have been infected by smallpox. Depending on the quality of information, different values for vaccine efficacy can be calculated. Including all contacts intended to be investigated (n = 464), and excluding those cases whose vaccination status could not be determined (n = 16), vaccine efficacy was 90% (16/331 vs. 54/112). The cases clustered in children aged under 10 years (71/91, 78%) that were located within a few compounds since more than half of the attacks occurred in four families (50/91, 55%), whereas nearly half of the families did not report secondary cases. Surprisingly, vaccine efficacy cited in the table of Fenner et al. is 97% (3/238 vs. 10/22). The origin of these numbers is unclear, as they could neither be found nor reproduced in the cited article (Table 1).

Sheikhupura:⁷ 121 smallpox outbreaks with 956 secondary cases were investigated in the district Sheikhupura in West Pakistan. Vaccination status and personal data of index and contact cases were documented meticulously. Travelling characteristics of the villagers were analysed, and the rural transmission of the scourge was described. The huge inquiry resulted in three publications.^7,10,11 However, only the data of a group of 51 secondary cases were reported in the table of Fenner et al.,¹ displaying a vaccine efficacy of 92% (13/180 vs. 38/43) (Table 1). Assuming a 92% vaccine efficacy, we reconstructed the raw data for the entire cohort of secondary cases (n = 956) through surveillance data and attack rates given in the publication. The reconstructed data revealed that more vaccinated than unvaccinated individuals must have caught smallpox in the largest part of the Pakistani society, the near-illiterate stratum of the population (295/98,390 vs. 251/6615, vaccine efficacy 92%). Considering additional information given in the publication, it can be speculated that vaccine efficacy was significantly lower than 92% in major subgroups of the rural communities.

Calcutta:⁸ 741 contacts of 43 index cases were retrospectively investigated in Calcutta, India. The third part of the index cases (15/43, 35%) and nearly half of the secondary cases (47/108, 44%) were vaccinated. Notwithstanding, as most of the population was vaccinated, vaccine efficacy was calculated to be 91% (47/661 vs. 61/80). Although an aim of the study was to assess the impact of different exposure for the risk of infection, e.g. in single family compounds compared to multiple family compounds, data are presented in a pooled form only. Thus, data for determining clustering and foci of transmission could not be traced.

It is important to note that in all five publications – although vaccine efficacy was calculated to approach full protection – the impact of vaccination was explicitly modified by quoting factors other than vaccination having an impact in the control of smallpox. However, these were considered only superficially and were not analysed in detail as it would be required in nowadays evidence-based vaccinology. Exposure and socioeconomic status were specifically mentioned as risk factors for infection: ‘there seems to be … some other factors also playing a role in making a person susceptible or resistant to infection besides the vaccinial status … ’.⁴ ‘The close association of exposure factors with infection is one of the most striking findings of this study’.⁵ ‘Under conditions of effective exposure, the vast majority of contacts will become infected, regardless of their immune status’.⁶ ‘Systematic factors other than herd immunity must be important in determining the extend of spread’.⁷ ‘In Calcutta, as in many other endemic cities, the real home of smallpox is usually the areas inhabited by people of the lower socioeconomic groups, where resistance to vaccination and unhygienic living conditions, especially overcrowding, help in the maintenance and spread of infection’.⁸

Discussion

At first sight, the nearly complete vaccine efficacy of the smallpox vaccine seemed to be beyond reproach, because many unvaccinated individuals caught the infection, whereas attack rates in vaccinated individuals were low. However, numbers of infected vaccinated individuals were often reduced to small attack rates due to huge denominators. If the denominator population was at such a high risk, it is important that this parameter be assessed carefully.

Exposure as a substantial risk factor for infection was analysed in all five publications. Differences on the micro-social, intrafamilial level seemed to be important. For example, persons in constant close contact, such as mothers, had a higher risk for infection.⁶ Moreover, it was known that ‘the poorer class of people usually escape vaccination’.⁴ Whether domestic servants, who tended to be less vaccinated,¹² were more exposed, was not investigated. One study indicates that access to healthcare was not equal, as only a few of the unvaccinated were vaccinated (2/93, 2%) within seven days of exposure, whereas 16% (46/285) of the vaccinated were revaccinated.⁶ In short, pockets of transmission, where high exposure was paralleled by negative vaccination status, could not be excluded in any of the studies. Although some of these aspects were reflected in the discussion sections, they were not included in the vaccine efficacy calculations.

One reason for the lack of inclusion of confounders could be that the authors assumed a protective efficacy of 100%. Thus, the interpretation of the data was limited. For example, low exposure was discussed as the reason why unvaccinated individuals escaped infection. Conversely, high exposure was, therefore, assumed to be the causative factor for explaining how vaccinated individuals acquired smallpox.⁶ Alternative interpretations, such as those suggesting that the vaccinated individuals could have escaped the disease because they were less exposed, and, vice versa, that the unvaccinated caught smallpox because they were more exposed, appear to have been rejected.⁶

Moreover, low risks of infection among the educated were explained by the high level of vaccination and not by more favourable behaviour in health-related issues and better living conditions.⁷ However, as smallpox was almost always transmitted at the bedside of the source,¹³ awareness of the transmission modus and subsequent behavioural changes in those interacting with the diseased was likely a major obstacle for spreading of the virus. This phenomenon was explained elsewhere with the example of indigenous people: ‘there was a very rapid spread of smallpox through the Native American community. The reason for that is that the Native Americans had no concept of infectiousness. They were not frightened by someone covered in a rash and did not shun them the way we would today. Every doctor should recognise the characteristic appearance of smallpox, and once the first case appears, everyone will recognise it from the media coverage. No one's going to rush up and hug a highly infectious person covered in rash – they'll run the other way.’¹⁴

The partial interpretation of the data in the five seminal publications must be considered in the historical context of the last century. It is important to contextualise that the eradication campaign was ongoing and promised to be successful during the time of these studies. Certainly, it would have been counterproductive for operational reasons if the protective capacity of the vaccine would have been questioned. Thus, campaign-associated effects on smallpox transmission brought about by the isolation element of the containment concept, along with behavioural changes brought about by health educators and vaccinators that reduced exposure, were unmeasured consequences that were ascribed to the vaccine. We suppose that the primary reason for the suboptimal processing of the data was due to the lack of more advanced methodologies available at that time.

We hypothesise, that the efficacy of the smallpox vaccine was considerably lower than that reported in Fenners table. As a first step, we showed how a plausible reduction of the population at risk changed the denominator, and hence vaccine efficacy in one study (Table 1). In another study, today’s standard procedures in terms of inclusion and exclusion criteria were applied (Table 1). A much greater impact on vaccine efficacy is expected when confounding factors are integrated adequately into the assessments. Also, the duration of protection should become a subject of statistical revision. To provide a general estimate, we assume that vaccine efficacy was less than 60%. We presuppose that the study data are applicable to state-of-the-art statistics. Applying statistical tools allowing for confounders, we speculate that some of the co-factors will show a similar effect as the vaccination itself.

In hindsight, it is hard to establish the immunological mechanisms by which the vaccinia virus conferred protection from infection and disease, but it can be assumed that immune reactions varied substantially due to genetic and environmental differences in vaccinees. As smallpox was eradicated prior to the development of modern immunology, the role of T-cells was never evaluated with clinical endpoints. With respect to humoral responses, a titre >1:32 of antibodies neutralising smallpox plaques in cell cultures was estimated to be protective.¹⁵ However, numbers in the few underlying non-randomised prospective investigations were small and, in fact, the role of the humoral response remained unclear.^16,17 Above all, it was difficult to distinguish immune responses raised by vaccination from those acquired by natural contact with human smallpox in an endemic area.¹⁶ Immune responses against smallpox vaccination seemed to be rather long-lived and robust. High amounts of antiviral antibodies might have been associated with higher T-cell memory making them a useful biomarker regardless of whether protection was mediated by B-cells, T-cells or a combination of both.¹⁸

In the studies presented here, we identified the smallpox vaccine as a ‘leaky vaccine’. The issue of leaky vaccines – as, for example, the RTS,S malaria vaccine – belongs to the normal procedures and debate topics of modern vaccinology.^19,20 Leaky vaccines are defined as those modifying per-exposure infection rates for all subjects equally, whereas ‘all or nothing’ vaccines completely protect some subjects and have no effect on the others.²¹ Concisely, every individual becomes infected when immunised with a leaky vaccine if the level of exposition is high enough. Though leaky vaccines may not lead to protection in the vaccinated person, they instead may reduce disease severity or protect a part of the population in scenarios of low transmission.²² These features characterised the smallpox vaccine and have been described in numerous reports since the 19th century.¹ In contrast to an ‘all or nothing’ vaccine, exposure is a key element in order to understand the vaccine efficacy of a leaky vaccine. Therefore, the claimed vaccine efficacy from former smallpox vaccine studies is challenged if exposure data are not integrated.

If the bar is raised too high, the grapes will turn sour. In the public vaccine discourse, the smallpox vaccine is celebrated as an ‘all or nothing’ vaccine with almost complete efficacy. The perception of a perfect vaccine entails the prospect of an ‘one size fits all’ approach for the eradication of infectious agents. However, modern epidemiology suggests that such an approach is not adequate for most pathogens. In this context, it is important to realise that the eradication of smallpox was anything but the result of a ‘one size fits all’ campaign. Based on the perspectives offered, we hypothesise that a respective revision of the historical effectiveness data would reveal a more realistic appraisal of the legendary smallpox eradication campaign and would generate fruitful debate. Important information can be obtained from the experiences pertaining to how activities were tailored to local needs. An understanding that smallpox eradication was possible with the use of a less effective vaccine will facilitate strategic orientations of actual (and practical) eradication efforts for measles, polio and malaria, and provide insight for other challenges such as the control of an ebola outbreak.

The matter of Dr Jenners cowpox inoculation has become a strong metaphor for full vaccine protection in the history of medicine and in collective memory. As such, our hypothesis may inadvertently face strong rejection. In order to reject or accept the hypothesis presented above, we propose that the special smallpox archives that were established in the WHO headquarters in 1980¹ be made publically available online with the data anonymised and accessible in the raw form so that statistical ‘cloud’ intelligence can be applied.

Declarations

Competing interest

None declared.

Funding

The paper was funded by institutional core funding of the Institute of Tropical Medicine of the University of Tübingen.

Ethical approval

Not applicable.

Guarantor

WM.

Contributorship

WGM had the idea and drafted the manuscript. CK and BGM contributed to all parts of the manuscript during the preparation process. All authors revised the final version of the manuscript.

Acknowledgement

We thank Dr Douglas J Perkins and Dr Lauren Kaplan for proofreading and commenting on the content of this manuscript.

Provenance

Not commissioned; peer-reviewed by Elizabeth Haworth.