The World Health Organization Measles Programmatic Risk Assessment Tool—Pilot Testing in India, 2014

Abstract

Measles is a leading cause of child mortality, and reduction of child mortality is a key Millennium Development Goal. In 2014, the World Health Organization and the U.S. Centers for Disease Control and Prevention developed a measles programmatic risk assessment tool to support country measles elimination efforts. The tool was pilot tested in the State of Uttarakhand in August 2014 to assess its utility in India. The tool assessed measles risk for the 13 districts of Uttarakhand as a function of indicator scores in four categories: population immunity, surveillance quality, program delivery performance, and threat. The highest potential overall score was 100. Scores from each category were totaled to assign an overall risk score for each district. From this risk score, districts were categorized as low, medium, high, or very high risk. Of the 13 districts in Uttarakhand in 2014, the tool classified one district (Haridwar) as very high risk and three districts (Almora, Champawat, and Pauri Garhwal) as high risk. The measles risk in these four districts was largely due to low population immunity from high MCV1–MCV2 drop-out rates, low MCV1 and MCV2 coverage, and the lack of a supplementary immunization activity (SIA) within the past three years. This tool can be used to support measles elimination in India by identifying districts that might be at risk for measles outbreaks, and to guide risk mitigation efforts, including strengthening routine immunization services and implementing SIAs.

1. INTRODUCTION

In 2012, the World Health Organization (WHO) Global Vaccine Action Plan set a goal for measles elimination in five of the six WHO regions by 2020.⁽¹⁾ In September 2013, the WHO Southeast Asia Region (SEAR) adopted a measles elimination goal by 2020 and for the first time, all six WHO regions have a goal for measles elimination by 2020 or earlier.⁽²⁾ The government of India (GoI) is committed to achieving measles elimination by 2020 and is implementing measles elimination strategies in all 29 states and six union territories (UTs), which are a type of administrative division in the Republic of India.⁽³⁾

In India, the routine first dose of measles-containing vaccine (MCV1) at nine months of age was introduced through the Universal Immunization Program in 1985. In 2008, the India National Technical Advisory Group on Immunization (NTAGI) recommended the introduction of a routine second dose of measles-containing vaccine (MCV2) in states with ≥80% MCV1 coverage, and implementation of a mass vaccination campaign (also referred to as a supplemental immunization activity [SIA]) in states with <80% MCV1 coverage.⁽⁴⁾ In 2010, following NTAGI recommendations, the GoI started implementing a strategy for providing two doses of measles vaccine to all children.⁽⁵⁾ GoI started introducing MCV2 in the 21 states and UTs with >80% MCV1 coverage for children 16–24 months of age in 2010 and completed in 2012. In the 14 states with <80% MCV1 coverage,⁽⁶⁾ a measles catch-up SIA targeting children nine months to nine years of age was implemented, followed by routine MCV2 introduction for children 16–24 months of age, which started in 2010 and completed in 2013 in phased manner.⁽⁴⁾ The MCV2 introduction started in 2010 and by 2012, all states including all UTs had introduced MCV2. The measles catch-up SIA started in 2010, and was completed in 2013.

To support countries in their measles elimination efforts, the WHO and the U.S. Centers for Disease Control and Prevention developed a measles programmatic risk assessment tool in 2014 to identify areas off-track from meeting programmatic targets for achieving measles elimination and to prioritize efforts to strengthen implementation of measles elimination strategies.⁽⁷⁾ The tool was designed to assist national program managers to identify districts at risk for measles outbreaks and to provide recommended actions to address programmatic weaknesses. To assess the utility of the WHO measles risk assessment tool in India, the tool was pilot tested in the State of Uttarakhand in August 2014.

2. MATERIALS AND METHODS

2.1. Setting of the Pilot Testing

In July 2014, the India Epidemic Intelligence Service (EIS) assisted the Ministry of Health and Family Welfare (MoHFW) to conduct a measles risk assessment in Uttarakhand and pilot test the recently developed WHO measles risk assessment tool.

2.2. Application of the WHO Measles Risk Assessment Tool

The development and methods of the global measles risk assessment tool were previously described in detail and published.⁽⁸⁾ The tool was designed to assess measles risk for each district as a function of indicator scores in four assessment categories: (1) population immunity, (2) surveillance quality, (3) program delivery performance, and (4) threat (Table I). The highest potential overall risk score was 100. The population immunity category indicators that assess measles susceptibility among the district population had a total of 40 possible points. The scoring used administrative vaccination data for providing measures of district-level MCV1 and MCV2 coverage, and measles SIAs conducted within the past three years.

Table I.

Maximum Risk Points by Component of the World Health Organization (WHO) Measles Programmatic Assessment Tool for Risk of Measles Virus Transmission

Components	Points	Cut-Off Criteria (Risk Points)
Population Immunity	(40)
District MCV1 coveragea	8	≥95% (+0);90–94% (+2); 85–89% (+4); 80–84% (+6); <80% (+8)
Proportion of neighboring districts with <80% MCV1	4	<50% (+0); 50–74% (+2); ≥75% (+4)
District MCV2 coveragea	8	Same as MCV1 coverage
Measles SIA conducted within the past three yearsb	8	Yes: ≥95% coverage (+0); 90–94% coverage (+2); 85–89% coverage (+4); <85% coverage (+6); No coverage data (+6); No SIA (+8)
Target age group of measles SIA conducted within the past three years	2	Wide age group (+0); Narrow age group (+2); No SIA (+2)
Years since the last measles SIA	4	≤1 year (+0); 2 years (+2); ≤3 years (+4)
Proportion of suspected cases who are unvaccinated or have unknown vaccination status	6	<20% (+0); ≥20% (+6)
Surveillance Quality	(20)
Nonmeasles discarded rate	8	≥2 per 100,000 (+0); <2 per 100,000 (+4); <1 per 100,000 (+8)
Proportion of measles cases with adequate investigation	4	≥80% (+0); <80% (+4)
Proportion of measles cases with adequate specimens collection	4	≥80% (+0); <80% (+4)
Proportion of measles cases with laboratory results available in a timely manner	4	≥80% (+0); <80% (+4)
Program Performance	(16)
Trends in MCV1 coverage	4	Increasing or same (+0); ≤10% decline (+2); >10% decline (+4)
Trends in MCV2 coverage	4	Same as MCV1 trend
MCV1-MCV2 drop-out rate	4	≤10% (+0); >10% (+4)
DPT1-MCV1 drop-out rate	4	Same as MCV1-MCV2 dropout rate
Threat Assessment	(24)
≥1 measles case reported among children <5 years	4	No (+0); Yes (+4)
≥1 measles case reported among persons 5–14 years	3	No (+0); Yes (+3)
≥1 measles case reported among persons ≥15 years	3	No (+0); Yes (+3)
Population density	4	0-50/km2 (+0); 51–100/km2 (+1); 101–300/km2 (+2); 301–1000/km2 (+3); > 1000/km2 (+4)
≥1 measles case reported in a bordering district within the past 12 months	2	No (+0); Yes (+2)
Presence of vulnerable groupsc	8	No vulnerable groups (+0); One risk point for each vulnerable group present (up to maximum of +8)
Total possible points	(100)

DPT1 = first dose in series for diphtheria, pertussis, and tetanus vaccination; MCV1 = first dose in series for measles-containing vaccination; MCV2 = second dose in series for measles-containing vaccination; SIA = supplemental immunization activity.

^aVaccination coverage estimates from surveys if conducted within past three years and includes birth cohorts of recent three years can be used to replace administrative coverage.

^bOutbreak response immunization (ORI) campaign coverage data can be considered if an SIA was not conducted within the past three years and if the ORI targeted a geographical area that included the entire district.

^cPresence of vulnerable groups includes any of the following: (1) migrant population, internally displaced population, slums, or tribal communities; (2) communities resistant to vaccination (e.g., religious, cultural, philosophical reasons); (3) security and safety concerns; (4) areas frequented by calamities/disasters; (5) poor access to health services due to terrain/transportation issues; (6) lack of local political support; (7) high-traffic transportation hubs/major roads or bordering large urban areas (within and across countries); (8) areas with mass gatherings (i.e., trade/commerce, fairs, markets, sporting events, high density of tourists).

To assess the ability of surveillance in a district to rapidly detect and confirm cases, the tool uses standard measles case-based surveillance performance indicators, including the nonmeasles discarded rate, proportions of suspected measles cases with adequate investigation, adequate specimen collection, and laboratory results available in a timely manner. The surveillance quality indicators had a total of 20 possible points. Because measles outbreak surveillance rather than measles case-based surveillance was implemented in Uttarakhand, data were unavailable to calculate data inputs needed for case-based surveillance performance indicators and variables related to the age and vaccination status of cases. To adjust for this lack of data, the tool was applied using a rule for settings that have not fully implemented case-based surveillance for measles. For outbreak surveillance, the type of surveillance used in Uttarakhand, the tool assigns half the total possible risk points for each variable requiring measles case-based surveillance data.

The program delivery performance indicators assessed specific aspects of routine immunization services and accounted for a total of 16 points. Program delivery performance indicators included trends in MCV1 and MCV2 administrative coverage, as well as drop-out rates from MCV1 to MCV2 and from the first dose of diphtheria, pertussis, and tetanus vaccine (DPT1) to MCV1 using vaccination administrative coverage data.

The threat assessment indicators accounted for factors that might influence the risk for measles virus exposure and transmission in the population and had a total of 24 points. Indicators in this category that required the use of measles case-based data—including reported measles cases among persons aged <5 years, 5–14 years, and ≥15 years, and measles cases reported in a bordering district within the past 12 months—were automatically assigned half the total risk points for each surveillance-related variable. Threat assessment indicators also include population density and presence of vulnerable populations in the district.

Scores from each assessment category were totaled to assign an overall risk score for each district. Districts were categorized as low, medium, high, or very high risk based on the overall score and cutoffs set at the 50th, 75th, and 90th percentiles of the distribution of all possible combinations of scores from each indicator.

2.3. Calculation of Vaccination Coverage Figures

To apply the tool in Uttarakhand, we calculated vaccination coverage estimates using the number of doses of MCV1, MCV2, and DPT1 administered per year during 2011–2014 that were obtained from the Health Management Information System (HMIS) web-based portal maintained by the GoI MoHFW.⁽⁸⁾ These HMIS data were reported by the immunization program for each calendar year (April to March) in India. To align the assessment results with those from other countries, data were reorganized by the calendar year January to December. After obtaining the number of doses of vaccines administered, we estimated the total target population for MCV1, MCV2, and DPT1 by calculating the infant population using MoHFW population data for each district based on 2011 census projections and an exponential growth rate. The target infant population is provided to all states by the MoHFW. The target infant population for each district was calculated by considering the crude birth rate and infant mortality rate.

Vaccination coverage estimates were then calculated by dividing the total number of doses administered by the target infant population and multiplying by 100 (e.g., MCV1 coverage = [doses of MCV1 administered/target infants] x 100). Risk points for MCV1 and MCV2 coverage were assigned based on the average administrative coverage in each district from the past three years (2011–2013). The indicator “MCV drop-out rate” uses the formula ([MCV1 coverage – MCV2 coverage] / MCV1 coverage) x 100. The drop-out rates for MCV vaccine were computed based on coverage of MCV1 and MCV2 and presented as a percentage. If districts had administrative coverage reported as >100%, then entries were capped at 100% in an effort to minimize the bias.

2.4. Scoring of Threat Assessment

Population density was calculated using the estimated population based on the 2011 census and the geographic area data found in the shape file. The vulnerable population groups for each district were identified at the Immunization Program Office, Directorate of MoHFW, Uttarakhand on August 7, 2014 at the state capital Dehradun. A six-member team from the Directorate of MoHFW, Uttarakhand identified vulnerable groups for each district. The six-member team consisted of the state immunization officer, state cold chain officer, additional research officer, a computer assistant, a data entry operator, and an India EIS officer. The team conducted a three-hour discussion on each threat category variable based on the local knowledge, experience, and expertise of each member. If applicable, one risk point was assigned for each vulnerable population group present in a district. Presence of vulnerable groups includes any of the following: (1) migrant and internally displaced populations, slums, or tribal communities; (2) communities resistant to vaccination (e.g., religious, cultural reasons); (3) security and safety concerns; (4) frequented by calamities/disasters; (5) poor access to health services due to terrain/transportation issues; (6) lack of local political support; (7) presence of high-traffic transportation hubs/major roads or bordering large urban areas; and (8) presence of areas with mass gatherings (i.e., trade/commerce, fairs, markets, sporting events, high density of tourists).

2.5. Uttarakhand and State Measles Risk by District in 2014

Risk assessment data were entered using Microsoft Excel version 7.0. The WHO measles risk assessment tool is an Excel-based tool with embedded formulas that calculate the risk score for each district. The tool is designed to allow the user to observe the risk points for each variable, the four main categories, and the total for each district, and examine the primary drivers of the risk score. These detailed assessments by district can show potential reasons why a district is at risk for measles and lead to formulation of recommended action to mitigate risk for each district. Risk assessment results were mapped using ArcGIS version 10.1 (ESRI).

3. RESULTS

3.1. Setting of the Pilot Test

Uttarakhand, located in northern India, has a long international border with Nepal, mountainous and difficult-to-access areas, and an estimated population of 13.7 million in 2014 (Fig. 1). The state is administratively organized into 13 districts. The Expanded Programme on Immunization (EPI) introduced MCV1 in 1985 and MCV2 in 2011. Starting in January 2014, the WHO-National Polio Surveillance Project and MoHFW have implemented measles outbreak surveillance to detect and serologically confirm measles outbreaks.

Fig. 1.

Location of Uttarakhand (marked in red; colors visible in on-line version) in India.

3.2. Population Immunity and Program Delivery

Among the 13 districts in Uttarakhand, the lowest average MCV1 coverage (72%) during 2011–2013 was in Haridwar district, which received the maximum eight risk points, whereas the districts Almora, Champawat, and Pauri Garhwal had average MCV1 coverage of 82–83% during 2011–2013 and received six risk points (Table II). Bageshwar, Dehradun, Nainital, Pithoragarh, Rudraprayag, Tehri Garhwal, and Udham Singh Nagar, districts had average MCV1 coverage of 99–112% during 2011–2013, and received zero risk points for this indicator. All 13 districts had average MCV2 coverage of 17–69% during 2011–2013 with the maximum eight risk points.

Table II.

Administrative Vaccination Coverage and Drop-Out Rates by District, Uttarakhand State, India, 2011–2013

District	MCV1 Administrative Coverage (%)				MCV2 Administrative Coverage (%)				Drop-Out MCV1–MCV2	DPT1/Penta1 Coverage	Drop-Out Rate DPT1– MCV1
Uttarakhand	2011	2012	2013	Avg	2011	2012	2013	Avg	2013	2013	2013
Almora	88	77	83	83	3	17	35	18	58%	70	−19%
Bageshwar	95	100	100	98	21	86	99	69	1%	93	−8%
Chamoli	96	87	95	93	8	40	61	36	36%	78	−22%
Champawat	82	80	83	82	4	35	61	33	26%	85	3%
Dehradun	95	91	100	95	0	16	59	25	41%	100	0%
Haridwar	66	68	83	72	3	20	39	21	53%	94	12%
Nainital	96	100	100	99	0	21	62	28	38%	100	0%
Pauri Garhwal	87	76	84	82	9	38	63	37	24%	77	−9%
Pithoragarh	100	100	99	100	2	45	65	37	35%	73	−36%
Rudraprayag	100	90	100	97	1	1	48	17	52%	92	−9%
Tehri Garhwal	100	98	100	99	10	38	88	46	12%	83	−20%
Udham Singh Nagar	100	100	100	100	0	19	59	26	41%	100	0%
Uttarkashi	86	85	95	89	7	35	80	41	15%	87	−9%
Uttarakhand State	93	90	98	94	3	26	58	29	41%	92	−7%

MCV1 = first dose in series for measles-containing vaccination; MCV2 = second dose in series for measles-containing vaccination; SIA = supplementary immunization activity; DPT1 = first dose in series for diphtheria, pertussis, and tetanus vaccination.

Among the 13 districts, the lowest MCV1–MCV2 (6%) drop-out rate was in Bageshwar district with zero risk points. The remaining 12 districts had drop-out rates MCV1–MCV2 ranging from 15% to 58% with four risk points each. These large MCV1–MCV2 drop-out rates were primarily caused by low MCV2 coverages. Among the 13 districts, the maximum DPT1-MCV1 drop-out rate (12%) was in Haridwar district with four risk points, whereas the remaining 12 districts had DPT1-MCV1 drop-out rates ranging from −36% to 5% with zero risk points each.

3.3. Surveillance Quality and Threat Assessment

On the basis of the rule applied for settings that have not fully implemented case-based measles surveillance, the tool assigned half the total possible risk points for all surveillance quality indicators requiring measles case-based data. Consequently, all 13 districts in Uttarakhand had 10 points for surveillance quality.

Points for threat assessment ranged from 11 to 15 points (Table III). Most of the districts (11 of 13, or 85%) received either 11 or 12 points for threat assessment. Haridwar received 15 points. Twelve of the possible 24 points for threat assessment were based on case-based data, so all districts received half the possible risk points for those indicators. Indicators with heterogeneity included population density, which ranged from one to three points, and presence of vulnerable populations, which ranged from three to six points. Haridwar, the district with the highest points for vulnerable populations, is the site of mass gatherings related to Hindu pilgrimage to the Ganges River. Other districts, including Dehradun, Nainital, and Udham Singh Nagar, have similar mass gatherings.

Table III.

Measles Programmatic Risk Assessment Points and Category by District, Uttarakhand State, India, 2014

District	Population Immunity Points	Surveillance Quality Points	Program Delivery Performance Points	Threat Assessment Points	Total Points	Risk Category
Almora	29	10	6	11	56	HR
Bageshwar	23	10	0	11	44	LR
Chamoli	25	10	6	11	52	MR
Champawat	29	10	4	12	55	HR
Dehradun	23	10	4	12	49	MR
Haridwar	31	10	8	15	64	VHR
Nainital	23	10	4	12	49	MR
Pauri Garhwal	29	10	6	11	56	HR
Pithoragarh	23	10	6	12	51	MR
Rudraprayag	23	10	6	12	51	MR
Tehri Garhwal	23	10	6	12	51	MR
Udham Singh Nagar	23	10	6	14	53	MR
Uttarkashi	27	10	4	11	52	MR

LR = low risk; MR = medium risk; HR = high risk; VHR = very high risk.

3.4. Uttarakhand Measles Risk in 2014

The risk assessment tool classified 4 of the 13 districts in Uttarakhand as high risk or very high risk in 2014 (Table III; Fig. 2). Haridwar district was categorized as very high risk, driven primarily by low population immunity and suboptimal program delivery performance combined with a high threat score. The high risk in these four districts was due to large MCV1–MCV2 drop-out rates, and low population immunity primarily due to low MCV1 and MCV2 coverage combined with the lack of an SIA within the past three years.

Fig. 2.

Measles programmatic risk category by district, Uttarakhand State, India, 2014.

Note: In 2014, measles case-based surveillance was not conducted in India. Therefore, this map was based on a version of the risk assessment tool that assigned half of the possible points for each indicator that required case-based surveillance data.

4. DISCUSSION

The results of the pilot testing of the WHO measles programmatic risk assessment tool showed the tool was able to synthesize routinely collected administrative data in India to assign each district a risk category. The assessment process brought together key immunization and surveillance staff to identify vulnerable populations and areas for the threat category, and to discuss measles elimination efforts. Mapping districts by measles risk assessment category was helpful to show the clustering of the at-risk districts. Results of the tool were used to identify areas in need of routine immunization strengthening and SIAs. Other interventions that could be prioritized in high-risk districts could include measles case-based surveillance reviews to strengthen surveillance, routine immunization service reviews to increase routine vaccination coverage and detailed micro-planning and supervision of SIAs, outbreak response immunization planning, and advocacy and resource planning.

The pilot test results should be considered in light of some limitations. First, the WHO tool was designed to use administrative data; therefore, suboptimal data quality and inaccurate data inputs might result in an over- or underestimation of risk. In particular, during the assessment in Uttarakhand, multiple districts had administrative vaccination coverage >100% and likely led to an overestimation of population immunity and, therefore, an underestimation of the level of risk in those districts. Second, because of the lack of measles case-based surveillance in India, the tool was applied using a rule assigning half the total possible risk points for each case-based variable in settings where outbreak surveillance was used, and all of the total possible risk points for each case-based variable in settings where no measles surveillance was conducted. In Uttarakhand, this rule affected several indicators, including one population immunity indicator (percent of suspected measles cases unvaccinated), four surveillance quality indicators (nonmeasles discarded rate, percent of suspected cases with adequate investigation, percent of suspected cases with adequate blood specimen collection, percent of suspected cases with timely availability of laboratory results), and four threat assessment indicators (recent measles cases among <5 years, recent measles cases among 5–14 years, recent measles cases among ≥15 years, bordering area with measles case in the past 12 months). Use of this rule likely decreased the utility of the tool by discounting 19 total possible points and could have resulted in an underestimation of measles risk in each district and reduced heterogeneity in risk across districts.

Although the WHO tool was developed to use measles case-based data, these data were unavailable for the pilot test because measles case-based surveillance was not yet implemented in India as of 2014. By the end of 2016, the GoI is planning to have fully operational nationwide case-based, laboratory supported measles and rubella surveillance with strong links to outbreak investigations and inclusion of line-listed cases from confirmed outbreaks in the case-based system.⁽⁹⁾ In the future, the availability of measles case-based data will further enhance the performance of the tool to accurately assess measles risk in India.

The mapping of results by district showed the geographic clustering of the four high-risk or very-high-risk districts within Uttarakhand in 2014. The ability of the assessment tool to identify and show heterogeneity of population immunity and clustering of measles risk provides an important finding for program managers. If an annual risk assessment is conducted in all states and UTs, regional or nationwide maps of measles risk could be generated, which would provide useful information for identifying contiguous areas of measles risk across borders to synchronize state-level measles elimination efforts, including mass vaccination activities.