A Performance Management Initiative for Local Health Department Vector Control Programs

Abstract

Local health department (LHD) vector control programs have experienced reductions in funding and capacity. Acknowledging this situation and its potential effect on the ability to respond to vector-borne diseases, the U.S. Centers for Disease Control and Prevention and the Public Health Foundation partnered on a performance management initiative for LHD vector control programs. The initiative involved 14 programs that conducted a performance assessment using the Environmental Public Health Performance Standards. The programs, assisted by quality improvement (QI) experts, used the assessment results to prioritize improvement areas that were addressed with QI projects intended to increase effectiveness and efficiency in the delivery of services such as responding to mosquito complaints and educating the public about vector-borne disease prevention. This article describes the initiative as a process LHD vector control programs may adapt to meet their performance management needs. This study also reviews aggregate performance assessment results and QI projects, which may reveal common aspects of LHD vector control program performance and priority improvement areas. LHD vector control programs interested in performance assessment and improvement may benefit from engaging in an approach similar to this performance management initiative.

Introduction

Almost half of the approximately 2,800 local health departments (LHDs) in the U.S. provide vector control services. LHD, as defined by the National Association of County and City Health Officials, refers to “an administrative or service unit of local or state government, concerned with health, and carrying some responsibility for the health of a jurisdiction smaller than the state.”¹ LHD vector control programs are commonly considered a responsibility of environmental public health, a foundational area of public health.^2,3 Vector control programs provide services to address vectors, primarily mosquitoes, that may include performing site assessments, providing pesticide treatments, establishing policy and regulations, educating the public, and conducting surveillance to detect and prevent vector-borne disease transmission.

LHD vector control programs have experienced reduced capacity resulting from decreased budgets.⁴ Decreased funding to support state health department and LHD West Nile virus and other arbovirus surveillance activities could be a factor impacting capacity.⁵ These reductions in funding and capacity may affect the ability of vector control programs to identify and respond to existing and emerging threats, including the spread of the mosquito-borne Chikungunya and Zika viruses in the Americas.^6,7 Considering these impacts, the U.S. Centers for Disease Control and Prevention (CDC) and Public Health Foundation (PHF), a nonprofit organization that seeks to improve the quality and performance of public health practice, supported an LHD vector control program performance management initiative. The primary objectives of the initiative were to support a performance assessment process and use the results to inform and implement quality improvement (QI) projects. This study describes the initiative as a process LHD vector control programs may adapt to meet their performance management needs. It also reviews aggregate performance assessment results and QI projects, which may reveal common aspects of LHD vector control program performance and priority improvement areas.

This initiative began in November 2014, as 14 participating programs conducted self-facilitated assessments using the Environmental Public Health Performance Standards (EnvPHPS) Version 2.0 with the goal of determining the level at which they provided the 10 Essential Environmental Public Health Services (EEPHS). The EnvPHPS provides a set of standards describing activities an environmental public health program should conduct to perform the EEPHS. The EnvPHPS can be used for assessments at varying organizational levels, including environmental public health systems, which involve a broad range of entities and partners, departments, and specific programs such as vector control.⁸ Environmental public health departments have previously used EnvPHPS and the EEPHS to assess their performance and inform QI efforts.^9–11 PHF assigned a QI expert to each vector control program to assist with using EnvPHPS assessment results to identify and prioritize improvement areas and implement QI projects to address these areas. The QI experts emphasized aspects of efficiency such as reducing mosquito complaint response time and effectiveness, including enhancing vector-borne disease prevention outreach and educational efforts.

Methods

Participants

A total of 14 LHD vector control programs from 13 states participated in the initiative from November 2014 through May 2015. The majority of the programs were identified by contacting Environmental Public Health Leadership Institute (EPHLI) alumni from LHD and state health department. EPHLI, supported by the CDC from 2005 to 2012, accepted approximately 30 practicing environmental health professionals each year into the yearlong program designed to enhance leadership skills and abilities.¹² The alumni indicated whether their department had a vector control program and whether they were interested in participating in the initiative. In some cases, the alumni were able to recommend another point of contact within the LHD or another jurisdiction if they were not personally involved with the vector control program or a program did not exist at their department. Additionally, internet searches identified LHD vector control programs in geographic areas were not represented by the EPHLI alumni.

While mosquito control agencies or districts not operated by an LHD serve many jurisdictions across the nation, this initiative focused specifically on LHD vector control programs that may address a wider range of vectors and provide a broader scope of public health services. Two of the LHD vector control programs that participated in the initiative were collocated with a mosquito control agency in their jurisdiction. The vector control programs that took part in the initiative represented different regions of the country and variation in population size served, organizational characteristics, and services. For example, two vector control programs were located in the Midwest, three in the Northeast, eight in the South, and one in the West U.S. Census Regions. A limited number of LHD vector control programs were identified in the West, which may have resulted from a seemingly high prevalence of mosquito control agencies in this region. One vector control program served a population <100,000, five served populations of 100,000–500,000, four served populations of 500,001–1,000,000, and four served populations >1,000,000. Four programs had budgets <$1,000 per 10,000 population, five between $1,000 and 9,999, three between $10,000 and 20,000, and one had a budget of > $20,000 per 10,000 population (data were missing for one program). Nine programs had less than one vector control full-time equivalent (FTE) per 100,000 population, four had 1–3 FTEs per 100,000, and one program had approximately five FTEs per 100,000. The scope of services provided by the vector control programs varied, yet mosquito-related services and activities were common among all the participating programs.

Performance assessment

Each of the 14 LHD vector control programs conducted self-facilitated EnvPHPS assessments in a group setting, with participation from three to eight staff members. The EnvPHPS and supporting resources, including a tool for analyzing results, are publicly available on the CDC website.¹³ An instructional webinar provided each program with guidance on using the EnvPHPS and conducting the performance assessment. During the assessment, staff members engaged in discussion while considering and responding to 64 measures organized by EnvPHPS model standards that correspond to the 10 EEPHS (see Table 1 for a list of the 10 EEPHS and corresponding EnvPHPS Model Standards). Responses were based on a scale of five activity levels: no activity (0%), minimal activity (>0%–25%), moderate activity (>25%–50%), significant activity (>50%–75%), and optimal activity (>75%–100%). Each vector control program assigned a recorder that indicated the responses on a data collection instrument and noted major discussion points during the self-assessment for later reference when reviewing results and determining QI project topics. The programs emailed the completed instruments to the PHF project manager for entry into the EnvPHPS tool for analyzing results, which computed an overall activity level as a percentage for each EEPHS. Assessment results were provided to the vector control programs for their review and consideration, as they entered the QI phase of the initiative. EnvPHPS performance assessment results for all 14 vector control programs were aggregated to reveal average activity levels (reported as a percentage) according to each EEPHS.

Table 1.

10 Essential environmental public health services and corresponding environmental public health performance standards model standards.

EEPHSa	ENVPHPSb MODEL STANDARDS
1. Monitor environmental and health status to identify and solve community environmental public health problems	1.1: Community Environmental Health Profile 1.2: Current Technology for Data Collection, Storage, and Analysis 1.3: Enhanced Environmental Health Databases and Plan
2. Diagnose and investigate environmental public health problems and health hazards in the community	2.1: Identification and Surveillance of Environmental Health Threats 2.2: Investigation and Response to Environmental Health Threats and Emergencie 2.3: Laboratory Accessibility
3. Inform, educate, and empower people about environmental public health issues	3.1: Health Education, Health Promotion and Health Communications 3.2: Crisis Communications
4. Mobilize community partnerships and actions to identify and solve environmental health problems	4.1: Constituency Development 4.2: Community Partnerships
5. Develop policies and plans that support individual and community environmental public health efforts	5.1: Governmental Presence at the Local Level 5.2: Public Health Policy Development 5.3: Community Environmental Health Improvement Process and Strategic Planning 5.4: Plan for Environmental Health Emergencies
6. Enforce laws and regulations that protect environmental public health and ensure safety	6.1: Review and Evaluation of Current Laws, Regulations and Ordinances 6.2: Involvement in Improvement of Laws, Regulations and Ordinances 6.3: Enforcement of Laws, Regulations, and Ordinances
7. Link people to needed environmental public health services and assure the provision of environmental public health services when otherwise unavailable	7.1: Identification of Environmental Public Health Service Needs of the Population 7.2: Assuring the Linkage of People to Environmental Public Health Services
8. Assure a competent environmental public health workforce	8.1: Workforce Assessment, Planning and Development 8.2: Environmental Health Workforce Standards 8.3: Life-Long Learning through Continuing Education, Training, and Mentoring 8.4: Environmental Health Leadership Development
9. Evaluate effectiveness, accessibility, and quality of personal and population-based environmental public health services	9.1: Evaluation of Environmental Public Health Services 9.2: Evaluation of Local Environmental Health System
10. Research for new insights and innovative solutions to environmental public health problems	10.1: Fostering Innovation 10.2: Linkage with Institutions of Higher Learning or Research

Notes:

^aEssential environmental public health service.

^bEnvironmental public health performance standards.

Quality improvement

Between January 1 and May 31, 2015, the vector control programs completed QI projects as QI experts provided guidance and consultation during four webinars, regular phone discussions, and two day-long site visits. PHF’s QI experts are individuals who have served in senior leadership roles in public health, health care, and international business, are or have been faculty at renowned universities, and have extensive experience developing, refining, and improving QI tools and techniques for public health practitioners. Each program received two copies of the Public Health Quality Improvement Encyclopedia, written by PHF’s QI experts, which provides explanations and examples of 75 QI tools.¹⁴ One of these experts assisted each vector control program with a review of their EnvPHPS performance assessment results, identifying and prioritizing improvement areas according to the 10 EEPHS and implementing one QI project to address the program’s top priority. Programs were encouraged to choose a project that could be completed within the initiative time frame. The QI experts then facilitated the use of QI tools such as Gantt charts, cause-and-effect diagrams, flow charts, and a population health driver diagram framework to assist programs leverage partnerships. The QI experts assisted with the establishment of goals and objectives that programs sought to accomplish over the five-month project period.

Results

Performance assessment

Aggregate EnvPHPS performance assessment results of all 14 programs indicated the average level of activity for each of the 10 EEPHS (see Table 2 for aggregate EnvPHPS performance assessment results). On average, vector control programs performed higher in EEPHS 2 diagnose and investigate (µ = 76.3%, SD = 11.7) and EEPHS 3 inform, educate, and empower (µ = 71.1%, SD = 18.9). Lower assessment results were identified with EEPHS 1 monitor environmental and health status (µ = 43.3%, SD = 30.1) and EEPHS 9 evaluate effectiveness, accessibility, and quality (µ = 43.4%, SD = 26.9). A moderate (>25%–50%) to significant (>50%–75%) level of activity was noted for EEPHS 7 link to services (µ = 47.9%, SD = 27.7), EEPHS 10 research (µ = 53.6%, SD = 18.5), EEPHS 6 enforce laws and regulations (µ = 53.8%, SD = 28.9), EEPHS 4 mobilize partnerships (µ = 56.9%, SD = 26.9), EEPHS 5 develop policies and plans (µ = 57.1%, SD = 15.7), and EEPHS 8 assure a competent workforce (µ = 62.0%, SD = 21.0).

Table 2.

Aggregate performance assessment results for 14 local health department vector control programs using the environmental public health performance standards.

	EEPHSa	MEAN (%)	SD	RANGE (%)
1.	Monitor status	43.3	30.1	0–95
2.	Diagnose & investigate	76.3	11.7	52–98
3.	Inform, educate, empower	71.1	18.9	38–100
4.	Mobilize partnerships	56.9	26.9	19–100
5.	Develop policies & plans	57.1	15.7	29–81
6.	Enforce laws & regulations	53.8	28.9	3–100
7.	Link to services	47.9	27.7	8–100
8.	Assure competent workforce	62.0	21.0	34–100
9.	Evaluate	43.4	26.9	4–100
10.	Research	53.6	18.5	25–88

Note:

^aEssential environmental public health service.

Quality improvement

The vector control programs completed QI projects focused on a range of topics that spanned from increasing community outreach and educational efforts to decreasing the time required for resolving complaints. The majority of projects involved mosquito-related services and activities, while one project broadened a community-based rodent survey to include mosquito control aspects and another reviewed rabies investigation processes to examine staff workloads. All vector control programs made accomplishments, yet not all fully achieved established QI project goals and objectives within the initiative time frame. Because the QI projects sometimes reflected more than one EEPHS, the authors reviewed each of the projects and agreed upon the primary EEPHS addressed. Overall, the projects addressed seven of the 10 EEPHS. EEPHS 5 (develop policies and plans), EEPHS 6 (enforce laws and regulations), and EEPHS 10 (research) were not addressed. Projects most frequently addressed EEPHS 2 (diagnose and investigate), EEPHS 4 (mobilize partnerships), and EEPHS 9 (evaluate effectiveness, accessibility, and quality). All 14 programs strengthened existing or newly built partnerships internal or external to the LHD (see Table 3 for a description of the QI projects).

Table 3.

Local health department vector control program quality improvement project descriptions.

OBJECTIVE	ACCOMPLISHMENTS	PARTNERSHIPS	PRIMARY EEPHS ADDRESSEDa
Broaden an annual community-based rodent survey to include inspections of mosquito breeding areas and educating residents about vector and household pest control	Used the expanded survey for exterior inspection of >900 homes and 50 home interior inspections	LHDb Healthy Homes Program	1
Build lab capacity within the LHD to detect arboviral disease and to increase the effective rate for results (receiving results within four days of mosquito collection) Improve mosquito control program data reporting systems to decrease complaint response time	Built lab capacity with the goal of reaching 85% effective rate. 10% was achieved at conclusion of the project with intentions to continue to increase the rate Increased complaint data availability and decreased average complaint resolution time by 18 days compared with 25 days during the previous year	State department of health LHD information technology program	2
Increase public availability of adult mosquito West Nile virus surveillance data	Completed a database for surveillance data with plans to develop a website interface for public use	LHD surveillance and informatics program, community officials	2
Educate tire dealers about proper tire storage to eliminate collection of water and prevent mosquito breeding	Used the protocol to conduct inspections and provide education to >175 tire dealerships and related businesses	Tire dealerships and businesses, city code enforcement, county environmental officer	3
Create a multi-agency vector-borne borne disease taskforce	Identified members and established a taskforce that was expanded as a state-wide arbovirus network	Various state, tribal, local and federal agencies (CDC, border health, and USDA)	4
Create partnerships with community organizations to increase public messaging and outreach concerning mosquito breeding sites	Disseminated information at public events, distribution of flyers, and press releases, and radio and news interviews	City staff, public radio station, event organizers, and seven community organizations	4
Increase collaboration among partners and stakeholders to improve communication about mosquito control and mosquito-borne illness	Improved communication between stake-holders through development of flowcharts and flyers about mosquito-borne illness	LHD environmental health, county mosquito control section, healthcare facilities	4
Identify vulnerable populations with higher risk of exposure to mosquito-borne illness and develop ways to address barriers to services	Compared geographic and population data with areas of high mosquito presence to identify vulnerable populations. Outreach about the vector control program and mosquito-borne illness will be enhanced to the identified populations	Other LHD programs, city agencies, local medical reserve corps	7
Examine vector control program capacity and staffing to address an increasing workload	Analyzed time spent on rabies investigations and reviewed resulting increased workload. Submitted request for an additional FTE and increased capacity for tracking time spent on vector-borne disease related investigations	LHD communicable disease staff	8
Increase the number of mosquito control program staff with pesticide application certification	Supported and encouraged staff to seek and obtain certification. Work was underway to meet a goal of 75% of staff credentialed as the project concluded	LHD staff and human resources department	8
Conduct a community survey to assess satisfaction and knowledge of the mosquito control program and use the results to inform future activities	Distributed a survey to 800 residents with a goal of 20% response rate. Achieved 44% and used results to improve services, such as the timing of pesticide applications	State department of health, another LHD	9
Institute a process to measure mosquito control program effectiveness	Developed a dataset including variables such as weather, public complaints, mosquito pool counts, West Nile virus human cases and positive mosquitoes, and mosquito treatment data. Data were analyzed and results informed selection of effective treatment options	LHD data management program	9
Conduct community survey to determine effectiveness of aerial mosquito spraying notifications	Distributed the survey to approximately 900 residents with a goal of 10% response rate. Achieved 31% and planned to analyze results following conclusion of the project period	LHD emergency preparedness coordinator, another LHD	9

Notes:

^aEssential environmental public health service.

^bLocal health department.

Discussion

Aggregate performance assessment results of all 14 vector control programs indicated the highest levels of performance in EEPHS 2 (diagnose and investigate) and EEPHS 3 (inform, educate, and empower). This likely represented the emphasis that vector control programs placed upon surveillance as related to EEPHS 2 and outreach and community awareness efforts pertaining to EEPHS 3. EEPHS 1 (monitor environmental and health status) and EEPHS 9 (evaluate effectiveness, accessibility, and quality) revealed the lowest performance levels. Three QI projects addressed EEPHS 9 by emphasizing the evaluation of community satisfaction and effectiveness of services. Lower assessment results in EEPHS 9 may show a need for increased support for evaluating vector control services and determining their efficacy. Vector control programs interested in addressing EEPHS 1 may benefit from efforts to enhance capabilities for conducting community assessments and utilizing technologies such as geographic information systems to monitor vector concerns. The absence of QI projects addressing EEPHS 5 (develop policies and plans), EEPHS 6 (enforce laws and regulations), and EEPHS 10 (research) may reveal lower priority assigned to improve related vector control services, perhaps influenced by higher levels of activity as aggregate results were significant (>50%–75%) for each of these EEPHS. Nearly all of the QI projects focused on mosquitoes. The LHD vector control program that did not address a topic related to mosquitoes was collocated with a mosquito control agency. The presence of the mosquito control agency may have contributed to the LHD vector control program’s decision to emphasize a QI project that did not focus on mosquitoes.

Performance assessment results varied among the programs, which may have resulted from wide variation in population size served and organizational characteristics (eg, staffing levels and funding). Participation was voluntary, and results cannot be generalized to represent all U.S. LHD vector control programs. Effective prioritization of assessment results and detailed project planning were essential because of the short five-month duration of the project period. Time limitations, especially as the spring months and peak vector season approached, likely influenced project selection in favor of approaches that could be accomplished within the initiative time frame and supported by existing financial and staffing resources. This situation may have caused programs to avoid addressing EEPHS 1 and EEPHS 9 with monitoring and evaluation-related QI projects.

In the absence of funding to participate in the initiative, the dedicated programs were able to identify means for improvement within the boundaries of available resources. LHD vector control programs may be able to conduct performance management activities, similar to this initiative, without additional funding to support their efforts. All programs indicated that engaging in this initiative resulted in stronger collaboration with partners within the LHD, other governmental agencies, and private entities. This result could be particularly important for supporting collaborative approaches between LHD vector control programs and mosquito control agencies when these two types of organizations are collocated within a jurisdiction.

This performance management initiative provided insight into vector control program performance according to the 10 EEPHS and demonstrated how the EnvPHPS can serve as a performance assessment framework for environmental public health programs, such as vector control, to produce results for informing QI projects. The QI experts provided valuable guidance for the vector control programs; however, programs undertaking similar performance assessment and QI improvement activities may leverage the multitude of resources publicly available online or in references such as the PHF QI encyclopedia. LHD vector control programs interested in performance assessment and improvement may benefit from engaging in an approach similar to this performance management initiative.