Laboratory Services in Support of Public Health: A Status Report

SYNOPSIS

Objectives

To assess Healthy People 2010 Objective 23-13 and its related sub-objectives measuring comprehensive laboratory services in support of essential public health programs, the Association of Public Health Laboratories (APHL) collaborated with the Centers for Disease Control and Prevention (CDC) to create and administer a survey of state public health laboratories (PHLs).

Methods

A committee of APHL, with representation from CDC, constructed the survey based on the 11 Core Functions of State Public Health Laboratories (hereafter, Core Functions)—the premise being that the extent to which they fulfilled these Core Functions would represent their level of providing or assuring comprehensive laboratory services in support of public health. The survey was distributed biennially to all state health agencies from 2004 to 2008, and respondents were given two months to complete it.

Results

The response rate for all surveys was ≥90.2%. State PHLs were more likely to meet the sub-objectives relating to traditional functions (e.g., disease surveillance and reference testing) than other areas (e.g., food safety and environmental testing). Emergency preparedness fell in between. Overall, but most notably in the areas of food safety and training and education, there was improvement from 2006 to 2008, with the percentage of respondents who met more than half of the sub-objectives increasing from 58.7% in 2006 to 61.2% in 2008.

Conclusions

The comprehensive laboratory services survey has been a valuable tool in measuring the laboratory infrastructure that underpins public health in the U.S. It will be necessary to continue monitoring laboratory infrastructure in this way to determine where the gaps in services exist and how they can best be addressed.

When Healthy People (HP) 2000 was promulgated by the U.S. Department of Health and Human Services (HHS) in 1991, it contained no focus area on public health infrastructure.¹ As a result, it did not include objectives measuring critical components of public health, such as laboratory services. During the 1990s, there was a collective effort to formally define public health practice in the U.S., culminating in the 10 Essential Public Health Services (hereafter, Essential Services).² It became clear that to address the Essential Services, a robust public health infrastructure was required. In 1992, the Association of State and Territorial Public Health Laboratory Directors (now the Association of Public Health Laboratories [APHL]) published Laboratory Initiatives for the Year 2000 (LIFT 2000),³ which demonstrated that all Essential Services required laboratory infrastructure.

As a result of these efforts, HP 2010 included a section (focus area 23) on public health infrastructure.⁴ Within focus area 23 was an objective (HP 23-13) to “increase the proportion of tribal, state, and local health agencies that provide or assure comprehensive laboratory services to support essential public health services.” At the outset, this objective was identified as a developmental—rather than measurable—objective, as there was no data source available to measure progress. Among the organizations identified as potentially being able to develop a tool for measuring comprehensive laboratory services were APHL and the Centers for Disease Control and Prevention (CDC). In 2002, a committee of APHL, including representation from CDC's Division of Laboratory Systems (DLS), began to devise a survey that could be used to transform HP 23-13 from a developmental objective to a measurable objective.

In 2002, APHL and CDC co-published a report⁵ defining the 11 Core Functions and Capabilities of State Public Health Laboratories (hereafter, Core Functions). The APHL committee, in continued collaboration with CDC DLS, proposed that measuring the extent to which state health agencies were fulfilling those Core Functions would be a reasonable metric for the provision or assurance of comprehensive laboratory services. The committee also determined it would be more appropriate to measure each of the Core Functions independently. This resulted in a request to HHS, which oversees the HP process, to add 11 sub-objectives under HP 23-13. At that time, it was also clear that no comparable description of the Core Functions had ever been established for the large, diverse population of local public health laboratories (PHLs) in the United States. As a result, a modified HP 23-13 was written, removing the term “local” and adding 11 sub-objectives corresponding to the 11 Core Functions.

In 2004, when HHS had approved the modifications, APHL conducted the first Comprehensive Laboratory Services Survey (CLSS), the results of which have been published.⁶ A major finding was that many states did not meet several sub-objective goals, including those for food safety, emergency response, and environmental health and protection. Given there had been no prior survey tool to measure comprehensive laboratory services, the 2004 version of the CLSS served well as a pilot. The planned collection and comparison of data on a biennial basis would allow for the measurement of progress toward HP 23-13 throughout the decade. Results for all HP objectives are available through the CDC Data 2010 website and are updated as new data are made available.⁷

Following a review of CLSS 2004's results, and in preparation for conducting CLSS 2006, the committee completed a thorough analysis of the survey instrument and feedback from the respondents. The analysis revealed that some survey questions were unclear and needed revisions, some terms used in the survey were not consistently understood, and the process for scoring the responses required changes. In addition, CLSS 2004 did not distinguish between providing or assuring specific laboratory services. The result of this analysis was a significantly improved CLSS 2006. The challenge of trying to reconcile the two surveys proved so formidable that ultimately the data collected for CLSS 2004 were used in a stand-alone format and the data obtained from CLSS 2006 were established as new baseline data. A third survey conducted in 2008, essentially equivalent to CLSS 2006, allowed for a comparison between the findings of those two years. This article compares the progress (or lack thereof) made within the 11 sub-objectives from 2006 to 2008.

METHODS

The new version of HP 23-13, revised in 2004, included 11 sub-objectives, which corresponded to the 11 Core Functions (Figure 1).

Figure 1.

Healthy People 2010 Objective 23-13^a and its 11 sub-objectives^b

^aRevised in 2004

^bThe Association of Public Health Laboratories used these sub-objectives to measure comprehensive laboratory services in support of public health programs in its 2004, 2006, and 2008 Comprehensive Laboratory Services Surveys.

The original CLSS 2004 contained 89 questions covering the 11 sub-objectives. The questions were established after first searching for applicable and accepted standards of practice for each Core Function being measured. The committee also reviewed existing survey instruments for potential questions that may have been used previously by APHL or other organizations to assess specific types of laboratory services (e.g., food safety or emergency preparedness). Ultimately, the 89 questions used were selected as representative indicators of performance within each sub-objective. The number of questions per sub-objective ranged from two to 22, with a mean of eight. For many questions, multiple items were scored. The committee determined whether a responding state PHL met the standard of performance within a given sub-objective by calculating a numerical score. The traditionally accepted academic score of 70% (equivalent to a letter grade of C) was used as the cutoff for “minimally fulfilling the Core Function” and thereby meeting the representative sub-objective. Each sub-objective was scored independently.

In CLSS 2006, a concerted effort was made to give more balance to the assessment of all 11 sub-objectives and to improve the scoring process. CLSS 2006 had a total of 84 questions spread among the 11 sub-objectives. The number of questions per sub-objective ranged from one to 22, with a mean of eight. Questions with multiple components were assigned a score value proportionate to their complexity. The mean potential point total per sub-objective was 11, with a range of five to 25. This modification of scoring gave better balance to all sub-objectives. Ultimately, CLSS 2006, compared with CLSS 2004, balanced the assessment of all sub-objectives, used clearer wording through the inclusion of a glossary, and distinguished between providing and assuring a laboratory service.

The modifications made in CLSS 2008 compared with CLSS 2006 were minimal. CLSS 2008 had a total of 83 questions, ranging from one to 29 questions per sub-objective, with a mean of eight. As with CLSS 2006, the method of scoring for the 11 sub-objectives was more balanced. The total potential score for CLSS 2008 was 124. The mean potential point total per sub-objective was 11, with a range of six to 26.

The original results from CLSS 2004 yielded baseline values for each sub-objective. The committee established targets for each sub-objective based on expert opinion—an acceptable target-setting method recognized by HHS in the absence of previously established targets or national standards. The targets represented the collective views of the APHL committee, which comprised senior laboratory scientists including CDC representation. After the committee determined that results of CLSS 2004 could not be compared with those of CLSS 2006, it created new baseline values using the 2006 data. With new baselines came the need for new targets. Under the guidance and direction of the HP 2010 Steering Committee, it was determined that the proportional approach—the committee's standard method for adjusting targets—should be used to create new HP 23-13 targets. For each sub-objective, a proportion (ratio) was determined based upon the ratio between the previous (expert opinion) targets and the 2004 baseline for that particular sub-objective. The proportionality ratio was then multiplied by the new 2006 baselines. APHL and CDC maintained that no sub-objective should have a target less than 50%. The HP 2010 Steering Committee concurred. Thus, all revised HP 2010 targets now range between 50% and 100% for each sub-objective.

In both 2006 and 2008, the committee beta tested the survey among a small group of state laboratory directors who completed the survey and provided feedback. Following minor corrections and clarifications, the surveys were then distributed to all 50 state PHLs, plus the District of Columbia PHL. Collection of data was completed within two months of distribution.

For the sake of comparing results, the committee grouped laboratories into three cohorts based on the number of full-time equivalent (FTE) employees. Laboratories were categorized as small (23–73 FTE employees), medium (74–140 FTE employees), or large (>140 FTE employees). Individual state performance reports were provided offering comparisons within the respective cohorts. APHL compiled the results and distributed findings to each laboratory director. Each state's performance was reported confidentially to that state only. Aggregate results without state identifiers were compiled to measure the overall performance of the National Laboratory System as it collectively supports the Essential Services.

RESULTS

The committee distributed CLSS 2006 to 50 state PHLs and the District of Columbia PHL. Forty-six laboratories responded, yielding a response rate of 90.2%. CLSS 2008 was distributed to the same 51 laboratories, and 49 laboratories (96.1%) responded. Forty-four laboratories (86.3%) responded to both surveys. Figure 2 depicts the CLSS 2006 results, the CLSS 2008 results, and the 2010 targets for each sub-objective in HP 23-13 based on the percentage of all states (irrespective of size) meeting each sub-objective.

Figure 2.

Progress toward HP 2010 Objective 23-13 goals, by sub-objective, among state public health laboratories responding to the Association of Public Health Laboratories' CLSS in 2006 and 2008

CLSS = Comprehensive Laboratory Services Survey

HP = Healthy People

Based on the results of CLSS 2006, the state health agencies and their laboratories collectively demonstrated their greatest strengths (more than two-thirds met the sub-objective) in the areas of:

• Disease prevention, control, and surveillance;

• Reference and specialized testing; and

• Emergency response.

They were less strong (one- to two-thirds met the sub-objective) in the areas of:

• Integrated data management,

• Environmental health and protection,

• Laboratory improvement and regulation,

• Policy development, and

• Partnerships and communications.

The weakest areas (fewer than one-third met the sub-objective) were:

• Food safety,

• Public health-related research, and

• Training and education.

Based on the findings of CLSS 2008, the state health agencies and their laboratories collectively were strongest (more than two-thirds met the sub-objective) in the areas of:

• Disease prevention, control, and surveillance;

• Reference and specialized testing; and

• Policy development.

Overall, state PHLs were not as strong (one- to two-thirds met the sub-objective) in the areas of:

• Integrated data management,

• Environmental health and protection,

• Laboratory improvement and regulation,

• Emergency response,

• Training and education, and

• Partnerships and communications.

The weakest areas for state PHLs (fewer than one-third met the sub-objective) remained:

• Food safety and

• Public health-related research.

Comparing the results of CLSS 2006 with those of CLSS 2008 reveals that, for most sub-objectives, the changes in the percentage of states meeting the respective sub-objectives were only slight to moderate. In the case of two sub-objectives, however, there was a notable shift in the positive direction. These two sub-objectives measured the Core Functions related to food safety and training and education. Of the sub-objectives that shifted in the negative direction, the most notable was the change in emergency response.

In addition to determining that the collective performance of state health agencies and their laboratories maintained or improved their status within the 11 sub-objectives for HP 23-13, the two surveys allowed us to determine whether performance was improving at the individual state level. The Table compares the number of states that met between zero and 11 sub-objectives in 2006 compared with 2008. In 2006, 27 of 46 respondents (58.7%) met more than half of the sub-objectives, compared with 30 of 49 respondents (61.2%) in 2008.

Table.

Number of Healthy People 2010 Objective 23-13 sub-objectives met by state public health laboratories responding to the Association of Public Health Laboratories' Comprehensive Laboratory Services Surveys in 2006 and 2008

DISCUSSION

The CLSS, developed by APHL in collaboration with CDC DLS, is a valid means of measuring HP 23-13. It has been administered twice since the 2004 pilot test, with modifications and enhancements to the survey instrument in 2006. As a result, only the data collected in 2006 and 2008 can be compared for the purpose of measuring progress toward meeting HP 23-13 and its 11 sub-objectives.

HP 23-13 originally stated that the objective was to measure “comprehensive laboratory services to support essential public health services.”⁴ Given that there was an established set of Core Functions published in 2002, and a similar set of functions was not available to measure performance at the local or tribal level, HP 23-13 has focused on state health agency performance only since 2004. APHL has previously reported on the intersection between the Core Functions and the Essential Services.⁸ Comprehensive laboratory services represent more than just laboratory testing and analyses. Although these are key to comprehensive laboratory services, it is clear that other activities, such as policy development and partnerships, are also critical.

The data collected in 2006 and 2008 reveal that the provision or assurance of comprehensive laboratory services varies by type of service (Core Function) and state. Services also varied somewhat, depending upon the size of the state PHL. Although there was no discernable pattern when comparing the size of the state PHL with the likelihood of meeting particular sub-objectives (data not shown), it is appropriate to continue to evaluate services offered compared with state laboratory size in future surveys. A traditional strength of state PHLs—disease prevention, control, and surveillance—was identified as strong in both 2006 and 2008, despite a slight drop in the percentage of states that met this sub-objective. Another area of strength for state PHLs has been reference and specialized testing. Although it is recognized that this service may not be as strong nationally, given that smaller state laboratories cannot meet all reference testing needs, the data still show that two-thirds of states met the standard in 2006, and 77.6% met it in 2008. This is close to the 2010 target of 82.0%.

In other sub-objectives, the states did not fare as well. Some of the sub-objectives, while appearing to be less critical than testing capabilities, represent the underpinnings that influence a state's ability to meet overall laboratory service needs. For example, sub-objective b measures integrated data management, sub-objective f measures laboratory improvement and regulation, and sub-objective j measures training and education. In each of these areas, there was slight progress toward meeting the 2010 targets, although none of the targets for these sub-objectives is 100%.

Two areas previously identified as a concern⁶ that persist are food safety and environmental health and protection. In most states, the responsibility of food safety is spread across several agencies, as it is at the federal level. As a result, the state PHL does not necessarily have the responsibility to provide or assure laboratory services in this area. This may lead to service gaps, which must be met through better partnerships and coordination. Despite the fact that fewer than half of the states met the food-safety sub-objective in 2008, there was still a notable improvement in the number of states that met the sub-objective compared with 2006; but there is still a great distance to meet the 2010 target. It is reasonable to assume that food-safety initiatives such as FoodNet and PulseNet, as well as the testing requirements incorporated into the bioterrorism grants, have contributed to this improvement.^9,10 In the area of environmental health and protection, most states have a separate environmental protection agency. As with food safety, it will be necessary for state health agencies and their laboratories to continuously assure that stronger partnerships and coordination of services reduce the likelihood of environmental vulnerability with respect to laboratory services.

Two areas that have not been the traditional focus of state PHLs are policy development and public health research. The laboratory leadership in each state has the potential to impact health policy if they find ways to become part of the policy-development process within their states. With respect to research, the state PHLs need to tap into the ongoing research agendas within their respective state government structure and the local academic community. Such engagement can prove reciprocally beneficial, particularly as we move toward a focus on clinical and translational research.

Emergency response was recognized as a critical role for PHLs after the anthrax events of 2001 and the subsequent focus on biological, chemical, and radiological agents of terrorism. The overall performance in this area of comprehensive laboratory services diminished somewhat from 2006 to 2008. It is unclear what caused this apparent diminution of emergency response services; however, recent reductions in funding of laboratory infrastructure at both the state and federal level may have contributed. It is imperative that emergency response services remain a strong and well-supported part of a laboratory's infrastructure.

One area of progress that particularly bodes well for public health is the sub-objective related to partnerships and communication. It has been recognized for a decade or more that state PHLs need to create and strengthen laboratory systems within each state. Indeed, State Public Health Laboratory Systems must encompass their established Core Functions, as well as the functions and contributions of local PHLs and other system partners. To better effect these changes, states have been encouraged to create Laboratory Program Advisor¹¹ positions to conduct outreach and link state health needs with the greater statewide community.

As we proceed toward 2020, it is clear that the CLSS findings should guide policy and funding decisions as they relate to comprehensive laboratory services nationally. As we enter a new decade, it is important to not only continue to measure comprehensive laboratory services, but also find ways to measure how different state laboratory systems function, what factors influence their ability to provide vs. assure laboratory services, and which are the best mechanisms for measuring the quality of laboratory services delivered.

CONCLUSIONS

The CLSS has been administered three times and is a valid means of measuring HP 2010 Objective 23-13. It has helped to identify areas of comprehensive laboratory services that are relatively strong, as well as areas that are relatively weak. Some areas have shown slight improvement from 2006 to 2008, and it will be important to see where those specific sub-objectives are, relative to their targets, in 2010. The CLSS monitoring of PHL infrastructure within the U.S. should prove beneficial with regard to influencing public health policy and assisting with decision-making regarding resource allocation.