Benefits and Barriers to Electronic Laboratory Results Reporting for Notifiable Diseases: The New York City Department of Health and Mental Hygiene Experience

Abstract

Objectives. Despite national support for electronic laboratory reporting (ELR), the transition from paper to electronic reporting has been slow both nationally and locally. We assessed the ELR experience of New York City’s surveillance programs to identify barriers to ELR implementation and generalizable lessons about automated electronic notifiable disease surveillance.

Methods. We conducted interviews with key staff of the New York City Department of Health and Mental Hygiene to evaluate ELR implementation. A review of paper and ELR disease reports enabled a comparison of the reporting systems.

Results. The completeness and timeliness of ELR were similar to, and sometimes better than, paper reporting for certain diseases. Incorporating electronic data into surveillance databases created new problems with data quality, shifted work demands, and required additional skills for data monitoring. ELR improved the handling of high-volume and time-sensitive diseases but did not completely automate reporting for diseases that required complicated assessments by staff.

Conclusions. Although ELR streamlines data processing, electronic reporting has its own limitations. A more successful use of ELR can be achieved by understanding its strengths and limitations for different disease types.

To identify outbreaks and prevent transmission of communicable diseases, accurate, complete, and timely reporting of notifiable diseases is critical, but traditional methods of paper reporting are slow and depend on active participation of laboratory and clinical staff. Electronic laboratory reporting (ELR) can increase the volume of tests reported to surveillance programs and improve the timeliness and completeness of reports.1^–3 Because of the promise of increased efficiency, ELR has been promoted as being integral to improving disease surveillance.4

Over the past decade, much progress was made in understanding and addressing the challenges of translating local laboratory codes into public health standards5 and creating secure systems for electronic data transmission. To accelerate adoption of ELR, the Centers for Disease Control and Prevention advanced standards for vocabulary, format, and messaging; funded the development of software; and conducted an extensive outreach campaign to state and local health departments to increase use of the software. Despite these efforts, the transition from paper to electronic reporting has been slow both nationally and locally. By April 2005, nearly 5 years after states received federal funding to initiate plans for ELR, only 35 state and 2 municipal health departments had begun developing or implementing an electronic system for disease surveillance.6

Comparisons between electronic and paper reporting have focused on expected improvements to the completeness and timeliness of reporting.7^–11 Less attention has been paid to understanding the barriers and benefits of adopting ELR by health departments and, in particular, the challenges of incorporating electronic data into surveillance databases in an automated fashion. The experience of the New York City Department of Health and Mental Hygiene (NYC DOHMH) can be illustrative. In 2002, NYC DOHMH began receiving electronic reports through the Electronic Clinical Laboratory Reporting System (ECLRS). Because of concerns about the quality and completeness of ECLRS, laboratories were required to undergo a certification process during which they temporarily sent electronic and paper reports in tandem, for comparison, before any electronic reports were uploaded into routine surveillance databases. Beginning in July 2006, the NYC Board of Health legally mandated ELR of notifiable diseases to NYC DOHMH. We sought to assess the benefits and disadvantages of ECLRS experienced during the premandate time frame from the perspective of the NYC DOHMH disease surveillance programs.

METHODS

We conducted in-depth interviews with key NYC DOHMH informatics and surveillance staff to elicit their experiences in using ECLRS and certifying licensed clinical laboratories conducting tests for patients residing in New York City or seen at New York City hospitals. Of those laboratories for which a sample of paper and electronic reports were compared, we examined the data to measure the completeness and timeliness of paper versus ECLRS reporting for communicable diseases, sexually transmitted diseases (STDs), and tuberculosis (TB).

NYC DOHMH staff feedback regarding ECLRS certification and implementation was summarized according to the challenges that they experienced and the benefits to surveillance that they identified. Completeness was assessed first as the coverage of electronic reporting, which is the proportion of laboratories certified to use ECLRS by the communicable disease, STD, and TB surveillance programs for individual diseases. Because laboratories vary in the amount of testing that they conduct, we also calculated the proportion of all of the laboratory reports received through ECLRS as a second measure of the completeness of ECLRS coverage. Timeliness of reporting was measured as the time between the specimen collection date and the date when the health department was notified of a positive result.

RESULTS

Completeness of ECLRS Coverage and Timeliness

As of August 2005, a total of 106 laboratories (62 hospital based and 44 commercial) performed tests for communicable diseases, STDs, and TB that were reportable to NYC DOHMH. Nearly one third of all of the laboratories were ECLRS-certified for STD reporting, a higher proportion than for CD or TB reporting (Table 1▶). ECLRS coverage of STD case reports was more complete than coverage of communicable disease or TB reports (Table 2▶). Compared with paper reporting, electronic reporting was almost as complete (only missing few reports that were submitted on paper), and, for certain diseases (e.g., giardia and salmonella), included 70% to 76% more cases than had been submitted on paper. Overall, ECLRS reports reached NYC DOHMH faster (median: 6 days from date of specimen collection) than paper reports (median: 25 days). The median improvement was 11 days (range: 3–42 days). After arrival at NYC DOHMH, data were immediately available for upload, although some data uploading was delayed if data had to first be transformed to meet the specifications of a particular surveillance registry. Regardless, for priority diseases, including rare illnesses, automated alarms signaled staff to immediately review specific reports.

TABLE 1—

Proportion of Laboratories Certified to Report Communicable Diseases, STDs, and TB Through ECLRS as of August 2005: New York City

Surveillance program	Total No. of Laboratories	No. of ECLRS-Certified Laboratories	Proportion of ECLRS-Certified Laboratories, %
Communicable Diseases	66	12	18
STDs
Chlamydia	45	14	31
Gonorrhea	49	16	33
Syphilis	53	16	30
TB	37	2	5

Note. STD = sexually transmitted disease; TB = tuberculosis; ECLRS = Electronic Clinical Laboratory Reporting System.

TABLE 2—

Proportion of Laboratory Reports for Communicable Diseases, Chlamydia, Gonorrhea, and TB Transmitted Through the ECLRS: New York City, April 1, 2004–June 30, 2005

Reported Disease	No. of Case Reports	No. of ECLRS-Submitted Case Reports	ECLRS Cases, %
Communicable Diseases	25 726	9940	39
Chlamydia and Gonorrhea	12 174	6206	51
TBa	574	16	3

Note. TB = tuberculosis; ECLRS = Electronic Clinical Laboratory Reporting System.

^a TB data are from January 1, 2005, through July 31, 2005.

Experiences With Certification

Although the initial goal of the ECLRS certification process was to ensure that the completeness, timeliness, and accuracy of electronic reporting was equivalent to or better than paper reporting, staff soon recognized that the process of providing feedback to the laboratories was a rare opportunity to negotiate improvements in the level of detail of disease reports. Whereas most information legally mandated by New York State Public Health Law 2102 was reported consistently (e.g., patient name, test type, and provider name), some laboratories sent electronic reports lacking required elements that had not been consistently sent on paper (e.g., patient address). Therefore, at the risk of losing the cooperation of the laboratories, NYC DOHMH sometimes delayed certification until the electronic disease reports included more of the required information.

Laboratories were certified when the level of transmitted detail was considered sufficient for appropriate public health response. In some instances, staff would not certify a laboratory from which they needed specific details, such as patient address for field follow-up, even in cases when this information was not readily available; the possibility still existed that the laboratory could obtain the information either from the requesting provider or another database (e.g., hospital billing). Yet, other laboratories were certified without such detail if NYC DOHMH staff thought that they were getting the most information possible from a laboratory’s own database, as long as the electronic reports were at least as complete as the paper reports. This approach made certification challenging for both laboratory and health department staff.

Surveillance staff reported that certifying and monitoring laboratory data was tedious and cumbersome. Without the consistent urging and encouragement by the NYC DOHMH staff for laboratory staff to communicate and collaborate with their information technology department, test results would frequently be coded incorrectly. Even after laboratories were certified, NYC DOHMH informatics and surveillance staff had to continually monitor data for quality-assurance purposes. Although complicated, quality assurance was a process that had not been easily conducted using paper reports. Furthermore, automated electronic analyses of the quantity of laboratory reports were implemented to flag any erratic deviations from the expected number of reports, which created new tasks for health department staff and new communication needs with the laboratories.

Prioritizing Laboratory Certification for Certain Diseases

The health department staff recognized that the primary benefits of ELR were improved timeliness and automatic data upload (when possible within existing work-flows), thereby eliminating substantial data entry needs. In 2005, 65% of the 43 568 hepatitis C cases and 52% of the 35 814 chlamydia cases were entered into the health department surveillance databases through electronic reporting, which equated to 47204 reports that staff would have otherwise had to hand enter. As a result, more staff resources could be dedicated to conducting field work, which was especially important for diseases when the immediate treatment of the patients and prophylaxis of their contacts could prevent further sequelae and transmission. Thus, the surveillance programs identified high-volume diseases (e.g., hepatitis B, hepatitis C, and chlamydia) and time-sensitive diseases requiring field follow-up (e.g., hepatitis A and syphilis) as priority targets for ECLRS certification.

Limitations of Electronic Reporting

Despite enhancements to reporting, ECLRS did not equally improve surveillance of all diseases. For some diseases, such as gonorrhea, ECLRS markedly reduced work burden, because a positive test report equates to a new case. Yet, despite the reduction of data entry time for other, more complex diseases, reporting was still complicated because of the intricacies related to testing. Numerous TB tests, for example, are conducted and reported on multiple specimens (i.e., smears, cultures, rapid diagnostic tests, and susceptibility results) collected over several months, so updating case information electronically was difficult. Similarly, determining the case status of complex diseases requires comparisons with other patient data and might necessitate field follow-up to collect more information than what is reported electronically. For example, before adding a syphilis test report to the registry, staff must review multiple past tests and treatments. Therefore, for these complex diseases, ECLRS had less impact, because the work burden could not be fully automated.

ECLRS also created unique problems that could not be consistently monitored under quality-assurance protocols. False reports of rare diseases could be caused by incorrect automapping of electronic codes. For example, an electronic report for the relatively rare disease trichinosis led to an investigation of a case that turned out to be an error of reading in the common STD, trichomoniasis, which is not a reportable disease. In addition, the burden of obtaining basic patient details sometimes shifted from the laboratory staff to health department personnel when data were unavailable from electronic laboratory information systems. Finally, the lack of uniformly applied standards for representing laboratory results carried the risk that some negative or equivocal results (e.g., “positive for multiple contaminants”) could be reported through ECLRS. Validating the specificity of automated filters (software programs) that limit electronic reporting to reportable cases could be difficult for both NYC DOHMH and laboratories, particularly for high-volume conditions that are not manually reviewed or investigated.

DISCUSSION

NYC DOHMH staff appreciated the enhancements that ECLRS made to disease reporting but also recognized its limitations. Electronic reporting created new technological needs, shifted communication lines to include informatics staff, and required new staff skills in data monitoring and quality assurance. Only for certain diseases did ECLRS uniformly increase the volume of reports, reduce the time for notification, and enable automatic data uploading. The transition to electronic reporting was slow, because NYC DOHMH was cautious in certifying laboratories because of new problems experienced with ECLRS and the potential loss of necessary patient information. Therefore, to maximize the benefits of ELR, surveillance programs must have realistic expectations and allocate sufficient resources toward its implementation.

ELR does not necessarily enhance surveillance for all diseases to the same extent. The advantages are obvious for reporting diseases where volume is high and timeliness is a concern. However, raw laboratory data are not always suitable for automatic uploading into surveillance databases. In fact, some laboratories that sent complete paper reports in turn had difficulty with preparing equally complete electronic reports, often because of limited staff resources dedicated to data entry or coding. For a disease that is as complicated to report as TB,12 electronic reporting can remove 1 step of data entry but does not automate the entire reporting and classification process, partly because of the inability to automate complex judgments.13 Complete reporting requires very close collaboration between laboratory and health department staff.

Surveillance programs might want to follow the example of the certification process that NYC DOHMH undertook. Having the opportunity to educate laboratories about reporting requirements was particularly valuable. Several laboratories did not know that certain diseases were reportable, whereas many more were not clear about the details that they had to send in their reports, which in part explains the differences that we identified in completeness of reporting by disease category. Many laboratory information management systems do not include patient information required by health departments, thereby shifting labor from data entry at the laboratories to information gathering by health department staff.3 Laboratories might need additional resources (e.g., technical expertise and increased staff time) to create interfaces or change work-flows that improve their reports and address the problems that hinder their certification.

Although ELR eliminates paper burden and the need for data entry staff, additional support is required at the health department and laboratories to establish and maintain the system. Furthermore, health departments will need a higher level of technological support and skilled surveillance staff that can assure the quality of data, identify problems rapidly, and provide continual feedback to laboratories. To assure that ELR is integrated into public health surveillance workflows, close attention must be paid to understanding the limitations, as well as the benefits, of automated ELR for public health staff.