Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance

Abstract

With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013–2014 NYC Health and Nutrition Examination Survey.

NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators.

We discuss NYC’s experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.

With better sanitation, antibiotics, and improved critical care medicine, average life expectancy has greatly increased in the past century in most parts of the world. In the United States and in many countries, chronic diseases have replaced infectious diseases as leading causes of death, yet most local public health agencies have limited information that can be used to inform prevention and control efforts. One emerging source for surveillance of chronic disease risk factors and conditions is data from electronic health records (EHRs).

Although data from EHRs are collected primarily to manage individuals’ health care, they have the potential to be used for monitoring the health of populations. EHR use in the United States has increased from 21% of office-based physicians in 2004 to 87% in 2014.¹ This increased use is partly because of federal and state regulations. Federal meaningful use² incentives have encouraged adoption of EHRs and standardization of how data are entered. In New York State, electronic prescriptions are now legally mandated³ and other states are following suit. Beyond individual patient management, researchers and clinicians have mostly used EHR data for clinical quality improvement initiatives, comparative effectiveness research, and monitoring outbreaks of infectious disease.^4,5 However, in the past few years, there has been increasing interest in using EHRs for population health surveillance, that is, for monitoring the disease burden in a defined geographic area.

Several jurisdictions have made progress in accessing and integrating electronic health data from various sources. One example is the Chicago Health Atlas, which combines hospital EHR data with public health and social service data⁶ for chronic diseases (e.g., diabetes, asthma, breast cancer), infectious diseases (e.g., tuberculosis, chlamydia), environmental exposures such as lead, and birth and deaths aggregated at the zip code level. Another example is the Colorado Health Observation Regional Data Service,⁷ which collects and integrates data from 10 health care systems to produce a registry for obesity, hypertension, and hyperlipdemia. In Massachusetts, MDPHnet allows health departments to obtain aggregate data from practices’ EHRs for key health outcomes.⁸ Elsewhere, Regional Health Information Organizations are facilitating the collection and analysis of EHR data from multiple sites.

Using aggregated data to generate population prevalence estimates of chronic diseases is still in its early stages.⁹ There is poor understanding of the validity and reliability of EHR data for this purpose and whether EHR data represent the population in a particular geographic area. We briefly describe one recently developed municipal EHR surveillance system and summarize validation study findings published to date.

NEW YORK CITY MACROSCOPE DEVELOPMENT

In 2012, a cross-disciplinary team including epidemiologists, health information technology experts, and clinicians from the NYC Department of Health and Mental Hygiene (DOHMH) and the City University of New York School of Public Health (now at NYU), with the support of several foundations, developed an EHR surveillance system known as the NYC Macroscope.¹⁰ The NYC Macroscope uses data from a large, distributed EHR network in NYC. This process included defining health indicators, generating prevalence estimates, and validating the estimates.

NYC Macroscope uses EHR data from the DOHMH’s Primary Care Information Project (PCIP). PCIP assists ambulatory practices to adopt and use EHRs to increase delivery of clinical preventive services, reduce chronic disease risk factors, and improve disease management.¹¹ As part of this program, a subset of 700 practices using eClinicalWorks agreed to share aggregate data with the NYC DOHMH.¹² This virtual network, created in collaboration with eClinicalWorks, allows the NYC DOHMH to monitor the health of nearly 1.5 million patients, about one in six New Yorkers. The NYC DOHMH uses these data to create reports for providers that highlight opportunities for delivering clinical preventive services. The reports also allow providers to compare their aggregate patient data with similar providers in the city,¹³ enabling them to quickly identify what is working well in their practices and areas for improvement. At the same time, the NYC DOHMH can use aggregate data to assess population health and to guide policy and programmatic initiatives. In many ways, PCIP serves as a bridge between clinical care and public health.

In developing the NYC Macroscope, we selected and defined indicators (Table 1; Appendix, available as a supplement to the online version of this article at http://www.ajph.org as) important to public health surveillance. These indicators included prevalence of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination as well as treatment and control of diagnosed diabetes, hyperlipidemia, and hypertension (Table 1). We developed queries to extract information from PCIP practices aggregated by provider and used information captured by these queries to calculate the NYC Macroscope estimates.

TABLE 1—

Selected New York City Macroscope 2013 Indicators Important to Public Health Surveillance

Indicator and Type	Definition
Smoking prevalence	Current smoker recorded in structured fields (checkboxes)a
Obesity prevalence	Body mass index ≥ 30 kg/m2 from most recent height and weighta in vitals
Depression prevalence	ICD diagnosis of depression or Patient Health Questionnaire-9 score of 10–27
Influenza vaccination prevalence	CVX, CPT, or ICD code indicating receipt of influenza vaccinationa
Hypertension
Prevalence	ICD diagnosis of hypertension
Augmented prevalence	Most recent blood pressurea systolic ≥ 140 mm Hg or diastolic ≥ 90 mm Hg or ICD diagnosis of hypertension or medication prescribed for hypertensiona
Treatment	Medication prescribeda among those with ICD diagnosis of hypertension
Control	Most recent blood pressure < 140/90a mm Hg among those with ICD diagnosis of hypertension
High cholesterol
Prevalence	ICD diagnosis of high cholesterol
Augmented prevalence	Most recent total cholesterol ≥ 240b mg/dL or ICD diagnosis of high cholesterol or medication prescribed for high cholesterola
Treatment	Medication prescribeda among those with ICD diagnosis of high cholesterol
Control	Most recent total cholesterol < 240b mg/dL among those with ICD diagnosis of high cholesterol
Diabetes
Prevalence	ICD diagnosis of diabetes
Augmented prevalence	Most recent hemoglobin A1c ≥ 6.5b or ICD diagnosis of diabetes or medication prescribed for diabetesa
Treatment	Medication prescribeda among those with ICD diagnosis of diabetes
Poor control	Most recent A1c > 9b among those with ICD diagnosis of diabetes

Note. CPT = current procedural terminology; CVX = vaccine administered; ICD = International Classification of Diseases International Classification of Diseases, Ninth Revision, Clinical Modification (Hyattsville, MD: National Center for Health Statistics; 1980) DHHS publication PHS 80-1260.

^aIn calendar year.

^bIn calendar year or previous calendar year.

We designed the NYC Macroscope to be generalizable to the NYC adult population who sought primary care in the past year because of known health differences between this “in-care” population and those who were not in care in the past year.¹⁴ An important advantage of PCIP’s aggregate data model is the absence of personal identifying information, which ensures patient privacy. However, this inability to identify individuals prevented us from eliminating duplicate records across practices. To reduce duplicate records and minimize bias from specialists seeing sicker patients, we limited the NYC Macroscope to primary care providers. We also limited the surveillance system to a group of providers that met minimum criteria for EHR data completeness and quality.

We used estimates from two well-established reference surveys to validate the NYC Macroscope estimates: the gold standard NYC Health and Nutrition Examination Survey (NYC HANES)¹⁵ and the NYC Community Health Survey.¹⁶ NYC HANES is a population-based, in-person examination survey modeled on the national HANES, and the Community Health Survey is an annual random digit dial telephone survey in NYC modeled on the national Behavioral Risk Factor Surveillance System.¹⁷ For comparability, we limited all data sources to those who reported seeking primary care in the past year. To determine if estimates could be used for population health surveillance, we established a priori goodness-of-fit criteria to compare estimates from the NYC Macroscope with those from the two surveys. Criteria included prevalence difference (within five points), prevalence ratio (0.85–1.15), a Spearman correlation of ≥ 0.80, t test, and the two one-sided test of equivalence.¹⁸

In addition to validating the overall estimates from the NYC Macroscope, we conducted a chart review on a subset of NYC HANES participants who had sought primary care in the past year to assess the validity of EHR measures at the individual level (61% consented to chart review, and 27% of those signed HIPAA [the Health Insurance Portability and Accountability Act of 1996] release forms and met eligibility criteria). We abstracted information from the EHRs from 48 NYC HANES participants whose providers contributed data to the NYC Macroscope and 142 participants whose providers did not (the latter included charts from 22 different EHR vendors). For each study participant, we compared the indicator estimates from primary care EHR data to those from NYC HANES and computed sensitivity and specificity. We also stratified EHR-based indicator estimates by whether the participant’s provider was part of the NYC Macroscope to examine generalizability to other EHR surveillance systems.

NYC MACROSCOPE VALIDATION RESULTS

Detailed results of the NYC Macroscope validation have been described previously.^19,20 In brief, NYC Macroscope indicators for diabetes, hypertension, smoking, and obesity prevalence performed well according to a priori criteria for agreement (Table 2), and each of these demonstrated high sensitivity and specificity (0.89–1.00) in the chart review study of records from providers contributing to the NYC Macroscope. Although hyperlipidemia prevalence met many of the a priori criteria, sensitivity and specificity were not as good (0.72 and 0.58, respectively).

TABLE 2—

Results of Indicator Validation by Comparing NYC Macroscope 2013 (n = 648 816) with NYC HANES 2013–2014 (n = 1135) and the 2013 Community Health Survey (n = 6166) for NYC Adults in Care in the Past Year

	Indicator
	Hypertension	Diabetes	Smoking	Obesity	Hypercholesterolemia	Depression	Influenza Vaccination
NYC Macroscope, % (95% CI)	32.3 (32.2, 32.4)	13.9 (13.8, 14.0)	15.2 (15.1, 15.3)	27.9 (27.7, 27.9)	49.3 (49.1, 49.5)	8.2 (8.1, 8.2)	20.9 (20.8, 21.0)
NYC HANES, % (95% CI)	32.5 (29.4, 35.7)	12.6 (10.6, 14.8)	17.7 (15.1, 20.8)	31.3 (28.5, 34.5)	46.9 (42.6, 51.3)	19.0 (16.4, 21.9)	47.6 (44.0, 51.3)
Community Health Survey, % (95% CI)	31.6 (30.2, 33.0)	12.5 (11.5, 13.6)	14.9 (13.6, 16.3)	24.7 (23.2, 26.3)	47.9 (45.7, 50.1)	NA	47.3 (45.5, 49.0)
NYC Macroscope vs NYC HANES
Absolute difference < 5	✓ (0.15)	✓ (1.36)	✓ (2.55)	✓ (3.46)	✓ (2.36)	X (10.80)	X (26.71)
Prevalence ratio of 0.85–1.15	✓(1.00)	✓ (1.11)	✓ (0.86)	✓ (0.89)	✓ (1.05)	X (0.43)	X (0.44)
Test of difference (t test), P ≥ .05	✓(P = .93)	✓ (P = .19)	✓ (P = .08)	X (P = .02)	✓ (P = .29)	X (P < .01)	X (P < .01)
Test of equivalence, P < .05	✓ (P < .01)	✓ (P < .001)	✓ (P = .04)	X (P = .14)	X (P = .12)	X (P = .99)	X (P = .99)
Recommendation	Ready for use	Ready for use	Ready for use	Ready for use	Use with cautiona	Not ready for use	Not ready for use

Note. ✓ = criterion met; X = criterion not met; CI = confidence interval; HANES = Health and Nutrition Examination Survey; NA = not available in this data set; NYC = New York City.

^aCaution for hypercholesterolemia prevalence is on the basis of the tests shown here and lower sensitivity and specify in the chart review study (data not shown).

Indicators for treatment and control measures for diabetes, hypertension, and hyperlipidemia were less concordant,²⁰ which might be explained by bias attributable to PCIP’s interactions with physicians, including newsletters, summary dashboards for providers, and EHR-based reminders to perform clinical preventive services and for hyperlipdemia, a change in clinical guidelines in 2013.²¹ Small NYC HANES sample sizes could also have contributed to poor precision and lower agreement. The NYC Macroscope prevalence estimates for depression and influenza vaccination were substantially lower than were survey estimates and had poor sensitivity.¹⁹ Many providers are not routinely using a standardized screening tool to assess depression (eClinicalWorks software uses the Patient Health Questionnaire), and influenza vaccines given at pharmacies or workplaces may not be documented in primary care settings.

Results of the broader chart review study involving multiple EHR vendors are forthcoming, but preliminary results suggest that obesity and diabetes prevalence estimates translate well across primary care EHR software systems. Smoking and hypertension prevalence also performed well and can likely be used for population health surveillance by other jurisdictions, although it may be helpful to conduct local validation against an established data source if possible. For these indicators, generalizability across EHR software systems may improve with strategic restrictions, such as including only providers who have attested to Meaningful Use standards.

Future validation efforts will focus on evaluating accuracy of neighborhood-level estimates as well as estimates stratified by race/ethnicity to explore how the NYC Macroscope can be used to monitor health inequities. We have also initiated an evaluation of the system’s capacity to monitor trends from 2012 to 2015 and have begun to explore additional indicators, including obesity in young children. Ongoing validation will be important to monitor changes in validity as the EHR network matures and to assess new indicators.

The NYC Macroscope complements existing chronic disease monitoring systems and can be used to help the NYC DOHMH and its local partners establish priorities and inform programming. In NYC and other jurisdictions, survey response rates are declining, and results from traditional surveillance systems are often unavailable for smaller cities, towns, or neighborhoods because these areas do not have sample sizes large enough to produce reliable estimates. EHR-based surveillance systems can be cost effective and timely and can provide prevalence estimates for local communities and smaller subpopulations. Especially when used in conjunction with other data sources, they can provide a comprehensive and accurate picture of the health of a defined population.

ADVANCING POPULATION HEALTH NATIONALLY

Validation findings from the NYC Macroscope and early results from similar emerging systems^22,23 suggest that prevalence of diabetes, smoking, hypertension, and obesity are good indicators for EHR-based surveillance in jurisdictions with functioning EHR networks (Table 2). Methodological decisions made in developing the NYC Macroscope may be helpful to other jurisdictions establishing similar EHR-based surveillance systems; for example, they can help them develop methods to weight results for generalizability to a larger population, establish inclusion criteria, define indicators with attention to how diagnostic criteria may differ from epidemiological definitions, and select statistical methods for comparing estimates across data sources.¹⁰

To share ideas and further advance the field of population health surveillance using EHRs, the NYC DOHMH has fostered collaborative meetings with other early adopter agencies and select organizations to advance EHR surveillance methodology. The DOHMH hosted a small conference in October 2014 that was attended by representatives from health departments, federal agencies, clinical data research networks, and health care delivery networks. Common issues discussed included governance models, the need for high level buy-in, support from partnering institutions, legal and technical challenges in establishing a network and permitting data sharing, and difficulties of sustainability and ensuring data quality.

In January 2017, AcademyHealth and the DOHMH, with foundation support, sponsored a workshop to explore methodologies for small area estimation using EHR networks. We also partnered with AcademyHealth to develop a Population Health Community of Practice²⁴ for sharing ideas and challenges and for encouraging collaboration among those using EHRs for population health.²⁴ As EHR networks mature, the active sharing of lessons learned can stimulate and improve data-driven initiatives to bridge primary care and population health²⁵ and help achieve the full potential of EHRs for population health surveillance.