Abstract
Electronic health records (EHRs) provide an alternative to traditional public health surveillance surveys and administrative data for measuring the prevalence and impact of chronic health conditions in populations. As the infrastructure for secondary use of EHR data improves, many stakeholders are poised to benefit from data partnerships for regional access to information. EHRs can be transformed into a common data model that facilitates data sharing across multiple organizations and allows data to be used for surveillance. The Colorado Health Observation Regional Data Service (CHORDS), a regional distributed data network, has assembled diverse data partnerships, flexible infrastructure, and transparent governance practices to better understand the health of communities through EHR-based, public health surveillance. This paper describes attributes of regional distributed data networks using EHR data, and the history and design of CHORDS as an emerging public health surveillance tool for chronic health conditions. CHORDS and our experience may serve as a model for other regions interested in similar surveillance efforts. While benefits from EHR-based surveillance are described, a number of technology, partnership and value proposition challenges remain.
Introduction
Chronic health conditions (e.g., heart disease and diabetes) are leading causes of death in the United States.1 Local public health agencies (LPHAs) developing strategies to identify and address chronic conditions build on a strong communicable disease surveillance foundation.2 Historically, chronic disease data available to LPHAs have been vital statistics, state or local surveys (e.g. Behavioral Risk Factor Surveillance System [BRFSS]), and administrative or claims data (e.g., Medicare or Medicaid).3,4 Administrative data describe care-seeking behaviors of populations, but often lack clinical measurements (e.g., blood pressure, height and weight measures). Representative population surveys assess disease prevalence and behaviors influencing chronic conditions, but rely on respondent self-reported data and also lack relevant clinical observations to assess severity and disease control. Surveys are expensive, time consuming, insufficiently represent small geographic areas or subpopulations, and rarely provide rapid feedback.5 Consequently, some LPHAs have utilized electronic health record (EHR) data collected through clinical care for surveillance.6,7
Recent federal policies and technology have expanded EHR data availability for public health surveillance.8,9 Selection bias and referral patterns may affect generalizability of information from a single healthcare provider. By assembling EHR data from multiple healthcare practice settings, LPHAs may more accurately represent chronic disease prevalence within their jurisdiction. One monitoring strategy for chronic conditions is through regional distributed data networks (DDNs), which facilitate data sharing across sites by transforming site-specific data to a common data model (CDM) without relying on a centralized data repository. In these DDNs, sites can maintain data autonomy and assure patient confidentiality.10
Regional DDNs require willing healthcare provider and LPHA partnerships to govern and standardize data across sites.11 While challenging to create and maintain, a regional DDN provides LPHAs with more granular, place-based surveillance, timely data access, insights into clinic-based preventive care, and a dual-purpose infrastructure serving public health surveillance and clinical research.5,6 Additionally, DDNs have incorporated social and environmental indicators into surveillance, chronic disease management, and intervention planning12,13 with potential to inform healthcare providers, community partners, and LPHAs about their populations.
This paper describes the creation and maintenance of a regional, EHR-based, public health surveillance DDN in Colorado. The Colorado Health Observation Regional Data Service (CHORDS), a DDN of health and mental health care providers, has engaged state and local public health agencies while leveraging technical expertise of a health information exchange ([HIE], Colorado Regional Health Information Organization [CORHIO]) and large university (University of Colorado Anschutz Medical Campus [UCAMC]). This paper will outline key regional DDN attributes, and early developmental approaches, infrastructure, and challenges faced during CHORDS implementation and expansion. The paper concludes with an assessment of CHORDS’ accomplishments and challenges five years after inception, and prospects for sustainability.
Attributes of EHR-based data systems for chronic condition surveillance
Accessed through a regional DDN, EHR data may facilitate surveillance of incidence, prevalence, and management of chronic conditions through rich clinical data (e.g., vital signs, diagnostic, treatment, and laboratory results), which are often too costly to collect from health surveys. Diagnosis codes identify incidence and prevalence of conditions as observed by clinicians, while laboratory results (e.g., hemoglobin A1C) and anthropometric measures (e.g., blood pressure and weight) confirm diagnoses and assess severity and control. EHRs capture multiple measures for a patient over time and store data longitudinally. Whereas surveys often reflect self-reported conditions, structured EHR queries for common health conditions incorporating multiple data sources provide rapid, accurate and potentially less biased and objective measures to support LPHA decision-making and novel use cases.
Accessing multiple systems’ EHR data through DDNs represents an extension of nation-wide clinical comparative effectiveness and drug safety surveillance research tools, and is increasingly useful for place-based surveillance.14 Similar to these federal efforts, a regional public health surveillance DDN should be comprised of healthcare organizations and one or more LPHAs serving a defined geographic area (e.g., contiguous counties). Combined data from multiple provider organizations improve representativeness by reducing selection or referral bias from any one provider. Combined data also expand the DDN’s observed population, broaden capacity to study subpopulations, and produce sub-county area estimates with greater statistical power and precision. Optimizing representativeness typically requires engaging non-profit, public and private providers, and in particular the inclusion of safety-net partners such as Federally Qualified Health Centers (FQHCs).
Data Quality and Validity
Public health surveillance efforts have long relied on imperfect data. While EHRs face challenges to internal and external validity, they address some limitations of surveys or administrative records. EHRs improve measurement and status of chronic health conditions by incorporating diagnoses, laboratory results, vital signs, pharmaceutical data, and medical procedures. Structured clinical data are less likely to under-diagnose certain conditions compared with health surveys, particularly for asymptomatic patients (e.g., uncomplicated hypertension).15
EHRs provide more flexibility for specific LPHA needs than static surveys or administrative data. “Living” EHR data may be manipulated and provide flexibility in how conditions are measured and reported over time. For example, survey data may contain self-reported diabetes information, but EHRs could assess across a continuum; type, severity, treatment and control of diabetes are relevant for LPHA action planning. EHRs provide some clear benefits regarding data quality, yet challenges regarding data quality and validity remain and are discussed later.
Historical Context
The network that would become CHORDS emerged around 2009 in response to technological innovations, policy incentives, and ongoing collaborations among Denver Metropolitan area institutions. Figure 1 provides a timeline of key CHORDS development and implementation events, ranging from national policies to specific technological developments. After 2009, many healthcare systems adopted and implemented certified EHR technology, which created an environment to aggregate structured EHR data across systems.9 National networks16 had invested and created technology to support DDNs (e.g., CDM and distributed data sharing methods) for cost effectiveness studies, quality improvement and drug safety surveillance.14,17
Federal health information technology investments led to infrastructure and project-based grants requiring data sharing partnerships. In 2007, UCAMC researchers received a Clinical and Translational Science Award (CTSA) through the National Institutes of Health. These researchers envisioned a research HIE based on early clinical data sharing for treatment/operations between regional health care providers. The CTSA project required collaboration from the local healthcare system to develop infrastructure that could support surveillance and research. CHORDS began as an informatics component of the CTSA-funded project.
The initial CHORDS network faced numerous barriers to full implementation during the five-year CTSA grant period. No site was clearly identified as the technology or administrative owner. Insufficient funds were available to support the concept. The selected federated query technology (caGRID)18 proved difficult to implement. Without specific research questions to be answered, cross-site research infrastructure collaborations were nebulous and tenuous. Lack of a clear purpose challenged progress on well-defined governance processes for research studies.
Between 2009 and 2015, several organizations participating in CHORDS received grants and projects to incrementally improve technical and governance processes and procedures. Individual healthcare organizations, researchers and leadership from the Denver Metropolitan area gradually created more funded opportunities to support CHORDS’ development.
A few specific projects were noteworthy in advancing CHORDS’ public health surveillance capabilities. In 2010, Kaiser Permanente of Colorado (KPCO) received funding to collaborate with Denver Health (DH) to build and pilot test a DDN to answer specific research questions related to attention-deficit/hyperactivity disorder (ADHD) and obesity.19 That work informed subsequent CHORDS activities in three ways. First, since both KPCO and DH were primary CHORDS sites, the project demonstrated how EHR data within a DDN could be used to address regional health concerns. This DDN produced tangible deliverables, including an initial real-world PopMedNet ([PMN], www.popmednet.org) application, the federated query system that replaced the caGRID-based query mechanism. The DDN’s use of PMN facilitated subsequent CHORDS PMN deployment by directly transferring knowledge and experience. Second, the DDN tested standardized menu-driven summary data requests; these results confirmed that a federated DDN could conduct surveillance of chronic health conditions.20 Third, the DDN formalized governance policies and practices (subsequently leveraged by the Patient Centered-Research Network [PCORnet]); these ultimately provided a robust template for CHORDS governance.21
In 2011, DH received Colorado Department of Public Health and Environment (CDPHE) funding to support a comprehensive, technology-driven, tobacco cessation program (Q-LINe). Using a population-based approach, electronic referrals to the local Quitline were combined with EHR-derived data from CHORDS to monitor tobacco use/cessation referral trends.22
In 2012, eight data partners (e.g., managed care organizations, safety net providers, and community health clinics) launched the Colorado Body Mass Index (BMI) Monitoring System to utilize EHR-recorded and extracted height and weight measures obtained during routine care to calculate BMI. Prevalence estimates for overweight and obesity were visualized through maps and data tables for dissemination to counties in the Denver Metropolitan area and Prowers County, Colorado.23,24 The Colorado BMI Monitoring System provided a model for disseminating chronic health condition information to LPHAs.
In 2013, DH and KPCO received Agency for Healthcare Research and Quality funding to combine their data to understand depression prevalence variation across Denver’s census tracts. Census tract level differences in depression prevalence were statistically validated, and results were disseminated to community organizations.25,26
Through these projects, CHORDS technical and governance practices gradually became more formal and established, while CHORDS-based information began to demonstrate potential value to communities. Concomitantly, Denver Metropolitan area LPHAs and CDPHE were discussing how CHORDS might address regional public health issues. Between 2012 and 2015, CHORDS recruited eight healthcare organizations as data partners who signed data use agreements defining a CDM.17
In 2015, facilitated by the Colorado Health Institute (CHI), LPHAs secured a two-year grant (renewed through 2020) from The Colorado Health Foundation to fund CHORDS expansion efforts. Funded goals included: reinforce and further develop CHORDS governance, expand data users beyond the five Denver Metropolitan area LPHAs, recruit new data partners including mental health centers, improve data sharing technologies, begin the transition for scaling and housing a statewide DDN, and promote CHORDS utility from a public health and policy perspective.
Since 2016, the CHORDS Governance Committee has overseen all aspects of CHORDS implementation. A standardized data request system was designed and implemented. Staff have engaged and guided local mental health centers joining the network. By 2017, LPHAs began submitting CHORDS data requests for depression, diabetes, obesity, and hypertension prevalence estimates in the Denver Metropolitan area.
In 2012, the local institutional review board deemed the HIPAA public health surveillance exemption used for communicable disease surveillance and reporting was applicable to non-communicable disease. While CHORDS received non-human subject research designation when applied to public health surveillance, the investigational review board continues to specifically review and approve each non-LPHA initiated, research protocol. This broadens potential CHORDS use beyond public health surveillance; some researchers use CHORDS for cohort health services research (e.g., asthma and congenital heart disease) while others investigate network methods (e.g., record linkage and deduplication).
System Design
Data model development
To standardize data across sites, DDNs must agree on a CDM. A CDM is essential for efficient data storage and sharing across propriety systems. CDM development requires standardizing variable names, codes, and storage location so each site easily contributes data in response to requests. Given DH’s and KPCO’s prior shared DDN experience, that CDM was adopted.17 By 2017, the CHORDS CDM had 17 tables nearly identical to an existing standard.27
CHORDS staff assisted sites in extracting, transforming, and loading data with CDM technical tools (i.e., SQL data definition language) and partnered with Colorado Community Managed Care Network (CCMCN) to bring five FQHCs into the network using an existing data warehouse.28
Federated query system
Figure 2 demonstrates the data request and response process. A key CHORDS requirement and PMN feature was preservation of data autonomy. Data partners must acknowledge and review codified queries and results after processing data requests and may withhold results from multi-institution aggregation, although as of early 2018 none have been withheld. Specific information about data partners is not published to maintain patient privacy and reduce competition-based use of information. CHORDS data sharing rules also protect patient and site confidentiality by suppressing small cells when implementing structured requests.
Table 1 describes health data types available through CHORDS, including surveillance questions relevant to chronic conditions, and demonstrates how standardized data tables as part of the CDM may be used across data partners. Although PMN might support highly customized queries, CHORDS has focused on developing structured, parameterized queries. Time- and resource-intensive to develop and validate, structured requests are re-usable and ideal for repeated LPHA surveillance, (e.g., annual tobacco prevalence reports). Four priority conditions – obesity, depression, diabetes and hypertension prevalence- have been initially selected. Each request uses validated structured query language (SQL) code to define a care seeking population (the denominator), identify a cohort (the numerator) (e.g., based on ICD-9/ICD-10 diagnosis codes), and calculate prevalence using a computable phenotype.7 Once developed, established structured requests require little technical expertise to execute, increase responsiveness to LPHA data requests and radically expand LPHA surveillance capacity. Structured requests ensure that indicators such as hypertension prevalence are estimated using a consistent definition.
Table 1.
Data type | Description | Relevance | Example |
---|---|---|---|
Patient demographics | Age, gender, race/ethnicity, primary language | Examines prevalence, management, and disparities in chronic conditions by individual characteristics | What percent of children are diagnosed with asthma? |
Geographic location and demographics | County and census tract of residence, American Community Survey measures | Examines how differences in social and environmental characteristics within a region are associated with chronic conditions | How is neighborhood poverty associated with treatment adherence for diabetes? |
Encounters | Date, and encounter type | Examines how frequency and timing of patient visits affect prevention and management of chronic conditions | How frequently are adults with depression seen? |
Diagnoses and procedures | ICD-9/10 diagnosis code, date of diagnosis, procedure type and date | Calculates prevalence of diagnosed chronic health conditions and related procedures | What percent of adults have been diagnosed with high blood pressure? |
Laboratory results | Test type, date of test, result | Captures undiagnosed chronic conditions and examine screening rates | What percent of Denver adults have been tested for hepatitis C virus? |
Pharmacy | Outpatient prescription orders and dispenses | Examines relationship between diagnosis of chronic conditions and pharmaceutical treatment for these conditions | What percent of adults with high cholesterol are prescribed cholesterol-lowering medication? |
Vital signs | Objectively-measured height, weight, blood pressure | Adds to diagnostic information to examine relationships of vital signs to chronic condition prevalence | What percent of adults are overweight or obese? What percent of adults with hypertension are well controlled? |
Social History | Self-reported data on tobacco use and exposure, alcohol/drug use | Examines the relationship between health behaviors and chronic conditions | What percent of children are exposed to second hand smoke? |
Note: Table is organized to the common data model (CDM); only a portion of the 17 CDM tables are presented here.
Governance and organizational structure
Effective governance fosters a culture of trust among data partners while developing and assuring procedures that protect patient privacy and institutional data.21 Early on, CHORDS leaders sought transparent governance guided by principles of partner data autonomy and collaborative decision-making. To promote surveillance and research activities, by June 2015, the CHORDS formalized governance process had established monthly Governance Committee meetings with data partners, users, and technology service providers to create an organizational structure, policies and standard work.
As the neutral convening organization, CHI has administered and coordinated CHORDS funds and provided an “organizational home” for governance activities including convening stakeholders, overseeing governance structure, communicating and disseminating results, and policy development resulting in a trusting environment for data partners.29
Governance policies and guidelines define how data are accessed, analyzed and disseminated. The CHORDS Governance Committee, responsible for CHORDS governance, developed three Work Groups: 1) Data, 2) Network Operations, and 3) Project Development, which address data model stewardship, network structure, and potential projects and requests, respectively. The Work Groups focus on specific tasks and report back to the larger governance committee. Additionally two Advisory Councils focus on research, community engagement and data dissemination. The Advisory Councils and Work Groups provide broad guidance and oversight with specific and complementary tasks. Table 2 (supplemental digital table) describes the roles of each group that are either underway or proposed.
Table 2.
Group | Tasks |
---|---|
Data Work Group |
|
Network Operations Work Group |
|
Project Development Work Group |
|
Advisory Council – Research |
|
Advisory Council – Community |
|
Demographic Characteristics and Coverage
The CHORDS team is currently working on an extensive evaluation of coverage, comparing the CHORDS patient population to the population represented in local health surveys and the U.S. Census Bureau’s American Community Survey (ACS). Table 3 outlines basic demographic and coverage information for the CHORDS’ population compared to ACS Denver Metropolitan area estimates. Overall, as of 2015 the CHORDS network patients encompassed roughly 30% of the seven-county population. CHORDS network patients were slightly younger and had more women and patients of color compared to population estimates. Examining CHORDS patient race/ethnicity characteristics highlights a strength and challenge; the network covers patients in racial/ethnic subgroups typically underrepresented in health surveys or small area ACS estimates, limited by sample size. However, CHORDS network partners variably collect race/ethnicity data leaving 8% of CHORDS’ patient population with unknown racial/ethnicity. Furthermore, a known coding error misclassified some patients as Hawaiian or Pacific Islander exceeding the reported Hawaiian or Pacific Islander population in ACS. While CHORDS captures key demographic categories to monitor chronic conditions for the more than 3 million residents in the Denver Metropolitan area, these sporadically identified coding errors drive continuous quality improvement through data quality and validity efforts.
Table 3.
ACS population estimate | Clinical observations | Coverage rate | |||
---|---|---|---|---|---|
n | (%) | n | (%) | % | |
Total | 3,077,326 | (100) | 945,270 | (100) | 31 |
Age | |||||
<18 | 714,448 | (23) | 360,270 | (38) | 50 |
18-24 | 280,388 | (9) | 75,284 | (8) | 27 |
25-64 | 1,715,528 | (56) | 405,988 | (43) | 24 |
65+ | 366,962 | (12) | 103,728 | (11) | 28 |
Gender | |||||
Male | 1,533,085 | (50) | 437,196 | (46) | 29 |
Female | 1,544,241 | (50) | 508,067 | (54) | 33 |
Ethnicity & Race | |||||
White | 2,013,151 | (65) | 470,650 | (50) | 23 |
Black | 152,190 | (5) | 73,575 | (8) | 48 |
Hispanic | 686,195 | (22) | 285,189 | (30) | 42 |
American Indian Alaska Native | 10,429 | (<1) | 5,379 | (1) | 52 |
Asian | 121,145 | (4) | 23,628 | (2) | 20 |
Hawaiian or Pacific Islander | 3,034 | (<1) | 8,983 | (1) | 296 |
Multiple Race | 81,931 | (3) | 6,772 | (1) | 8 |
Race/ethnicity unknown | NA | NA | 71,094 | (8) | NA |
Adams, Arapahoe, Boulder, Broomfield, Denver, Douglas and Jefferson Counties
Challenges of EHR-based surveillance of chronic conditions
Implementing CHORDS has highlighted a number of challenges and lessons that likely apply to developing regional DDNs in other parts of the country.
Technology
Instituting a CDM across diverse data partners with varying technological capabilities has been a challenge. Smaller healthcare organizations have struggled to build and populate their CDM, resulting in more financial and technical investment than initially anticipated. Existing data warehouses were leveraged for some participating FQHCs, but other safety net providers or small group practices typically have limited experience with data manipulation, warehousing and aggregation.30,31 After the initial load, some sites have had challenges refreshing data, which limits capacity for trend analyses using repeated measures.
Threats to validity
By integrating EHRs from multiple health systems, CHORDS aims to represent the healthcare seeking population in metropolitan Denver. All data sources face challenges in population representativeness as even randomly sampled, survey cohorts have non-response bias, particularly among vulnerable populations.32 Surveillance is improved with accurate coverage, but perfect population representation is likely elusive. Thus, CHORDS representativeness challenges should not be cause to reject its use for chronic condition surveillance but as complementary or comparable to other sources.
Over time, patients may interact with multiple providers in a region leading to representing individuals more than once in aggregated reports across providers. A near-term CHORDS goal is patient record de-duplication by incorporating a unique master patient identifier into the CDM, leveraging work of an existing HIE. Privacy protecting de-duplication for a single patient has been challenging because key identifying data fields are not easily shared across a DDN or are incompletely populated across providers.
Inter-institutional
CHORDS data partners have agreed to collaborate in public health surveillance and research activities. Challenges in recruiting data partners and maintaining strong partnerships have also been noted in other models, such as HIEs.33 CHORDS leveraged existing data sharing partnerships between LPHAs and healthcare providers to create a DDN based on established trust, similar to existing models (e.g., National Syndromic Surveillance Program [NSSP]).34 These collaborations have benefited from a regional LPHA approach to achieve surveillance goals. Several LPHAs conduct routine community health needs assessments, often depending on strong relationships among regional healthcare partners. Relationships that foster trust, identify common goals and monitor mutual success are essential for a sustainable regional DDN. Although data partners are willing to share data with LPHAs for common benefit, they are reluctant to share data in ways that offer direct comparisons between delivery systems. Governance policy stipulates CHORDS technology will never provide site-specific information. As the federated query tool receives site-specific results, it returns only aggregated results for the geographic region (e.g., stratified by county or census tract) or population of interest.
Responsiveness and Timeliness
CHORDS faces the challenge of timely responses to data requests. Particularly for grant applications, well-timed responses need to balance speed with required time to properly vet, and review results as established through governance policies. While currently focused on a relatively standard set of LPHA chronic condition requests, methods for rapid deployment of tailored or unique requests will require time and resources to develop and implement. Competing demands and financing opportunities may require prospective management of requestor expectations.
Defining and Measuring the Value Proposition
The success of regional DDNs for public health surveillance depends on the network’s ability to demonstrate value to public health departments, healthcare providers, and communities. Identifying policy or program data needs of these groups and delivering timely, accessible, and useful health analyses are essential. Key to demonstrating CHORDS value is measuring how summary information is consistent with or complements existing disease/condition prevalence estimates in the same population. Is CHORDS consistent with BRFSS? What weighting methods would enhance CHORDS population representativeness and estimates? How might EHR-based estimates complement longstanding value and credence we place on BRFSS? These are some of the questions that the CHORDS team will have to answer going forward.
Funding and financial sustainability
A financial model for long-term CHORDS sustainability is currently being assessed and will be described in a future paper. Funded through 2020 by The Colorado Health Foundation, the team has a mandate for financial sustainability.
Overall, startup costs for CHORDS were substantial. Some data partners needed extensive resources for CDM development. CHORDS has relied on diverse funding sources, and this grant and research funding model may continue going forward. A goal is to establish a demonstrated need among regional partners and accountable care organizations that may offer consistent financial support for monitoring improvement among populations.
Conclusion
Federal, state, and local investments in EHRs have opened the door to new public health surveillance opportunities for chronic health conditions. Regional DDNs that incorporate information from EHRs for public health surveillance are relatively new and moderately rare, but demonstrate a complementary alternative to traditional population-based health surveys and administrative data. CHORDS is an operational DDN in the Denver Metropolitan area building surveillance capacity for the State of Colorado. CHORDS addresses the needs of state and local public health departments by incorporating diverse data partners, instituting governance practices that protect patients and data partners, allowing data partners to maintain data autonomy, hosting a federated query tool, and providing timely, standardized information requested by LPHAs. Going forward, CHORDS is continuously addressing challenges related to data validity, governance, technology solutions, funding, and operations while extending its geographic reach across the state.
This article outlined the strengths and challenges of CHORDS as a regional DDN. Developing EHR-based public health surveillance tools may be an opportunity for other regions to consider. Fundamental to CHORDS’ success has been the development of respectful relationships between data partners and users. Novel EHR-based surveillance systems are not founded on statute but rather trust and collaboration, fostered through commitment to meaningful relationships between researchers, practitioners, administrators, and public health officials across the targeted region.
Implications for Policy and Practice.
The Colorado Health Observation Regional Data Service (CHORDS), an EHR-based regional distributed data network, provides a novel method for public health surveillance of chronic health conditions and diseases.
Leveraging federal investments, CHORDS implemented a common data model, federated query system, and governance structure of potential value to other communities interested in replicating or building similar partnerships.
Attributes of a regional distributed data network for public health surveillance include: place-based surveillance, more timely data collection and reporting, measurement of clinically-based preventive care (e.g., rates of tobacco screening and cessation referral).
While challenges abound, this regional distributed data network creates opportunities for meaningful relationships with healthcare providers; building this public health surveillance infrastructure has capacity to serve and strengthen clinical research partnerships across a jurisdiction.
Acknowledgments
This work was supported by The Colorado Health Foundation, NIH/NCATS Colorado CTSA grant number UL1 TR001082, the Colorado Department of Public Health and Environment (CDPHE) Q-LINe grant number PO FLA PPG1348289A, CDC Denver Cardiovascular disease Reduction: A Focused Transformation (CRAFT) grant number 1U58DP003493-01 CO11 and AHRQ grant number 5R24HS0122143. Contents are the authors’ sole responsibility and do not necessarily represent official views of the funding organizations. We would like to acknowledge Sara Schmitt and the Colorado Health Institute for their support and resources in writing this manuscript. We would also like to acknowledge the Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCHORDS) and the University of Colorado Cancer Center at the Anschutz Medical Campus for hosting CHORDS staff and technology resources.
Contributor Information
Emily Bacon, Department of Sociology and Population Program, Institute of Behavioral Science, University of Colorado Boulder.
Gregory Budney, Denver Public Health, Denver Health.
Jessica Bondy, Colorado School of Public Health, University of Colorado Denver Anschutz Medical Campus.
Michael G. Kahn, Department of Pediatrics, University of Colorado Denver Anschutz Medical Campus.
Emily V. McCormick, Denver Public Health, Denver Health.
John F. Steiner, Kaiser Permanente Colorado Institute for Health Research.
David Tabano, Kaiser Permanente Colorado Institute for Health Research, University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences.
Jeanette A. Waxmonsky, Jefferson Center for Mental Health, Department of Family Medicine, University of Colorado Medical Anschutz Campus.
Rachel Zucker, University of Colorado Anschutz Medical Campus.
Arthur J. Davidson, Denver Public Health, Denver Health.
References
- 1.Kochanek KD, Murphy SL, Xu JQ, Tejada-Vera B. Deaths: final data for 2014. Natl Vital Stat Rep. 2016;65(4):1–122. [PubMed] [Google Scholar]
- 2.Bauer UE, Briss PA, Goodman RA, Bowman BA. Prevention of chronic disease in the 21st century: elimination of the leading preventable causes of premature death and disability in the USA. Lancet. 2014;384(9937):45–52. doi: 10.1016/S0140-6736(14)60648-6. [DOI] [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System: turning information into health. Atlanta, GA: US Department of Health and Human Services, CDC; 2004. Available at http://www.cdc.gov/brfss. [Google Scholar]
- 4.Virnig BA, McBean M. Administrative data for public health surveillance and planning. Annu Rev Public Health. 2001;22(1):213–230. doi: 10.1146/annurev.publhealth.22.1.213. [DOI] [PubMed] [Google Scholar]
- 5.Casey JA, Schwartz BS, Stewart WF, Adler NE. Using electronic health records for population health research: a review of methods and applications. Annu Rev Public Health. 2016;37:61–81. doi: 10.1146/annurev-publhealth-032315-021353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Birkhead GS, Klompas M, Shah NR. Uses of electronic health records for public health surveillance to advance public health. Ann Rev Public Health. 2015;36:345–359. doi: 10.1146/annurev-publhealth-031914-122747. [DOI] [PubMed] [Google Scholar]
- 7.Thorpe LE, McVeigh KH, Perlman S, et al. Monitoring prevalence, treatment, and control of metabolic conditions in New York City adults using 2013 primary care electronic health records: a surveillance validation study. eGEMs. 2016;4(1) doi: 10.13063/2327-9214.1266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Collins FS, Hudson KL, Briggs JP, Lauer MS. PCORnet: turning a dream into reality. J Am Med Inform Assoc. 2014 doi: 10.1136/amiajnl-2014-002864. Available at: http://jamia.bmj.com/content/early/2014/05/12/amiajnl-2014-002864.full. [DOI] [PMC free article] [PubMed]
- 9.Blumenthal D. Implementation of the federal health information technology initiative. N Engl J Med. 2011;365(25):2426–2431. doi: 10.1056/NEJMsr1112158. [DOI] [PubMed] [Google Scholar]
- 10.Tabano DC, Cole E, Holve E, Davidson AJ. Distributed Data Networks That Support Public Health Information Needs. J Public Health Manag Pract. 2017 doi: 10.1097/PHH.0000000000000614. [DOI] [PubMed] [Google Scholar]
- 11.Newton-Dame R, McVeigh KH, Schreibstein L, et al. Design of the New York City Macroscope: innovations in population health surveillance using electronic health records. eGEMs. 2016;4(1):1–17. doi: 10.13063/2327-9214.1265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Adler NE, Stead WW. Patients in context—EHR capture of social and behavioral determinants of health. N Engl J Med. 2015;372:698–701. doi: 10.1056/NEJMp1413945. [DOI] [PubMed] [Google Scholar]
- 13.Palacio A, Suarez M, Tamariz L, Seo DA. Road Map to Integrate Social Determinants of Health into Electronic Health Records. Popul Health Manag. 2017 doi: 10.1089/pop.2017.0019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med Care. 2010;48(6):S45–S51. doi: 10.1097/MLR.0b013e3181d9919f. [DOI] [PubMed] [Google Scholar]
- 15.Molenaar EA, Van Ameijden EJ, Grobbee DE, Numans ME. Comparison of routine care self-reported and biometrical data on hypertension and diabetes: results of the Utrecht Health Project. Eur J Pub Health. 2007;17(2):199–205. doi: 10.1093/eurpub/ckl113. [DOI] [PubMed] [Google Scholar]
- 16.Behrman RE, Benner JS, Brown JS, McClellan M, Woodcock J, Platt R. Developing the Sentinel System—a national resource for evidence development. N Engl J Med. 2011;364(6):498–499. doi: 10.1056/NEJMp1014427. [DOI] [PubMed] [Google Scholar]
- 17.Ross TR, Ng D, Brown JS, et al. The HMO Research Network Virtual Data Warehouse: a public data model to support collaboration. eGEMs. 2014;2(1):1049. doi: 10.13063/2327-9214.1049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Saltz J, Oster S, Hastings S, et al. caGrid: design and implementation of the core architecture of the cancer biomedical informatics grid. Bioinformatics. 2006;22(15):1910–1916. doi: 10.1093/bioinformatics/btl272. [DOI] [PubMed] [Google Scholar]
- 19.Daley M. Scalable Partnering Network for Comparative Effectiveness Research Across Lifespan, Conditions, and Settings (SPAN) 2013 Available online: http://www.span-network.org/
- 20.Ames MJ, Bondy J, Johnson SC, Wade TD, Davidson A, Kahn MG. AMIA. Summit on Clinical Research Informatics San Francisco; CA: 2013. Analysis of Federated Data Sharing Platforms for a Regional Data Sharing Network. [Internet]. [cited 2017 Jun 21]; Available from: http://www.popmednet.org/wp-content/uploads/2015/10/CHORDS-AMIA-Poster-2013-lowres.pdf. [Google Scholar]
- 21.Paolino AR, McGlynn EA, Lieu T, et al. Building a Governance Strategy for CER: The Patient Outcomes Research To Advance Learning (PORTAL) Network Experience. eGEMs. 2016;4(2):1–17. doi: 10.13063/2327-9214.1216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Adsit RT, Fox BM, Tsiolis T, Ogland C, Simerson M, Vind LM, et al. Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project. Transl Behav Med. 2014;4(3):324–332. doi: 10.1007/s13142-014-0259-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Colorado BMI Monitoring System. Available online: http://www.cohealthmaps.dphe.state.co.us/colorado_bmi_monitoring_system/.
- 24.Gutilla MJ, Davidson AJ, Daley MF, Anderson GB, Marshall JA, Magzamen S. Data for Community Health Assessment in Rural Colorado: A Comparison of Electronic Health Records to Public Health Surveys to Describe Childhood Obesity. J Public Health Manag Pract. 2017;23:S53–S62. doi: 10.1097/PHH.0000000000000589. [DOI] [PubMed] [Google Scholar]
- 25.Beck A, Davidson AJ, Xu S, et al. A multilevel analysis of individual, health system and neighborhood factors associated with depression within a large metropolitan area. J Urban Health. doi: 10.1007/s11524-017-0190-x. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Davidson AJ, Xu S, Oronce CIA, Durfee MJ, McCormick EV, Steiner JF, Havranek E, Beck A. Monitoring Depression Rates in an Urban Community: Use of Electronic Health Records. J Public Health Manag Pract. doi: 10.1097/PHH.0000000000000751. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.PCORnet. PCORnet common data model specification. Washington, DC: PCORI; 2014. Version 1.0. Available online: http://www.popmednet.org/. Accessed May 23, 2017. [Google Scholar]
- 28.Colorado Associated Community Health Information Exchange (CACHIE) (Colorado) AHRQ National Resource Center; Health Information Technology: Best Practices Transforming Quality, Safety, and Efficiency. https://healthit.ahrq.gov/ahrq-funded-projects/colorado-associated-community-health-information-exchange-cachie. Accessed January 29, 2018.
- 29.Colorado Health Institute. Colorado Health Observation Regional Data Service. 2017 Available online: http://www.coloradohealthinstitute.org/research/colorado-health-observation-regional-data-service.
- 30.Ong TC, Pradhananga R, Holve E, Kahn MG. A Framework for Classification of Electronic Health Data Extraction-Transformation-Loading Challenges in Data Network Participation. eGEMs. 2017;5(1) doi: 10.5334/egems.222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Davidson AJ, Shilling L, Tuteur D, Greer J, Brown M. Colorado Associated Community Health Information Exchange Final grant report prepared for The Agency for Healthcare Research and Quality (AHRQ) US Department of Health and Human Services by Denver Health and Hospital Authority. 2010 [Google Scholar]
- 32.Galea S, Tracy M. Participation rates in epidemiologic studies. Ann epidemiol. 2007;17(9) doi: 10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]
- 33.Vest JR, Gamm LD. Health information exchange: persistent challenges and new strategies. J Am Med Inform Assoc. 2010 May 1;17(3) doi: 10.1136/jamia.2010.003673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Centers for Disease Control and Prevention. National Syndromic Surveillance Program (NSSP) Atlanta, GA: US Department of Health and Human Services, CDC; 2017. Available at https://www.cdc.gov/nssp/biosense/index.html. [Google Scholar]