Abstract
Objectives
We compared an electronic health record-based influenza-like illness (ILI) surveillance system with manual sentinel surveillance and virologic data to evaluate the utility of the automated system for routine ILI surveillance.
Methods
We obtained weekly aggregate ILI reports from the Electronic medical record Support for Public Health (ESP) disease-detection and reporting system, which used an automated algorithm to identify ILI visits among a patient population of about 700,000 in Eastern Massachusetts. The percentage of total visits for ILI (“percent ILI”) in ESP, percent ILI in the Massachusetts Department of Public Health's sentinel surveillance system, and percentage of laboratory specimens submitted to participating Massachusetts laboratories that tested positive for influenza were compared for the period October 2007–September 2011. We calculated Spearman's correlation coefficients and compared ESP and sentinel surveillance systems qualitatively, in terms of simplicity, flexibility, data quality, acceptability, timeliness, and usefulness.
Results
ESP and sentinel surveillance percent ILI always peaked within one week of each other. There was 80% correlation between the two and 71%–73% correlation with laboratory data. Sentinel surveillance percent ILI was higher than ESP percent ILI during influenza seasons. The amplitude of variation in ESP percent ILI was greatest for 5- to 49-year-olds and typically peaked for the 5- to 24-year-old age group before the others.
Conclusions
The ESP system produces percent ILI data of similar quality to sentinel surveillance and offers the advantages of shifting disease reporting burden from clinicians to information systems, allowing tracking of disease by age group, facilitating efficient surveillance for very large populations, and producing consistent and timely reports.
The percentage of outpatient visits to health-care providers for influenza-like illness (ILI) is one of the key indicators used by the Centers for Disease Control and Prevention (CDC) to monitor influenza activity.1 On a weekly basis, more than 2,700 health-care providers in about 1,800 outpatient care sites around the United States provide data on the total number of patients seen and the number with ILI by age group through the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet).2 Together with virologic surveillance, mortality surveillance through the 122 Cities Mortality Reporting System and the Influenza-Associated Pediatric Mortality Surveillance System, hospitalization surveillance through FluSurv-NET, and summaries of influenza geographic spread from state and territorial epidemiologists' reports,2 the weekly “percent ILI” measure derived from these reports helps local and national public health authorities to continually assess the temporal, geographic, and demographic patterns of influenza activity, so as to respond optimally. Traditionally, percent ILI reporting has been conducted manually by volunteer sentinel clinicians recruited each year by state and local health departments. However, sentinel surveillance for ILI imposes a burden on both sentinel clinicians and health departments, resulting in variable levels of participation and reporting delays.3
Automated ILI surveillance using routinely collected electronic health record (EHR) data is a promising alternative to traditional sentinel ILI surveillance. Electronic systems have the potential to reduce the burden of traditional surveillance on both clinicians and public health personnel and to ensure consistent and complete reporting, timeliness, year-round coverage, and more stable estimates of percent ILI due to the larger numbers of patient visits they can process compared with manual systems. In addition, automated EHR-based systems can provide age-specific percent ILI reporting, which is not currently requested of sentinel clinicians because it is too burdensome for providers to manually collect age-specific denominators.
We report on an EHR-based surveillance system that has been prospectively reporting percent ILI from 30 ambulatory care centers in Eastern Massachusetts to the Massachusetts Department of Public Health (MDPH) on a weekly basis since June 2010, using an automated ILI detection algorithm. Several years of historical data prior to June 2010 are also available. This article provides both a quantitative and qualitative evaluation of the system. The quantitative evaluation compares ILI rates from the automated system with (1) concurrent rates generated by the volunteer sentinel clinicians using manual methods and (2) state virologic surveillance data, for four influenza seasons. The qualitative evaluation analyzes and compares the automated EHR-based ILI surveillance with traditional sentinel surveillance, using recommended criteria for evaluating public health surveillance systems, including simplicity, flexibility, data quality, acceptability, timeliness, and usefulness.4
METHODS
Electronic medical record Support for Public Health
The Electronic medical record Support for Public Health (ESP) application is an open-source disease-detection and reporting system currently operating in Massachusetts and Ohio that uses a fully automated data flow from commercial EHR systems.5–8 ESP's surveillance modules automatically execute disease-detection algorithms in EHR data, providing timely surveillance for public health agencies without placing extra demands on clinicians.9 Algorithms have been implemented for a wide range of infectious and chronic diseases and conditions. Technical features of the ESP system8 and its performance in reporting notifiable diseases to the state health department7,9–11 have been described previously. ESP source code is available free of charge from esphealth.org. We evaluated ESP's weekly aggregate ILI reporting module.
Population covered by surveillance systems and evaluation period
The Massachusetts ESP surveillance system covers the 700,000 patients in Eastern Massachusetts who use any of five of the six nonprofit community-based physician groups in the Atrius Health alliance, all five of which use EpicCare EHRs (Epic Systems, Verona, Wisconsin). Of Atrius Health's 50 practice locations, 30 have pediatric, family medicine, internal medicine, or urgent care departments that are likely to receive patients seeking care for ILI. These departments are the only ones that are included in the surveillance system. The age distribution of Atrius patients is as follows: 0–19 years (21.5%), 20–39 years (31.1%), 40–59 years (28.9%), and ≥60 years (18.5%). Of patients declaring their race/ethnicity, 79% are Caucasian, 8% are black, 6% are Asian, 4% are Hispanic, and 3% identified as “other.” The age and ethnic composition is very similar to that of Massachusetts as a whole.12,13 However, the Atrius population may not be representative of the uninsured or of more rural subpopulations.
The other two data sources used for comparative purposes—traditional sentinel surveillance for ILI and combined laboratory data from the Massachusetts laboratories belonging to the National Respiratory and Enteric Virus Surveillance System (NREVSS) and from the William A. Hinton State Laboratory Institute (SLI)—draw from the population of the whole state, approximately 6.5 million people. The evaluation covered the four-year period September 30, 2007–October 1, 2011. NREVSS data were not available for the last three months of this period.
Available demographic and geographic detail
ESP ILI surveillance data were aggregated into the age groups used by ILINet: 0–4, 5–24, 25–49, 50–64, and ≥65 years of age. Traditional sentinel surveillance data indicated ILI visits by age group but total visits only for all ages combined; thus, percent ILI by age group was not available for traditional surveillance. Age was not available in the summarized laboratory data.
The ILI data sources provided substate geographic detail, based on medical practice location, allowing comparison among regions, although data were sparse for some regions. The seven MDPH bioterrorism preparedness regions were used. Region was not available for the combined laboratory data.
ILI surveillance and reporting using ESP
To produce weekly percent ILI reports, the ESP system first identified cases of ILI in daily data, using an electronic algorithm developed in collaboration with MDPH and since used in other studies.14,15 The algorithm required (1) a measured temperature of ≥100°F or, if no measured temperature was available, an International Classification of Diseases, Ninth Revision (ICD-9)16 code for fever (780.6); and (2) at least one of the following respiratory illness symptoms, identified by ICD-9 code: viral infection (079.3, 079.89, 079.99), acute pharyngitis (460, 462), acute laryngitis and tracheitis (464.0, 464.1, 464.2, 465), acute bronchitis and bronchiolitis (466.0, 466.19), other diseases of the upper respiratory tract (478.9), pneumonia (480.8, 480.9, 481, 482.40, 482.41, 482.49, 484.8, 485, 486), influenza (487), throat pain (784.1), or cough (786.2). Daily counts of ILI by site and age group were extracted for the 30 Atrius sites. The daily counts were aggregated into Sunday-to-Saturday weekly reports of ILI visits, total visits (only clinic and urgent care visits—no telephone calls or prescription refills), and the ratio of these two counts (percent ILI), by site and age group. Aggregation was conducted retrospectively for the historical period. Since June 2010, weekly reports for all 30 sites have been sent to MDPH, usually every Tuesday, three days after the end of the week featured.
ILI surveillance and reporting using traditional sentinel surveillance
ILI sentinel surveillance in Massachusetts has been operational since the 1997–1998 influenza season. Sentinel providers report to MDPH on a weekly basis the total number of patients seen for any reason and the number of those patients with ILI, the latter by age group. ILI was defined as a temperature of ≥100°F plus a cough and/or sore throat in the absence of a known cause other than influenza. From these total and ILI visit counts, MDPH obtained the percentage of patients of all ages seen for ILI, by site. Sentinel providers were asked to report during weeks 40 through 20 (i.e., beginning of October through mid-May), with some sites reporting through the summer. During the summer of 2009, due to the H1N1 pandemic, all sites were asked to report throughout the year.
MDPH surpassed CDC's specification for state-level ILI surveillance of at least one regularly reporting sentinel provider per 250,000 population. With a target number of 26 regularly reporting providers, Massachusetts recruited 39, 42, 46, and 45 ILI sentinel providers in 2007–2008, 2008–2009, 2009–2010, and 2010–2011, respectively. Of these providers, 32/39 (82%), 37/42 (88%), 39/46 (85%), and 42/45 (93%) submitted reports for ≥16 weeks (defined as “regular reporting”) during the traditional reporting period of October through mid-May each season. Sixty-five percent of sentinel providers reported within four days for ≥16 weeks during 2010–2011. Of the 55 providers reporting in any of the four seasons, 29 (53%) participated in all four of them.
Laboratory data
NREVSS is maintained by CDC and includes weekly data for influenza from approximately 60 laboratories throughout the country, most of which are hospital-based. The number of Massachusetts laboratories participating in NREVSS varied from 10 to 12 per influenza season during the four influenza seasons studied, with eight reporting in all four seasons. (It was noteworthy that a single pediatric hospital conducted more tests than other reporting laboratories, contributing 75% of the total number of Massachusetts specimens tested for influenza and submitted to NREVSS, and 69% of the ones testing positive for influenza. Possible explanations for this occurrence are that the catchment area of this large children's hospital includes the whole state; most NREVSS specimens appear to be from the pediatric population, particularly due to respiratory syncytial virus [RSV] testing among very young children, based on personal communication with NREVSS staff; and NREVSS is a voluntary reporting system, which contributes to differences among hospitals with respect to testing volume.) The participating laboratories are able to report on tests performed for influenza, parainfluenza, RSV, and adenovirus, including typing and subtyping of influenza, using antigen detection, virus isolation, and polymerase chain reaction (PCR). Number of specimens tested, number positive for each influenza strain, and percent positive for any influenza were tracked weekly.
The SLI, one of about 80 World Health Organization collaborating laboratories in the U.S., conducts testing for influenza, including typing and subtyping, using PCR and viral culture. Number of specimens tested for influenza and number positive for the various types and subtypes were tracked weekly, from which the weekly proportion positive for any strain of influenza (“percent positive”) was calculated. Specimens tested at SLI included those submitted by sentinel clinicians who are asked to submit specimens throughout the season to support virologic surveillance.
Overlap between NREVSS and SLI data was manually identified and removed; we then combined these data by adding weekly numbers of specimens positive for influenza from the two systems and adding weekly total numbers of specimens tested from the two systems. In most influenza seasons, the number of positive influenza specimens in the SLI data was about 10%–30% of the total number; however, in the first wave of the H1N1 pandemic in spring 2009, SLI had 3.6 times as many influenza specimens as NREVSS, and its proportion of specimens in the 2008–2009 season was almost 60%. (Due to unmanageably high volume and consistent results, SLI sent out a clinical/laboratory advisory on June 5, 2009, announcing that SLI would no longer accept specimens for routine influenza testing as of June 12, 2009.)
Other Massachusetts laboratories report positive influenza tests to MDPH but without the total number of specimens tested for influenza; therefore, no percent positive is available, and their data were not included in this evaluation.
Comparisons and analysis
Primary comparisons and analyses were for all age groups and regions combined. We made comparisons among three data types: (1) ESP percent ILI, (2) traditional sentinel surveillance (Massachusetts ILINet sites) percent ILI, and (3) percent positive for any strain of influenza in combined NREVSS and SLI laboratory data. All data types were graphed to show temporal patterns during the 2007–2011 period. Spearman's correlation coefficients were calculated for all pairs of the three data types as well as for ESP percent ILI in 5- to 49-year-olds vs. percent positive in the combined laboratory data. Age- and region-specific temporal patterns were visually compared for the two ILI data sources, which contained age and geographic detail.
In the purely qualitative analysis, we compared the ESP and traditional ILI surveillance systems in terms of CDC guidelines for evaluating public health surveillance systems.4 The CDC criteria were simplicity, flexibility, data quality, acceptability, sensitivity, positive predictive value, representativeness, timeliness, stability, usefulness, portability, and cost. We made this comparison without surveying external stakeholders, but we considered it fair, especially given the fact that some of us were responsible for traditional sentinel surveillance and others were responsible for ESP-based surveillance.
RESULTS
Graphical comparisons
Figure 1 shows the trajectories of percent ILI in each of the two ILI surveillance systems and of percent positive influenza specimens in the combined laboratory data during the course of the four influenza seasons, for all ages combined and all regions combined, as well as ESP percent ILI for those aged 5–49 years. The timing of the five main peaks in each of the four data types never differed by more than one week. ESP percent ILI for those aged 5–49 years was never later than the others.
Figure 1.
Comparison of ESP percent ILI in all ages and in 5- to 49-year-olds, traditional sentinel surveillance percent ILI in all ages, and percent of laboratory specimens testing positive for influenza: Massachusetts, September 30, 2007–October 1, 2011
ESP = Electronic medical record Support for Public health
ILI = influenza-like illness
However, the magnitude of the peaks and the ratios of peaks to baselines differed among data types. Percent ILI was lower for the ESP all-ages group than in the traditional sentinel surveillance system during influenza seasons but similar to or slightly higher than sentinel percent ILI between influenza seasons. There was a more marked difference during the fall 2009 pandemic wave: the peak percent ILI was 9% for sentinel surveillance vs. <3% for ESP (Figure 1).
In ESP, percent ILI was consistently higher for those aged 0–4 years and 5–24 years than for those in older age groups (Figure 2). Notably, the amplitude of variation in percent ILI over time was greatest for the age groups 5–24 and 25–49 years, followed by 50- to 64-year-olds. The first two of these age groups are combined in Figure 1, and this combined group displays generally more pronounced peaks than the all-ages group, with similar values to traditional surveillance in three of the five peaks. Percent ILI typically peaked for the 5- to 24-year-old age group before the others, never after; in the first wave of the pandemic, this group peaked four weeks earlier than the 0- to 4-year-old group (Figure 2). Those aged 0–4 years experienced a peak in percent ILI in the week ending February 6, 2010, of equal magnitude as the preceding 2008–2009 and two 2009 pandemic peaks (Figure 2). This pattern was not seen for any other age group.
Figure 2.
Comparison of ESP percent ILI by age group: Massachusetts, September 30, 2007–October 1, 2011
ESP = Electronic medical record Support for Public health
ILI = influenza-like illness
The regions were similar in their gross temporal patterns, and there was no evidence of influenza or ILI appearing first in one region of the state and arriving in others later. However, the data were unstable for some regions, making it difficult to discern any such tendencies.
Correlations
ESP and traditional percent ILI were correlated, with a Spearman's correlation coefficient of 80%. ESP and traditional percent ILI were similar in their degree of correlation with the laboratory data—ESP percent ILI and traditional percent ILI had correlation coefficients of 73% and 71%, respectively, with percent positive laboratory tests. When restricted to those aged 5–49 years, ESP percent ILI was more highly correlated with the laboratory data, with a correlation coefficient of 83% (data not shown).
Qualitative comparison of ESP and traditional sentinel surveillance systems
Figure 3 shows the ESP and traditional ILI surveillance systems side by side with respect to criteria for evaluating public health surveillance systems.4 Regarding cost, requirements for installing ESP included purchasing a server, installing ESP software onto the server, programming the host practice's EHR to export raw data to ESP, mapping local laboratory test codes to universal concepts (not necessary for ILI surveillance but required by ESP's notifiable disease and chronic disease surveillance modules), and validating a sample of cases. The total cost varies widely depending on the host practice's informatics resources and expertise but is likely in the tens of thousands of dollars.
Figure 3.
Comparison of traditional sentinel surveillance and ESP (EHR-based) surveillance for ILI in Massachusetts, with respect to criteria in CDC guidelines for evaluating public health surveillance systems
ESP = Electronic medical record Support for Public health
EHR = electronic health record
ILI = influenza-like illness
CDC = Centers for Disease Control and Prevention
ICD-9 = International Classification of Diseases, Ninth Revision
DISCUSSION
ESP-generated and traditional sentinel surveillance weekly percent ILI reports derived from routinely collected data in EHRs agreed well in the timing of their five major peaks, always peaking within one week of each other. These ILI data sources were also similar in terms of their degree of correlation with the laboratory data. However, the two ILI data sources differed in the magnitude of percent ILI. Sentinel surveillance percent ILI was higher than ESP percent ILI during influenza seasons—markedly so during the second wave of the H1N1 pandemic—but was often slightly lower than ESP percent ILI at other times.
To understand why percent ILI was higher for sentinel surveillance than for ESP during influenza seasons, we considered possible differences in how numerators and denominators were obtained and in the age structure of the populations in each of the surveillance systems. The age groups 0–4 and 5–24 years contributed the preponderance of ILI visits in both systems. One possible explanation is that sentinel providers could use recent history of fever to determine ILI, whereas ESP would require fever to be perceived and entered as a measured temperature or a diagnosis of fever during the visit for a case to be captured as ILI. As for why sentinel surveillance had a much higher peak than ESP in the second wave of the pandemic, it is possible that direct clinical examination is simply a more sensitive detection method than running algorithms on diagnostic codes. But it is also possible that, because of the great amount of publicity concerning novel H1N1, sentinel clinicians were more likely than at other times to attribute any of a broad array of clinical presentations, even mild ones, to influenza, classifying them all as ILI. The publicity may well have influenced coding of visits captured by ESP as well, but perhaps a clinician would be more easily influenced by hype about a pandemic when counting cases of ILI than when entering specific diagnoses into the EHR system.
The availability of percent ILI in specific age groups is a major advantage of the ESP system -compared with traditional sentinel surveillance, allowing a more detailed examination of ILI epidemiology. For example, we noted a peak in percent ILI in the age group 0–4 years in the week ending February 6, 2010, which was not seen for other age groups. This peak coincided with a distinct two-month peak in percent positive for RSV at a Boston pediatric hospital participating in NREVSS, December 20, 2009–February 20, 2010, suggesting that the peak percent ILI in young children may have been due to RSV infection. We also noted that the signal-to-noise ratio was highest for those in the age groups 5–24 and 25–49 years and that the correlation between percent ILI and percent positive in the laboratory data was higher for those aged 5–49 years (83%) than for all ages combined (73%). The finding that percent ILI among those aged 5–24 years appeared to peak earlier than in other groups suggests that surveillance efforts seeking to detect peaks clearly and early might be well served to focus primarily on the school-aged and young adult populations.
In the side-by-side qualitative comparison of the two ILI surveillance systems, there was no clear winner. Major strengths of the ESP system, other than the availability of age-specific percent ILI, are the fact that it does not impose a burden on clinicians or public health staff; it offers consistent, timely, year-round reports; and it is potentially scalable to be truly population-based as EHR systems become more widely available. Downsides of ESP ILI surveillance compared with sentinel surveillance are the possibility that the sensitivity and/or positive predictive value of ILI identified by an electronic algorithm is lower than that of ILI diagnosed by direct clinical examination and possible lower sensitivity to differences in ILI activity among seasons (e.g., traditional surveillance percent ILI and laboratory percent positive both had high peaks in the fall of 2009, while ESP percent ILI did not). In addition, the initial cost of installing and validating ESP, typically borne by either the medical practice or the health department, is high; however, for this expenditure, ESP provides additional benefits, such as automated reporting of other diseases.
CONCLUSIONS
As EHR systems are adopted by more physician practices, automated, integrated disease surveillance and reporting systems such as ESP hold great promise for disease surveillance in effectively shifting the burden of disease reporting from clinicians to information systems. The ESP system also allows tracking of disease by age group, can include a bigger population, operates year-round, produces consistent and timely reports, and allows system-wide changes to be implemented centrally.
Footnotes
Research support was provided by Cooperative Agreement #U01GM076672 from the National Institute of General Medical Sciences under the Models of Infectious Disease Agent Study program, and Cooperative Agreement #1P01HK00088 from the Centers for Disease Control and Prevention (CDC) Centers for Excellence in Public Health Informatics program. The authors thank Joyce Cohen, Inna Dashevsky, Allyson Kats, Julie Lankiewicz, and Dr. Susan Lett for their contributions.
This study received Institutional Review Board approval from Harvard Pilgrim Health Care. The findings and conclusions in this article are those of the authors and do not necessarily represent the opinions of CDC.
REFERENCES
- 1.Brammer TL, Murray EL, Fukuda K, Hall HE, Klimov A, Cox NJ. Surveillance for influenza—United States, 1997–98, 1998–99, and 1999–00 seasons. MMWR Surveill Summ. 2002;51(7):1–10. [PubMed] [Google Scholar]
- 2.Centers for Disease Control and Prevention (US) Overview of influenza surveillance in the United States [cited 2013 May 3] Available from: URL: http://www.cdc.gov/flu/weekly/overview.htm.
- 3.Olson DR, Heffernan RT, Paladini M, Konty K, Weiss D, Mostashari F. Monitoring the impact of influenza by age: emergency department fever and respiratory complaint surveillance in New York City. PLoS Med. 2007;4:1349–61. doi: 10.1371/journal.pmed.0040247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep. 2001;50(RR-13):1–35. [PubMed] [Google Scholar]
- 5.Klompas M, Lazarus R, Daniel J, Haney G, Hou X, Campion FX, et al. Electronic medical record Support for Public health (ESP): automated detection and reporting of statutory notifiable diseases to public health authorities. Adv Dis Surveill. 2007;3:1–5. [Google Scholar]
- 6.Klompas M, Lazarus R, Haney G, Hou X, Daniel J, Campion F, et al. The Electronic Support for Public health (ESP) project: automated detection and electronic reporting of notifiable diseases. Adv Dis Surveill. 2007;4:52. [Google Scholar]
- 7.Klompas M, McVetta J, Lazarus R, Eggleston E, Haney G, Kruskal BA, et al. Integrating clinical practice and public health surveillance using electronic medical record systems. Am J Prev Med. 2012;42(6 Suppl 2):S154–62. doi: 10.1016/j.amepre.2012.04.005. [DOI] [PubMed] [Google Scholar]
- 8.Lazarus R, Klompas M, Campion FX, McNabb SJN, Hou X, Daniel J, et al. Electronic Support for Public health: validated case finding and reporting for notifiable diseases using electronic medical data. J Am Med Inform Assoc. 2009;16:18–24. doi: 10.1197/jamia.M2848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Klompas M, Lazarus R, Platt R, Hou X, Campion FX, Kruskal B, et al. Automated detection and reporting of notifiable diseases using electronic medical records versus passive surveillance—Massachusetts, June 2006–July 2007. MMWR Morb Mortal Wkly Rep. 2008;57(14):373–6. [PubMed] [Google Scholar]
- 10.Klompas M, Haney G, Church D, Lazarus R, Hou X, Platt R. Automated identification of acute hepatitis B using electronic medical record data to facilitate public health surveillance. PLoS One. 2008;3:e2626. doi: 10.1371/journal.pone.0002626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Calderwood MS, Platt R, Hou X, Malenfant J, Haney G, Kruskal B, et al. Real-time surveillance for tuberculosis using electronic health record data from an ambulatory practice in Eastern Massachusetts. Public Health Rep. 2010;125:843–50. doi: 10.1177/003335491012500611. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Census Bureau (US) Community facts [cited 2013 Sep 1] Available from: URL: http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=bkmk.
- 13.Census Bureau (US) State and county quickfacts: Massachusetts [cited 2013 Sep 1] Available from: URL: http://quickfacts.census.gov/qfd/states/25000.html.
- 14.Yih WK, Deshpande S, Fuller C, Heisey-Grove D, Hsu J, Kruskal BA, et al. Evaluating real-time syndromic surveillance signals from ambulatory care data in four states. Public Health Rep. 2010;125:111–20. doi: 10.1177/003335491012500115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Greene SK, Kulldorff M, Huang J, Brand RJ, Kleinman KP, Hsu J, et al. Timely detection of localized excess influenza activity in Northern California across patient care, prescription, and laboratory data. Stat Med. 2011;30:549–59. doi: 10.1002/sim.3883. [DOI] [PMC free article] [PubMed] [Google Scholar]