Abstract
Objective
To estimate a commercially available ambulatory electronic health record’s (EHR’s) impact on workflow and financial measures.
Data Sources/Study Setting
Administrative, payroll, and billing data were collected for 26 primary care practices in a fee-for-service network that rolled out an EHR on a staggered schedule from June 2006 through December 2008.
Study Design
An interrupted time series design was used. Staffing, visit intensity, productivity, volume, practice expense, payments received, and net income data were collected monthly for 2004–2009. Changes were evaluated 1–6, 7–12, and >12 months postimplementation.
Data Collection/Extraction Methods
Data were accessed through a SQLserver database, transformed into SAS®, and aggregated by practice. Practice-level data were divided by full-time physician equivalents for comparisons across practices by month.
Principal Findings
Staffing and practice expenses increased following EHR implementation (3 and 6 percent after 12 months). Productivity, volume, and net income decreased initially but recovered to/close to preimplementation levels after 12 months. Visit intensity did not change significantly, and a secular trend offset the decrease in payments received.
Conclusions
Expenses increased and productivity decreased following EHR implementation, but not as much or as persistently as might be expected. Longer term effects still need to be examined.
Keywords: Electronic health records, workflow, financial performance
Despite electronic health records’ (EHRs’) perceived potential to improve care, the 2011 National Ambulatory Medical Care Survey revealed that only 55 percent of U.S. physicians had adopted an EHR (Jamoom et al. 2012). Of these physicians, 74 percent believe it enhances patient care (Jamoom et al. 2012). However, physicians frequently cite financial barriers to adoption: the lack of reliable information about return on investment, decreased productivity, and that costs are borne by the practice but most potential savings accrue to payers (Boonstra and Broekhuis 2010; Police, Foster, and Wong 2010; Yan, Gardner, and Baier 2012). The context for EHR adoption has changed substantially with the Medicare and Medicaid Meaningful Use incentives, and it is now frequently viewed as inevitable (Song et al. 2011). Nevertheless, the experience of practices that adopted EHRs prior to these incentives remains relevant, providing information on the impact on productivity, volume, staffing, and income (Song et al. 2011).
We previously reported the hardware, software, and time-and-effort costs of implementing a commercially developed EHR in 26 primary care practices (Fleming et al. 2011). We now report the impact on productivity, staffing, and financial measures in those same practices.
Methods
Setting
HealthTexas Provider Network (HTPN) is the fee-for-service ambulatory care provider network affiliated with Baylor Health Care System in Dallas-Fort Worth, Texas. HTPN comprises >100 primary care, specialty care, and senior health centers, and >450 physicians. As of July 2013, HTPN had 400 physicians eligible for Medicare Meaningful Use incentives, all of whom were in Stage 1.
Given the differences in work flow and care provided between specialties, we limited our study to the 26 primary care practices—those with physicians specializing in family or internal medicine—that implemented the EHR between June 2006 and December 2008.
Intervention
HTPN’s web-based, externally hosted EHR package comprises GE Centricity Physician Office—EMR 2005, Clinical Content Consultants advanced forms, and Kryptiq Secure Messaging and Docutrack. These components integrate clinical and demographic information and incorporate clinical content and decision support, secure physician-physician messaging, and integrated scanning. The EHR does not subsume practice management functions, so the Misys PM and Vision practice management and billing systems HTPN has used since 2002 remained in place.
EHR implementation occurred on a staggered schedule (Figure 1), based on practices’ technical readiness and willingness (Fullerton et al. 2006). HTPN policy made EHR adoption, and its use for all patient encounters following a practice’s launch date, mandatory.
Study Outcomes
Workflow: Staff per Physician Full-time Equivalent (FTE) (staffing), Work Relative Value Unit (RVU) per Visit (intensity), Work RVU per Physician FTE (productivity), and Visits per Physician FTE (volume). “Staff” included medical assistants and office staff: since precise roles vary between practices and by daily demand, we did not separate these job categories.
Financial: Practice Expense per Work RVU, Payment Received per Work RVU, Net Income per Work RVU, and Net Income per Physician FTE.
Study Covariates
For each practice, we collected data on patient characteristics (mean age, percentage female), physician characteristics (number of physicians, length of time in HTPN, practice type [family medicine, internal medicine, or “other,” which represents combined primary care specialties]), and year of adoption (2006/2007 vs. 2008).
Data Collection
Data were collected monthly from January 2004 to December 2009, providing ≥30 months preimplementation and ≥12 months postimplementation for all practices.
Data related to individual visits and revenues were collected from the Misys PM billing system. These included patient demographic information and detailed visit component information (e.g., CPT-4 codes). Charges were captured at the procedure code level and linked to RVU values, obtained from Ingenix (Ingenix). The 2009 RVU scale was used for all years to eliminate the impact of changes in nominal RVU values for specific CPT-4 codes.
Practice expense and staffing data were obtained from the general ledger and payroll systems. Payroll data include hours and pay information, along with cost centers and accounts. The collections balance to the general ledger at the visit level. Provider number is linked to the general ledger cost centers and accounts; thus, payroll and practice expense data were merged with billing system data at cost center and practice levels. Billing system accounts receivable is reconciled to the general ledger through regular external audits.
Data were accessed through a SQL server database and analyzed in SAS (Cary, NC, USA). Data were aggregated by practice on a monthly basis then divided by the number of physician FTEs to enable comparisons across practices.
Statistical Analysis
We used a random intercept and random slope statistical model that provides the flexibility for analysis of repeated data—here, 72 months of observations in 26 practices—by allowing each practice to have its own intercept and slope for the trend variable (Fitzmaurice, Laird, and Ware 2004). We estimated the preimplementation linear trend for each measure and assumed postimplementation persistence. They were used to account for increases related to price (measured by the medical component of the Consumer Price Index), as the sensitivity of the trends provides the best method for adjusting financial data. While micro-level (e.g., patient visit) financial data often require logarithmic methods with extreme distributions (Glick et al. 2007), monthly, practice-level data are less likely to be distributed non-normally. We conducted tests of normality on the resulting residual to ensure our methods were appropriate.
We estimated the effects for the following linear model: Yit = β0 + βAEHR*EHR + βT*Tit + βH*H + εit where Yit is the workflow or financial measure for practice i (i = 1–26 practices) at time t (months since January 2004) and H is a vector of patient- and practice-level covariates (those listed in Table 1 and adopting group [2006/2007 vs 2008]). β0 represents the preimplementation secular trend. Using the preimplementation period as the reference, we tested H0: βAEHR = 0 for each of three postimplementation periods (1–6 months, 7–12 months, and >12 months) to determine if the EHR affected these measures beyond any difference attributable to the secular trend. For net income per work RVU and net income per physician FTE, the trends appeared curvilinear; we used linear regression splines with four knots to smooth the data rather than forcing assumptions of linearity (Verbyla et al. 1999; Ruppert, Wand, and Carroll 2003). The coefficients represent the shift in the intercept for practices with parallel random slopes parallel to the preimplementation secular trend for the three postimplementation periods. Percentage change from preimplementation was calculated by dividing the estimated coefficients by the overall mean values of the workflow and financial measures. We examined multiple postimplementation periods since interventions often have a “burn-in” effect. We also included an implementation group effect, accounting for the early and late adopters in the staggered implementation: two practices implemented the EHR in late 2006, 14 in 2007, and 10 in 2008. We dichotomized this variable as 2006/2007 versus 2008.
Table 1.
Mean (SE) | |
---|---|
Physician variables | |
No. of physicians—n | 5.20 (0.12) |
Time in HTPN—years | 5.96 (0.06) |
Patient variables | |
Percentage of female patients—% | 61.07 (0.15) |
Age of patients—years | 47.67 (0.17) |
Practice type | N (%) |
Internal medicine | 8 (30.8) |
Family medicine | 13 (50.0) |
Other | 5 (19.2) |
EHR exposure | Practice-months (%) |
0 months | 1,192 (64.6) |
1–6 months | 156 (8.5) |
7–12 months | 156 (8.5) |
>12 months | 340 (18.4) |
Results
Table 1 shows the characteristics of the 26 practices. Table 2 shows the means and standard errors for the workflow and financial measures from 2004 to 2009. Table 3 shows the regression coefficients, representing the change in each measure in relation to the preimplementation level. Practice-type coefficients were not statistically significant—likely because of high correlation with the mean number of physicians in the practice.
Table 2.
Practice-Months (n) | Overall | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 |
---|---|---|---|---|---|---|---|
1,844 | 302 | 312 | 312 | 312 | 309 | 297 | |
Mean (SE) | Mean (SE) | Mean (SE) | Mean (SE) | Mean (SE) | Mean (SE) | Mean (SE) | |
Workflow | |||||||
Staff per physician FTE (n) | 3.42 (0.03) | 3.55 (0.07) | 3.35 (0.06) | 3.37 (0.06) | 3.43 (0.06) | 3.45 (0.06) | 3.38 (0.06) |
Work RVU per visit (RVU) | 1.05 (0.00) | 1.05 (0.01) | 1.06 (0.01) | 1.07 (0.0) | 1.05 (0.01) | 1.04 (0.01) | 1.05 (0.01) |
Visits per physician FTE (RVU) | 396.34 (2.43) | 396.77 (6.23) | 390.56 (5.90) | 402.51 (5.87) | 395.69 (6.05) | 393.22 (5.663 | 399.42 (5.95) |
Work RVU per physician FTE (RVU) | 412.29 (2.356) | 412.42 (5.94) | 410.77 (5.74) | 423.09 (5.60) | 408.65 (5.76) | 403.63 (5.65) | 415.21 (5.93) |
Financial | |||||||
Practice expense per work RVU ($) | 70.35 (0.31) | 65.61 (0.82) | 66.83 (0.76) | 67.79 (0.65) | 71.35 (0.69) | 74.94 (0.78) | 75.71 (0.63) |
Payment received per work RVU ($) | 107.44 (0.40) | 102.44 (0.99) | 103.69 (0.93) | 103.38 (0.95) | 110.26 (0.90) | 109.15 (0.92) | 111.78 (1.00) |
Net income per work RVU ($) | 35.28 (0.37) | 34.21 (0.82) | 34.38 (0.89) | 37.59 (0.77) | 37.85 (0.87) | 33.14 (1.05) | 34.41 (1.01) |
Net income per physician FTE ($) | 15,155.0 (185.59) | 14,592.4 (427.89) | 14,788.6 (433.29) | 16,395.1 (407.91) | 15,980.1 (468.55) | 14,083.6 (495.11) | 15,057.4 (479.79) |
Table 3.
1–6 months | 7–12 months | >12 months | |||||||
---|---|---|---|---|---|---|---|---|---|
Change from Preimplementation* (%) | Regression Coefficient (SE) | p-value | Change from Preimplementation* (%) | Regression Coefficient (SE) | p-value | Change from Preimplementation* (%) | Regression Coefficient (SE) | p-value | |
Workflow | |||||||||
Staff per physician FTE | 5.56 | 0.19 (0.04) | <.01 | 2.92 | 0.10 (0.04) | .02 | 3.51 | 0.12 (0.05) | .01 |
Work RVU per visit | −0.10 | −0.001 (0.006) | .92 | 1.90 | 0.02 (0.01) | .02 | 0.29 | 0.003 (0.001) | .68 |
Visits per physician FTE | −8.07 | −31.99 (4.70) | <.01 | −7.48 | −29.63 (5.08) | <.01 | −4.51 | −17.86 (5.36) | <.01 |
Work RVU per physician FTE | −7.97 | −32.84 (4.49) | <.01 | −5.41 | −22.29 (4.86) | <.01 | −4.03 | −16.62 (5.15) | <.01 |
Financial | |||||||||
Practice expense per work RVU | 5.42 | 3.81 (0.66) | <.01 | 5.76 | 4.05 (0.71) | <.01 | 5.96 | 4.19 (0.75) | <.01 |
Payment received per work RVU | −2.82 | −3.03 (0.47) | <.01 | −3.27 | −3.51 (0.51) | <.01 | −4.37 | −4.70 (0.54) | <.01 |
Net income per work RVU | −11.08 | −3.91 (1.49) | <.01 | −12.05 | −4.25 (1.74) | .146 | −11.11 | −3.92 (2.05) | .06 |
Net income per physician FTE | −16.46 | −2,494.37 (710.65) | <.01 | −11.65 | −1,765.59 (828.86) | .03 | −7.91 | −1,199.12 (982.89) | .22 |
Percentage changes from preimplementation was calculated by dividing the coefficients by the overall mean values (from Table 2) of the workflow and financial measures.
Workflow Measures
No statistically significant preimplementation trends were observed in work-flow measures (all p-values >.10).
Staff per physician FTE (staffing) increased during the first 6 months postimplementation, then dropped closer to preimplementation levels. Relative to the overall mean of 3.42 staff per physician FTE (Table 2), the increase was small—approximately 6 percent in the first 6 months and 3 percent after 12 months.
Work RVUs per visit (intensity) showed no significant differences from preimplementation during the first 6 months, or after 12 months. The 0.02 increase in months 7–12 (Table 3) was statistically significant but nominal compared with the mean of 1.05 (Table 2).
Work RVUs per physician FTE (productivity) showed statistically significant decreases postimplementation. Productivity was lowest during the first 6 months (8 percent decrease), but it regained half this ground by 12 months.
Visits per physician FTE (volume) followed a similar pattern, dropping 8 percent during the first 6 months, but recovering 3.5 percent after 12 months.
Financial Measures
Statistically significant preimplementation trends were seen for practice expense per work RVU ($0.21 per month, p < .001), payments received per work RVU ($0.29 per month, p < .001), and net income per physician FTE (-$68 per month, p < .001).
Practice expense per work RVU increased $3.81–$4.19 per month beyond the secular trend, depending on time from implementation (Table 3). Based on the monthly mean of 412.29 work RVUs per physician FTE (Table 2), the increased expense is approximately $1,650 per physician FTE per month.
Decreases in payment received per work RVU increased with time from EHR implementation (Table 3) but were largely offset by the preimplementation trend. An additional model containing trend confirmed persistence of the secular trend at $0.23 per month (p < .001) postimplementation.
Net income per work RVU decreased significantly during the first year postimplementation (11.1-12.1 percent) Net income per physician FTE showed a significant decrease beyond the negative secular trend during the first 6 months (16.5 percent); but after 12 months was not significantly different (p = .22).
Sensitivity Analysis
Figure 2 shows the distribution of practice sizes. We conducted the same analyses, excluding the largest practice, to see if our results differed for “small” practices (≤10 physicians), and observed no material difference.
Discussion
While expenses do increase and productivity does decrease following EHR implementation, the effects are not as large or persistent as might be expected. We saw increased practice expenses of approximately $1,650 per physician FTE per month, which corresponds closely to the monthly $1,425 per physician cost of EHR maintenance ($1,225 for software licensing, networking, and hosting and technical support through a third-party vendor, and $200 of additional maintenance personnel support) reported in the evaluation of the costs associated with HTPN’s EHR implementation (Fleming et al. 2011). The other contributing expense was the 3 percent increase in staffing.
We also observed decreased productivity and patient volume postimplementation, but by 12 months, performance was only approximately 4 percent below preimplementation, suggesting full recovery could be achieved. Net income per physician also decreased but rebounded, suggesting little long-term detriment. As intensity of services did not change significantly, the initial drop in net income was likely related to increased expenses and decreased productivity, rather than changes in case mix.
Decreased payment per work RVU could cause decreased net income per physician. However, this decrease was offset by an increasing secular trend, suggesting that annual contracting rates play a determinative role. The overall lack of change in payment per work RVU during our study suggests a flattening of reimbursement rates (best seen in Table 2 for years 2007–2009), such that these should have little impact on changes in the net income measures.
A decrease in net income could result from lower net collection—for example, if integrating the EHR with the practice management system proved problematic, affecting practices’ ability to collect the charges billed. This is unlikely here: HTPN continued using its preexisting practice management system, which was not integrated with the EHR; the codes/charges for services rendered continued to be entered manually. Furthermore, collection typically only affects income from self-pay patients, which constitute only 3 percent of HTPN patients.
One potential issue in interpreting our results is the impact of the 2008 recession. Its effect was mitigated by (but also confounded with) the early versus late adopter variable in our models. The 1-year postimplementation activity for early adopters occurred before the recession, and a subgroup analysis found their productivity and net income recovered completely at >12 months. The late adopters did not recover during the study period, but we cannot tell how much of this difference is attributable to differences between early and late adopters versus the recession. In addition, since the impact of the recession was not uniform across the United States (Semega 2009), the effects may not be broadly generalizable.
Our results show both consistencies and inconsistencies with similar evaluations. Like the 2010 MGMA Electronic Health Record Impacts on Revenue, Costs, and Staffing report, we saw increased costs for equipment and software maintenance and support staff (Gans 2010). We did not observe that the revenue increases MGMA reports, which could be attributable to our shorter postimplementation time (<2 years vs. >5 years). One academic pediatric primary care center reported increased revenue at 2 years postimplementation, but this was explained by a 13 percent increase in coding for detailed level visits and a 7 percent decrease for problem-focused visits (Samaan et al. 2009). While we did not look at visit coding, visit intensity did not change significantly, suggesting no similar change. This is interesting given the concern that EHRs result in visit “upcoding” (Abelson and Creswell 2013). Our results suggest this does not necessarily follow when the practice management system remains separate and consistent with that used prior to EHR implementation. However, lack of integration between the EHR and practice management system may also represent an opportunity missed to achieve greater productivity and efficiency through more efficient charge capture, and reduced time and effort for manual charge entry.
Differences in preimplementation practice management systems and coding practices may also explain why we did not see the significantly higher average monthly patient visits and work RVUs per physician with EHR adoption reported by the Weill Cornell physician group (Cheriff et al. 2010). They also observed a significant decrease in work RVU per visit, leading to speculation that the EHR captured and recorded low-intensity services that were previously undocumented and unbilled, making it appear that volume and productivity had increased (Cheriff et al. 2010). Finally, EHR implementation in five University of Rochester Medical Center ambulatory care practices had a neutral impact on efficiency and billing (similar to the small effects we saw on visits per physician FTE and payments received per work RVU), but total savings of $14,055 per provider, ongoing annual savings of $9,983 per provider, and a reduction of 1 staff FTE per physician FTE (Grieger, Cohen, and Krusch 2007). The savings were primarily due to reductions in chart pulls and support staff (Grieger, Cohen, and Krusch 2007). Since we did not separate chart pulls from the activities accounted for under “staff per physician FTE,” it is hard to compare our results. However, the opposite effect on support staff needs suggests that they play different roles in the practices, and detail on their activities is needed to truly determine the impact of the EHR.
Our observational study carries the inherent limitation of possible imbalance in unobserved differences that confound the outcomes. However, all practices ultimately implemented the EHR, and ≥12 months of data pre- and postimplementation were included for each practice, minimizing any imbalance. In addition, the interrupted time series design reduces the threat of historical events and selection bias by looking at an intervention occurring at different times across all included practices (Mercer et al. 2007).
The other substantial limitation is that we evaluated a product from a single vendor in a single network, meaning our results may not be generalizable to dissimilar settings—for example, managed care—or products. When HTPN chose its EHR in 2004–2005, GE Centricity Office typically ranked in the top three products, as reported through such mechanisms as the Towards the Electronic Patient Record annual conference. In 2006, when implementation began, it was the most widely used ambulatory care EHR in the United States and ranked fifth by KLAS (Enrado 2006). As such, HTPN’s experience should have been similar or superior to that of other practices implementing an EHR. Like other high-ranked products, GE Centricity Office incorporated CPOE, e-prescribing, and electronic documenting and charting, and had the ability to interface with disparate hospital systems (Enrado 2006); it differed in that it did not integrate the billing and scheduling applications. GE Centricity Office’s successor was rated as “average” across all categories at the end of our study period (KLAS).
Having studied only primary care practices also limits the generalizability. While subspecialties that function similarly to primary care might have a similar experience, those that do not (e.g., surgical subspecialties with high patient volumes, which would magnify any per visit time losses or gains; or cardiology, which relies heavily on ancillary testing services) may see very different effects. Furthermore, we evaluated an EHR geared toward general ambulatory care rather than any of the specialty-specific EHRs that have been developed.
Neither our results nor other recent reports show persistent substantial decreases in productivity or financial performance following EHR implementation. Nevertheless, the physicians whose bottom lines are affected may find the changes practically significant and must weigh them against the evidence of EHRs’ impact on quality of care (Crosson et al. 2012; Reed et al. 2012; Walsh et al. 2012). Future research should examine staff roles and visit capture by paper-based records: if practices considering EHR implementation can identify which models their practices follow, they will be better able to predict the EHR’s impact. Research is also needed on long-term effects: additional experience may enable practices to realize gains in productivity and net income.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: This work was funded by the Agency for Health Care Research and Quality (grant no.: AHRQ 1R-03 HS018220-01S). The authors thank Phil Aponte, M.D., for technical advice regarding the EHR; Ashley Collinsworth, M.P.H., E.L.S., for project management; Rustam Kudyakov, M.D., M.P.H., for data management; and Rebecca Roper, the AHRQ project officer for the grant that funded this work, for her support. The authors have no conflicts of interest.
Disclosures: None.
Disclaimers: None.
Supporting Information
Additional Supporting Information may be found in the online version of this article at the publisher’s web-site:
References
- Abelson R, Creswell J. 2013. U.S. Warning to Hospitals on Medicare Bill Abuses ” [accessed on June 28, 2013]. Available at http://www.nytimes.com/2012/09/25/business/us-warns-hospitals-on-medicare-billing.html.
- Boonstra A, Broekhuis M. Barriers to the Acceptance of Electronic Medical Records by Physicians from Systematic Review to Taxonomy and Interventions. BMC Health Services Research. 2010;10:231. doi: 10.1186/1472-6963-10-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cheriff AD, Kapur AG, Qiu M, Cole CL. Physician Productivity and the Ambulatory EHR in a Large Academic Multi-Specialty Physician Group. International Journal of Medical Informatics. 2010;79(7):492–500. doi: 10.1016/j.ijmedinf.2010.04.006. [DOI] [PubMed] [Google Scholar]
- Crosson JC, Ohman-Strickland PA, Cohen DJ, Clark EC, Crabtree BF. Typical Electronic Health Record Use in Primary Care Practices and the Quality of Diabetes Care. Annals of Family Medicine. 2012;10(3):221–7. doi: 10.1370/afm.1370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Enrado P. 2006. Buyers Guide: Ambulatory EMRs ” [accessed on June 28, 2006]. Available at http://www.healthcareitnews.com/news/buyers-guide-ambulatory-emrs.
- Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis. Hoboken, NJ: John Wiley & Sons Inc; 2004. [Google Scholar]
- Fleming NS, Culler SD, McCorkle R, Becker ER, Ballard DJ. The Financial and Nonfinancial Costs Of Implementing Electronic Health Records in Primary Care Practices. Health Affairs (Millwood) 2011;30(3):481–9. doi: 10.1377/hlthaff.2010.0768. [DOI] [PubMed] [Google Scholar]
- Fullerton C, Aponte P, Hopkins R, Bragg D, Ballard DJ. Lessons Learned from Pilot Site Implementation of an Ambulatory Electronic Health Record. Proceedings (Baylor University Medical Center) 2006;19(4):303–10. doi: 10.1080/08998280.2006.11928188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gans DN. With Time Comes Performance. MGMA Connexion. 2010;10(10):23–4. [PubMed] [Google Scholar]
- Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic Evaluation in Clinical Trials. Oxford: Oxford University Press; 2007. [Google Scholar]
- Grieger DL, Cohen SH, Krusch DA. A Pilot Study to Document the Return on Investment for Implementing an Ambulatory Electronic Health Record at an Academic Medical Center. Journal of the American College of Surgeons. 2007;205(1):89–96. doi: 10.1016/j.jamcollsurg.2007.02.074. [DOI] [PubMed] [Google Scholar]
- Ingenix. [accessed on November 21, 2012]. Available at http://www.ingenix.com/physicians/overview/
- Jamoom E, Beatty P, Bercovitz A, Woodwell D, Palso K, Rechtsteiner E. Physician Adoption of Electronic Health Record Systems: United States, 2011. NCHS Data Brief. 2012;July(98):1–8. [PubMed] [Google Scholar]
- KLAS. GE Healthcare Centricity Practice Solutions EMR: Current Performance ” [accessed on June 28, 2013]. Available at http://www.klasresearch.com/products/overview/547.
- Mercer SL, DeVinney BJ, Fine LJ, Green LW, Dougherty D. Study Designs for Effectiveness and Translation Research: Identifying Trade-Offs. American Journal of Preventive Medicine. 2007;33(2):139–54. doi: 10.1016/j.amepre.2007.04.005. [DOI] [PubMed] [Google Scholar]
- Police RL, Foster T, Wong KS. Adoption and Use of Health Information Technology in Physician Practice Organisations: Systematic Review. Informatics in Primary Care. 2010;18(4):245–58. doi: 10.14236/jhi.v18i4.780. [DOI] [PubMed] [Google Scholar]
- Reed M, Huang J, Graetz I, Brand R, Hsu J, Fireman B, Jaffe M. Outpatient Electronic Health Records and the Clinical Care and Outcomes of Patients with Diabetes Mellitus. Annals of Internal Medicine. 2012;157(7):482–9. doi: 10.7326/0003-4819-157-7-201210020-00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruppert D, Wand MP, Carroll RJ. Semiparametric Regression. New York: Cambridge University Press; 2003. [Google Scholar]
- Samaan ZM, Klein MD, Mansour ME, DeWitt TG. The Impact of the Electronic Health Record on an Academic Pediatric Primary Care Center. Journal of Ambulatory Care Management. 2009;32(3):180–7. doi: 10.1097/JAC.0b013e3181ac9667. [DOI] [PubMed] [Google Scholar]
- Semega J. 2009. Median Household Income for States: 2007 and 2008 American Community Surveys ” [accessed on November 21, 2009]. Available at http://www.census.gov/prod/2009pubs/acsbr08-2.pdf.
- Song PH, McAlearney AS, Robbins J, McCullough JS. Exploring the Business Case for Ambulatory Electronic Health Record System Adoption. Journal of Healthcare Management. 2011;56(3):169–80. discussion 81-2. [PubMed] [Google Scholar]
- Verbyla AP, Cullis BR, Kenward MG, Welham SJ. The Analysis of Designed Experiments and Longitudinal Data by Using Smoothing Splines (with Discussion) Journal of the Royal Statistics Society, Series C. 1999;48:269–312. [Google Scholar]
- Walsh MN, Albert NM, Curtis AB, Gheorghiade M, Heywood JT, Liu Y, Mehra MR, O’Connor CM, Reynolds D, Yancy CW, Fonarow GC. Lack of Association between Electronic Health Record Systems and Improvement in Use of Evidence-Based Heart Failure Therapies in Outpatient Cardiology Practices. Clinical Cardiology. 2012;35(3):187–96. doi: 10.1002/clc.21971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yan H, Gardner R, Baier R. Beyond the Focus Group: Understanding Physicians’ Barriers to Electronic Medical Records. Joint Commission Journal on Quality and Patient Safety. 2012;38(4):184–91. doi: 10.1016/s1553-7250(12)38024-0. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.