Abstract
OBJECTIVE
To test the hypothesis that electronic patient-provider messaging is associated with high care quality for diabetes and lower outpatient utilization.
RESEARCH DESIGN AND METHODS
We conducted a cross-sectional analysis of electronic patient-provider messaging over a 15-month period between 1 January 2004 and 31 March 2005. The study was set at Group Health Cooperative—a consumer-governed, nonprofit health care system that operates in Washington and Idaho. Participants included all patients aged ≥18 years with a diagnosis of diabetes. In addition to usual care, all patients had the option to use electronic messaging to communicate with their care providers. The primary outcome measures were diabetes-related quality-of-care indicators (A1C, blood pressure, and LDL cholesterol) and outpatient visits (primary care, specialty care, and emergency).
RESULTS
Nineteen percent of patients with diabetes used electronic messaging to communicate with their care providers during the study period (n = 2,924) (overall study cohort: 15,427 subjects). In multivariate models, frequent use of electronic messaging was associated with A1C <7% (relative risk [RR] 1.36 [95% CI 1.16–1.58]). Contrary to our hypothesis, frequent use of electronic messaging was also associated with a higher rate of outpatient visits (1.39 [1.26–1.53]).
CONCLUSIONS
Frequent use of electronic secure messaging is associated with better glycemic control and increased outpatient utilization. Electronic patient-provider communication may represent one strategy to meet the health care needs of this unique population. More research is necessary to assess the effect of electronic messaging on care quality and utilization.
Online patient-provider communication has the potential to fill unmet needs of patients with chronic conditions. The Institute of Medicine has suggested a shift in care toward “continuous healing relationships” supported by access to care outside in-person office visits such as over the Internet and by telephone (1). Although electronic mail has been slow to diffuse into clinical settings, its use is increasing in response to the Institute of Medicine report and strong consumer demand (2). Despite the promise of electronic patient-provider communication to improve care, there is a paucity of prior research in the area and, consequently, little is understood about the relationship between electronic communication and care quality. Specifically, it is unclear whether electronic communication is used as a complement to or a substitute for traditional outpatient utilization.
To explore these questions, we have conducted a cross-sectional analysis of electronic patient-provider messaging at a large health care delivery system. Our analysis focused on diabetes care as a result of the need for frequent communication and care coordination in this condition (3). We hypothesized that electronic messaging would be associated with improved care quality and lower utilization of in-person services (4,5). Although causation cannot directly be inferred from cross-sectional data, demonstration of an association between electronic messaging and care quality or altered patterns of utilization would suggest that electronic messaging may interact with care processes in important ways and provide justification for further study.
RESEARCH DESIGN AND METHODS
We conducted a cross-sectional analysis of electronic messaging activity at Group Health Cooperative between 1 January 2004 and 31 March 2005. All study variables were abstracted from clinical and administrative data repositories using SAS, version 8. STATA, version 10, was used for statistical analyses.
The study was conducted at Group Health Cooperative—a mixed-model health care financing and delivery organization in Washington and north Idaho. Over 300,000 members receive care through Group Health Cooperative Integrated Delivery System, which includes 20 Group Health Cooperative–owned facilities and over 500 Group Health Cooperative physicians. Beginning in August 2003, all patients in the Integrated Delivery System were able to access patient Web services through the MyGroupHealth Web site. These services include prescription refills, appointment scheduling, medical record access, and secure messaging to contact health care team members as previously described (6). The MyGroupHealth patient Web site has two levels of security (Table 1). At the initial level (registration only), a patient created a password-protected account on the Web site. At this level, the user could not exchange personally identifiable health information with Group Health Cooperative providers but could access discussion groups on several topics. Group Health Cooperative discouraged disclosure of personally identifiable information in the discussion groups. A facilitator was employed to review all content and remove any personally identifiable content. A second, higher level of security provides access to the medical record, secure messaging, and other advanced Web services. Access to this security level required each patient to complete an additional step verifying the patient's identity. At this level of security, patients and providers shared clinical and other personal health information. Patients could obtain identity verification through an online request or in person at a Group Health Cooperative clinic with a driver's license or a passport.
Table 1.
Patient services on the MyGroupHealth Web site
| Level of access |
||
|---|---|---|
| Registration only | Identity verification | |
| Healthwise knowledge base | X | X |
| Discussion groups | X | X |
| Health assessment tools | X | X |
| Choose a PCP | X | X |
| Appointment requests | X | |
| Shared medical record | ||
| Pharmacy refills and list of medications | X | |
| Secure messaging to and from health care team | X | |
| Medical test results | X | |
| After-visit summaries | X | |
| Medical conditions | X | |
| List of allergies | X | |
| Immunization history | X | |
Inclusion criteria and cohort definitions
The study was restricted to diabetic patients aged >18 years who were continuously enrolled in Group Health Cooperative's Integrated Group Practice during the study period. Patients were identified as having diabetes by three or more outpatient ICD-9 diagnoses for diabetes during 2 years before the study period. We divided the study population into several groups, depending on their level of Web service use (Fig. 1). Of the baseline population of adults with diabetes at Group Health Cooperative, 34% completed the process of identity verification to gain access to secure messaging and other advanced services. This group was further divided into two subgroups: members who used secure messaging at least once during the study period (secure messaging users) and members who had obtained identity verification but did not use secure messaging during the study period (identity verified). The identity-verified group subjects were selected as the primary comparison cohort because they were expected to be most similar to the secure messaging users in terms of unmeasured characteristics (Fig. 1). The group of patients who had no prior MyGroupHealth registration or who elected to obtain only basic username and password access was designated as a secondary comparison cohort (nonverified).
Figure 1.
Overview of study cohorts..
Measurement of primary outcome variables
Three diabetes-related quality-of-care indicators were selected to assess care quality: A1C <7%, blood pressure ≤130/80 mmHg, and LDL cholesterol <100 mg/dl (7). Outpatient visits were identified and categorized using Current Procedural Terminology and department codes.
Measurement of secure messaging
Message threads, rather than individual messages, were used to quantify secure messaging activity. A thread was defined as the set of messages related to an original message by successive replies. Threads could be initiated by patients or providers. A prior analysis of patient-provider messaging patterns at Group Health Cooperative has suggested that a message thread is most conceptually similar to a single episode of clinical care. For example, during 2004, 96% of all threads contained 5 or fewer messages, 86% spanned ≤3 calendar days, and 99.7% contained fewer than 10 messages (8).
Patient population characteristics
Patient age and sex were abstracted from administrative databases. Twelve adjusted diagnostic groups were selected a priori to control for medical comorbidity (9). ICD-9 data were used to generate a count of complications to control for diabetes severity (10). Depression severity was modeled as an ordinal variable with four levels (G. Simon, personal communication). A history of depression was defined by three or more outpatient visits with an ICD-9 diagnosis of depression in the year before the study. Moderate depression was defined by any diagnosis of depression by a psychiatrist or other mental health specialist, and severe depression was defined by any inpatient mental health admission. Insurance was grouped into the following three categories: commercial, Medicare, or Medicaid. Members with Medicaid insurance were grouped with members in Washington State's Basic Health Plan—an insurance program for low-income individuals not qualifying for Medicaid. Primary care provider (PCP) participation in secure messaging was measured as a percent of total outpatient encounters. Neighborhood-level race, income, and educational attainment were abstracted from U.S. Census data. High neighborhood racial diversity was defined as a nonwhite population of over 20%. Low neighborhood socioeconomic status was defined as a neighborhood where at least 20% of the population earned <20,000 USD per year or at least 25% of adults aged >25 years had less than a high-school education (11). Census data were further used to determine rural or urban residence, and home address records were used to calculate the distance from home to clinic.
PCP characteristics
Provider's sex and specialty, quintile rank of age, the proportion of secure messaging threads that were provider initiated, and the average provider response time were abstracted from administrative databases.
Statistical analysis
Multivariable regression models were used to examine the association between secure messaging use and care quality. A second set of models was generated to examine the association between secure messaging use and the number of outpatient visits. The patient was the unit of analysis for all regression models, with clustering by PCP. Log-linear models were used to estimate relative risks (RRs) or rate ratios. Models were adjusted for age, sex, overall medical comorbidity, diabetes severity, depression severity, insurance, and PCP's age, sex, and participation in secure messaging. To ensure a flexible specification and minimize residual confounding, age was modeled using linear splines. Neighborhood-level racial/ethnic and socioeconomic status measures were not significant in preliminary models and were dropped from the final models. Generalized estimating equations were used to estimate regression coefficients (12), and the Huber-White robust sandwich estimator was used for variance estimates (13). The Wald tests were used to assess the significance of explanatory variables, using two-sided P values evaluated at the 0.05 significance level.
RESULTS
Thirty-four percent of the study cohort completed the process of identity verification for use of advanced Web services including secure messaging (n = 5,274) (overall study cohort included 15,427 subjects). Of the patients who obtained identity verification, 55% went on to use secure messaging (n = 2,924). Secure messaging users participated in an average of 5.3 threads, consisting of 11.8 individual messages. Altogether, 86.9% of threads were patient initiated.
The mean age of the secure messaging cohort was 58 years, whereas the mean age of the nonmessaging subgroups was 63 years (Table 2). Among the subgroup with at least 12 secure messaging threads per year, 77% had high or very high expected resource use compared with 44% of the identity-verified cohort.
Table 2.
Population characteristics
| Nonverified* | MyGroupHealth identity verified† | Secure messaging users |
|||
|---|---|---|---|---|---|
| 1–3 threads | 4–11 threads | ≥12 threads | |||
| n | 10,153 | 2,350 | 1,892 | 814 | 218 |
| Male sex | 52 | 45 | 46 | 49 | 49 |
| Age (years) | 64 ± 13 | 60 ± 12 | 58 ± 12 | 57 ± 12 | 57 ± 11 |
| <35 | 1.9 | 2.2 | 3.2 | 3.1 | 4.1 |
| 35–49 | 12 | 15 | 17 | 24 | 21 |
| 50–64 | 36 | 46 | 52 | 50 | 53 |
| ≥65 | 50 | 37 | 28 | 23 | 22 |
| Low neighborhood SES | 25 | 20 | 20 | 19 | 24 |
| Distance to clinic (miles) | 9.6 ± 69 | 10 ± 85 | 13 ± 108 | 7.8 ± 9.7 | 7.9 ± 11 |
| Rural | 3.0 | 3.2 | 3.4 | 4.0 | 3.7 |
| Insurance | |||||
| Commercial | 46 | 58 | 68 | 73 | 65 |
| Medicare | 52 | 40 | 31 | 26 | 33 |
| Medicaid, Basic Health Plan | 1.9 | 2.0 | 0.9 | 0.4 | 1.8 |
| Expected resource use | |||||
| None or low | 5.8 | 5.8 | 3.8 | 2.7 | 0.5 |
| Moderate | 48 | 50 | 51 | 46 | 22 |
| High | 26 | 26 | 28 | 30 | 38 |
| Very high | 21 | 18 | 18 | 21 | 39 |
| Depression visits‡ | 0.69 ± 2.7 | 0.86 ± 3.6 | 0.83 ± 4.1 | 1.1 ± 4.3 | 2.5 ± 8.9 |
| Diabetes complications | 1.5 ± 1.1 | 1.4 ± 1.1 | 1.3 ± 1.1 | 1.3 ± 1.1 | 1.8 ± 1.0 |
| Quarters with any secure messaging | N/A | 0 | 1.5 ± 0.67 | 3.2 ± 1.0 | 4.4 ± 0.85 |
| Patient's ratio of secure messaging threads: all encounters | N/A | 0 | 0.20 ± 0.14 | 0.41 ± 0.17 | 0.54 ± 0.16 |
| PCP ratio of secure messaging threads: all encounters | 0.14 ± 0.07 | 0.14 ± 0.07 | 0.16 ± 0.08 | 0.18 ± 0.09 | 0.19 ± 0.10 |
| PCP male sex | 72 | 73 | 71 | 68 | 67 |
| Panel size | 1,408 ± 363 | 1,434 ± 366 | 1,403 ± 356 | 1,375 ± 356 | 1,382 ± 351 |
| Tenure with PCP (years) | 6.2 ± 5.2 | 6.1 ± 5.2 | 6.2 ± 5.2 | 6.1 ± 5.3 | 6.2 ± 5.1 |
| A1C (%) | 7.8 ± 1.6 | 7.7 ± 1.6 | 7.6 ± 1.5 | 7.6 ± 1.5 | 7.5 ± 1.6 |
| <7 | 34 | 36 | 37 | 38 | 45 |
| Blood pressure (mmHg) | 134/74 ± 20/11 | 133/75 ± 19/11 | 132/75 ± 18/11 | 132/76 ± 18/11 | 132/75 ± 19/11 |
| <130/80 | 33 | 35 | 36 | 34 | 36 |
| LDL cholesterol (mg/dl) | 96 ± 35 | 95 ± 34 | 95 ± 35 | 94 ± 36 | 93 ± 34 |
| <100 | 80 | 82 | 81 | 81 | 83 |
| Outpatient visits‡ | 9.5 ± 14 | 9.4 ± 14 | 9.7 ± 14 | 12 ± 15 | 18 ± 20 |
| Primary care | 49 | 48 | 47 | 45 | 42 |
| Specialty care | 47 | 49 | 51 | 53 | 55 |
| Emergency | 4.6 | 3.6 | 3.1 | 3.2 | 4.1 |
Data are % or means ± SD.
*Patients with no MyGroupHealth Web site registration or only basic username/password registration.
†Patients who registered to use the MyGroupHealth Web site and completed secondary identity verification but never used secure messaging.
‡Annualized visits. SES, socioeconomic status.
Sixty-four percent of PCPs with high secure messaging use were female compared with 37% of PCPs with low secure messaging use (Table 3). Ten percent of PCPs with high secure messaging use and 32% of PCPs with low secure messaging use had panels of over 2,000 patients.
Table 3.
Characteristics of PCPs
| All providers | Secure messaging <20% of all encounters | Secure messaging ≥20% of all encounters | |
|---|---|---|---|
| Total number of providers | 186 | 136 | 50 |
| Male sex | 56 | 63 | 36 |
| Age (years) | |||
| 29–41 | 20 | 16 | 30 |
| 42–49 | 20 | 21 | 17 |
| 50–52 | 21 | 20 | 23 |
| 53–56 | 19 | 19 | 19 |
| 57–63 | 20 | 24 | 11 |
| Panel size | |||
| <500 | 11 | 12 | 8.0 |
| 500–2,000 | 63 | 56 | 82 |
| >2,000 | 26 | 32 | 10 |
| Provider-initiated threads | |||
| None | 15 | 18 | 4 |
| <15 | 62 | 68 | 46 |
| >15 | 23 | 13 | 50 |
| Time to respond (h) | 8.2 ± 4.6 | 8.8 ± 4.9 | 6.5 ± 3.2 |
Data are means ± SD or %.
Unadjusted RRs are presented in Table 4 (Model A). In multivariable models, the rate of A1C <7% was 36% higher in patients with the highest rate of secure messaging use (≥12 threads per year) compared with that in the nonmessaging identity-verified comparison cohort (RR 1.36 [95% CI 1.16–1.58]) (Table 4). In contrast, secure messaging use was not associated with control of blood pressure <130/80 mmHg. There was a small but statistically significant association between secure messaging and LDL cholesterol <100 mg/dl. For all three quality measures, the nonmessaging identity-verified cohort had slightly better outcomes than the nonmessaging non-verified cohort.
Table 4.
Multivariable regression results
| Outcome | Secure messaging intensity | Model A |
Model B |
||||||
|---|---|---|---|---|---|---|---|---|---|
| n | RR† | (95% CI) | P * | n | RR† | (95% CI) | P * | ||
| A1C <7% | 14,075 | <0.001 | 13,908 | <0.001 | |||||
| Nonverified | 0.95 | (0.90–1.01) | 0.92 | (0.87–0.98) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 1.06 | (0.97–1.15) | 1.08 | (1.00–1.18) | |||||
| 4–12 threads | 1.08 | (0.98–1.20) | 1.13 | (1.02–1.25) | |||||
| ≥12 threads | 1.29 | (1.10–1.50) | 1.36 | (1.16–1.58) | |||||
| Blood pressure <130/80 mmHg | 10,181 | 0.150 | 10,079 | 0.155 | |||||
| Nonverified | 0.92 | (0.86–0.99) | 0.92 | (0.87–0.99) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 1.00 | (0.92–1.10) | 1.01 | (0.92–1.11) | |||||
| 4–12 threads | 0.97 | (0.87–1.09) | 0.98 | (0.87–1.09) | |||||
| ≥12 threads | 1.00 | (0.80–1.26) | 0.98 | (0.78–1.24) | |||||
| LDL cholesterol <100 mg/dl | 11,637 | 0.255 | 11,487 | <0.001 | |||||
| Nonverified | 0.97 | (0.93–1.01) | 0.94 | (0.90–0.98) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 0.97 | (0.91–1.02) | 0.99 | (0.93–1.05) | |||||
| 4–12 threads | 1.02 | (0.96–1.10) | 1.07 | (1.00–1.15) | |||||
| ≥12 threads | 0.99 | (0.87–1.13) | 1.01 | (0.89–1.14) | |||||
| Total outpatient visits | 15,427 | <0.001 | 15,237 | <0.001 | |||||
| Nonverified | 1.01 | (0.97–1.06) | 0.98 | (0.95–1.02) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 1.03 | (0.98–1.09) | 1.05 | (1.01–1.10) | |||||
| 4–12 threads | 1.27 | (1.18–1.36) | 1.19 | (1.13–1.25) | |||||
| ≥12 threads | 1.97 | (1.76–2.20) | 1.39 | (1.26–1.53) | |||||
| Primary care visits | 15,427 | <0.001 | 15,237 | <0.001 | |||||
| Nonverified | 1.04 | (0.99–1.08) | 0.99 | (0.96–1.03) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 1.01 | (0.96–1.07) | 1.02 | (0.98–1.07) | |||||
| 4–12 threads | 1.18 | (1.10–1.26) | 1.11 | (1.05–1.18) | |||||
| ≥12 threads | 1.74 | (1.57–1.94) | 1.32 | (1.19–1.45) | |||||
| Specialty care visits | 15,427 | <0.001 | 15,237 | <0.001 | |||||
| Nonverified | 0.97 | (0.92–1.03) | 0.96 | (0.91–1.01) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 1.06 | (0.98–1.14) | 1.08 | (1.01–1.15) | |||||
| 4–12 threads | 1.36 | (1.24–1.48) | 1.26 | (1.16–1.36) | |||||
| ≥12 threads | 2.17 | (1.87–2.51) | 1.43 | (1.25–1.64) | |||||
| Emergency visits | 15,427 | <0.001 | 15,237 | <0.001 | |||||
| Nonverified | 1.33 | (1.16–1.54) | 1.17 | (1.05–1.32) | |||||
| 0 threads, identity verified | 1 | — | 1 | — | |||||
| 1–3 threads | 0.90 | (0.76–1.07) | 1.04 | (0.90–1.21) | |||||
| 4–12 threads | 1.19 | (0.94–1.51) | 1.19 | (0.98–1.44) | |||||
| ≥12 threads | 2.34 | (1.61–3.41) | 1.66 | (1.23–2.26) | |||||
Model A is unadjusted. Model B is adjusted for age, sex, overall medical comorbidity, diabetes severity, depression severity, insurance type, PCP's participation in secure messaging, PCP's age, PCP's sex, and PCP panel size.
*Wald tests for composite linear hypotheses.
†Rate ratio.
The primary care visit rate was 32% higher among patients with high use of secure messaging compared with that in the nonmessaging comparison cohort (RR 1.32 [95% CI 1.19–1.45]) (Table 4). This translates to approximately three to four additional outpatient office visits, given a baseline visit rate of nine visits per year. High secure messaging users also had more outpatient specialty visits (1.43 [1.25–1.64]) (Table 4) and emergency care visits (1.66 [1.23–2.26]) (Table 4) than the nonmessaging comparison cohort.
CONCLUSIONS
Secure messaging and glycemic control
In this cross-sectional study, patients who used more secure messaging had better glycemic control. Secure messaging may have enabled better glycemic control by filling unmet needs for care. Care providers may have used secure messaging to recommend medication changes between in-person visits, thereby optimizing treatment regimens more quickly. Medication intensification has previously been found to be the most significant predictor of variation in A1C (14). By increasing the frequency of contact, secure messaging might also strengthen continuity of care, which has also been associated with improved glycemic control (15).
As with all observational studies, unmeasured differences between users and nonusers of secure messaging may also explain our results. Users of secure messaging may have had higher health literacy, education, or other characteristics that are also found among individuals with better glycemic control. Patients who engage in secure messaging may also be more engaged in self-care behaviors that promote better glycemic control.
Finally, it is possible that use of advanced Web services other than secure messaging, which included prescription refills, appointment scheduling, and medical record access, may explain some of these results. Among patients who did not use secure messaging, the cohort who had access to advanced Web services exhibited slightly better control of A1C, blood pressure, and LDL cholesterol than the cohort without access.
Secure messaging and outpatient visit rates
Contrary to our hypothesis, we found a positive association between secure messaging use and the number of outpatient visits. Two prior longitudinal studies of physicians have found electronic messaging systems to be associated with a reduction in outpatient visit rates (4,5). These studies suggest that electronic consultations may substitute for traditional in-person office visits. Our results may differ from these studies because of our focus on diabetes. Patients with diabetes who use secure messaging may be more proactive with care providers both online and in person and may use secure messaging as a complement to care rather than as a substitute. Secure messaging use may also raise unmet needs in this population that require further engagement either in person or through secure messaging.
Strengths and weaknesses of the study
This is the one of the first studies to examine the association among electronic patient-provider messaging, diabetes-related health outcomes, and outpatient utilization in a large health care delivery system. Although several important studies have been published in this area over the past 5 years, most are limited by small sample sizes. In contrast, a key strength of our study is its setting: a large nonprofit health care system that serves ∼300,000 members. Limitations include the cross-sectional design of the study, the short duration of exposure to secure messaging, and potential self-selection bias. Secure messaging users may have differed from nonusers by unmeasured factors such as self-efficacy, attitudes toward medical care, race, socioeconomic status, and health literacy. Given the cross-sectional design of this study, it is impossible to conclude that electronic messaging caused the outcomes that we observed. Randomized controlled trials or longitudinal analyses will be required to assess the causal relationship among secure messaging and utilization, care quality, and health outcomes.
Unanswered questions and future research
As physicians and health care organizations consider how and when to support electronic communication with patients, we must understand how this new care environment differs from traditional in-person care. Secure messaging may serve as an important part of care for patients with diabetes and an opportunity to support them in self-management outside of routine visits. Our findings suggest that patients with diabetes who are frequent users of electronic patient-provider messaging systems may represent a unique patient population, characterized by better glycemic control and a greater use of outpatient visits.
Acknowledgments
This study was funded in part by the Agency for Healthcare Research and Quality (R03 HS014625-01). L.T.H. was supported by a Public Health Informatics Training grant (3 T15 LM007442-06S1) from the Robert Wood Johnson Foundation.
No potential conflicts of interest relevant to this article were reported.
We would like to acknowledge the contributions of David Carrell, PhD; Gregory Simon, MD, MPH; Julia Hecht, PhD; and David Rubanowice.
Footnotes
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked “advertisement” in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
References
- 1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC, National Academies Press, 2001 [PubMed] [Google Scholar]
- 2. Harris Interactive. Patient/physician online communication: many patients want it, would pay for it, and it would influence their choice of doctors and health plans. Health Care News 2002; 2: 1– 4 [Google Scholar]
- 3. Wielawski IM: Improving chronic illness care. In To Improve Health and Health Care, The Robert Wood Johnson Foundation Anthology. Vol. X. Rochester, NY, Jossey-Bass, 2006 [Google Scholar]
- 4. Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL: Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care 2007; 13: 418– 424 [PubMed] [Google Scholar]
- 5. Bergmo TS, Kummervold PE, Gammon D, Dahl LB: Electronic patient-provider communication: will it offset office visits and telephone consultations in primary care? Int J Med Inform 2005; 74: 705– 710 [DOI] [PubMed] [Google Scholar]
- 6. Ralston JD, Carrell D, Reid R, Anderson M, Moran M, Hereford J: Patient web services integrated with a shared medical record: patient use and satisfaction. J Am Med Inform Assoc 2007; 14: 798– 806 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. American Diabetes Association. Standards of medical care for patients with diabetes mellitus (Position Statement). Diabetes Care 2003; 26( Suppl. 1): S33– S50 [DOI] [PubMed] [Google Scholar]
- 8. Carrell D, Ralston J: Messages, strands and threads: measuring use of electronic patient-provider messaging. AMIA Annu Symp Proc 2005; 913. [PMC free article] [PubMed] [Google Scholar]
- 9. Johns Hopkins University. Adjusted Clinical Groups (ACG) Assignment Software. Version 7.0 for Windows. Baltimore, MD, Johns Hopkins University, 1990–2006 [Google Scholar]
- 10. Simon G, Katon W, Lin E, Ludman E, Von Korff M, Ciechanowski P, Young B: Diabetes complications and depression as predictors of health care costs. Gen Hosp Psychiatry 2005; 27: 344– 351 [DOI] [PubMed] [Google Scholar]
- 11. Kreiger N: Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health 1992; 82: 703– 710 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Liang KY, Zeger SL: Longitudinal data analysis using generalized linear models. Biometrika 1989; 73: 13– 22 [Google Scholar]
- 13. White HA: Heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 1980; 48: 817– 830 [Google Scholar]
- 14. O'Connor PJ, Rush WA, Davidson G, Louis TA, Solberg LI, Crain L, Johnson PE, Whitebird RR: Variation in quality of diabetes care at the levels of patient, physician, and clinic. Prev Chronic Dis 2008; 5: A15. [PMC free article] [PubMed] [Google Scholar]
- 15. Sperl-Hillen JM, Solberg LI, Hroscikoski MC, Crain AL, Engebretson KI, O'Connor PJ: The effect of advanced access implementation on quality of diabetes care. Prev Chronic Dis 2008; 5: A16. [PMC free article] [PubMed] [Google Scholar]