Patient-centeredness and timeliness in a primary care network: baseline analysis and power assessment for detection of the effects of an electronic health record

Abstract

Electronic health records are expected to improve all six dimensions of quality care identified by the Institute of Medicine (safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness). HealthTexas Provider Network, the ambulatory care network affiliated with the Baylor Health Care System in Dallas–Fort Worth, Texas, is implementing a networkwide ambulatory electronic health record (AEHR). To evaluate the quality of care and financial impact of the AEHR implementation, we examined the available indicators for quantitatively measuring performance in each dimension of quality. For patient-centeredness, the primary data source available is the patient satisfaction survey. To achieve a broad view of patient-centeredness, we identified two measures of satisfaction (overall satisfaction with the physician and willingness to refer the physician) to be examined individually and used additional survey items to construct physician interaction and organizational scales. These scales showed good reliability (Cronbach alpha = 0.95 and 0.89, respectively) and predictive ability ranging from 77% to 93% when applied to the overall satisfaction measures. Data from September 2003 to June 2006 showed mean pre-AEHR implementation baseline performance of 22.9 (±3.3) on the 25-point physician interaction scale and 38.0 (±5.8) on the 45-point organizational scale; 70.9% of patients reported excellent satisfaction with their physician, and 97.6% of patients reported willingness to refer. Timeliness data were collected using the same survey. Baseline performance showed that 43.4% of patients waited <2 days between making and keeping an appointment, and 50.6% of patients waited <5 minutes past appointment time. However, 12.5% waited >30 days between making and keeping an appointment, and 14.0% waited >30 minutes past appointment time. The power to detect changes in the patient-centeredness and timeliness measures in the 3-year multiple time series evaluation of the quality and financial impact of the AEHR was investigated and showed that even small changes in these measures will be detectable.

Health Texas Provider Network (HTPN), the ambulatory care physician network affiliated with the Baylor Health Care System in North Texas, is in the process of implementing a networkwide ambulatory electronic health record (AEHR). We will evaluate the effect of this implementation on all six dimensions of quality identified by the Institute of Medicine (IOM) in the 2001 report, Crossing the Quality Chasm (1): safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness, in a 3-year multiple time series research project.

Finding objective measures for the dimension of patient-centeredness has been challenging. The IOM defined patient-centered care as care that is respectful of and responsive to individual patient preferences and needs and that is guided by patient values (1); this definition encompasses a broad range of concepts and requires different actions for different patients. Although patient satisfaction does not encompass all features of patient-centeredness, satisfaction surveys that address multiple aspects of the patient's health care experience can provide valuable insights into how a health care provider is performing on patient-centeredness and in which areas improvements can be made.

In the context of computer-based patient record systems, a systematic review found only two studies addressing patient satisfaction, and these did not examine patients' perceptions of changes to the physician-patient relationship (2). The authors noted the need to investigate patient sentiments towards health information technology and its impact on their relationship with the provider. Considering the myriad of information technology solutions becoming available, from electronic visits to electronic patient education, the impact of these new capabilities must be evaluated in the greater context of the physician-patient relationship.

In this article, we report the pre-AEHR implementation levels of patient-centeredness for 31 HTPN primary care practices using existing data that address as many aspects of the IOM definition of patient-centeredness as possible. Additionally, since timeliness data are collected using the same instrument as patient-centeredness data, we report pre-AEHR implementation performance on timeliness measures for these same practices.

METHODS

Measures

We developed four patient-centeredness measures based on information collected in the HTPN patient satisfaction survey. Two of these measures were questions taken directly from the survey (Figure): “Would you recommend your doctor?” (yes/no) and “Overall satisfaction with the doctor” (excellent/very good/good/fair/poor). Specifically, we examined the proportion of patients reporting that they would refer their physician and the proportion of patients reporting excellent overall satisfaction.

Figure.

The patient satisfaction survey used by the HealthTexas Provider Network.

The remaining two patient-centeredness measures were a physician interaction score and an organizational score based on patient satisfaction survey questions assessing these dimensions of the patient's experience of care. The physician interaction questions were based on the Medical Outcome Study visit- specific instrument, the properties of which have been described elsewhere (3–5). This 25-point scale is based on questions addressing 1) thoroughness of treatment, 2) clarity and completeness of physician's explanations, 3) attention given to what the patient said, 4) respect shown to the patient by the doctor, and 5) amount of time the patient had with doctors and staff during the visit. The organizational measures were similar to those in an instrument developed by the Quality Improvement Committee at the Indiana School of Medicine (4). A 45-point scale was constructed based on 1) ease of making appointments by phone, 2) length of time between appointment and visit, 3) friendliness and courtesy shown by the receptionist, 4) convenience of the office location, 5) hours when the office is open, 6) cleanliness and attractiveness of the doctor's office, 7) billing process, 8) politeness and helpfulness of the nurse/medical assistant, and 9) overall satisfaction with the nurse/medical assistant.

These four patient-centeredness measures addressed two important dimensions identified by the IOM: “respect for patients' values, preferences, and expressed needs” and “information, communication, and education” (1).

The two timeliness indicators were taken directly from the patient satisfaction survey: days between making and keeping appointment (same day/1–2 days/3–7 days/8–14 days/15–30 days/>30 days) and time waited past appointment (did not wait/?5 minutes/6–15 min/16–30 min/>30 min).

Data collection

For each physician, patient satisfaction surveys were mailed monthly to approximately 15 patients who had a visit in the preceding calendar month. The surveys were administered by DSS Research, a national market research firm with 20 years of experience and multiple clients across the USA. To establish the pre-AEHR implementation performance on the patient-centeredness and timeliness measures, data from the patient satisfaction surveys collected from September 2003 to June 2006 were evaluated.

Analysis

Reliability and validation of physician interaction and organizational scores

Inter-item correlation within the physician interaction and organizational scores was assessed using Cronbach alpha (6). The relationships between the physician interaction and organizational scores and overall satisfaction and willingness to refer were assessed using binomial logistic regression models. The predictive validity and discrimination of the constructed scales were investigated using an approach that has been previously applied to patient satisfaction measures in primary care (7). This approach involves predicting overall, discrete patient satisfaction measures (in this case, overall satisfaction and willingness to refer) as a function of the constructed scales, to determine their ability to predict and discriminate. To determine the ability of the scales to discriminate between likelihood of satisfaction and willingness to refer, observed proportions of patients who were satisfied/willing to refer were examined for different points along the continuous measures.

Pre-AEHR implementation baseline performance on patient- centeredness and timeliness

Mean (SD) performance on the physician interaction and organizational scales was calculated for the patient satisfaction surveys returned for patients seen at the 31 HTPN primary care practices between September 2003 and June 2006. The proportions of patients reporting willingness to refer their physician and excellent, very good, good, fair, and poor overall satisfaction with their physician were calculated. To assess whether the six measures showed a trend over the baseline period, we estimated hierarchical generalized linear models with each measure as the dependent variable and time from 2003 as the independent variable, with error terms for patient, physician, and practice.

Power to detect changes in patient-centeredness and timeliness following AEHR implementation

Based on the existing patient satisfaction survey data collection scheme, in the planned 3-year study, we will have ∼18,000 surveys before AEHR implementation (including existing data from September 2003 to June 2006) and 12,000 surveys after AEHR implementation. Patient-centeredness measures will be analyzed using 1) linear regression, 2) ordered multinomial regression, and 3) logistic regression. Random effects hierarchical generalized linear models will be used to compare patient satisfaction measures between AEHR groups. Thus, if Y_ijkl is a continuous-scale measure of patient-centeredness, we will estimate

Y_ijkl = β₀ + β_AEHR × AEHR + β_T × T_ijkl + v_jkl + u_kl + η_l

where β_AEHR is the log (odds ratio) of a patient treated by an AEHR practice vs a non-AEHR practice reporting, for instance, excellent satisfaction vs poor satisfaction or willing to refer the physician vs not willing to refer; AEHR indicates if the patient was treated in a practice with the AEHR implemented; β_T is the secular effect; T_ijkl represents calendar time; and v_jkl, u_kl, and β_l are error terms reflecting the random effects at patient, physician, and practice levels, respectively. Testing H₀: β_AEHR = 0, we can determine if AEHR affects patient-centeredness.

To test the hypothesis that AEHR implementation improves overall patient-centeredness (and timeliness of care) as measured by appropriate patient survey responses, we will use ordered multinomial models that account for correlation of responses within provider. For example, let Y_ijklm indicate whether the response for the i^th survey of j^th patient seen by the k^th physician at the l^th practice is equal to m (m = 1, …, 5). Then we estimate an ordered multinomial model:

γ_ijkl1 = η_ijkl1

γ_ijkl1 = η_ijkl1 + η_ijkl2

γ_ijkl1 = η_ijkl1 + η_ijkl2 + η_ijkl3

γ_ijkl1 = η_ijkl1 + η_ijkl2 + η_ijkl3 + η_ijkl4

and γ_ijkl1 = η_ijkl1 + η_ijkl2 + η_ijkl3 + η_ijkl4 + η_ijkl5

and for each γ_ijklm (m = 1, … 5) we estimate simultaneously:

logit(γ_ijklm) = β_0m + β_AEHR × AEHR + β_T × T_ijkl + v_jkl + u_kl + η_l

We can test the hypothesis that AEHR affects overall satisfaction by testing H₀: β_AEHR = 0. The model for willingness to refer is a logistic regression model for a dichotomous response (yes vs no), which is specified exactly the same as for the ordered multinomial model.

Minimum detectable effects, at power 1 – β = 80% and α= 0.05, for changes in patient-centeredness and timeliness measures following AEHR implementation were calculated using rank sum tests (for measures based on Likert responses), chi-square tests (for measures based on dichotomous responses), and t tests (for multi-item scale measures), all corrected for clustering of patients within physician with intraclass correlation coefficients estimated for the specific measures within practices.

RESULTS

Reliability and validation of physician interaction and organizational scores

Both the physician interaction and organizational scales demonstrated very strong reliability, i.e., lack of measurement error (Cronbach α = 0.95 and 0.89, respectively). The physician interaction score, organizational score, overall satisfaction, and willingness to refer all showed significant positive relationships, and the physician interaction and organizational scores showed good predictive validity and strength of discrimination (see R² and c statistic values in Table 1).

Table 1.

Influence of physician interaction and organizational scores on overall satisfaction and willingness to refer the physician, and the predictive validity and ability to discriminate on these measures

Overall satisfaction

Based on the c statistics for overall satisfaction, the physician interaction score ordered the pairs of satisfied vs dissatisfied patients correctly 93% of the time, and the organizational score ordered the pairs correctly 84% of the time. Using physician interaction scores, the observed probability of reporting excellent overall satisfaction—96.4% for patients with a score of 25 (n = 10,143) but only 14.7% of patients with a score of 20 (n = 1939)—indicated reasonable strength of discrimination. Using organizational scores, the observed probability of reporting excellent overall satisfaction—96.9% for patients with a score of 45 (n = 2977) but only 85.3% for patients with a score of 40 (n = 830) and 52.4% for those with a score of 30 (n = 826)—indicated reasonable strength of discrimination. All models became even more predictive and discriminating when the patient's response to the 5-point rating of overall health was included (Table 1).

Willingness to refer

Based on the c statistics for willingness to refer, the physician interaction score ordered the pairs of satisfied vs dissatisfied patients correctly 88% of the time, and the organizational score ordered the pairs correctly 77% of the time. Using physician interaction scores, the observed probability of reporting willingness to refer—99.5% for patients with a score of 25 (n = 10,088) but 92.7% for patients with a score of 15 (n = 437)—indicated reasonable strength of discrimination. Using organizational scores, the observed probability of reporting willingness to refer—99.4% for patients with an organizational score of 45 (n = 2948) but 97.2% for patients with a score of 35 (n = 820)—indicated reasonable strength of discrimination. All models became even more predictive and discriminating when the patient's response to the 5-point rating of overall health was included (Table 1).

Pre-AEHR implementation baseline performance on patient-centeredness and timeliness

Baseline data from September 2003 to June 2006 (N = 13,279) showed a mean (SD) score of 22.9 (3.3) on the physician interaction scale and 38.0 (5.8) on the organizational scale. Baseline performance on willingness to refer the physician and overall satisfaction with the physician are shown in Table 2. Performance on timeliness measures are shown in Table 3.

Table 2.

Overall satisfaction measures from the patient satisfaction survey conducted from September 2003 to June 2006 in the HealthTexas Provider Network

	Responses
Measure	Excellent	Very good	Good	Fair	Poor
Overall satisfaction with the doctor	70.9%	21.4%	6.0%	1.3%	0.5%
	Yes				No
Would you recommend your doctor?	97.6%				2.4%

Table 3.

Timeliness of care measures from the patient satisfaction survey conducted from September 2003 to June 2006 in the Healthtexas Provider Network

	Responses
Measure	Same day	1–2 days	3–7 days	8–14 days	15–30 days	>30 days
Days between making and keeping appointment	17.9%	25.5%	12.1%	22.5%	9.5%	12.5%
	Did not wait	5 min or less	6–15 min	16–30 min	>30 min
Time waited past appointment	11.4%	39.2%	12.5%	22.9%	14.0%

Two of the patient-centeredness measures—organizational score (P = 0.0004) and satisfaction with physician (P = 0.0047)—showed significant improvement over the 4 years. The mean organizational score improved from 37.8 in the last 6 months of 2003 to 38.1 in the first 6 months of 2006. The proportion of patients reporting excellent satisfaction with their physician increased in the same time period from 69.1% to 71.5%.

Power to detect changes in patient- centeredness and timeliness following AEHR implementation

Minimum detectable effects, at power 1 – β = 80% and α= 0.05, for all patient- centeredness and timeliness measures are shown in Table 4.

Table 4.

Minimum detectable effects for study measures^∗

Dimension	Measure	ICC	SD	%	Test	Effect†
Timeliness
Time for appointment	Likert scale	0.0798	1.63	Rank sum	0.311
Waiting room time	Likert scale	0.0235	1.27	Rank sum	0.136
Patient-centeredness
Physician interaction	25-point scale (mean of 5 Likert items)	0.0406	3.68	t test	0.545
Organizational	45-point scale (mean of 9 Likert items)	0.0350	5.79	t test	0.800
Overall satisfaction	Likert 5-point scale	0.0386	0.701	Rank sum	0.095
Willingness to refer	Dichotomous, percentage of yes/no	0.0135	97.6%	Chi-square	1.8%

∗All tests adjusted for clustering of measures within practice, with intrapractice correlation = 0.05.

†Effect measured in unit of scale.

ICC indicates intraclass correlation coefficients.

DISCUSSION

The baseline data show good performance on all the patient-centeredness and timeliness measures examined, but also some room for improvement. The power calculations indicate that the 3-year multiple time series evaluation of the impact of the AEHR on quality of care in primary care practices will be able to detect even small changes in performance on the patient-centeredness and timeliness measures. The identification of significant upward trends in two of the measures during this baseline period is important, as it will allow us to separate the effects of secular trend from the effects of the AEHR on these measures. Failure to recognize the preimplementation trend could result in improvements being erroneously attributed to the AEHR implementation.

The reliability of the physician interaction and organizational scales developed as patient-centeredness measures for this study is approximately equivalent to that demonstrated by Fan et al (7) for the humanistic and organizational scales used in their study of patient satisfaction and compares favorably with the reliability demonstrated by Roblin et al (9) for their physician interaction scale. These researchers developed the scales to examine the effects of provider characteristics on patients' experiences—finding, for example, that continuity of care was strongly predictive of both humanistic and organizational components of patient satisfaction (7) and that provider type (physician vs midlevel provider) influenced the provider interaction component (9). We will use the physician interaction and organizational scales developed here essentially to examine the effect of a practice characteristic—presence or absence of the AEHR—on these components of the patient's experience.

While there might be some debate concerning the inclusion of questions relating to nurses/medical assistants in the organizational rather than the physician interaction scale, inter-item analysis found the Cronbach alpha for the physician interaction scale to decrease 0.04 with the inclusion of these items and the Cronbach alpha for the organizational scale to decrease 0.02 with the exclusion of these items.

The identification of timeliness measures that are both relevant and practical can be challenging in many health care environments, especially when the use of historical data to establish a baseline level of performance necessitates the use of existing data sources. The Institute for Healthcare Improvement suggests employing the measure “availability of third next appointment” (10). This measure is less susceptible to fluctuations caused by last-minute cancellations than “next available appointment” and “time between making and keeping the appointment.” However, availability of the third next appointment has not been historically collected by HTPN, making it difficult to establish a baseline performance level. Instituting the measure for the purposes of evaluating the effect of the AEHR is being considered, as the staggered roll-out of the AEHR would provide preimplementation data for at least the later-adopting clinics. The significant disadvantage to the third next appointment measure is the resource-intensive data collection required: hiring “mystery shoppers” to regularly contact all practices in the network to establish appointment availability, which pre-sents feasibility issues for many health care organizations. The two timeliness measures used in this study—time between making and keeping an appointment and time waited past the appointment time—have been well studied by health services researchers for over 30 years (11). Besides the fact that these two measures could contribute to complaints and poor patient satisfaction, appointment delay could reduce the demand for services (11) that should be received in a timely way for optimal health outcomes.

This baseline evaluation of patient-centeredness and timeliness measures in the HTPN primary care practices contributes to our understanding of the current state of quality of care in a typical fee-for-service setting and provides information about areas in which quality can be improved and should, as such, be targeted by improvement initiatives. Additionally, since the HTPN primary care practices have an average of 5 physicians, our results provide information about the current quality of care in small practices. This is important in the context of AEHR, which has been adopted more slowly in smaller ambulatory care practices than in larger practices (12). Information regarding both the need for and potential impact of AEHR implementation in such settings will help inform these practices' decisions on adopting AEHR.

CONCLUSIONS

Our results demonstrate the validity of the physician and organizational scores we developed from the HTPN patient satisfaction surveys as measures of IOM's dimension of patient-centeredness. These scores, as well as the other measures of patient-centeredness and timeliness taken directly from the patient satisfaction survey, provide a very important baseline performance level from which we can track the impact of AEHR. The almost 3 years of historical preimplementation data with ∼18,000 surveys substantially enhances the power of the analyses, i.e., the capacity to detect statistically significant differences from baseline, under the anticipated collection of an additional 12,000 surveys during the course of the study. As these data have previously benefited HTPN by tracking patient satisfaction, they will also provide a context in which practices can compare their performance with their own individual history and the performance of other practices. Such analyses will provide important feedback for the organization, helping target future improvement initiatives in the areas of patient-centeredness and timeliness of care.

Acknowledgments

The authors thank Briget da Graca, MS, for writing and editorial assistance on this manuscript; Chris Felton, RN, BSN, for data resourcing and background information about the survey; and Andrew Hayes, for creating the analytic files.