Office‐Based Tools and Primary Care Visit Communication, Length, and Preventive Service Delivery

Abstract

Background

The use of physician office‐based tools such as electronic health records (EHRs), health risk appraisal (HRA) instruments, and written patient reminder lists is encouraged to support efficient, high‐quality, patient‐centered care. We evaluate the association of exam room use of EHRs, HRA instruments, and self‐generated written patient reminder lists with patient–physician communication behaviors, recommended preventive health service delivery, and visit length.

Research Methods

Observational study of 485 office visits with 64 primary care physicians practicing in a health system serving the Detroit metropolitan area. Study data were obtained from patient surveys, direct observation, office visit audio‐recordings, and automated health system records. Outcome measures included visit length in minutes, patient use of active communication behaviors, physician use of supportive talk and partnership‐building communication behaviors, and percentage of delivered guideline‐recommended preventive health services for which patients are eligible and due. Simultaneous linear regression models were used to evaluate associations between tool use and outcomes. Adjusted models controlled for patient characteristics, physician characteristics, characteristics of the relationship between the patient and physician, and characteristics of the environment in which the visit took place.

Results

Prior to adjusting for other factors, visits in which the EHR was used on average were significantly (p < .05) longer (27.6 vs. 23.8 minutes) and contained fewer preventive services for which patients were eligible and due (56.5 percent vs. 62.7 percent) compared to those without EHR use. Patient written reminder lists were also significantly associated with longer visits (30.0 vs. 26.5 minutes), and less use of physician communication behaviors facilitating patient involvement (2.1 vs. 2.6 occurrences), but more use of active patient communication behaviors (4.4 vs. 2.6). Likewise, HRA use was significantly associated with increased preventive services delivery (62.1 percent vs. 57.0 percent). All relationships remained significant (p > .05) in adjusted models with the exception of that between HRA use and preventive service delivery.

Dissemination and Implementation Implications

Office‐based tools intended to facilitate the implementation of desired primary care practice redesign are associated with both positive and negative cost and quality outcomes. Findings highlight the need for monitoring both intended and unintended consequences of office‐based tools commonly used in primary care practice redesign.

Health care in the United States is undergoing dramatic changes. The 2009 Health Information Technology for Economic and Clinical Health Act and the 2010 Affordable Care Act have accelerated practice redesign efforts and increased attention to the need for a paradigm shift from physician‐centric, inefficient care models to those that are patient‐centered, efficient, and supportive of active patient engagement. Perhaps nowhere are these changes more felt than in primary care (Kikano et al. 2000). With these changes has come a focus not only on reimbursement incentives and team‐based care but also the increased adoption of physician office‐based tools, such as electronic health records (EHRs), health risk appraisals (HRAs), and patient‐generated reminder lists.

Electronic health record adoption has grown from under 50 percent in 2009 to 72 percent in 2012 (Hsiao and Hing 2012), and EHRs are now integral to many primary care practices (Holroyd‐Leduc et al. 2011). EHRs have the potential to provide comprehensive and organized medical information during office visits, thereby improving care quality and efficiency (Bates et al. 2003). However, they also introduce another “interactant,” the computer, into the office visit, thereby potentially obstructing the rhythm and flow of clinician–patient exchanges and diverting attention away from patients (Ventres et al. 2006; Street et al. 2014). A prior systematic review concluded that EHRs have the potential to improve preventive health service delivery as well as to impact visit length both positively and negatively (Chaudhry et al. 2006). Others, including a more recent systematic review, have documented the gap between postulated and empirically demonstrated benefits of eHealth technology, including that of EHRs (Linder et al. 2007; Irani et al. 2009; Shachak and Reis 2009; Buntin et al. 2011; Holroyd‐Leduc et al. 2011).

Other office‐based tools that are growing in popularity include the use of HRA instruments to systematically gather information on patients' specific health behaviors and risk factors (Bellows, McMenamin, and Halpin 2010; Goetzel et al. 2011), and patient‐generated written lists to support patients expressing concerns and asking questions during office visits (AHRQ, 2013). Just as with EHRs, these tools could have positive and negative influences on patient–physician exchanges, care quality, and efficiency. Use of HRA instruments could potentially facilitate timely identification of patients who may benefit from counseling or other interventions, but they may also disrupt history taking, and their impact on cost and quality outcomes is not well understood (Dickey, Gemson, and Carney 1999). The use of patient‐generated written lists may facilitate timely attention to the patient's agenda and has led to improvements in patients' psychosocial and health outcomes (Kaplan, Greenfield, and Ware 1989; Rost et al. 1991; Stewart 1995; Belkora et al. 2013). Yet these tools could interfere with patient–clinician rapport building or, as others have found, serve to slow the timely progression of office visits (Schrager and Gaard 2009).

Thus, whether office‐based tools that are commonly associated with primary care practice redesign are facilitating or hindering the desired paradigm shift to care that is efficient, high quality, and patient‐centered remains unknown. To fill this void, we evaluated the association of three such office‐based tools (i.e., EHRs, HRA instruments, and patient written lists) simultaneously on visit length, patient–physician communication behaviors, and the delivery of evidence‐based preventive health services.

Methods

Study Setting

Study subjects were identified from 26 primary care practices of a salaried, multispecialty medical group in southeast Michigan. At the time of study, while no centralized HRA was used by the medical group, the group did use an EHR that was available in all clinical exam rooms and that included a gender‐ and age‐targeted prompt for evidence‐based routine preventive health screening (i.e., cervical cancer, cholesterol, breast cancer, hypertension, and colorectal cancer screening), counseling (i.e., obesity), and immunization (i.e., influenza, Tetanus, and Pneumococcal). A “stoplight” alert was red if a patient was due for any one included service, and yellow if he or she was coming due in the next 30 days. When a user opened the stoplight window, a list of services and due status/date was visible. Although a desktop computer with EHR access was available within each exam room, physician preference/discretion determined EHR access during office visits. Likewise, physician preference (and not organizational policy) determined the type and whether an HRA instrument was used. Patient preference/choice determined use of a patient‐generated written list.

Participant Eligibility Criteria and Recruitment

Eligible clinicians and patients were those enrolled in an observational study of patient–physician communication surrounding physician recommendations for colorectal cancer screening (Wunderlich et al. 2010; Lafata et al. 2011). An administrative supplement broadened the original focus to include evaluation of patient–physician communication behaviors and other office visit characteristics associated with the delivery of guideline‐recommended preventive health services (Shires et al. 2012). The parent study recruited 77 family and general internal medicine physicians and 500 of their HMO‐insured patients aged 50–80 years and due for colorectal cancer screening at the time of a scheduled periodic health examination (PHE). This resulted in 485 audible office visit recordings with 64 primary care physicians.

Physician and patient participants/nonparticipants have been previously described (Wunderlich et al. 2010). Briefly, 47 percent of physicians and 50 percent of patients agreed to participate. Physician participants did not differ from nonparticipants in age or gender, but they were significantly more likely to be African American or a practicing family medicine physician. Patient participants did not differ from nonparticipants in race or marital status but were significantly younger and more likely to be female. The Institutional Review Boards of the participating organizations approved all aspects of the study and its supplement.

Data Sources

Physician characteristics (age, gender, race, and medical specialty) were obtained from medical group administrative records. Patient sociodemographic characteristics (age, gender, race, education) as well as tobacco use (CDC, 2006), alcohol use (Vinson et al. 2004; CDC, 2006), and depressive symptoms (Kroenke et al. 2009) were obtained via a previsit patient survey. These data were joined with EHR data to identify which patients had seen the same physician within a year, and to construct the Charlson Comorbidity Index (Deyo, Cherkin, and Ciol 1992), the Framingham General Cardiovascular Disease Risk Score (D'Agostino et al. 2008), and body mass index for each patient.

For each visit, a research assistant completed an observer checklist that included time the patient was roomed and discharged; time the physician spent in the exam room; physician use of the EHR within the exam room, and physician use of an HRA instrument; and patient use of any type of self‐generated written reminder list. The latter included, but was not limited to, lists of symptoms, questions, home monitoring results, and current medications. Transcribed office visit audio‐recordings were used to capture patient–physician communication behaviors and preventive health service delivery as described below.

Outcome Variables

We considered four outcomes: (1) visit length; (2) patient engagement communication behavior; (3) physician–patient‐centered communication behavior; and (4) physician delivery of evidence‐based preventive health services. The first, visit length, represented the face‐to‐face interaction time in minutes between patients and physicians. For the communication behavior variables, research assistants coded transcripts using the Street Patient Activation Coding System (Street and Millay 2001). Using this system, occurrences of patient question asking, assertive responses, and expressions of concern are identified and summed. Likewise, occurrences of physician use of partnership building and supportive talk are identified and summed. For each, we summed the total number of unprompted occurrences during the visit. As such, each communication variable reflects a count of the self‐initiated or facilitative communication used by the physician and patient during the visit. Interrater reliability for the communication variables was previously assessed with a random sample of approximately 10 percent of visits. Cohen's kappa (weighted) was on average 0.68 (median = 0.72, range 0.66–0.74) for these variables (Shay et al. 2012).

The fourth outcome variable was the percentage of delivered guideline‐recommended preventive health services for which the patient was eligible and due at the time of the PHE. For each service, delivery was defined by any of the following: (1) a recommendation by the physician to change the behavior or receive the service; (2) a suggestion by the physician to think about changing the behavior or receiving the service; (3) reinforcement by the physician of current or future planned behavior change; or (4) actual service delivery (Shires et al. 2012). Preventive health services for which each patient was eligible and due were determined as recommended by the U.S. Preventive Services Task Force (www.uspreventiveservicestaskforce.org) and Advisory Committee on Immunization Practices (www.cdc.gov/vaccines/recs/acip) relative to the patient's individual risk factor status (e.g., age, gender, tobacco use, etc.). Services of interest (with EHR prompt indicated by “*”) included screening (cervical cancer*, colorectal cancer*, breast cancer*, hypertension*, cholesterol*, diabetes, osteoporosis, vision, and hearing); counseling (aspirin, tobacco, alcohol, calcium*, mental health, obesity*, diet); and immunizations (pneumococcal*, tetanus*, influenza*). Details regarding determination of eligibility can be found elsewhere (Shires et al. 2012). Interrater reliability for these variables was previously assessed with a random sample of approximately 10 percent of visits. Cohen's kappa (weighted) was on average 0.73 (median = 0.81, range 0.54–0.78) (Shires et al. 2012).

Statistical Methods

Associations between EHR, HRA, and patient‐generated written list use and outcomes of interest were assessed using Mplus to simultaneously estimate four linear regression models (one for each of: visit length, physician–patient‐centered communication, patient engagement communication, and preventive health service delivery). Results therefore account for correlations among dependent variables. Both unadjusted and adjusted models were estimated using full information maximum likelihood estimation. Standard error estimates correct for clustering of patients by physician. Variable inclusion for the adjusted model was guided by the Institute of Medicine's framework for patient‐centeredness (Institute of Medicine, 2001). According to this framework, patient‐centered care depends collectively on clinicians, patients, relationships (clinical and social), and health services. As such, our adjusted models control for (a) patient sociodemographic and health need characteristics, (b) physician characteristics, (c) the relationship between the patient and physician, and (e) characteristics of the environment in which the visit took place.

Results

Characteristics of the Study Visits

Sample characteristics are shown in Table 1. Physicians accessed the EHR in the exam room in the majority of visits (81 percent), but an HRA instrument was used in only 16 percent of visits, and only 11 percent of patients used a preprepared written list during the visit. We found few statistically significant differences in patient, physician, and visit characteristics by whether the EHR was accessed during the visit or the patient brought a written list to the visit (Table 1). We did, however, find a number of differences in patient characteristics between visits in which an HRA instrument was used compared to those in which one was not used (Table 1).

Table 1.

Sample Characteristics (N = 485 Visits)

Characteristics	All Visits (N = 485)	Office‐Based Tool
		EHR		HRA		Patient List
		Yes (n = 392)	No (n = 93)	Yes (n = 61)	No (n = 424)	Yes (n = 52)	No (n = 433)
Age	58.7 (8.3)	58.8 (8.4)	58.3 (7.7)	59.3 (8.6)	58.7 (8.2)	60.4 (9.1)	58.5 (8.2)
Female (%)	65.2	64.5	67.7	62.3	65.6	57.7	66.1
Race (%)				a
Black	27.8	26.0	35.5	16.4	29.5	19.2	28.9
White	65.6	67.9	55.9	70.5	64.9	76.9	64.2
Other	6.6	6.1	8.6	13.1	5.7	3.8	6.9
Educational level (%)
Less than high school	3.9	4.4	2.2	1.6	4.3	1.9	4.2
High school diploma	24.3	25.7	18.3	19.7	24.9	19.2	24.9
Some college or more	71.8	69.9	79.6	78.7	70.8	78.8	70.9
Charlson score	0.8 (1.3)	0.8 (1.3)a	0.5 (0.9)	0.7 (1.1)	0.8 (1.3)	1.0 (1.6)	0.7 (1.2)
CVD risk score	17.0 (9.3)	16.9 (9.2)	17.6 (9.6)	15.5 (9.2)	17.2 (9.3)	19.0 (10.1)	16.8 (9.2)
Depressive symptoms (%)	18.1	18.6	16.1	8.2a	19.6	17.3	18.2
BMI	31.0 (7.3)	31.2 (7.5)	30.2 (6.1)	29.5 (6.3)	31.2 (7.4)	29.5 (6.3)	31.2 (7.4)
Current smoker (%)	18.8	17.3	24.7	11.5	19.8	13.5	19.4
Problem drinker (%)	15.3	15.9	13.0	14.8	15.4	15.4	15.3
No. preventive services due	5.5 (2.3)	5.5 (2.3)	5.5 (2.3)	4.8 (2.1)a	5.6 (2.3)	5.4 (2.4)	5.5 (2.3)
Physician age	49.3 (7.8)	49.9 (7.8)	46.9 (7.6)	49.9 (7.3)	49.3 (7.9)	49.3 (7.4)	49.3 (7.9)
Female physician (%)	57.3	56.6	60.2	62.3	56.6	44.2a	58.9
Physician race (%)
Black	14.4	14.8	12.9	8.2	15.3	7.7	15.2
White	51.3	49.2	60.2	47.5	51.9	61.5	50.1
Other	34.2	36.0	26.9	44.3	32.8	30.8	34.6
Physician specialty (%)		a
General internal medicine	68.0	64.0	84.9	50.8	70.5	65.4	68.4
Family medicine	32.0	36.0	15.1	49.2	29.5	34.6	31.6
Race concordant Pt/MD (%)	49.3	48.0	54.8	42.6	50.2	53.8	48.7
Gender concordant Pt/MD (%)	72.8	72.2	75.3	80.3	71.7	67.3	73.4
Pt/MD visit within 12 months (%)	82.4	81.5	86.0	83.6	82.2	76.9	83.0
Minutes late physician arrived	23.9 (18.0)	25.3 (18.4)a	18.0 (14.7)	32.9 (19.5)a	22.6 (17.4)	25.6 (18.6)	23.7 (17.9)

BMI, body mass index; CVD, cardiovascular disease; EHR, electronic health record; HRA, health risk appraisal; MD, medical doctor; Pt, patient.

^a p < .05.

Visit Length, Patient–Physician Communication Behaviors, and Preventive Health Service Delivery

On average, physicians spent almost 27 minutes with the patient (SD = 10 minutes) (Table 2). Ninety‐three percent of visits included at least one instance of the physician using a patient‐centered communication behavior that was not prompted by something the patient said. Across all visits, the mean number of instances per visit of such physician facilitative verbal communication behaviors was 2.6 (SD = 1.8) (Table 2). Eighty‐seven percent of visits contained at least one instance of engagement communication behavior by the patient that was not prompted by something the physician said, and, on average, each visit contained 2.7 (SD = 3.1) instances of such patient engagement communication behaviors (Table 2). On average, patients were eligible and due for three guideline‐recommended preventive health services at the time of their visit (SD = 1.6) and, on average, 57.5 percent (SD = 25.0) of those services were delivered (Table 2).

Table 2.

Mean (Standard Deviation) Visit Length, Communication Behaviors, and Preventive Health Service Delivery by Office‐Based Tool Use

Outcomes	All Visits (N = 485)	Office‐Based Tool Use
		EHR		HRA Instrument		Patient List
		Yes (n = 392)	No (n = 93)	Yes (n = 61)	No (n = 424)	Yes (n = 52)	No (n = 433)
Visit length (in minutes)	26.9 (10.0)	27.7 (10.3)	23.9 (7.9)	29.2 (11.7)	26.6 (9.6)	30.0 (8.1)	26.5 (10.1)
		p = .02		p = .16		p < .001
Physician facilitativecommunication	2.6 (1.8)	2.5 (1.8)	3.0 (1.7)	2.9 (2.1)	2.5 (1.7)	2.1 (1.4)	2.6 (1.8)
		p = .06		p = .21		p = .007
Patient engagementcommunication	2.7 (3.1)	2.8 (3.2)	2.5 (2.7)	3.7 (4.2)	2.6 (2.9)	4.4 (4.6)	2.6 (2.8)
		p = .39		p = .06		p = .004
Percent of due services delivered	57.7 (25.0)	56.5 (25.5)	62.7 (22.2)	62.1 (24.0)	57.0 (25.1)	58.6 (22.7)	57.6 (25.3)
		p = .03		p = .001		p = .63

p‐values represent unadjusted linear regression models.

EHR, electronic health record; HRA, health risk appraisal.

Associations between Office‐Based Tools and Care Quality and Other Outcomes

Prior to adjusting for other factors (Table 2), mean visit length was significantly longer for patients who used a self‐generated written reminder list compared to those patients who did not use such a list (30.0 vs. 26.5 minutes). Visit length was also significantly longer when the EHR was accessed in the exam room compared to those visits in which the EHR was not accessed in the exam room (27.7 vs. 23.9 minutes). Physicians used less facilitative communication when patients brought a written reminder list to the appointment (2.1 vs. 2.6 instances of use), while patients used more engagement communication when they brought a written reminder list (4.4 vs. 2.6 instances of use). Patients were delivered more of the preventive health services for which they were eligible and due if an HRA instrument was available (62.1 percent vs. 57.0 percent), but were delivered significantly fewer of those services if the EHR was accessed in the exam room (56.5 percent vs. 62.7 percent).

In adjusted models (Table 3), patient use of a self‐generated written list was associated with just under a 3‐minute increase in visit length, a decrease in the use of patient‐centered facilitative communication by the physician, and an increase in the use of engagement communication by the patient. We also found a positive association between exam room–based EHR use and visit length, with those visits that included exam room–based use of the EHR lasting, on average, just over 3 minutes more than visits in which the EHR was not accessed in the exam room. Furthermore, EHR use was negatively associated with the percent of due preventive services delivered, with patients delivered, on average, 8 percent fewer of the preventive services for which they were due if the visit included in‐room EHR use. No other significant relationships were identified between the use of office‐based tools and the outcomes evaluated.

Table 3.

Simultaneous Multivariable Regression Results (N = 484)

Office‐Based Tool	Visit Length (in Minutes)	Physician FacilitativeCommunication	Patient EngagementCommunication	Percent of Due Services Delivered
	β (SE)	β (SE)	β (SE)	β (SE)
EHR	3.19a (1.23)	−0.47 (0.32)	0.24 (0.27)	−8.28a (2.69)
HRA instrument	1.60 (1.63)	0.44 (0.28)	0.84 (0.53)	0.85 (2.98)
Patient list	2.98a (1.01)	−0.61a (0.19)	1.69a (0.64)	1.79 (2.90)

All models control for patient age, gender, race, education, Charlson comorbidity score, Framingham cardiovascular disease risk, depressive symptoms, body mass index (BMI), smoking and drinking status, number of preventive services due; physician age, gender, race, and specialty; and the gender and race concordance of the patient–physician dyad, whether the patient had seen the same physician in the prior year, and the minutes after the scheduled appointment time the visit started.

EHR, electronic health record; HRA, health risk appraisal.

^a p < .01.

Discussion

Our results illustrate the potential for both intended and unintended consequences of office‐based tools. Of particular note is the influence of self‐generated patient written lists. As intended, we found that patient lists were associated with patients' engagement in office visit conversations, albeit at the cost of increased visit length and a reduction in physician facilitative communication. We also found that exam room use of EHRs was associated with longer visit length and delivery of a smaller percentage of guideline‐recommended preventive health services for which the patient was due. On the other hand, the use of an HRA instrument was neither positively nor negatively associated with visit length, patient–physician communication, or preventive service delivery. Our observational study findings of mixed implications from the use of office‐based tools common to practice redesign are consistent with the mixed findings from recent randomized trials evaluating patient‐centered medical homes (PCMHs) (DeVries et al. 2012; Hoff, Weller, and DePuccio 2012; Fifield et al. 2013; Jackson et al. 2013; Landon 2013; Friedberg et al. 2014).

Given the increasing demands on physicians' time (Yarnall et al. 2003), perhaps most striking are findings regarding the association of office‐based tools and visit length. Two of the three office‐based tools evaluated were associated with increased visit length, for an average of almost 6 minutes more combined. The third office‐based tool (HRA instrument), while not associated with increased visit length, was not associated with decreased length either. Such findings are consistent with those from a recent study finding physician perceptions of lost time due to EHRs (McDonald et al. 2014), and highlight the potential for well‐intended office‐based tools, often being incorporated into PCMHs, to add to the time constraints already faced in primary care.

Our findings that patient self‐generated written lists were associated with increased visit length is consistent with a prior study that also found that such lists increase visit length (Middleton, McKinley, and Gillies 2006). But this result is in conflict with other findings that structured question prompt lists do not alter consultation length (Brown et al. 2001). It may be that providing patients with a question prompt list that contains a range of prespecified, salient questions from which the patient can select ahead of time helps to focus the patient's agenda and thus ensuing conversation. Or it may be that patients who bring self‐generated written lists with them are inherently different from those who do not bring such lists in a way that is not captured by our data.

While our findings highlight concerns regarding the potential for office‐based tools to add to the time pressures faced by primary care physicians, consistent with some prior studies (Frankel et al. 2005; Hsu et al. 2005; Nagy and Kanter 2007), we did not find that EHR use altered either patient or physician communication behaviors. On the other hand, others have found that patients rate physician communication as less effective when physicians spend more time looking at the computer (Street et al. 2014). Nonetheless, in our study only the use of patient‐generated written lists was associated with changes in physician or patient communication behaviors. While patients' use of written lists was associated with a decrease in physician facilitative communication behaviors, or those behaviors designed to support patient engagement, it was simultaneously associated with an increase in unprompted patient engagement communication during the visit. It is unknown whether fewer facilitative physician communication behaviors are needed to achieve the same outcome when patients come to visits already activated. As physicians and other providers are continually encouraged to use patient‐centered communication (Sheridan, Harris, and Woolf 2004; Epstein and Street 2007), findings here highlight the importance of understanding the pathways through which communication heals (Street et al. 2009). Until these pathways are better understood, it will remain difficult to disentangle the ultimate impact of patient written lists.

Finally, contrary to prior findings (Chaudhry et al. 2006), we found in‐office use of EHRs to be associated with a decrease in the percent of eligible and due preventive health services the patient received. While most previous studies have found EHR prompts to be positively associated with delivery of prompted services, to our knowledge prior studies have not specifically considered either in‐office EHR access or its impact on a bundle of prompted and unprompted services collectively. Our study, on the other hand, evaluated receipt of a bundle of preventive services regardless of whether there was a specific prompt for the service and we could consider only EHR use that occurred within the exam room. Prior studies, however, have found that EHRs can hinder history taking during clinical encounters (Lown and Rodriguez 2012) and lead to “screen‐driven” information gathering (Patel, Arocha, and Kushniruk 2002), which may in turn lead to missed opportunities for relevant risk factor discovery within a patient's own narrative thread.

Results should be considered in the context of a number of limitations. For example, although we know whether an EHR, HRA instrument, and patient written list was used, we considered the use of each tool generally, and not any specific characteristic of the tool. Nor did we consider the amount of time or attention given to them, how they were integrated within care processes, or the skill with which they were used. This latter factor may be particularly important in the case of EHRs (Booth, Robinson, and Kohannejad 2004; Ventres et al. 2005; McGrath, Arar, and Pugh 2007; Rouf et al. 2007). Furthermore, we were not able to consider EHR use that occurred outside of the exam room (whether by the physician or another clinic staff member in support of the physician's activities). In addition, while we estimated the regression equations simultaneously and controlled for a number of factors, because physician preference/discretion determined EHR and HRA use during office visits, and patient preference/choice determined use of a patient‐generated written list, our results are subject to selectivity bias. Such omissions and potential selectivity biases illustrate the ongoing challenges faced when studying the impact of these and other office‐based tools. In addition, care should be taken when generalizing findings to other settings as findings may be a result of nuances specific to the office‐based tools used in this setting, the skills with which they were used, or other characteristics of, as well as the size of, the available sample. Nonetheless, to our knowledge the setting, data, and methods used here represented a unique opportunity to consider simultaneously the influences of a number of primary care office visit tools, many of which are being advocated as important components of practice redesign, on both cost and quality outcomes.

Conclusions

Our results indicate that office‐based tools commonly being introduced into primary care clinics are associated with both positive and negative quality and efficiency outcomes. Findings therefore highlight the challenges faced when redesigning complex systems such as those found within primary care clinics as well as the ongoing need for well‐designed studies—that either via research design or statistical control—account for potential selectivity biases and the impact of contextual factors when evaluating not only the intended consequences of primary care practice redesign tools but also their potential unintended consequences.

Supporting information

Appendix SA1: Author Matrix.