Abstract
Objective
To examine the relationship between medical home transformation and patient experience of chronic illness care.
Study Setting
Thirteen safety net clinics located in five states enrolled in the Safety Net Medical Home Initiative.
Study Design
Repeated cross‐sectional surveys of randomly selected adult patients were completed at baseline (n = 303) and postintervention (n = 271).
Data Collection Methods
Questions from the Patient Assessment of Chronic Illness Care (PACIC) (100‐point scale) were used to capture patient experience of chronic illness care. Generalized estimating equation methods were used to (i) estimate how differential improvement in patient‐centered medical home (PCMH) capability affected differences in modified PACIC scores between baseline and postintervention, and (ii) to examine cross‐sectional associations between PCMH capability and modified PACIC scores for patients at completion of the intervention.
Principal Findings
In adjusted analyses, high PCMH improvement (above median) was only marginally associated with a larger increase in total modified PACIC score (adjusted β = 7.7, 95 percent confidence interval [CI]: −1.1 to 16.5). At completion of the intervention, a 10‐point higher PCMH capability score was associated with an 8.9‐point higher total modified PACIC score (95 percent CI: 3.1–14.7) and higher scores in four of five subdomains (patient activation, delivery system design, contextual care, and follow‐up/coordination).
Conclusions
We report that sustained, 5‐year medical home transformation may be associated with modest improvement in patient experience of chronic illness care for vulnerable populations in safety net clinics.
Keywords: Primary care, primary care redesign, medical home, patient‐centered medical home, chronic disease, chronic illness, chronic illness care, patient experience, patient satisfaction, patient‐oriented measures, safety net clinics, vulnerable populations
The patient‐centered medical home (PCMH) has gained widespread attention as an important model to redesign U.S. primary care and address population health. Broadly, the PCMH is a physician‐directed medical practice that can enhance access, coordinate care, and improve safety and quality through whole‐person oriented care (American College of Physicians 2007). Part of the policy interest in the medical home movement can be attributed to poor patient experiences with chronic illness care (Berenson et al. 2008; Sidorov 2008), which have been associated with higher health care costs and poorer health outcomes (Doyle, Lennox, and Bell 2013). Longitudinal studies examining patient experiences have shown a general decline in the quality of provider–patient interactions among Medicare beneficiaries, many of whom have chronic illnesses (Safran 2003). Through implementation of the PCMH, health systems have attempted to improve patient experiences and outcomes—thus transforming health care delivery from reactionary, acute‐care models to preventative, chronic‐care models that emphasize a patient‐centered approach.
Patient experience is most fully defined as “the sum of interactions, shaped by culture, that influence a patient's perceptions across the continuum of care” (Wolf et al. 2014). Despite a theoretical framework that identifies patient experience as central to PCMH practice redesign, prior studies evaluating the effectiveness of PCMH models to address chronic illness care have often focused on process measures (e.g., measurement of HbA1c), clinical outcomes (e.g., glycemic control, healthy body weight), and cost (e.g., Medicare savings) (Flottemesch et al. 2012; Nocon et al. 2012; Peikes et al. 2012; Jackson et al. 2013; Markovitz et al. 2015). These studies have demonstrated mixed findings, and recent systematic reviews have described the evidence on process measures to be largely inconclusive (Peikes et al. 2012; Jackson et al. 2013). Berenson et al. (2008) cautioned against this bias toward practice‐based measures in his commentary, “A House Is Not a Home,” describing a tendency to overemphasize the structural redesign of practices while losing focus on the patient‐centered aspects that hastened redesign in the first place.
Fewer studies have examined patient experience measures, or they have limited evaluation to only a small subset of patient experience measures (Glasgow, Peeples, and Skovlund 2008). First, studies have often used general patient‐oriented metrics that are not necessarily specific to discrete components of “patient experience,” such as patient satisfaction (McFarland et al. 2014; Nelson et al. 2014). However, “patient experience” is thought to capture a more robust set of constructs that connect the lived experience of illness to more specific and practical elements of clinical care (e.g., “Were you helped to make a treatment plan that you could carry out in your daily life?”) (Martsolf et al. 2012; Wolf et al. 2014). Second, studies using patient experience measures have often restricted evaluations to only one or two domains as part of a larger analysis with additional process or outcome measures. The findings from these studies have been mixed, with several demonstrating isolated improvements in patient–provider communication (Pourat, Lavarreda, and Snyder 2013; Heyworth et al. 2014), patient activation or empowerment (Reid et al. 2009; Pourat, Lavarreda, and Snyder 2013; Nocon et al. 2014), and shared decision making (Reid et al. 2009). Other studies have demonstrated no improvement in patient‐rated outcomes, including interpersonal exchange and goal setting (Jaen et al. 2010; Martsolf et al. 2012). Several have applied a broader set of patient experience measures to evaluate chronic illness care (Reid et al. 2009, 2010), but fewer have applied these measures in safety net clinics (Sugarman et al. 2014).
This study examined whether PCMH transformation improved patient experience of chronic illness care in safety net clinics. We used primary data obtained from the Safety Net Medical Home Initiative (SNMHI), a 5‐year national initiative launched in 2008 to transform safety net clinics into medical homes. Our dataset is one of the few to apply a broad set of patient experience measures to evaluate chronic illness care (Glasgow et al. 2005), and one of the first to evaluate a PCMH demonstration program lasting more than 2 years in practices caring for vulnerable populations. Specifically, we examined whether (i) differential improvement in PCMH capability was associated with differences in patient experience of chronic illness care before and after the SNMHI intervention, and (ii) higher PCMH capability was associated with better patient experience of chronic illness care at completion of the SNMHI intervention.
Methods
Study Design
The SNMHI was a 5‐year demonstration supported by the Commonwealth Fund (Sugarman et al. 2014). For our study, we used a repeated cross‐sectional design with distinct random samples of patients taken at baseline and post‐intervention from participating clinics. Due to financial constraints, 24 clinics were randomly selected to administer patient surveys from the initial 65 clinics participating in the SNMHI intervention. The random selection process was completed as a two‐stage sample to avoid random selection of multiple clinic sites from within the same health center organization. The first stage consisted of randomly ordering each health center organization within each state. For health center organizations with greater than one clinic participating in the study, the second stage consisted of randomly ordering each clinic within each health center organization. Clinics were selected from each of the five states (Colorado, Idaho, Massachusetts, Oregon, and Pennsylvania) until 24 clinics were selected for participation. Of the 24 clinics included at baseline, 13 completed both baseline and postintervention surveys and were included in our analyses. Of the 11 clinics who did not participate in the post‐intervention survey, one clinic discontinued participation in the overall SNMHI, while other clinics cited lack of time due to other competing administrative and patient care priorities. Characteristics of participating and nonparticipating clinics are included in Appendix A.
Implementation of the intervention was led by Qualis Health and the MacColl Center for Health Care Innovation at the Group Health Research Institute and has been described in previously published work (Sugarman et al. 2014). Briefly, practice transformation was guided by eight “change concepts” (e.g., patient‐centered interactions, enhanced access), based on medical home principles specifically tailored to the safety net setting. These changes were supported locally by a Regional Coordinating Center with practice coaches to organize the intervention (Sugarman et al. 2014).
This study was approved by the University of Chicago Institutional Review Board (IRB).
Data and Measures
Patient Survey
Participating patients were randomly selected at each clinic to complete the survey. To minimize the amount of patient information shared during this study, individual clinics took responsibility for applying patient names and addresses to prestamped envelopes. Self‐administered surveys were mailed to 70 patients (age ≥18) at each clinic, representing distinct samples at baseline and postintervention. Surveys were translated into the patient's preferred language (English, Spanish, or Portuguese). In 2010, baseline surveys were mailed by clinics beginning 13 months after the intervention began; in 2014, postintervention surveys were mailed by clinics beginning 60 months after the intervention began. A one‐time incentive of $2 was included with initial mailings; nonresponders were mailed a follow‐up survey up to three additional times. We received 420 eligible responses at baseline and 401 eligible responses at postintervention, reflecting a 46.2 percent and 44.1 percent response rate, respectively.
Surveys included questions on patient‐level characteristics, including age, gender, race, education, insurance status, and self‐reported health status (i.e., “In general, how would you rate your overall health?”). To limit the sample to those reporting ongoing chronic illness, we used the following question: “Do you have any health issue that has required treatment by your regular provider over the past 12 months?” Of note, this question was designed to capture chronic illness care, a construct that is uniquely distinct from chronic disease management. By definition, chronic illness refers to the personal experience of living with an affliction (e.g., chronic pain, recurrent urinary tract infection, diabetes) that often includes or accompanies chronic disease but is not limited to chronic disease (Martin 2007). This construct is more broadly applicable to diverse patients (Glasgow et al. 2005).
To capture patient experience of chronic illness care, we used a subset of survey items from the Patient Assessment of Chronic Illness Care (PACIC; Appendix SA3) (Glasgow et al. 2005). The PACIC is an existing, validated, and comprehensive patient self‐report instrument developed by Glasgow and collaborators to evaluate whether patients with chronic illness experience care that aligns with models of chronic‐care delivery (Glasgow et al. 2005). The full instrument consists of 20 items across five subdomains (patient activation, delivery system support, goal setting, contextual care, and care coordination; definitions in Appendix SA3). We used 11 items across all five original subdomains, selecting questions most relevant to the PCMH intervention. Although originally validated in its entirety, others have validated short‐form versions of the PACIC (Cramm and Nieboer 2012); and each question in the PACIC assesses specific activities that “form the core of modern, patient‐centered self‐management support” (Glasgow et al. 2005). Therefore, our modified PACIC scale addresses the specific patient‐centered activities most likely to be impacted by PCMH implementation. Modified PACIC subdomain scores were derived from averaging the individual items in each subdomain, scored on a 1–5 point Likert‐type scale and subsequently converted into a single score on a 100‐point scale (0 indicates worst and 100 indicates best). The total modified PACIC score was derived from averaging the five subdomain scores and reported as a single score on a 100‐point scale. Participants completing any item(s) in any subdomain were included for analysis.
Provider and Staff Survey
Participating providers and staff were randomly selected at each clinic to complete the survey. Providers were defined as physicians, physician assistants, or nurse practitioners. Clinical staff were defined as behavioral health specialists, educators, certified medical assistants, counselors, dieticians, medical assistants, nurses (licensed practical nurse or registered nurse), or social workers. Self‐administered surveys were mailed to 15 providers and staff at each clinic at baseline and post‐intervention. In 2010, we mailed baseline surveys beginning 8 months after the intervention began. In 2013, we mailed postintervention surveys beginning 51 months after the intervention began. A one‐time incentive of $10 was included with initial mailings; nonresponders were mailed a follow‐up survey up to three additional times. From the 13 clinics included in this analysis, we received 124 eligible responses at baseline and 138 eligible responses at postintervention, reflecting a 63.6 percent and 70.8 percent response rate, respectively.
To capture medical home capability, we used a PCMH assessment tool developed to monitor progress and capture change in medical home capability during the SNMHI intervention period (Lewis et al. 2012; Sugarman et al. 2014). For this analysis, each clinic's PCMH score reflects aggregated provider and staff responses. The PCMH score was constructed based on the 2008 National Committee for Quality Assurance PCMH standards (https://www.ncqa.org/Portals/0/PCMH%20brochure-web.pdf). Survey items were organized along five domains (i.e., patient access and communication, provider communication, data tracking, care management, and quality improvement). Questions were adapted from national health care provider surveys (Linzer et al. 2009) and PCMH evaluation surveys (Nutting et al. 2009; Reid et al. 2009); a small subset were created by the SNMHI team. Questions were selected based on content validity; Cronbach's alpha for the five subscales ranged from 0.48 (five‐item access to care and communication with patients subscale) to 0.82 (seven‐item care management subscale), with an overall 0.87 for the total PCMH score. The full instrument consisted of 25 individual items scored on a 1–5 point Likert‐type scale (Appendix SA4). PCMH domain scores were derived from averaging the individual items in each domain and converted on a 100‐point scale (0 indicates worst and 100 indicates best). The total PCMH score was calculated by averaging all five PCMH domain scores and used to reflect provider and staff ratings of PCMH capability.
Organizational Survey
Participating organizational leaders at each clinic completed the survey. This survey was designed to provide an overview of each clinic's organizational capability and clinic characteristics (Birnberg et al. 2011). In 2009, we mailed baseline surveys beginning 2 months after the intervention began; in 2013, we mailed postintervention surveys beginning 54 months after the intervention began. We received 24 eligible responses at both baseline and postintervention, reflecting a 100 percent response rate.
Surveys included questions on clinic‐level characteristics, including state (i.e., state‐level variation in health policy), location/setting (i.e., regional variation in health insurance markets), provider full‐time equivalents (i.e., workforce capacity), and Electronic Health Record (EHR) adoption (i.e., technological infrastructure). In this study, this information was used for descriptive purposes only.
Timing of Surveys
Baseline surveys were mailed in a step‐wise fashion to capture organizational leadership (2 months after the intervention began), clinic staff/providers (8 months after the intervention began), and patients (13 months after the intervention began) after orientation to the intervention, but before substantive differences could be measured. As for most complex, large‐scale, multistate practice transformation efforts, the first year of the project focused on orientation, planning, basic formation, buy‐in from local teams, and the beginning of transformation projects. Given the scope and emphasis of this 5‐year demonstration on sustained implementation of the medical home, we felt that these were adequate baseline timeframes to compare practice transformation at baseline to practice transformation after the 5‐year intervention period. In addition, since each of these surveys required some work on part of each study clinic to administer the surveys, staggering the administration of surveys allowed clinics to participate in multiple parts of the study without excessive disruption at any given time.
Analysis
Descriptive statistics were calculated for both clinic and patient characteristics. We calculated modified PACIC scores and PCMH scores for each clinic at baseline and postintervention.
Change in Patient Experience of Chronic Illness Care by Improvement in Provider/Staff Ratings of Medical Home Capability
To analyze the effects of improving medical home capability on patient experience of chronic illness care, we used a difference‐in‐differences approach to compare patients in “high improvement” clinics to those in “low improvement” clinics. We defined the “high improvement” group as patients in clinics that achieved greater than or equal to median improvement (4.7‐point or greater improvement in PCMH capability) and the “low improvement” group as patients in clinics that achieved less than median improvement in PCMH capability over the 5‐year intervention period. We used generalized estimating equations (GEE) to model modified PACIC scores (total and subdomain) as a function of PCMH‐improvement group, survey period (baseline or postintervention), and their interaction (the difference‐in‐differences estimate). All models used an exchangeable correlation structure, identity link, and Gaussian family distribution, accounting for cluster effects of patients within clinics. Models were adjusted for compositional changes in the patient population from baseline to postintervention, including age group, insurance status, and self‐reported health status.
The Association between Patient Experience of Chronic Illness Care and Provider/Staff Ratings of Medical Home Capability at Completion of the SNMHI Intervention
To examine the cross‐sectional relationship between final PCMH capability and patient experience of chronic illness care, we performed a cross‐sectional analysis at postintervention. We used GEE with robust design‐based variance estimators (specified exactly as above) to model the association between modified PACIC scores (total and subdomain) and PCMH score (100‐point scale). We created one multivariable model for each dependent variable of interest, including total modified PACIC score and each subdomain score. We included all relevant sociodemographic characteristics in each model, including age, gender, race, education, and insurance status. We also included self‐reported health status and number of visits in the past 12 months.
We used a 10‐point higher PCMH score to describe the effect on patient experience of chronic illness care, which was found to be operationally meaningful and feasible in prior studies (Lewis et al. 2012; Nocon et al. 2014).
All analyses were performed using STATA, version 13.1 (StataCorp LP, College Station, Texas, USA).
Results
Characteristics of SNMHI Clinics and Responding Patients
Of the 24 baseline clinics, 13 completed the postintervention patient survey and were included in this study. The 11 clinics that did not complete the post‐intervention survey had similar clinic‐level characteristics and baseline PCMH scores (Appendix SA2); however, the baseline patient populations in those 11 nonparticipating clinics were younger, less educated, more likely to be uninsured, and had higher percentages of Hispanic persons. The 13 study clinics were representative of all states in the intervention. Over one‐third were from urban or suburban settings (38.5 percent). By the postintervention period, all clinics had adopted EHR (Table 1). Across the 13‐clinic study population, the majority of respondents were white non‐Hispanic (69.9 percent), female (64.0 percent), and 45–64 years old (45.2 percent). Compared to baseline, the postintervention sample consisted of significantly fewer young adults (−2.7 percent), more elderly adults (+4.6 percent), more respondents with fair or poor self‐rated health status (+12.6 percent), and fewer respondents with no insurance (−4.2 percent).
Table 1.
Clinic and Patient Characteristics: Safety Net Medical Home Initiative, United States, 2009–2014
| Clinic Characteristics (N = 13) | Baseline (n = 13) % | Postintervention (n = 13) % | p‐valuea |
|---|---|---|---|
| State | |||
| Colorado | 23.1 | 23.1 | – |
| Idaho | 15.4 | 15.4 | |
| Massachusetts | 30.8 | 30.8 | |
| Oregon | 23.1 | 23.1 | |
| Pennsylvania | 7.7 | 7.7 | |
| Location | |||
| Urban or suburban | 38.5 | 38.5 | – |
| Small town, rural, or frontier | 61.5 | 61.5 | |
| More than 8 provider full‐time equivalents | 46.2 | 38.5 | – |
| Electronic health record adoption | 61.5 | 100 | – |
| Patient‐centered medical home scoreb | |||
| Mean ± SD | 62.1 ± 7.0 | 66.1 ± 7.4 | – |
| Provider and Staff Characteristics | Baseline (n = 124) % | Postintervention (n = 149) % | |
| Provider or staff type | |||
| Physician | 29.8 | 28.7 | .12 |
| Nurse practitioner or physician assistant | 24.0 | 21.7 | |
| Registered nurse | 14.1 | 8.4 | |
| Licensed practical nurse or assistant | 19.8 | 19.6 | |
| Other | 12.4 | 21.7 | |
| Years employed at clinic | |||
| Median interquartile range [IQR] | 5.0 [2.0–10.8] | 4.8 [1.5–10.0] | .50 |
| Patient Characteristics | Baseline (n = 303) % | Postintervention (n = 271) % | |
| Age (years) | |||
| 18–24 | 4.6 | 1.9 | .11 |
| 25–44 | 15.9 | 19.3 | |
| 45–64 | 47.7 | 42.4 | |
| 65+ | 31.8 | 36.4 | |
| Female: Gender | 65.6 | 62.2 | .41 |
| Race | |||
| Black non‐Hispanic | 7.8 | 7.5 | .93 |
| Hispanic | 15.9 | 18.0 | |
| White non‐Hispanic | 70.9 | 68.9 | |
| Other | 5.4 | 5.6 | |
| Education | |||
| Less than secondary school graduation | 14.4 | 17.0 | .69 |
| Secondary school graduation or GEDc | 33.6 | 30.0 | |
| Some college or professional school | 34.2 | 36.3 | |
| College or more than college | 17.8 | 16.7 | |
| Patient Characteristics | Baseline (n = 303) % | Postintervention (n = 271) % | |
| Insurance | |||
| Medicaid/Medicare | 57.6 | 60.8 | .12 |
| Private/Other | 34.3 | 35.4 | |
| None | 8.1 | 3.9 | |
| Health status (self‐report) | |||
| Poor | 5.3 | 8.9 | .03 |
| Fair | 23.2 | 32.2 | |
| Good | 40.7 | 34.4 | |
| Very good | 22.2 | 17.8 | |
| Excellent | 8.6 | 6.7 | |
p‐values were not generated for clinic‐level data due to small sample size.
The Patient‐Centered Medical Home Score (100‐point scale: 0 = worst, 100 = best) was developed based on the 2008 National Committee for Quality Assurance PCMH standards, using five PCMH subscales: access to care and communication with patients, communication with other providers, tracking data, care management, and quality improvement (Sugarman et al. 2014).
General Educational Development (GED) test.
Change in Patient Experience of Chronic Illness Care by Improvement in Provider/Staff Ratings of Medical Home Capability
Mean PCMH score was 62.1 ± 7.0 at baseline and 66.1 ± 7.4 at post‐intervention (Table 1). Improvement in PCMH capability varied widely between clinics over the SNMHI intervention period, ranging from a 7.3‐point decline to an 11.6‐point improvement. Compared with the low PCMH‐improvement group, the high PCMH‐improvement group had lower modified PACIC scores at baseline in every category (Table 2). In adjusted analyses, high PCMH‐improvement was marginally associated with a 7.7‐point increase (ΔPACIC‐high − ΔPACIC‐low) in total modified PACIC score (95 percent CI: −1.1 to 16.5), 8.9‐point increase in patient activation score (95 percent CI: −1.2 to 19.0), and 7.6‐point increase in delivery system design score (95 percent CI: −0.7 to 16.0) over the intervention period (Table 2, Figure 1). Although adjusted difference‐in‐differences estimates for remaining subdomains suggested modest improvements in chronic illness care over the intervention period, these findings were not statistically significant.
Table 2.
Change in Patient Experience of Chronic Illness Care by Improvement in Provider/Staff Ratings of Medical Home Capability
| Modified PACICa Domain (N = 574) | Unadjusted Mean Score | ∆ Mean Score (95% CI) | ||||||
|---|---|---|---|---|---|---|---|---|
| Low PCMH‐Improvement Group (<Median ∆)b | High PCMH‐Improvement Group (≥Median ∆)b | Estimated Difference‐in‐Differences in PACIC Scorec | ||||||
| Baseline | Postintervention | Baseline | Postintervention | Unadjusted Estimate | p‐value | Adjusted Estimated | p‐value | |
| Overall PACIC (n = 485) | 59.4 | 58.9 | 54.6 | 60.6 | 6.5 (−1.8, 14.8) | .13 | 7.7 (−1.1, 16.5) | .09 |
| Patient activation (n = 517) | 70.4 | 67.2 | 67.0 | 72.0 | 7.3 (−2.2, 16.9) | .13 | 8.9 (−1.2, 19.0) | .08 |
| Delivery system design (n = 517) | 83.1 | 77.4 | 78.1 | 77.4 | 4.5 (−3.6, 12.6) | .28 | 7.6 (−0.7, 16.0) | .07 |
| Goal setting/tailoring (n = 508) | 47.2 | 48.6 | 44.0 | 51.5 | 6.2 (−5.1, 17.4) | .28 | 4.4 (−7.3, 16.0) | .46 |
| Contextual care (n = 512) | 65.5 | 67.2 | 59.6 | 66.5 | 5.2 (−5.8, 16.2) | .35 | 6.1 (−5.6, 17.7) | .31 |
| Follow‐up/coordination (n = 494) | 32.9 | 36.7 | 32.0 | 40.6 | 4.9 (−6.5, 16.4) | .40 | 3.7 (−8.3, 15.7) | .55 |
The Patient Assessment of Chronic Illness Care (PACIC) was developed by Glasgow and colleagues to evaluate whether patients with illness experience care that aligns with models of chronic‐care delivery (Glasgow et al. 2005). This version was modified for use in the Safety Net Medical Home Initiative.
The Patient‐Centered Medical Home Score (100‐point scale: 0 = worst, 100 = best) was developed based on the 2008 National Committee for Quality Assurance PCMH standards, using five PCMH sub‐scales: access to care and communication with patients, communication with other providers, tracking data, care management, and quality improvement (Sugarman et al. 2014).
Estimates were obtained from generalized estimating equation (GEE) methods.
Adjusted for age group, self‐reported health status, and insurance status.
Figure 1.
Change in Patient Experience of Chronic Illness Care in High PCMH‐Improvement versus Low PCMH‐Improvement Clinics
- Notes. *The Patient Assessment of Chronic Illness Care (PACIC) was developed by Glasgow and colleagues to evaluate whether patients with chronic illness experience care that aligns with models of chronic‐care delivery (Glasgow et al. 2005). This version was modified for use in the Safety Net Medical Home Initiative. **The Patient‐Centered Medical Home (PCMH) Score (100‐point scale: 0 = worst, 100 = best) was developed based on the 2008 National Committee for Quality Assurance PCMH standards, using five PCMH subscales: access to care and communication with patients, communication with other providers, tracking data, care management, and quality improvement (Sugarman et al. 2014). We defined “high improvement” clinics as those achieving ≥ median improvement and “low improvement” clinics as those achieving < median improvement in PCMH capability over the 5‐year intervention period.
The Association between Patient Experience of Chronic Illness Care and Provider/Staff Ratings of Medical Home Capability at Completion of the SNMHI Intervention
A 10‐point higher PCMH score at postintervention was significantly associated with an 8.9‐point higher total modified PACIC score (95 percent CI: 3.1–14.7; Table 3). A 10‐point higher PCMH score was also associated with a higher subdomain score for patient activation (adjusted β = 9.8, 95 percent CI: 1.4–18.1), delivery system design (adjusted β = 10.6, 95 percent CI: 2.9–18.4), contextual care (adjusted β = 9.2, 95 percent CI: 1.6–16.9), and follow‐up/coordination (adjusted β = 8.9, 95 percent CI: 5.4–12.3) subdomains. An association with the goal setting/tailoring subdomain was not demonstrated.
Table 3.
The Association between Patient Experience of Chronic Illness Care and Provider/Staff Ratings of Medical Home Capability at Completion of the SNMHI Intervention
| Modified PACICa Domain (N = 271) | Per 10‐Point Higher Mean Patient‐Centered Medical Home (PCMH) Scoreb | |||
|---|---|---|---|---|
| Unadjusted β c | Adjusted β c , d | |||
| ΔMean Score (95% CI) | p‐value | ΔMean Score (95% CI) | p‐value | |
| Overall PACIC (n = 236) | 8.8 (1.6, 15.9) | .02 | 8.9 (3.1, 14.7) | <.01 |
| Patient activation (n = 248) | 9.9 (0.1, 19.6) | .048 | 9.8 (1.4, 18.1) | .02 |
| Delivery system design (n = 246) | 10.1 (−0.2, 20.4) | .05 | 10.6 (2.9, 18.4) | <.01 |
| Goal setting/tailoring (n = 244) | 5.6 (−0.9, 12.2) | .09 | 5.1 (−1.0, 11.1) | .10 |
| Contextual care (n = 244) | 7.4 (−3.9, 18.7) | .20 | 9.2 (1.6, 16.9) | .02 |
| Follow‐up/coordination (n = 239) | 8.9 (4.3, 13.4) | <.01 | 8.9 (5.4, 12.3) | <.01 |
The Patient Assessment of Chronic Illness Care (PACIC) was developed by Glasgow and colleagues to evaluate whether patients with illness experience care that aligns with models of chronic‐care delivery (Glasgow et al. 2005). This version was modified for use in the Safety Net Medical Home Initiative.
The Patient‐Centered Medical Home Score (100‐point scale: 0 = worst, 100 = best) was developed based on the 2008 National Committee for Quality Assurance PCMH standards, using five PCMH sub scales: access to care and communication with patients, communication with other providers, tracking data, care management, and quality improvement (Sugarman et al. 2014); scores reflect final PCMH scores (postintervention).
Estimates were obtained from generalized estimating equation (GEE) methods.
Adjusted for age group, gender, education, race, insurance status, self‐reported health status, and number of visits in the past 12 months.
Discussion
As population health emerges as a national priority, the PCMH is an increasingly promising model for addressing the growing burden of chronic illness (Peikes et al. 2012). Our study found that improvement in PCMH capability had varying levels of association with improvement in patient experience of chronic illness care. In clinics that improved their PCMH scores above the median, their patient experience subdomain scores generally increased from baseline to postintervention (Table 2, Figure 1). However, their difference in total modified PACIC score only tended toward significance (p = .09) in final adjusted models (Table 2). In addition, prior studies have documented that a 12‐point or greater change in PACIC score (100‐point scale) is associated with improvement in chronic illness outcomes (Schmittdiel et al. 2008; Schillinger et al. 2009); however, our estimated PACIC improvements were smaller than this, suggesting modest improvement in patient experience with improvement in medical home capability.
We also examined the cross‐sectional association between final PCMH capability and patient experience of chronic illness care, reasoning that absolute PCMH capability at completion of the intervention may provide insight in addition to relative change in PCMH capability. We found that at completion of the intervention, higher PCMH rating was significantly associated with better patient experience; specifically, a higher total modified PACIC score and higher scores in four of five subdomains (Table 3).
These analyses ask two different, albeit related, research questions that both have value. The former (difference‐in‐differences analysis) evaluates change in PCMH capability over time (transformation), but it does not distinguish between absolute scores. The latter (cross‐sectional analysis) evaluates absolute PCMH score (post‐transformation), but it does not distinguish between degrees of transformation. The strength of the difference‐in‐differences analysis is that it reflects changes that policies can enact over time. In addition, the difference‐in‐differences analysis may be more valid if the cross‐sectional results are systematically biased due to unobserved time‐invariant characteristics intrinsic to each clinic, which may be independently correlated with both medical home capability and patient experience. For instance, it may be that a clinic's inherent culture and attitudes toward patient care are independently correlated with both constructs.
Alternatively, cross‐sectional analyses may capture important findings if those clinics completing the intervention with higher absolute PCMH scores had overcome a “threshold effect” to meaningfully improve patient experience. Our data may support this latter hypothesis, because clinics in the high PCMH‐improvement group had unilaterally lower PACIC scores at baseline. We speculate that high‐improvement clinics may have had “more room to improve” but potentially did not improve enough to achieve substantially better patient experience, thereby attenuating the results of our difference‐in‐differences analysis. Taken together, our findings suggest that sustained, 5‐year medical home transformation may be associated with at least modest improvement in patient experience, but larger studies with more variation in the degree of transformation (e.g., very high improvement, high improvement, moderate improvement, etc.) may help to elucidate and inform a meaningful threshold with which to target transformation efforts.
Our findings add to the current literature. First, we used a broader patient experience measure for chronic illness care. Increasingly, health care systems are interested in aligning structural and process improvements in quality with patient experience. Second, this study reflects a 5‐year demonstration. Prior studies have often taken place over less than 2 years (Jackson et al. 2013; Sugarman et al. 2014); our study takes place over a longer period of time, which is uniquely suited for analyzing safety net clinics with traditionally more complicated patient panels and fewer resources (Jones and Furukawa 2014). Finally, this is one of the few multiyear studies that evaluates patient experience of chronic illness care among vulnerable populations. Our study shows that relative improvement in PCMH capability over the SNMHI was marginally associated with modest improvement in patient experience, but higher absolute PCMH capability at conclusion of the SNMHI was significantly associated with better patient experience of chronic illness care.
This study has several important implications. First, to make primary care experiences more patient‐centered, interventions need to be designed with these goals in mind. Prior PCMH studies frequently overemphasized the structural components of “building houses” at the expense of truly creating patient‐centered homes and environments (Berenson et al. 2008). In our study, modest improvements in patient experience, despite increases in PCMH capability, may suggest that clinics focused on structural components of PCMH implementation rather than actions that would more directly enhance patient experience. Second, as policies tie PCMH capability to incentives (e.g., PCMH certification, pay‐for‐performance, branding as a medical home), it will be critical for policy makers to deliberate carefully about the metrics used to evaluate PCMH capability, and to include metrics that are meaningfully aligned with enhanced patient experience.
Our study has several limitations. First, due to IRB restrictions (i.e., feasibility of written informed consent across multiple safety net clinics and implementation by clinic staff), we were unable to follow individual patients over time, thereby restricting any panel or repeated measures analyses to the clinic level. Second, the PACIC was originally validated as a 20‐item survey instrument (Glasgow et al. 2005). However, we felt that some of the original survey items were not relevant for this particular intervention (Sugarman et al. 2014). Thus, we used a modified PACIC scale, similar to other validated short‐form versions (Cramm and Nieboer 2012), to purposefully reflect those items most closely related to the SNMHI and to minimize survey burden. We recommend interpreting these results in light of the individual items included in each subdomain (Appendix SA3), as individual items have been shown to have high factor loadings (i.e., strong association between the survey item and latent variable) in previously published work (Glasgow et al. 2005; Cramm and Nieboer 2012). Third, of the 24 clinics that completed baseline surveys, 13 completed postintervention surveys raising concern for potential selection bias for these 13 clinics (Appendix SA2). Consequently, the findings of this analysis are less generalizable to the full intervention and most closely apply to the 13 clinics completing both baseline and postintervention patient surveys. These clinics were located in less urban areas, had more provider full‐time equivalents, and served fewer uninsured patients.
In addition, we were limited by the elements captured in our PCMH capability score, which may not have captured all of the elements related to enhanced patient experience in safety net clinics. This may have led to smaller, nonsignificant effect sizes. Future studies, using composite measures of both objectively measured (e.g., full‐time equivalent clinical support or administrative staff) and survey‐measured (e.g., communication, willingness to change) clinic characteristics, may help to elucidate the extent to which different dimensions of PCMH capability are most closely associated with patient experience. Finally, the item used to identify patients with chronic illness may have included some patients with more subacute illness, such as a prolonged respiratory illness or ankle fracture; however, the PACIC scale was originally intended to “assess the recipe of patient‐centered care” relevant across diverse patients (Glasgow et al. 2005).
As policy makers seek to support and incentivize PCMH adoption broadly across health systems, careful and iterative evaluation will be required to recognize both the strengths and limitations of current implementation models. Our study suggests that sustained, 5‐year PCMH transformation in safety net clinics may be associated with modest overall improvement in patient experience of chronic illness care. Implementation models should undergo ongoing adjustment to prioritize patient experience and optimize patient‐centered chronic illness care among vulnerable populations.
Supporting information
Appendix SA1: Author Matrix.
Appendix SA2: Differences in Baseline Characteristics between Clinics Participating and Not Participating in the Postintervention Survey.
Appendix SA3: Modified Patient Assessment of Chronic Illness Care (PACIC), Subdomains and Questions.
Appendix SA4: Provider and Staff Survey of PCMH Capability, Subscales and Questions.
Acknowledgments
Joint Acknowledgment/Disclosure Statement: This project was supported by the Commonwealth Fund (grant 20080366). E. Tung and R. Nocon were supported by Agency for Healthcare Research and Quality training grants in health services research (AHRQ T32HS000078 and AHRQ T32HS000084). M. Chin, S.M. Lee, and M. Peek were supported by the Chicago Center for Diabetes Translation Research (NIDDK P30 DK092949). M. Chin was also supported by a NIDDK Midcareer Investigator Award in Patient‐Oriented Research (NIDDK K24 DK071933). E. Tung had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. This paper was presented in part at the Conference on the Science of Dissemination and Implementation in Health, Washington, DC, December 2015.
Disclosures: The authors report no other disclosures.
Disclaimers: None.
References
- American College of Physicians . 2007. “Joint Principles of a Patient‐Centered Medical Home” [accessed October 27, 2016]. Available at https://www.acponline.org/running_practice/delivery_and_payment_models/pcmh/
- Berenson, R. A. , Hammons T., Gans D. N., Zuckerman S., Merrell K., Underwood W. S., and Williams A. F.. 2008. “A House Is Not a Home: Keeping Patients at the Center of Practice Redesign.” Health Affairs (Millwood) 27 (5): 1219–30. [DOI] [PubMed] [Google Scholar]
- Birnberg, J. M. , Drum M. L., Huang E. S., Casalino L. P., Lewis S. E., Vable A. M., Tang H., Quinn M. T., Burnet D. L., Summerfelt T., and Chin M. H.. 2011. “Development of a Safety Net Medical Home Scale for Clinics.” Journal of General Internal Medicine 26 (12): 1418–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cramm, J. M. , and Nieboer A. P.. 2012. “Factorial Validation of the Patient Assessment of Chronic Illness Care (PACIC) and PACIC Short Version (PACIC‐S) among Cardiovascular Disease Patients in the Netherlands.” Health and Quality of Life Outcomes 10: 104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doyle, C. , Lennox L., and Bell D.. 2013. “A Systematic Review of Evidence on the Links between Patient Experience and Clinical Safety and Effectiveness.” British Medical Journal Open 3 (1): e001570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Flottemesch, T. J. , Anderson L. H., Solberg L. I., Fontaine P., and Asche S. E.. 2012. “Patient‐Centered Medical Home Cost Reductions Limited to Complex Patients.” American Journal of Managed Care 18 (11): 677–86. [PubMed] [Google Scholar]
- Glasgow, R. E. , Peeples M., and Skovlund S. E.. 2008. “Where is the Patient in Diabetes Performance Measures? The Case for Including Patient‐Centered and Self‐Management Measures.” Diabetes Care 31 (5): 1046–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glasgow, R. E. , Wagner E. H., Schaefer J., Mahoney L. D., Reid R. J., and Greene S. M.. 2005. “Development and Validation of the Patient Assessment of Chronic Illness Care (PACIC).” Medical Care 43 (5): 436–44. [DOI] [PubMed] [Google Scholar]
- Heyworth, L. , Bitton A., Lipsitz S. R., Schilling T., Schiff G. D., Bates D. W., and Simon S. R.. 2014. “Patient‐Centered Medical Home Transformation with Payment Reform: Patient Experience Outcomes.” American Journal of Managed Care 20 (1): 26–33. [PubMed] [Google Scholar]
- Jackson, G. L. , Powers B. J., Chatterjee R., Bettger J. P., Kemper A. R., Hasselblad V., Dolor R. J., Irvine R. J., Heidenfelder B. L., Kendrick A. S., Gray R., and Williams J. W.. 2013. “Improving Patient Care. The Patient Centered Medical Home. A Systematic Review.” Annals of Internal Medicine 158 (3): 169–78. [DOI] [PubMed] [Google Scholar]
- Jaen, C. R. , Ferrer R. L., Miller W. L., Palmer R. F., Wood R., Davila M., Stewart E. E., Crabtree B. F., Nutting P. A., and Stange K. C.. 2010. “Patient Outcomes at 26 Months in the Patient‐Centered Medical Home National Demonstration Project.” The Annals of Family Medicine 8 (Suppl 1): S57–67; s92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jones, E. B. , and Furukawa M. F.. 2014. “Adoption and Use of Electronic Health Records among Federally Qualified Health Centers Grew Substantially during 2010‐12.” Health Affairs (Millwood) 33 (7): 1254–61. [DOI] [PubMed] [Google Scholar]
- Lewis, S. E. , Nocon R. S., Tang H., Park S. Y., Vable A. M., Casalino L. P., Huang E. S., Quinn M. T., Burnet D. L., Summerfelt W. T., Birnberg J. M., and Chin M. H.. 2012. “Patient‐Centered Medical Home Characteristics and Staff Morale in Safety Net Clinics.” Archives of Internal Medicine 172 (1): 23–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linzer, M. , Manwell L. B., Williams E. S., Bobula J. A., Brown R. L., Varkey A. B., Man B., McMurray J. E., Maguire A., Horner‐Ibler B., and Schwartz M. D.. 2009. “Working Conditions in Primary Care: Physician Reactions and Care Quality.” Annals of Internal Medicine 151 (1): 28–36, w6‐9. [DOI] [PubMed] [Google Scholar]
- Markovitz, A. R. , Alexander J. A., Lantz P. M., and Paustian M. L.. 2015. “Patient‐Centered Medical Home Implementation and Use of Preventive Services: The Role of Practice Socioeconomic Context.” Journal of the American Medical Association Intern Med 175 (4): 598–606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martin, C. M. 2007. “Chronic Disease and Illness Care: Adding Principles of Family Medicine to Address Ongoing Health System Redesign.” Canadian Family Physician 53 (12): 2086–91. [PMC free article] [PubMed] [Google Scholar]
- Martsolf, G. R. , Alexander J. A., Shi Y., Casalino L. P., Rittenhouse D. R., Scanlon D. P., and Shortell S. M.. 2012. “The Patient‐Centered Medical Home and Patient Experience.” Health Services Research 47 (6): 2273–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McFarland, S. M. , Wallace J. P., Parra J., and Baker J.. 2014. “Evaluation of Patient Satisfaction with Diabetes Management Provided by Clinical Pharmacists in the Patient‐Centered Medical Home.” Patient 7 (1): 115–21. [DOI] [PubMed] [Google Scholar]
- Nelson, K. M. , Helfrich C., Sun H., Hebert P. L., Liu C. F., Dolan E., Taylor L., Wong E., Maynard C., Hernandez S. E., Sanders W., Randall I., Curtis I., Schectman G., Stark R., and Fihn S. D.. 2014. “Implementation of the Patient‐Centered Medical Home in the Veterans Health Administration: Associations with Patient Satisfaction, Quality of Care, Staff Burnout, and Hospital and Emergency Department Use.” Journal of the American Medical Association Intern Med 174 (8): 1350–8. [DOI] [PubMed] [Google Scholar]
- Nocon, R. S. , Sharma R., Birnberg J. M., Ngo‐Metzger Q., Lee S. M., and Chin M. H.. 2012. “Association between Patient‐Centered Medical Home Rating and Operating Cost at Federally Funded Health Centers.” Journal of the American Medical Association 308 (1): 60–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nocon, R. S. , Gao Y., Gunter K. E., Jin J., Casalino L. P., Quinn M. T., Derrett S., Summerfelt W. T., Huang E. S., Lee S. M., and Chin M. H.. 2014. “Associations between Medical Home Characteristics and Support for Patient Activation in the Safety Net: Understanding Differences by Race, Ethnicity, and Health Status.” Medical Care 52 (11 Suppl 4): S48–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nutting, P. A. , Miller W. L., Crabtree B. F., Jaen C. R., Stewart E. E., and Stange K. C.. 2009. “Initial Lessons from the First National Demonstration Project on Practice Transformation to a Patient‐Centered Medical Home.” The Annals of Family Medicine 7 (3): 254–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Peikes, D. , Zutshi A., Genevro J. L., Parchman M. L., and Meyers D. S.. 2012. “Early Evaluations of the Medical Home: Building on a Promising Start.” American Journal of Managed Care 18 (2): 105–16. [PubMed] [Google Scholar]
- Pourat, N. , Lavarreda S. A., and Snyder S.. 2013. “Patient‐Centered Medical Homes Improve Care for Adults with Chronic Conditions.” Policy Brief UCLA Center for Health Policy Research Pb2013‐3: 1–8. [PubMed] [Google Scholar]
- Reid, R. J. , Fishman P. A., Yu O., Ross T. R., Tufano J. T., Soman M. P., and Larson E. B.. 2009. “Patient‐Centered Medical Home Demonstration: A Prospective, Quasi‐Experimental, before and after Evaluation.” American Journal of Managed Care 15 (9): e71–87. [PubMed] [Google Scholar]
- Reid, R. J. , Coleman K., Johnson E. A., Fishman P. A., Hsu C., Soman M. P., Trescott C. E., Erikson M., and Larson E. B.. 2010. “The Group Health Medical Home at Year Two: Cost Savings, Higher Patient Satisfaction, and Less Burnout for Providers.” Health Affairs (Millwood) 29 (5): 835–43. [DOI] [PubMed] [Google Scholar]
- Safran, D. G. 2003. “Defining the Future of Primary Care: What Can We Learn from Patients?” Annals of Internal Medicine 138 (3): 248–55. [DOI] [PubMed] [Google Scholar]
- Schillinger, D. , Handley M., Wang F., and Hammer H.. 2009. “Effects of Self‐Management Support on Structure, Process, and Outcomes among Vulnerable Patients with Diabetes: A Three‐Arm Practical Clinical Trial.” Diabetes Care 32 (4): 559–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schmittdiel, J. , Mosen D. M., Glasgow R. E., Hibbard J., Remmers C., and Bellows J.. 2008. “Patient Assessment of Chronic Illness Care (PACIC) and Improved Patient‐Centered Outcomes for Chronic Conditions.” Journal of General Internal Medicine 23 (1): 77–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sidorov, J. E. 2008. “The Patient‐Centered Medical Home for Chronic Illness: Is It Ready for Prime Time?” Health Affairs (Millwood) 27 (5): 1231–4. [DOI] [PubMed] [Google Scholar]
- Sugarman, J. R. , Phillips K. E., Wagner E. H., Coleman K., and Abrams M. K.. 2014. “The Safety Net Medical Home Initiative: Transforming Care for Vulnerable Populations.” Medical Care 52 (11 Suppl 4): S1–10. [DOI] [PubMed] [Google Scholar]
- Wolf, J. A. , Niederhauser V., Marshburn D., and LaVela S. L.. 2014. “Defining Patient Experience.” Patient Experience Journal 1 (1): 7–19. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix SA1: Author Matrix.
Appendix SA2: Differences in Baseline Characteristics between Clinics Participating and Not Participating in the Postintervention Survey.
Appendix SA3: Modified Patient Assessment of Chronic Illness Care (PACIC), Subdomains and Questions.
Appendix SA4: Provider and Staff Survey of PCMH Capability, Subscales and Questions.