Performance of the Person Centered Primary Care Measure in Pediatric Continuity Clinic

Abstract

Objective:

Improvement efforts in pediatric primary care would benefit from measures that capture families’ holistic experience of the practice. We sought to assess the reliability and validity of the new Person-Centered Primary Care Measure (PCPCM) in a pediatric resident continuity clinic serving low-income families.

Methods:

We incorporated the 11-item PCPCM, stems adapted to reflect a parent responding about their child’s visit, into a telephone survey of 194 parents presenting for care in October 2019 at a pediatric resident continuity clinic in Cleveland Ohio (64% response rate). We evaluated PCPCM items using factor analysis and Rasch modeling, and assessed associations of the PCPCM with parents’ demographics and perceptions of specific elements of their child’s care.

Results:

In this sample of low-income families, the PCPCM had good reliability (Cronbach’s alpha 0.85). All items loaded onto a single factor in principal axes factor analysis. Of the 11 aspects of primary care represented in the scale, “shared experience” was most difficult for parents to endorse in Rasch modeling. All 11 items contributed significantly to the total scale score with corrected item-total correlations >0.4. The PCPCM score was independent of socio demographics and was associated with parent’s report that their child’s clinician spends enough time with them.

Conclusions:

The PCPCM performs well in a pediatric continuity clinic setting, warranting consideration for its use as a parsimonious parent-reported measure of what patients and clinicians say matters most in pediatric primary care.

Introduction

Health systems and payers are increasingly recognizing the importance of primary care’s role in coordinating and prioritizing care that considers patients and families as a whole,^1,2 by investing in practice improvement through provision of financial incentives tied to measurement of practice performance.^3,4 In the decade since codification of patient-centered medical home as a core model for primary care delivery by the ACA,^5,6 measurement of pediatric practice performance has largely relied on the piecemeal assessment of individual components of care or delivery of guideline-recommended services for specific conditions⁷ — for example, rates at which practices administer particular vaccines, screen for lead exposure among toddlers or depression among teens, assess asthma control, or discuss weight status as part of a well-child visit.^8-10 This approach may reduce the impetus to reflect more holistically on the experiences of individuals and families. A more holistic approach could seek to assess to extent to which primary care practices integrate a patient’s medical, behavioral-mental health, and social needs into a coherent plan of care.^11,12

Etz and colleagues used a multistep approach to develop a concise and parsimonious patient-reported measure for such use. ¹³ They began by engaging hundreds of patients, clinicians and payers iteratively to identify and refine a set of items assessing the processes and functions that comprise high-value primary care while emphasizing the patient perspective.¹³ The resulting instrument, the Person-Centered Primary Care Measure (PCPCM),¹⁴ assesses each of 11 aspects of high quality primary care (accessibility, advocacy, community context, comprehensiveness, coordination, family context, goal-oriented care, health promotion, integration, relationship, and shared experience) with a single item (Table 1).

Table 1.

Primary Care Domains, Functions, and Relationship to Principles of Patient Centered Medical Home (PCMH)^a

Primary Care Domain	Primary Care Function	PCMH Principle
Accessibility	Accessibility to the health care system	Enhanced access
Advocacy	Sustained partnership and relationship	Quality and Safety
Community Context	Family and community context	Neighborhood
Comprehensiveness	Accountability in addressing health care needs	Whole person orientation
Coordination	Coordination of care	Physician-directed team practice
		Coordination of care
Family Context	Comprehensiveness, Integration of Care	Whole person orientation
Goal-oriented Care	Accountability in addressing health care needs	Payment for added value
Health Promotion	Accountability in addressing health care needs	Whole person orientation
Integration	Integration of care	Integration of care
Relationship	Sustained partnership and relationship	Personal Physician
Shared Experience	Sustained partnership and relationship	Whole person orientation

^aAs adapted from Stange, KC, Nutting, PA, Miller, WL, Jaen, CR, Crabtree, BF, Flocke, SA, & Gill, J.M. (2010). Defining and Measuring the Patient-Centered Medical Home. J Gen Intern Med 25(6):601–12 DOI: 10.1007/s11606-010-1291-3.

The PCPCM demonstrated reliability and concurrent validity within a primarily commercially-insured adult population recruited from two internet samples and four clinical samples.¹⁴ While a few pediatric patients participated in the validation, the design of the initial validation study did not permit specific assessment of the measure’s performance in the pediatric population. It also did not distinguish amongst respondents (that is, parent responding on behalf of their child vs. child responding about their own experience). This paper seeks to advance the science supporting use of this metric for pediatric care by modifying lightly its stems to explicitly accommodate parents as proxy-reporters regarding their child’s care. We report reliability and concurrent validity analyses of the adapted PCPCM in a pediatric resident continuity clinic serving low-income families.

Methods

Study Participants and Recruitment

We embedded a modified version of the PCPCM into the 2019 Local Inventory of Needs and Knowledge (LINK) Survey. The LINK Survey is an annual intermittent cross-sectional telephone survey designed to generate a practice-level snapshot of patient-reported needs and experiences at the University Hospitals Rainbow Pediatric Practice, a pediatric resident continuity clinic in Cleveland Ohio. More than 85% of patients seen in the practice have Medicaid as their primary insurer, and approximately 90% identify as African American. The practice is staffed by 11 general pediatrics and adolescent medicine faculty, 3 advanced practice pediatric nurse practitioners, and approximately 90 resident trainees in categorical pediatrics, medicine-pediatrics, pediatric neurology, and pediatric genetics resident trainees. The practice has robust on-site ancillary and wrap around services, including integrated behavioral health support, medical-legal partnership, and social needs navigation.

LINK participants were defined as the adult (age 18 years or older) legal guardian of a child presenting to the clinic for a scheduled visit (hereafter referred to as “parents”). LINK participants were recruited in-person from a clinic visit over a two-week period in October 2019. Sampling for recruitment was structured to generate a representative cross-section of visits, accounting for patient age group (infant/toddler vs preschool vs school-aged vs adolescent), visit type (well-child vs. sick visit), visit mode (individual vs group), clinician type (attending physician vs. advanced practice nurse vs. resident trainee), day of the week, and time of day (morning vs afternoon). When there was more than one potential index child present (e.g., siblings scheduled together), we selected that child whose birthday was upcoming next to serve as the index child for the interview. In the case of multiples (e.g., twins, triplets), we asked the parent to select one child to serve as the index child. During the in-person recruitment interaction, we completed all informed consent procedures, and identified with the participant their preferred day of the week and time of the day to complete the interview, their preferred and alternate telephone contact numbers to reach them, and determined whether we had permission to send reminder text messages.

We completed the structured telephone interview with LINK participants within two weeks of the index visit. Interviews were conducted in English and lasted on average 25 minutes. Participants who completed the survey received a $10 gift card as a token of appreciation for their time. In all, 82 percent of potential participants agreed to enroll in the study, of whom 78 percent completed the telephone interview (overall response rate 64%). All procedures were reviewed and approved by the University Hospitals Cleveland Medical Center Institutional Review Board.

LINK Survey Instrument

The LINK survey assesses the following domains in the following order: (1) parent and child overall health status, top health concerns, and unmet needs for healthcare (e.g., healthcare visits, dental care, pharmacy, durable medical equipment and supplies, insurance coverage); (2) household experience of social determinants of health and related stressors (e.g., housing condition and stability, experience of food insecurity, access to government benefits and legal assistance, needs for childcare and recreational opportunities); (3) experiences with the practice (e.g., interactions with staff, awareness and use of with practice-based programs and services); (4) interactions with their child’s clinician (e.g., the extent to which the clinician listens carefully, spends enough time, PCPCM), and (5) child and family demographics (e.g., parent and child age and gender, parent relationship to child, household size and composition, parent’s education and employment status). The 11 items that comprise the PCPCM were asked together as a block at the end of the section addressing interactions with their child’s clinician.

Most LINK survey items were asked on either a 4-item or 5-item Likert scale. The answer set for the PCPCM was unchanged from the original four point scale of “not at all” to “definitely”. The adaptation of the PCPCM stems for parents to answer as proxies for their children (see Table 2), was based on feedback from 9 key informants (6 parents, 3 staff members from the practice) who participated in cognitive interviews as part of 2019 annual LINK survey development and pre-testing.

Table 2.

Modified PCPCM Items and Statistics Among Pediatric Continuity Clinic Parents (Data source: LINK 2019)

Item Stem	Primary Care Domain	Mean (SD)	Fact or Loading	Item- Total Correlations
This practice makes it easy for us to get care (mean (SD))	Accessibility	3.81 (0.58)	0.63	0.55
This practice is able to provide most of [child’s name]’s care (mean (SD))	Comprehensiveness	3.88 (0.44)	0.66	0.59
In caring for [child’s name], my child’s doctor considers all of the factors that affect their health (mean (SD))	Integration	3.88 (0.48)	0.61	0.54
This practice coordinates the care [child’s name] gets from multiple places (mean (SD))	Coordination	3.77 (0.71)	0.45	0.48
This doctor or practice knows [child’s name] as a person (mean (SD))	Relationship	3.47 (1.01)	0.66	0.67
[Child’s name]’s doctor and I have been through a lot together (mean (SD))	Shared experiences	2.58 (1.34)	0.50	0.50
My child’s doctor or practice stands up for my family (mean (SD))	Advocacy	3.31 (1.15)	0.72	0.74
The care [child’s name] gets takes into account knowledge of our family (mean (SD))	Family context	3.75 (0.71)	0.64	0.62
The care [child’s name] gets takes into account knowledge of our community (mean (SD))	Community Context	3.04 (1.26)	0.47	0.52
Over time, this practice helps [child’s name] and me to meet our goals (mean (SD))	Goal Oriented	3.68 (0.75)	0.73	0.66
Over time, this practice helps [child’s name] to stay healthy (mean (SD))	Health Promotion	3.81 (0.55)	0.60	0.52

Analyses

We used SPSS version 26 to generate descriptive statistics characterizing the study sample and to summarize responses to each of the 11 adapted PCPCM items. Consistent with the original validation of the instrument, individuals who answered at least eight PCPCM items were eligible for calculation of the PCPCM scale score. The scale score is defined as the mean across the completed items. We then conducted the following sequence of analyses to determine the extent of evidence of reliability and validity in this pediatric context:

Factor analysis of the 11-item correlation matrix assessed the number of factors on which these items loaded optimally. We used principal axes factor analysis, which generates three types of evidence regarding the number of constructs within the measure: 1) Eigen values (which are used to determine dimensionality) greater than 1.0 indicate a viable factor. If only one value is greater than 1.0 that is evidence of a single factor; 2) A scree plot of the Eigen values by factor allows visual respresentation to assess qualitatively whether there exists a large gap between the first and second Eigen values, which would provide additional evidence for a single factor; and 3) we reviewed the factor loadings, which represent the correlation of each item with the underlying factor. Generally, a factor loading greater than 0.4 is an indication that the item is contributing to the definition of the factor.^15,16 We interpreted factor loadings of >0.4 as evidence for good association, >0.6 as strong association, and >0.8 as very strong association between item and construct.

These steps suggested that a single factor was optimal. To assess internal reliability of the PCPCM (how consistent the results are across individual items in the measure), we then computed Cronbach’s alpha internal consistency reliability coefficient, which assesses the extent to which each item contributed to the scale’s reliability as well as the extent to which each item contributed to the total score. A second measure of internal reliability, the intra class correlation, was also computed and compared to Cronbach’s alpha.

We then used WINSTEPS 4.5.5 software to calculate Rasch item fit statistics. Closely related to the one-parameter Item Response Theory (IRT) model, Rasch item analysis is best when one plans to pool responses to items that involve only a single construct and to further corroborate that each item contributes to the single construct. A Rasch analysis of the PCPCM items provided goodness of fit statistics for each of the 11 items in the scale. When all items meet the Rasch criteria for goodness of fit (0.5-1.5), this is considered initial empirical evidence of the scale’s construct validity.^17,18 Rasch analysis further indicates an ordered likelihood for positive responses, referred to as item difficulty. Variability in item difficulty indicates that the scale can discriminate among responding populations. We interpreted Rasch item reliability >0.9 as evidence for excellent reproducibility of the item rank ordering.

Finally, to assess concurrent validity (how well the PCPCM compares to other parent-reported measures within the LINK survey), we explored whether there was evidence for an association among the PCPCM scale score and various participant characteristics and experiences. We used t-tests for binary variables and Analysis of Variance (ANOVA) for multi-categorical variables.

Results

Participant characteristics

A total of 194 parents and guardians participated in 2019 LINK, all of whom answered 8 or more of the adapted PCPCM items and were therefore eligible for PCPCM scoring. The vast majority were female (17 males, 9% of the sample), with a median age 29 years (IQR 25-34). Nearly all (96%) were the child’s biological parent. Highest education attainment for 47% was high school diploma or equivalent, with 14% not completing high school, and 39% completing at least some college. The median number of children in the household was 2 (IQR 1-3.75).

The majority of children (66%) were under the age of 5 years, with equal distribution among males and females. While nearly half of children were characterized by their parent as currently being in excellent overall health and only one third were characterized as being in excellent mental health. In terms of their experiences of care at the index visit, most parents (83%) reported that their child’s doctor or nurse “definitely” spent enough time with them. Despite all children having some form of insurance at the time of the interview (85% public), nearly one-third parents (32%) reported one or more unmet needs for healthcare for their child including medical visits, medications, durable medical equipment or supplies, dental care, and/or insurance coverage.

PCPCM Items

Of the 194 respondents, 62 rated all 11 PCPCM items “4” (“Definitely). Thus, individual item responses and their corresponding scale scores were substantially skewed toward the high end of the range, As shown in Table 2, mean responses to individual items ranged from 2.58 (shared experience) to 3.88 (comprehensiveness), with six of the eleven items averaging 3.75 or higher. The average PCPCM score was 3.55 (SD 0.53).

Psychometric Analyses

Exploratory principal axes factor analysis identified a single factor. The 11 PCPCM items together had an Eigen value of 4.73, accounting for 38% of the variance. Review of the scree plot of Eigen values visually demonstrated a large gap between first and second Eigen values. All individual item factor loadings were >0.4. Corrected item-total correlations were similarly robust (range 0.48-0.74, Table 2), with all items contributing significantly to the total scale score. Care coordination had the lowest item-total correlation at 0.48, while advocacy had the highest at 0.74. Thus, together, principal axes factor analyses provided strong evidence that the 11 items together reflect a single construct.

Overall internal reliability of the PCPCM in this population was excellent, as evidenced by Cronbach’s alpha coefficient of 0.85. A second measure of internal reliability, the intra class correlation, was computed to have a value of 0.81 and was thus comparable to that of Cronbach’s alpha.

Rasch analysis showed that even though 62 respondents rated all 11 PCPCM items “4” (“Definitely”), the remaining 68% of the sample made use of the full four-point response range for all 11 items. Figure 1 presents a Rasch person-item map of the items and scale scores. The person-side of the map (left and top axes of the figure) shows the distribution of person-level scale scores. The person-level scale scores are notably skewed towards the top of distribution, with a mean score of 1.23 (SD 0.85). This is contrast to an ideal distribution in which the person-level scale score would center around zero on the map’s left axis. The item-side of the map (right and bottom axes of the figure) shows the distribution of individual item means across the range of difficulty. There was some redundancy between items for “comprehensiveness” and “integration”. Nevertheless in total seven of 11 items had negative mean scores in Rasch analysis, indicating that it was relatively easy for parents to endorse more than half of the PCPCM items. The items that parents endorsed least often were was shared experience (“my child’s doctor and I have been through a lot together”), followed by community context (“The care [child’s name] gets takes into account knowledge of our community”).

Figure 1. Rasch Analysis Person-Item Map^a for Modified PCPCM^b Items and Scores.

^a Bars under Person heading (top axis) show distribution of scale scores. On the Person side of the map, mean = 1.23, 1 standard deviation (SD) = 0.85. Numbers in parentheses after each Item (bottom axis) show distribution of individual item means across the range of difficulty. On the Item side of the map, mean = 0.0, 1 standard deviation (SD) = 0.84.

^b PCPCM = Person Centered Primary Care Measure

We found evidence for significant association between PCPCM scale score and parent gender, with fathers on average rating PCPCM higher than mothers. We also found evidence for a significant association between PCPCM scale score and parent’s rating of the single item “This doctor or nurse spends enough time with my child”. Parents who rated this item as “definitely” true had significantly higher PCPCM scores than did parents who rated this item as “mostly” or “somewhat” true (F=10.56, p<0.0001). Otherwise, we did not find significant associations between PCPCM scale scores and other socio demographic features or parent-reported experiences of care.

Discussion

We found that the adapted PCPCM performed well in a predominantly low-income, African American population obtaining care in a pediatric resident continuity clinic. In this setting, the PCPCM demonstrated good internal reliability and psychometrics, and provided some preliminary evidence for concurrent validity. Importantly, we found no evidence for association between PCPCM score and most demographic or other features, in this specific population of parents. That is, we found that the PCPCM assessed aspects of care that are independent of parent- or child- age, parent educational attainment, household composition, primary insurer, parent relationship to child (bio-parent, adoptive parent, other legal guardian), or parents’ assessment of their child’s overall health and mental health status.

While we did find significant association among PCPCM scale score and parent gender, this finding needs to be interpreted with caution given the very few fathers or male guardians who participated in the study and the limited variability in their responses (PCPCM scale score range for males: 3.25-4). The developers of the PCPCM have not found evidence for gender differences in studies of PCPCM performance with adult populations in US or US-like settings abroad,¹⁴ though they do note some differences in response by gender in countries where gender roles are more proscribed.¹⁹

Given our survey sampling approach and robust survey response rate, we are confident that our respondents constitute a representative cross-section of parents typically seen in our pediatric resident continuity clinic. As such, the performance of individual PCPCM items may provide useful insights regarding delivery of care to this population.

The care coordination item was most problematic in terms of contribution to the overall construct, as evidenced by factor loading of 0.45 (corresponding to corrected item-total correlation of 0.48) Nevertheless, this performance was well above our threshold for significance in the overall measure. As such, we find that, while not all families may have care coordination needs (especially those with young and otherwise healthy children), these psychometric analyses suggest that, among those families for whom this domain applies, care coordination makes a meaningful contribution to their overall assessment of their experience of primary care.

The item for shared experience was the most challenging for parents to endorse. This may reflect that not all families have experienced the types of substantial health events captured by the phrase “my child’s doctor and I have been through a lot together”. Indeed, 61 parents rated this item “not at all” true, and an additional 13 did not know or declined to answer this item. Parents who endorsed shared experience as “definitely” true (n=78) were on average older (33 vs 29 years, p<0.001). However, compared to other respondents, these parents were no more likely to have older index children, were no less likely to rate their child as being in excellent health and were no more likely to report unmet needs for healthcare. We speculate that the shared experience item may also reflect the challenges to continuity and relationship-building posed by a busy academic practice that is staffed predominantly by trainees. Additional work is needed to explore the importance of this construct when assessing the care of generally healthy children. Longitudinal assessment may reveal the sensitivity of this item to other lived experiences.

The item representing clinicians’ appreciation for community context was also less common for parents to endorse, with 16 parents declining to rate this item at all. Of those that did, more than 1 in 5 (22%) reported that it was “not at all” true that the care their child receives takes into account knowledge of their community. This finding is notable, given that the practice from which parents were recruited is co-located with a number of services intended to address community-based determinants of health. This finding reminds us that clinicians need to inquire, listen, and reflect if they wish to understand their families’ lived experiences and life contexts. Assumptions and generalizations may mislead clinicians, hamper relationships,²⁰ and thus harm outcomes, even when communities appear to be relatively homogenous.

Our findings are limited by the restriction of our study population to only parents from a single practice at a single point in time. We asked PCPCM items as part of a larger survey exploring family needs and healthcare experiences, which may have influenced how parents responded to these items. It is possible that parents would respond differently to these items if asked in isolation, that is, without framing the PCPCM within the larger context of family, social, and other determinants of health. It is also possible that de novo development work specifically with parents and child-focused clinicians would have yielded a different set of items than those included here. We also cannot estimate clinic-level reliability from these data, though we do provide initial evidence of reliability for this adapted measure in our clinical setting. Replication of these results in other pediatric clinics and settings will be important to establish the measure’s generalizability. Future work is also needed to explore the sensitivity of the measure to changes in care delivery over time at both the level of individual (parent-clinician) relationships and the level of practice-wide performance.

Conclusion:

The PCPCM performed well in a pediatric resident continuity clinic setting, warranting consideration for further exploration of its use as a parsimonious parent-reported measure of what matters in primary care.

What’s New:

The Person Centered Primary Care Measure performs well in a low-income population of parents reporting what matters for their child’s primary care.