Patient-Centered Care in Small Primary Care Practices in New York City: Recognition Versus Reality

Abstract

Background: The Primary Care Information Project (PCIP) is a program administered by the New York City Department of Health and Mental Hygiene to help primary care providers adopt a fully functional electronic health record (EHR) and focus on population health. PCIP also offers practices assistance with the National Committee for Quality Assurance (NCQA) patient-centered medical home (PCMH) recognition application. The objectives of this study were to assess the presence of key dimensions of PCMH among PCIP practices with 5 or fewer providers and to determine whether and to what extent NCQA recognition was related to the presence of these dimensions. Methods: Analyses relied on data collected from a comprehensive practice assessment survey of PCIP practices administered in summer 2012. The survey was developed to assess discrete dimensions of the PCMH model and other practice characteristics. The study population includes practices for which survey results were available among PCIP practices with 5 or fewer providers (63% response rate; n = 83). Results: At the time of survey, 57% of practices had received some level of NCQA recognition (n = 47). Practices with recognition scored significantly higher on several dimensions, including whole person orientation, team-based care, care coordination and integration, and quality and safety. Conclusions: Results indicate that very small urban practices in New York City are implementing many key features of PCMH. In general, practices with NCQA recognition scored higher on PCMH constructs and domains relative to practices without recognition; however, there is room for improvement on construct and domain scores in both groups.

Background

The patient-centered medical home (PCMH) has become a popular model for providing primary care in the United States. In January 2008, the National Committee for Quality Assurance (NCQA) developed its original set of standards intended to measure patient-centeredness in primary care settings and began formally certifying practices as levels 1, 2, or 3. Although NCQA recognition is only one among many possible PCMH certifications, it has become the most widely accepted and sought after. In 2014, more than 10% of primary care practices in the United States, nearly 7,000 practices, were PCMH certified by NCQA.¹ Despite modest and mixed evidence of its success² and low adoption rates among even the most innovative medical practices,³ PCMH recognition has been promoted by policy makers, private foundations, physician groups, and payers as being synonymous with improving patient experience and outcomes, practice efficiency, and quality of care, and thus decreasing utilization and cost.

Financial incentive programs (typically per-member per-month) have been established to reward practices with PCMH recognition through NCQA. Practices with a level 3 designation have achieved the highest NCQA standard and therefore receive the highest level of incentives. The process of acquiring and maintaining even a level 1 certification, the lowest level of NCQA recognition, is onerous in terms of time and resources and providers often utilize costly external consultants to manage the application process.⁴ Limited resources, such as small staff and financial constraints, make it especially challenging for small practices to generate the documentation needed to acquire and maintain NCQA recognition.^5,6 Nonetheless, small practices cannot be ignored when it comes to primary care innovation and policies. In 2010, 68% of all office visits occurred at practices with 5 or fewer providers.⁷ In light of the recent surge in public interest and substantial funding supporting PCMH recognition,⁸ it is important to consider how small practices are faring relative to their larger counterparts. In this article, we examine the extent to which PCMH characteristics have been implemented in small New York City practices and explore the relationship between PCMH implementation and NCQA recognition.

This study was conducted by researchers from New York University (NYU) in collaboration with the Primary Care Information Project (PCIP), a bureau of the New York City Department of Health and Mental Hygiene (NYC DOHMH). Data from a survey of lead providers describe the practices in terms of patient, provider, and practice characteristics and self-reported aspects of the PCMH model present in the practices, as well as information about formal PCMH recognition through NCQA, including the reasons supporting providers’ decisions to apply for recognition or not.

Methods

Study Design and Sample Selection

Details of the study methodology have previously been published,⁹ and the research supporting this study was approved as exempt by the New York University School of Medicine Institutional Review Board. All practices in this study were selected via PCIP, a program established in 2005 to assist primary care practices in New York City with implementation and use of electronic health records (EHRs). PCIP’s mission is to improve population health and the quality of clinical preventive services and, to this end, participating practices were given heavily subsidized EHR software licenses as well as training and consultation on quality improvement processes (QI visits). Practices could also opt to apply for NCQA recognition through PCIP’s multisite application that streamlined some of the documentation required to meet minimum PCMH standards.

Primary care practices involved with PCIP were eligible for inclusion in this study if they had 5 or fewer clinicians, were serving a predominantly adult population, had used the EHR system provided through PCIP (eClinicalWorks) for at least 1 year as of October 2009, and received at least 2 QI visits from PCIP staff prior to October 2009 (or, for a small number of practices, were determined by PCIP staff to have advanced processes or capabilities in place that obviated the need for QI visits). The study team, with input from PCIP staff, determined that these criteria would limit the sample to those practices most likely to be implementing (or moving toward implementing) the PCMH model, regardless of NCQA status, in that they had the working capacity to monitor and track patients with the EHR, had been trained by a QI specialist on workflow design, had established practice processes and policies for care management, had been aware of or already using quality measurement, and had been able to conduct continuous clinical QI activities. We excluded specialty practices from the sample and eliminated practices that closed or merged during the study period.

The practice assessment survey was sent to the final sample of 131 practices. The survey was administered between June and September of 2012 using online survey software (Qualtrics) followed by paper copies for providers who did not complete the survey online after 3 reminders.¹⁰ This study focuses on the 83 practices whose sole primary care provider or a lead provider completed a survey (63% response rate). We assessed nonresponse bias by comparing respondents to the larger sample of practices invited to participate using data available from the NYC DOHMH in terms of practice, patient and PCIP participation characteristics. The 2 groups were quite similar suggesting participating practices were an unbiased subset of all eligible practices.

Measures and Analysis

The practice assessment survey was developed to measure 6 of the 7 principles underlying the PCMH model, namely: personal physician for each patient, whole person orientation, team-based care, coordination and integration of care across the health system, a focus on quality and safety, and timely access to care and communication. The seventh principle, payment reform, was not measured because individual practices have little control over health care financing mechanisms. Survey items were adapted from previously validated survey instruments related to primary care, chronic disease models, and PCMH, with items rephrased to be applicable to a small practice environment or items were developed de novo. In addition to PCMH dimensions, the survey included questions about other practice characteristics, including experiences prior to PCIP, practice finances, and details of NCQA application and award status. We pilot tested the survey for clarity and comprehensiveness among non-eligible PCIP providers working in small practices.

The survey was scored in accordance with the instrument structure: responses to multiple questions, mostly Likert-type–scaled, were aggregated to measure individual constructs, and multiple constructs in turn were combined to represent individual PCMH principles. All scores were standardized from 0 (low) to 1 (high) to facilitate comparisons across measures. Wilcoxon rank-sum tests were used to detect statistically significant differences. Effect size was examined by calculating the difference in mean scores and the standard deviations of the mean differences between practices with and without NCQA recognition. STATA/SE version 12.0 was used to perform all analyses.

Results

Sample Characteristics and PCMH Recognition

Practices in our sample with PCMH recognition were not significantly different than practices without recognition in terms of practice, provider, or patient population characteristics. More than half (n = 47) of the practices in our sample had received some level of NCQA recognition at the time of survey. Among practices with PCMH certification, 68% had level 1, 2% had level 2, and 30% had level 3. Table 1 presents provider-reported reasons for applying or not applying for recognition. A majority of practices with NCQA recognition reported that they applied because they were motivated to improve patient care (81%) and to increase practice revenue (81%). Reasons for not applying for recognition were more varied, but the most common was that providers did not know enough about PCMH to warrant the effort of applying (42%). Only 6% of practices without recognition reported that they planned to apply in the future.

Table 1.

Reasons for and Against Applying for NQCA Recognition.

	Percenta
Main motivations for applying among those with NCQA recognition (n = 47)
To improve patient care	81
To increase practice revenue	81
To improve patient experience or satisfaction	68
To improve provider and staff morale	40
Because the QI Specialist from DOH encouraged the practice to apply	30
Other/Don’t know	2
Reasons for not applying among those without NCQA recognition (n = 36)
Do not know much about PCMH	42
Other/Don’t know/No response	28
Already sufficiently patient centered and do not need the recognition	8
Would like to apply but do not have time to complete the application	8
Recognition is not worth the effort/cost of becoming a PCMH	6
Planning to apply	6
Not enough substance to the PCMH concept for the practice to apply	2

Abbreviations: NCQA, National Committee for Quality Assurance; QI, quality improvement; DOH, Department of Health; PCMH, patient-centered medical home.

^aProviders were asked to check all that apply; percentages may exceed 100% in total.

PCMH Dimensions

Table 2 shows the average standardized construct and principle scores for the full sample, scores stratified by NCQA recognition status, and differences in mean and standard deviation across groups. Full sample scores indicate that the small practices in our sample are implementing the key principles of PCMH to varying degrees, but that there is room for improvement on all measures. Overall, practices scored highest on “patient has a personal physician” and lowest on team-based care and quality and safety. The stratified analyses show that practices that have received NCQA recognition scored higher than those that have not on 4 of 6 PCMH principles.

Table 2.

Standardized PCMH Dimension Scores by PCMH Recognition Status.

	Mean Score (SD)			Between-Group Differences (NCQA − No NCQA)		Significance (Mean Scores)
	Full Sample (n = 83)	No NCQA Recognition (n = 36)	NCQA Recognition (n = 47)	Mean	SD of the Mean Differencea
1. Patient has a personal physician	0.77 (0.23)	0.80 (0.22)	0.74 (0.23)	0.06	–0.35	.29
Patient sees same provider/team	0.78 (0.32)	0.82 (0.29)	0.75 (0.33)	0.07	–0.28	.30
Patient can identify provider/team	0.75 (0.31)	0.78 (0.35)	0.72 (0.27)	0.06	–0.29	.19
2. Whole person orientation	0.54 (0.19)	0.48 (0.21)	0.59 (0.17)b	0.11	0.86	.02
Concern about nonmedical issues	0.54 (0.24)	0.50 (0.24)	0.57 (0.23)	0.07	0.40	.12
Engagement of patient’s family	0.49 (0.26)	0.48 (0.24)	0.50 (0.27)	0.02	0.10	.76
Availability of services on site	0.45 (0.35)	0.42 (0.39)	0.47 (0.32)	0.05	0.21	.45
Cultural competence	0.71 (0.30)	0.67 (0.33)	0.75 (0.28)	0.08	0.38	.24
Formal culture/communication training	0.34 (0.37)	0.22 (0.30)	0.44 (0.38)c	0.22	0.77	.01
3. Team-based care	0.44 (0.20)	0.39 (0.21)	0.48 (0.18)b	0.09	0.66	.05
Presence of internal teams	0.46 (0.33)	0.35 (0.33)	0.55 (0.30)c	0.20	0.88	<.00
Team communication	0.31 (0.30)	0.31 (0.31)	0.30 (0.29)	0.01	–0.04	.99
Interact with providers outside practice	0.43 (0.25)	0.40 (0.22)	0.46 (0.28)	0.06	0.28	.34
Top of skill set	0.40 (0.34)	0.37 (0.36)	0.42 (0.31)	0.05	0.21	.41
Regular staff meetings	0.48 (0.31)	0.45 (0.33)	0.51 (0.29)	0.06	0.27	.32
4. Care coordination, integration	0.55 (0.23)	0.47 (0.22)	0.61 (0.23)c	0.14	0.81	.01
Performs care coordination functions	0.75 (0.25)	0.67 (0.29)	0.80 (0.22)b	0.13	0.78	.02
Formal/informal care/case manager	0.36 (0.31)	0.31 (0.27)	0.39 (0.33)	0.08	0.32	.25
Use of patient registries/HIT	0.69 (0.26)	0.59 (0.27)	0.76 (0.24)c	0.17	0.94	<.00
Communication with hospitals	0.55 (0.23)	0.51 (0.24)	0.58 (0.22)	0.07	0.44	.22
Communicating with specialists	0.66 (0.21)	0.62 (0.20)	0.70 (0.21)	0.08	0.33	.06
Systematic appointment reminders	0.73 (0.32)	0.70 (0.34)	0.76 (0.32)	0.06	0.25	.40
Knowledge about, relationships with other community service providers	0.54 (0.24)	0.51 (0.22)	0.56 (0.24)	0.05	0.28	.29
5. Quality and safety	0.42 (0.20)	0.36 (0.20)	0.46 (0.19)b	0.10	0.70	.02
Evidence-based decision supports	0.33 (0.14)	0.31 (0.13)	0.34 (0.15)	0.03	0.27	.07
Use of EHR functions	0.73 (0.24)	0.66 (0.23)	0.79 (0.23)c	0.13	0.75	<.00
Use of performance feedback	0.59 (0.32)	0.60 (0.29)	0.59 (0.35)	0.01	–0.04	.81
Quality improvement (QI) activities	0.56 (0.30)	0.49 (0.30)	0.61 (0.30)	0.12	0.66	.07
Patient education	0.55 (0.25)	0.56 (0.26)	0.54 (0.24)	0.02	–0.11	.70
Use of patient feedback	0.31 (0.43)	0.24 (0.39)	0.36 (0.45)	0.12	0.35	.20
6. Access	0.51 (0.24)	0.46 (0.25)	0.54 (0.24)	0.08	0.44	.11
After hours care	0.51 (0.25)	0.50 (0.27)	0.52 (0.23)	0.02	0.12	.89
Open/advanced access	0.66 (0.48)	0.58 (0.50)	0.72 (0.45)	0.14	0.41	.18
Ease of access	0.53 (0.21)	0.57 (0.23)	0.51 (0.20)	0.06	–0.40	.09
Use of patient portal/email	0.45 (0.38)	0.35 (0.39)	0.52 (0.36)b	0.17	0.63	.03

Abbreviations: PCMH, patient-centered medical home; NCQA, National Committee for Quality Assurance; HIT, health information technology; EHR, electronic health record.

^aCalculated using Hedges’s g.

^bStatistically different than practices without recognition; Wilcoxon rank-sum test; P < .05.

^cStatistically different than practices without recognition; Wilcoxon rank-sum test; P < .01.

Practices with NCQA recognition achieved higher scores on whole person orientation than those with no recognition. This difference was primarily driven by having had more formal staff training on issues surrounding cultural competency and communication. Practices with recognition also scored higher on team-based care, with the largest difference between the 2 groups being in the dimension of care coordination: Practices with NCQA recognition reported higher levels of having someone who performs care coordination functions within their practice and use of patient registries or other health information technology. Although it did not quite reach statistical significance, NCQA recognized practices reported more communication with specialists than practices without recognition. Finally, the quality and safety dimension was higher for NCQA recognized than non-NCQA recognized practices. Differences in quality and safety were largely due to higher reported use of electronic health record functionality among practices with recognition. Two other constructs that comprised this principle—the use of evidence-based decision supports and engaging in QI activities—approached statistical significance. There were no statistically significant differences between the 2 groups in terms of patients having a personal physician or measures of access. However, interestingly, the personal physician domain is the only principle in which practices without NCQA recognition scored higher than practices with recognition. Mean differences range from −0.07 to 0.22, which is equivalent to a range of −0.40 to 0.94 in standard deviation units.

Conclusions

Results of this study indicate that very small urban practices in New York City are implementing many key features of PCMH to some degree, whether or not they have NCQA recognition. In general, practices with NCQA recognition scored higher on construct and principles relative to practices without recognition; however, there is room for improvement on construct and principle scores in both groups. It is also noteworthy that a substantial number of practices without formal PCMH recognition exhibit many key features of PCMH despite not receiving financial incentives to do so.

This study has several limitations. All of the small practices in our sample received assistance with implementing an EHR system and had the opportunity to benefit from PCIP assistance with the NCQA level 1 application. It is therefore not surprising that these practices have higher rates of recognition than is typical—47% in our sample compared with just over 10% nationally.¹ This study was not an evaluation of the PCIP program and we did not use a comparison group, therefore we cannot infer that the findings of this study extend beyond the sample of practices involved in PCIP. We did not have sufficient sample sizes to examine PCMH principles by NCQA certification level and so all our analyses treat level 1 the same as level 3. Finally, although not unique to this study, results rely exclusively on self-reported data collected through the practice assessment survey and therefore findings may over- or underestimate the extent to which practices have established PCMH characteristics.

In spite of these limitations, the results of this study have important policy implications. The proportion of practices in our sample that had achieved NCQA recognition is likely a best-case scenario for small practices given the application assistance that they received from PCIP. Continuing and expanding studies on small practices’ ability to achieve recognition is important given the populations they serve. Even still, a substantial proportion of practices in our sample had no intention of applying for NCQA recognition. Furthermore, there is some evidence to suggest that practices receiving assistance with their original application may still choose to not achieve NCQA recognition in subsequent certification cycles.¹¹ More research into what can be done to motivate and assist small practices to apply for NCQA recognition is warranted given that NCQA recognition was positively and statistically significantly associated with many PCMH principles and constructs in our study. More research is also needed to examine the association between implementation of PCMH constructs and patient health outcomes.

Author Biographies

Margaret M. Paul, MS is an associate research scientist in the Department of Population Health at New York University School of Medicine where she is part of team of researchers responsible for evaluating a variety of policy-relevant health care demonstration projects both locally in New York City and nationally. She is a doctoral candidate at NYU with an MS in bioethics, policy and law from Arizona State University and more than 7 years of research experience with mixed methods studies.

Stephanie L. Albert, MPA is a doctoral candidate in Public Health at UCLA. Her professional experience and academic training are in the areas of intervention development, implementation and evaluation, obesity prevention, health disparities, and community engagement. Before seeking a doctoral degree she directed research and evaluation projects related to health and health care for low-income and underserved populations.

Tod Mijanovich, PhD is a research associate professor of applied statistics and health policy in the Department of Applied Statistics, Social Science, and Humanities. He works primarily on evaluations of medical care management interventions for high-risk, high-utilizing populations.

Sarah C. Shih, MPH is the assistant commissioner for the Primary Care Information Project, a Bureau within the Division of Prevention and Primary Care in the New York City Department of Health and Mental Hygiene. She leads the bureau in initiatives in New York City to engage primary care settings in using health information technology to improve the delivery of clinical preventive services. She is responsible for overseeing various programs in the use of health information systems for transforming primary care and has also led and facilitated several program evaluation studies.

Carolyn A. Berry, PhD is a community psychologist who has been evaluating policies and policy relevant programs in public health, health care, and social services for over 20 years. Recently she has been deeply involved in studying and evaluating the Patient Centered Medical Home (PCMH) model and practice facilitation to enhance guideline adoption in small primary care practices in New York City, in a large urban hospital system serving primarily indigent and Medicaid recipients, and nationally in an initiative designed to reduce racial/ethnic disparities at the practice site.