Abstract
Although most patient-clinician interactions occur in ambulatory care, little research has addressed measuring ambulatory patient safety or how primary-care redesign such as the Patient-Centered Medical Home (PCMH) addresses patient safety. Our objectives were to identify PCMH standards relevant to patient safety, construct a measure of patient safety activity implementation, and examine differences in adoptions of these activities by practice and community characteristics. Using a consensus process, we selected elements among a widely-adopted, nationally representative PCMH program representing activities that, according to a physician panel, represented patient safety overall and in four domains (diagnosis, treatment delays, medications, and communication and coordination) and generated a score for each. We then evaluated this score among 5,007 practices with the highest PCMH recognition level. Implementation of patient safety activities varied; the few military practices (2.4%) had the highest, and community clinics the lowest, patient safety score, both overall (82.0 and 72.0, respectively, p<0.001) and across specific domains. Other practice and community characteristics were not associated with the patient safety score. Understanding better what factors are associated with implementation of patient safety activities may be a key step in improving ambulatory patient safety.
Keywords: Ambulatory Care, Patient Safety, Patient-Centered Medical Home
INTRODUCTION
Patient safety in ambulatory care settings is a critical but understudied issue. A recent technical brief for the Agency for Healthcare Research and Quality found little research in ambulatory safety, especially compared to the extensive research in inpatient care.1,2 In particular, there is limited reported data cataloguing performance in key domains relevant to ambulatory safety, including missed or incorrect diagnoses, delays in proper treatment or preventive services, medication errors and preventable adverse drug events, and defects in communication, information flow, and coordination of care.3–5 Ambulatory safety is increasingly important given the greater health care system dependency on office-based care6, a trend reinforced by value-based payment initiatives and an estimated 1 billion office visits annually in the US.7 As many as 5% of US adults may experience a missed or delayed diagnosis in the outpatient setting annually,8 and research suggests that breakdowns in workflow frequently contribute to these safety issues.9 Prescribing errors are highly prevalent in ambulatory practice,10 and poor coordination of care between outpatient clinicians has been a major safety concern noted in surveys of US patients; 30% report that coordination is a major problem for issues such as clinicians not having their medical records.11 Although metrics to quantify patient safety activities in ambulatory care are critical to evaluate and improve safety, there has been little research in this area.12
As recognized in the landmark Institute of Medicine (IOM) reports, To Err is Human13 and Crossing the Quality Chasm14, systems-level solutions are critical for promoting a culture of safety and patient engagement, and well-organized primary care plays a critical role in supporting safe, reliable, integrated and accessible care. The recent growth in adoption of the Patient-Centered Medical Home (PCMH), a key model for improving primary care, may offer an opportunity to achieve the goals laid out by the IOM reports. The PCMH is designed to address the core attributes of primary health care, including comprehensive, patient-centered, coordinated, and accessible care, with high quality and safety. There are more than 150 public and commercial programs designed to incentivize PCMH adoption nationwide.15 Patient safety has not always been highlighted as an explicit component of the PCMH model. However, PCMH practices regularly offer expanded access and continuity, robust patient engagement activities, and active care coordination, all of which support safe care, and so PCMHs may represent a best-case scenario in ambulatory safety.16
The purpose of this paper is to identify PCMH standards relevant to patient safety, construct a measure of patient safety activity implementation, and examine differences in adoptions of these activities by practice and community characteristics. Based on prior work in understanding care coordination among PCMH practices,17 we hypothesized that practices affiliated with health systems would have greater implementation of patient safety activities than those that are not.
METHODS
IRB Approval.
This study was approved by the IRB of our institution (No. 00007497).
Source.
We examined data collected by the first and most widely adopted formal assessment program for PCMHs. Our study focused on primary care practices that were recognized under this national PCMH program as of May 2017.
The PCMH recognition standards have 169 individual factors across 6 domains: 1) patient-centered access, 2) team-based care, 3) population health management, 4) care management and support, 5) care coordination and care transitions, and 6) performance measurement and quality improvement. PCMHs may select “Yes”, “No”, or “N/A” as responses to each factor. Trained reviewers employed by the PCMH program administrator assessed the documentation provided by practices applying for recognition. A random sample of approximately 5% of applications underwent audit to assess practice transformation. Audits were performed by email, teleconference, webinar, or onsite review.
Three levels of PCMH recognition were possible, reflecting the degree to which a practice met these recognition program standards. We included only Level 3 practices to examine performance among the most mature, fully-functioning PCMHs. After excluding 951 Level 1 and 2 practices, our sample included 84% of all recognized PCMH 2014 practices (n=5,958).
Consensus process.
Although attention to safety is not explicitly required in the program standards, PCMH practices must demonstrate competencies related to comprehensive, patient-centered, coordinated, and accessible care. These competencies overlap considerably with the four key domains of ambulatory safety, identified in the literature as: 1) missed or incorrect diagnoses, 2) delays in proper treatment or preventive services, 3) medication errors and preventable adverse drug events (ADEs), and 4) defects in communication, information flow, and coordination of care.3–5
Based on this understanding, three primary care physicians from two institutions with expertise in patient safety and the PCMH model then independently rated the relevance of each of the PCMH factors to these key ambulatory patient safety issues as “Low”, “Moderate”, or “High”. Participants unanimously identified 34 factors as highly relevant to one or more patient safety domain in the first round of review. These included standards related to medication reconciliation, monitoring and alerts; access to timely clinical advice; and coordination of care transitions after hospitalization.
Through two additional rounds of scoring and discussions, the physicians resolved discordance in ratings for 36 additional factors. This iterative consensus process resulted in 70 of the 169 PCMH factors being identified as relevant. A final step classified the 70 factors into the most relevant of the four ambulatory patient safety domains: medication errors and preventable ADEs (11 items); missed diagnoses (9 items); treatment delays (14 items); or communication and information flow defects (36 items).
We calculated the practice-level patient safety implementation score by: (1) determining the number of applicable factors achieved by each PCMH, (3) calculating a proportion (# achieved / 70 applicable factors), and (3) converting to an activity implementation score on a 0–100 scale (with 100 being the highest possible score). We calculated scores both overall and for each of the four domains.
We conducted preliminary testing of the score to establish its properties. More than 90% of Level 3 PCMH practices achieved 42 of the 70 patient safety factors. While some of these activities are deemed “must-pass”—mandatory to achieve PCMH recognition—the majority are elective. We decided to exclude these patient safety activities that were achieved by virtually all level 3 PCMHs in order to highlight activities achieved less frequently, and to maximize variation in patient safety implementation scores. These factors include: offering same-day appointments and clinical advice during and after office hours; decision-support tools for mental health, substance abuse, acute and chronic medication conditions; medication reconciliation during care transitions and review of drug-drug and drug-allergy interactions for all new prescriptions; and tracking of lab and imaging tests to notify patients of results and flag overdue orders for clinical follow-up. Thus, the final patient safety implementation score has 28 items.
Regression analysis.
To understand prevalence of patient safety activities across different PCMHs, we constructed hierarchical linear models with the overall patient safety implementation score and each of the four domain-specific scores as the dependent variables. The multilevel linear regressions accounted for practice- and county-level characteristics and nested practices within states, recognizing that regulatory differences and the public, commercial, and multi-payer incentive programs which drive PCMH adoption are frequently determined at a state level.
Independent variables included practice and community characteristics. Practice characteristics reported through the recognition program included population served (adult, pediatrics, or both), practice type and number of clinicians. We categorized practice types into 4 groups based on ownership: federally qualified health centers or community health centers (hereafter referred to as “community clinics”); practices owned by hospitals, health plans, or health systems (hereafter referred to as “health system practices”); facilities which provide health services to active duty and retired military personnel and their families (hereafter referred to as “military treatment facilities”); and independent or physician-owned practices (hereafter referred to as “private practices”). Community characteristics included urban/rural location of practices based on the Department of Agriculture’s Rural-Urban Continuum Code typology.18 Further, we used the Health Resources and Services Administration’s Area Health Resource File to identify practices in counties with a significant Black or African American population, a Health Professional Shortage Area designation, low employment, low education, or high poverty.19
RESULTS
Among our sample of 5,007 Level 3 PCMH practices, 3,341 practices (67%) treat both adult and pediatric populations (Table 1). Of the 5,007 practices, 2,370 (47%) are affiliated with a health system and 2,184 (44%) have 2 to 4 practicing clinicians. Of the 5,007 practices, 2,943 (58.7%) are located in a large metropolitan area (59%), and 4,393 (88%) are located in a county designated as having a partial primary care health professional shortage. Of the 5,007 practices, 1,369 (27%) PCMH practices are located in counties with a Black or African-American population greater than twenty percent. Approximately one-tenth of the 5,007 PCMH practices are in communities with significant economic strain, including 456 practices (9%) in counties with low employment, 437 practices (9%) in counties with low educational attainment (9%), and 620 practices (12%) in counties with high poverty.
Table 1:
Characteristics of Recognized Patient-Centered Medical Home Practices
| n (%) | |
|---|---|
| Total | 5,007 (100) |
| Type of Practice | |
| Community Clinic | 821 (16) |
| Health System | 2370 (47) |
| Military | 122 (2) |
| Private Practice | 1694 (33) |
| Population Served by Practice | |
| Adult Primary Only | 986 (20) |
| Adult and Pediatric | 3341 (67) |
| Pediatric Only | 680 (14) |
| Number of Clinicians | |
| 1 clinician | 813 (16) |
| 2 to 4 clinicians | 2184 (44) |
| 5 to 9 clinicians | 1333 (27) |
| Ten or more clinicians | 677 (14) |
| Urban/Rural a | |
| Metro, Population of 1m or More | 2943 (59) |
| Metro, Population of 250k to 1m | 1024 (21) |
| Metro, Population of <250k | 410 (8) |
| Nonmetro | 630 (13) |
| Race b | |
| >20% Black / African American Population | 1369 (27) |
| Health Professional Shortage Area b | |
| Primary Care – Whole County | 129 (3) |
| Primary Care – Part of County | 4393 (88) |
| Employment b | |
| Low Employment (>35% of population unemployed) | 456 (9) |
| Education b | |
| Low Education (>20% lack High School Diploma) | 437 (9) |
| Poverty b | |
| High Poverty (>20% of population below poverty line) | 620 (12) |
County-level community characteristics obtained through linkage to U.S. Department of Agriculture Economic Research Services 2013 Rural-Urban Continuum Codes.
County-level community characteristics obtained through linkage to Health Services & Resources Administration Area Health Resource File, 2016–2017 Release.
Table 2 shows the factors achieved by more than 90% of practices, which were therefore excluded from the final patient safety implementation score.
Table 2:
PCMH Factors Achieved by More than 90 Percent of Practices, by Safety Domain
| Decreasing medication safety issues (6 items) |
| Uses an electronic system which includes patient allergies, including medication allergies and adverse reactions, as structured, searchable data for more than 80 percent of patients. |
| Uses an electronic system which includes a list of prescription medication, including the date of updates, as structured, searchable data for more than 80 percent of patients. |
| Reviews and reconciles medications for more than 80 percent of patients received from care transitions. |
| Compares more than 50% of eligible prescriptions to drug formularies and sends electronically to pharmacies. |
| Enters electronic medication orders in the medical record for more than 60 percent of medications. |
| Performs patient-specific checks for drug-drug and drug-allergy interactions. |
| Reducing missed, delayed, or incorrect diagnoses (4 items) |
| Collects medical history of patient and family as part of a regularly updated comprehensive health assessment. |
| Tracks lab tests until results are available, flagging and following up on overdue results. |
| Tracks imaging tests until results are available, flagging and following up on overdue results. |
| Incorporates clinical lab test results electronically into structured fields in the medical record. |
| Preventing delays in treatment or prevention (11 items) |
| Provides same-day appointments for routine and urgent care. |
| Provides timely clinical advice by telephone. |
| Provides patients/families instructions for obtaining care and clinical advice during office hours and when the office is closed. |
| Records the patient’s Date of Birth as structured data in an electronic system for more than 80 percent of patients. |
| Collects information about risky or unhealthy behaviors as part of a regularly updated comprehensive health assessment. |
| Generates lists of patients who are overdue for an office visit or service, and acts to remind them. |
| Implements clinical decision support following evidence-based guidelines for at least one mental health or substance use disorder. |
| Implements clinical decision support following evidence-based guidelines for at least one chronic medical condition. |
| Implements clinical decision support following evidence-based guidelines for at least one acute medical condition. |
| Flags abnormal lab results, bringing them to the attention of the clinician. |
| Proactively identifies patients with unplanned hospital admissions or emergency department visits. |
| Improving communication, information flow, and coordination (21 items) |
| Provides continuity of medical record information for care and advice when the office is closed. |
| Documents clinical advice in patient records. |
| Informs patients/family about the role of the medical home and coordinates care across settings. |
| Records the patient’s telephone number as structured data in an electronic system for more than 80 percent of patients. |
| Records the patient’s preferred language as structured data in an electronic system for more than 80 percent of patients. |
| Provides interpretation or bilingual services to meet the language needs of its population. |
| Holds a scheduled patient care team meeting or has structured communication processes focused on individual patient care. |
| Records an up-to-date problem list with current and active diagnoses in an electronic system for more than 80% of patients. |
| Considers behavioral health conditions in process and criteria for identifying patients who may benefit from care management. |
| Considers high cost and high utilization in process and criteria for identifying patients who may benefit from care management. |
| Considers poorly controlled or complex conditions in process and criteria for identifying patients who may benefit from care management. |
| Identifies treatment goals for more than 75 percent of patients identified for care management. |
| Develops a self-management plan for more than 75 percent of patients identified for care management. |
| Provides educational materials and resources to patients. |
| Notifies patients/families of normal and abnormal lab and imaging test results. |
| Tracks referrals until the consultant or specialist’s report is available, flagging and following up on overdue reports. |
| Shares clinical information with admitting hospitals and emergency departments. |
| Obtains patient discharge summaries from hospitals and other facilities. |
| Proactively contacts patients/families for appropriate follow-up care within an appropriate period following a hospital admission or emergency department visit. |
| Evaluates patient/family experiences related to at least three of the following categories: Access, Communication, Coordination and Whole Person Care / Self-Management Support. |
| Conducts security risk analysis of its electronic health record system; implements security updates as necessary. |
Table 3 shows the factors included in the final patient safety implementation score.
Table 3:
Achievement of PCMH Activities, by Safety Domain, by Practice Type
| Patient Safety Domains and PCMH Factors | Number and Percentage of Practices Achieving Each Factor | ||||
|---|---|---|---|---|---|
| Overall | Community Clinic | Health System | Military | Private Practice | |
| n = 5,007 | n = 821 | n = 2,370 | n = 122 | n = 1,694 | |
| Decreasing medication safety issues (5 items) | |||||
| Generates lists of patients on specific medications in order to manage side effects, notify patients of a recall, remind patients about necessary monitoring, inform patients about drug-drug or dosage concerns, or identify patients using a brand name drug instead of a generic drug. | 3635 (73%) | 595 (73%) | 1737 (73%) | 73 (60%) | 1230 (73%) |
| Provides information about new prescriptions to more than 80 percent of patients/families/caregivers. | 2904 (58%) | 496 (60%) | 1228 (52%) | 108 (89%) | 1072 (63%) |
| Assesses understanding of medications for more than 50 percent of patients/families/caregivers, and dates the assessment. | 3915 (78%) | 615 (75%) | 1813 (77%) | 118 (97%) | 1365 (81%) |
| Assesses response to medications and barriers to adherence for more than 50 percent of patients, and dates the assessment. | 4147 (83%) | 661 (81%) | 1929 (81%) | 113 (93%) | 1469 (87%) |
| Documents over-the-counter medications, herbal therapies and supplements for more than 50 percent of patients, and dates updates. | 4466 (89%) | 703 (86%) | 2190 (92%) | 115 (94%) | 1459 (86%) |
| Reducing missed, delayed, or incorrect diagnoses (5 items) | |||||
| Assesses whether the patient and the patient’s family has mental health/behavioral conditions or substance abuse issues (e.g., stress, alcohol, prescription drug abuse, illegal drug use, maternal depression). | 3835 (77%) | 610 (74%) | 1825 (77%) | 95 (78%) | 1308 (77%) |
| Screens for depression in adults and adolescents using a standardized tool. | 3800 (76%) | 702 (86%) | 1581 (67%) | 111 (91%) | 1404 (83%) |
| Implements evidence-based decision support for overuse/appropriateness issues. | 3390 (68%) | 555 (68%) | 1723 (73%) | 105 (86%) | 1005 (59%) |
| Flags abnormal imaging results, bringing them to the attention of the clinician. | 4286 (86%) | 660 (80%) | 2019 (85%) | 121 (99%) | 1484 (88%) |
| Integrates imaging results that include a written report and may include images into the medical record electronically. | 3675 (73%) | 555 (68%) | 1858 (78.4%) | 116 (95%) | 1147 (68%) |
| Preventing delays in treatment or prevention (3 items) | |||||
| Provides routine and urgent-care appointments outside regular business hours. | 4341 (87%) | 730 (89%) | 2048 (86%) | 87 (71%) | 1477 (87%) |
| Provides alternative types of clinical encounters (e.g. group or virtual visits). | 1622 (32%) | 281 (34%) | 756 (32%) | 51 (42%) | 535 (32%) |
| Sets and tracks standards for availability of appointments. | 3946 (79%) | 555 (68%) | 1958 (83%) | 117 (96%) | 1316 (78%) |
| Improving communication, information flow, and coordination (15 items) | |||||
| Provides timely clinical advice using a secure, interactive electronic system. | 4366 (87%) | 639 (78%) | 2171 (92%) | 121 (99%) | 1437 (85%) |
| Involves care team staff in the practice’s performance evaluation and quality improvement activities. | 4411 (88%) | 742 (90%) | 2114 (89%) | 112 (92%) | 1442 (85%) |
| Documents advance directives in the medical record (NA for pediatric practices). | 1908 (38%) | 301 (37%) | 1019 (43%) | 59 (48%) | 527 (31%) |
| Documents the name and contact information of other health care professionals involved in patient’s care. | 3615 (72%) | 495 (60%) | 1782 (75%) | 96 (79%) | 1243 (73%) |
| Assesses the patient/family/caregiver’s ability to understand the concepts and care requirements associated with managing their health. | 2839 (57%) | 484 (59%) | 1287 (54%) | 88 (72%) | 977 (58%) |
| Considers referrals by outside organizations, practice staff, or family/caregiver in process and criteria for identifying patients who may benefit from care management. | 3305 (66%) | 448 (55%) | 1555 (66%) | 99 (81%) | 1203 (71%) |
| Ensures the care team and patient/family/caregiver collaborate (at relevant visits) to develop and update an individual care plan that includes an assessment and plan to address potential barriers to meeting goals. | 4326 (86%) | 692 (84%) | 1996 (84%) | 104 (85%) | 1536 (91%) |
| Provides individual care plans in writing to the patient/family/caregiver. | 4081 (82%) | 663 (81%) | 1958 (83%) | 77 (63%) | 1386 (82%) |
| Follows up with the inpatient facility about newborn hearing and newborn blood-spot screening (NA for adults). | 3039 (61%) | 571 (70%) | 1472 (62%) | 84 (69%) | 915 (54%) |
| Has the capacity for electronic exchange of key clinical information and provides an electronic summary of care record to another provider for more than 50 percent of referrals. | 3280 (66%) | 438 (53%) | 1583 (67%) | 100 (82%) | 1157 (68%) |
| Documents co-management arrangements in the patient’s medical record. | 3054 (61%) | 358 (44%) | 1564 (66%) | 87 (71%) | 1047 (62%) |
| Asks patients/families about self-referrals and requests reports from clinicians. | 3430 (69%) | 533 (65%) | 1519 (64%) | 111 (91%) | 1265 (75%) |
| Exchanges patient information with the hospital during a patient’s hospitalization. | 4286 (86%) | 673 (82%) | 2064 (87%) | 117 (96%) | 1431 (85%) |
| Exchanges key clinical information with facilities and provides an electronic summary-of-care record to another care facility for more than 50 percent of patient transitions of care. | 3345 (67%) | 523 (64%) | 1562 (66%) | 110 (90%) | 1150 (68%) |
| Measures or receives quantitative data on at least two measures related to the effectiveness of care coordination activities at least annually. | 4366 (87%) | 738 (89.9%) | 2159 (91%) | 72 (59%) | 1394 (82%) |
Achievement of specific factors included in the patient safety implementation score ranged from 32.4 (for “Provides alternative types of clinical encounters (e.g., group or virtual visits)” to 89.2 (for “Documents over-the-counter medications, herbal therapies and supplements for more than 50 percent of patients, and dates updates”) (Table 3). The overall mean patient safety implementation score was 74.0 (SD 11.9), and at the level of the domains, mean scores ranged from a low of 66.0 (SD 24.0) for “Preventing delays in proper treatment/ prevention” to a high of 76.4 (SD 20.3) for “Reducing missed, delayed or incorrect diagnoses”.
In unadjusted analyses (Table 4), the small number of military practices had the highest patient safety implementation scores among all practice types in all domains—reducing medication errors and preventable drug events (86.4), missed and delayed diagnoses (90.0)), delays in proper treatment (69.7), and defects in communication (80.3).
Table 4:
Average Patient Safety Implementation Score by Practice and Community Characteristics (n=5,007)
| n | Overall Patient Safety Score (28 items) Mean (SD) |
Decreasing Medication Safety Issues (5 items) Mean (SD) |
Reducing Missed, Delayed, or Incorrect Diagnoses (5 items) Mean (SD) |
Preventing Delays in Proper Treatment/ Prevention (3 items) Mean (SD) |
Improving Communication, Information Flow, Coordination (15 items) Mean (SD) |
|
|---|---|---|---|---|---|---|
| Total | 5007 | 74 (11.9) | 76.3 (23.5) | 76.4 (20.3) | 66 (24) | 74 (14.1) |
| Practice Characteristics | ||||||
| Type of Practice | ||||||
| Community Clinic | 821 | 70.2 (11.7) | 74.8 (24.1) | 75.6 (20.5) | 63.6 (25.8) | 68.1 (14.5) |
| Health System Owned | 2370 | 74.5 (11.3) | 75.1 (23.5) | 76.3 (19.6) | 67.0 (22.1) | 75.2 (13.3) |
| Military | 122 | 82.0 (8.5) | 86.4 (17.5) | 90.0 (14.6) | 69.7 (25.0) | 80.3 (11.4) |
| Private Practice | 1694 | 74.4 (12.3) | 77.9 (23.4) | 76.1 (21.1) | 65.5 (25.4) | 74.5 (14.3) |
| Population Served by Practice | ||||||
| Adult Primary Only | 986 | 75.0 (11.7) | 78.3 (22.4) | 75.6 (20.6) | 65.3 (24.1) | 75.7 (13.8) |
| Adult and Pediatric | 3341 | 73.7 (12.0) | 75.1 (23.8) | 76.9 (20.1) | 66.6 (23.9) | 73.6 (14.2) |
| Pediatric Only | 680 | 73.6 (11.2) | 78.9 (23.4) | 75.3 (20.6) | 64.1 (24.4) | 73.2 (13.7) |
| Number of Clinicians | ||||||
| 1 clinician | 813 | 73.4 (12.2) | 76.5 (23.8) | 75.0 (19.7) | 63.6 (25.2) | 73.7 (14.4) |
| 2 to 4 clinicians | 2184 | 73.7 (11.7) | 75.8 (23.5) | 76.5 (20.6) | 64.9 (23.9) | 73.8 (14.2) |
| 5 to 9 clinicians | 1333 | 74.5 (12.0) | 76.2 (23.8) | 77.4 (20.3) | 67.2 (23.4) | 74.4 (13.9) |
| Ten or more clinicians | 677 | 74.5 (11.7) | 77.7 (22.6) | 76.0 (19.9) | 70.1 (23.5) | 73.8 (13.7) |
| Community Characteristics | ||||||
| Urban/Rural a | ||||||
| Metro, Population of 1m or More | 2943 | 75.1 (11.6) | 78.3 (23.2) | 76.7 (19.7) | 66.7 (23.6) | 75.1 (14.0) |
| Metro, Population of 250k to 1m | 1024 | 72.9 (12.5) | 72.7 (23.9) | 76.4 (21.6) | 67.5 (24.2) | 72.9 (14.2) |
| Metro, Population of < 250k | 410 | 72.0 (11.5) | 74.7 (23.6) | 76.6 (20.4) | 61.1 (24.2) | 71.8 (13.6) |
| Nonmetro | 630 | 71.7 (11.5) | 73.6 (23.3) | 75.3 (20.6) | 63.4 (25.0) | 71.5 (14.0) |
| Race b | ||||||
| <20% Black / African American Population | 3638 | 73.9 (12.1) | 76.1 (23.6) | 76.3 (20.7) | 65.6 (23.9) | 74.0 (14.1) |
| >20% Black / African American Population | 1369 | 74.2 (11.3) | 76.8 (23.3) | 76.8 (18.9) | 67.0 (24.3) | 73.9 (14.1) |
| Health Professional Shortage Area b | ||||||
| Primary Care – Whole County | 129 | 73.6 (10.4) | 78 (19.2) | 76.0 (16.3) | 61.0 (27.0) | 73.8 (14.7) |
| Primary Care – Part of County | 4393 | 73.8 (11.9) | 76 (23.7) | 76.4 (20.4) | 66.0 (23.6) | 73.8 (14.2) |
| Primary Care – No Shortage | 405 | 75.2 (12.1) | 77.8 (22.7) | 77.3 (20.0) | 66.8 (26.2) | 75.3 (12.8) |
| Employment b | ||||||
| Low Employment | 456 | 70.3 (12.6) | 74.5 (24.3) | 74.0 (20.8) | 58.7 (27.2) | 70.0 (14.6) |
| Higher Employment | 4551 | 74.3 (11.7) | 76.4 (23.4) | 76.7 (20.2) | 66.7 (23.5) | 76.4 (23.4) |
| Education b | ||||||
| Low Education | 437 | 72.0 (11.3) | 75.2 (22.0) | 73.4 (18.1) | 65.1 (25.5) | 71.7 (14.1) |
| Higher Education | 4570 | 74.2 (11.9) | 76.4 (23.6) | 76.7 (20.4) | 66.1 (23.9) | 74.2 (14.1) |
| Poverty b | ||||||
| High Poverty | 620 | 72.0 (12.3) | 76.2 (22.9) | 74.2 (20.2) | 64.2 (26.0) | 71.3 (15.0) |
| Lower Poverty | 4387 | 74.2 (12.3) | 76.3 (23.6) | 76.8 (20.2) | 66.2 (23.7) | 74.3 (13.9) |
County-level community characteristics obtained through linkage to U.S. Department of Agriculture Economic Research Services 2013 Rural-Urban Continuum Codes.
County-level community characteristics obtained through linkage to Health Services & Resources Administration Area Health Resource File, 2016–2017 Release.
Differences in medication safety activities were primarily driven by the following factors: new prescription education (reported by 89% of military practices vs. 58% overall) and assessing patient understanding of medications (97% vs. 78%) (Table 3).
Differences in activities that could reduce missed and delayed diagnoses were driven by the ability to access imaging results electronically (95% vs. 73%) and using evidence-based decision support for overuse or appropriateness issues (86% vs. 68%). Military practices reported higher scores related to prevention of delays in treatment or prevention mainly as a result of setting and tracking standards for availability of appointments (96% vs. 79%).
Finally, military practices had higher-than-average scores on several activities related to communication and coordination of care: providing access to timely clinical advice using a secure electronic system (99% vs. 87%), and sharing electronic summary of care records when making a referral (82% vs. 66%).
Community clinics scored lowest of all practice types in all domains. Other practice characteristics - practice size, urban/rural, practice population (adult, pediatric, both) - had no statistically significant associations with patient safety implementation scores overall and across all domains.
Our multilevel analysis (Table 5) reinforced the finding that some practice-level characteristics have a significant association with achievement of ambulatory patient safety activities. Compared to health system practices, military practices achieved significantly higher scores overall (beta=8.1, p<.0001) and for three domains: decreasing medication safety issues (beta=12.9, p<.0001), reducing missed, delayed, or incorrect diagnoses (beta=5.91, p<.0001), and improving communication, information flow, and coordination (beta=5.9, p<.0001). Other practice and community characteristics had limited or inconsistent associations with patient safety activity implementation.
Table 5:
Regression Results for Patient Safety Implementation Scores, Overall and by Domain (n=5,007)
| Overall Patient Safety Score (28 items) |
Decreasing Medication Safety Issues (5 items) |
Reducing Missed, Delayed or Incorrect Diagnoses (5 items) |
Preventing Delays in Proper Treatment /Prevention (3 items) |
Improving Communication, Information Flow, Coordination (15 items) |
|
|---|---|---|---|---|---|
| Regression Coefficients | |||||
| Owner (RG: Health System) | |||||
| Community Clinic | −3.71*** | −0.28 | −0.27 | −2.36* | −6.27*** |
| Military | 8.06*** | 12.98*** | 13.62*** | 1.45 | 5.91*** |
| Private Practice | 1.11** | 1.78* | 2.24** | 0.84 | 0.51 |
| Number of Clinicians (RG: 5 to 9 clinicians) | |||||
| 1 clinician | −1.05* | −0.38 | −2.09* | −2.17* | −0.72 |
| 2 to 4 clinicians | −0.56 | −0.21 | −0.49 | −1.62* | −0.49 |
| Ten or more clinicians | 0.62 | 1.87 | −0.56 | 2.65 | 0.16 |
| Population Served by Practice (RG: Adult Primary Only) | |||||
| Adult and Pediatric | −0.17 | −1.84* | 0.88 | 2.73** | −0.56 |
| Pediatric Only | −1.7** | −0.04 | −1.35 | −1.09 | −2.47*** |
| Urban/Rural (RG: Metro, Population of 1M or More) a | |||||
| Metro, Population of 250k to 1M | −1.97*** | −5.33*** | −1.59 | 1.21 | −1.59** |
| Metro, Population of <250K | −1.81** | −3.21* | 0.12 | −3.77** | −1.67* |
| Nonmetro | −1.82** | −3.39** | −0.87 | −0.08 | −2.07** |
| Race (RG: < 20% Black / African American) b | |||||
| > 20% Black / African American | 0.49 | −1.78 | 1.37 | 3.52** | 0.33 |
| Health Professional Shortage Area (RG: Primary Care – Part of County) b | |||||
| Primary Care – No Shortage | 0.83 | 2.96* | 1.21 | 0.30 | 0.05 |
| Primary Care – Whole County | 1.22 | 3.28 | −0.13 | −1.22 | 1.56 |
| Employment (RG: Low Employment = 0) b | |||||
| Low Employment | −1.83* | −0.08 | −1.04 | −8.31*** | −1.34 |
| Education (RG: Low Education = 0) b | |||||
| Low Education | 0.01 | 0.52 | −0.73 | 1.37 | −0.26 |
| Poverty (RG: High Poverty = 0) b | |||||
| High Poverty | −0.65 | −0.15 | −0.84 | 0.32 | −0.97 |
Statistical significance is represented as follows:
p < 0.05
p < 0.01
p < 0.0001
County-level community characteristics obtained through linkage to U.S. Department of Agriculture Economic Research Services 2013 Rural-Urban Continuum Codes.
County-level community characteristics obtained through linkage to Health Services & Resources Administration Area Health Resource File, 2016–2017 Release.
LIMITATIONS
This study used available data on a large number of primary care practices with well-structured care delivery thought to be supportive of patient-centric access, continuity, comprehensiveness, coordination. The data are limited to descriptive and self-reported indicators of PCMH standard implementation. While this PCMH database is large and diverse, representing approximately 20% of the primary care workforce,20 it was not designed specifically to collect information about patient safety and may not address all patient safety practices in ambulatory care. The PCMH factors that we defined as most relevant to patient safety require further research to validate their relevance to actual patient safety in the ambulatory setting.
In addition, key contextual factors that may be associated with patient safety activity implementation include external factors such as financial or performance incentives; safety culture, teamwork and leadership; and availability of implementation tools, such as clinician and staff training,21 which our study was not able to account for. Further research should evaluate these additional factors in more detail to determine key facilitators and barriers to patient safety practice implementation.
DISCUSSION
To our knowledge, this is the first study to use a large data set based on consensus process to examine how different kinds of primary care practices address concerns related to ambulatory patient safety in the context of the medical home model.
We found that these practices have broadly implemented systems that should prevent harm and promote safety. Patient-centered medical homes have documented robust workflows to expand access to clinical information and advice, manage care across settings, and empower patients to manage their own health. These practices also address a diverse array of potential safety risks otherwise caused or exacerbated by fragmented, poorly-coordinated ambulatory care. Still, there were deficits in implementation of many patient safety activities. Activities with low adoption across all practice types may be of lower priority or less well-developed (e.g. documentation of advance directives, assessment of health literacy, and medication education). Other activities may be less valued in their utility to measure and address social risks or may lack sufficient resources or insurance coverage (e.g. mental health/substance abuse assessment). Other activities reflect persistent fragmentation of care (e.g. co-management arrangements, self-referrals). Finally, some variation appears to be a result of disparities in electronic capabilities (e.g. flagging abnormal imaging results, reviewing image results electronically, communicating with patients and providers electronically), consistent with other research that has found that effective electronic health record implementation is associated with better patient safety practice implementation.22
Our results also demonstrate that even PCMH practices with the highest level of recognition show room for improvement on key patient safety activities. Although the number of military practices was small, these practices had significantly higher, implementation of patient safety activities overall and generally across 4 key domains of ambulatory safety, in both univariate and multilevel regression analysis. These results suggest the Military Health System’s emphasis on systems-level solutions and continuous monitoring of staff perceptions of patient safety have contributed significantly to success in this area.23 Conversely, community clinics had significantly lower implementation of patient safety activities overall and across these 4 domains of ambulatory safety.
CONCLUSIONS
In this study, we developed a method to measure the implementation of patient safety activities in ambulatory care. We found that room for improvement exists for many of these activities, even among patient-centered medical homes achieving the highest level of recognition. Among practice characteristics, only military status was associated with significantly higher patient safety scores. Further research should evaluate other organizational and contextual factors and their associations with both these and a wider range of ambulatory patient safety implementation strategies. Finally, defining and evaluating key patient safety outcomes for primary care would improve understanding of the relationship between the environment of care and the ability of providers to maximize patient safety.
IMPLICATIONS
Health care delivery systems have an important opportunity to adopt effective process-improvement activities and tools which will drive cultural change in their organizations. It is notable that community clinics had lower performance all other practice types on all four patient safety domains: 1) missed or incorrect diagnoses, 2) delays in proper treatment or preventive services, 3) medication errors and preventable adverse drug events (ADEs), and 4) defects in communication, information flow, and coordination of care. This underperformance is particularly concerning given the vulnerable populations they serve, and suggests a need for additional technical assistance and financial support for patient safety activities among these practices. It is also notable that even among these practices that achieved the highest level of PCMH recognition, military practices outperformed their peers, although the number of practices was small. These practices had higher scores in several areas than all other practice types. Based on the methodology described in this manuscript, and given its extensive commitment to patient safety culture and rich history of providing coordinated health care services, the Military Health System may be a useful model to better understand how to implement activities supporting ambulatory patient safety.
Source of Funding:
This project was supported by grant number R01HS024859 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
Biographical Sketches:
Tyler Oberlander, BA, is Director of Research and Analysis, National Committee for Quality Assurance (NCQA), Washington, DC. He is responsible for planning, development and implementation of analytics for NCQA. He also participates in design and implementation of research projects using data provided by NCQA-recognized Patient-Centered Medical Home practices.
Sarah Hudson Scholle, DrPH is Vice President of Research and Analysis, National Committee for Quality Assurance (NCQA), Washington, D.C. She leads a broad portfolio of systems research and quality measurement projects addressing patient-centeredness, care coordination, behavioral health, disparities and patient experiences. Her research informed the development of NCQA’s Patient-Centered Medical Home program.
Jill Marsteller, PhD, is Professor, Health Policy and Management, Johns Hopkins, Baltimore, MD. She is jointly appointed to the Armstrong Institute for Patient Safety and Quality, where she leads the Research Facilitation Council, and at the Johns Hopkins Carey Business School. Her research focuses on organizational behavior and theory, specifically in estimating the influence of organizational variables and contextual measures on quality improvement activities in learning organizations.
Michael S. Barr, MD, MBA, MACP, FRCP is Executive Vice President of the Quality Measurement and Research Group, National Committee for Quality Assurance (NCQA), Washington, DC. His portfolio includes NCQA’s performance measurement, research, analysis, contracts and grants, and consulting activities. Dr. Barr also leads the digital measures strategy for HEDIS® and contributes to NCQA’s other strategic initiatives, public policy, and educational programs.
Sydney Morss Dy, MD, MS is Professor, Health Policy & Management, Medicine and Oncology, Johns Hopkins, Baltimore, MD. Dr. Dy is also a practicing primary care physician. Her research focuses on quality and safety measurement and improvement in primary care and for patients with serious illness.
Footnotes
Conflicts of Interest:
Tyler Oberlander, Sarah Hudson Scholle and Michael Barr are employed by the National Committee for Quality Assurance, which recognizes Patient-Centered Medical Homes. For the remaining authors none were declared.
IRB Approval:
This study was approved by the Johns Hopkins School of Public Health (JHSPH) Institutional Review Board (IRB) (No. 00007497).
Contributor Information
Tyler Oberlander, National Committee for Quality Assurance (NCQA), Washington, DC..
Sarah Hudson Scholle, National Committee for Quality Assurance (NCQA), Washington, D.C..
Jill Marsteller, Health Policy and Management, Johns Hopkins, Baltimore, MD..
Michael S. Barr, National Committee for Quality Assurance (NCQA), Washington, DC.
Sydney Morss Dy, Health Policy & Management, Medicine and Oncology, Johns Hopkins, Baltimore, MD..
CITATIONS
- 1.Shekelle PG, Wachter RM, Pronovost PJ, Schoelles K, Dy SM. Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices Rockville, MD: Agency for Healthcare Research and Quality (US); 2013. https://www.ncbi.nlm.nih.gov/pubmed/24423049. Accessed April 25, 2018. [PMC free article] [PubMed] [Google Scholar]
- 2.Shekelle PG, Sarkar U, Shojania K, Wachter RM, McDonald K, Motala A, et al. AHRQ comparative effectiveness technical briefs. Patient safety in ambulatory settings Rockville,MD: Agency for Healthcare Research and Quality (US); 2016. [PubMed] [Google Scholar]
- 3.Gandhi TK, Lee TH. Patient Safety beyond the Hospital. New England Journal of Medicine 2010;363(11):1001–1003. doi: 10.1056/nejmp1003294 [DOI] [PubMed] [Google Scholar]
- 4.Panesar SS, Desilva D, Carson-Stevens A, et al. How safe is primary care? A systematic review. BMJ Quality & Safety 2015;25(7):544–553. doi: 10.1136/bmjqs-2015-004178 [DOI] [PubMed] [Google Scholar]
- 5.Webster JSKH, Toomey LM, et al. Understanding quality and safety problems in the ambulatory environment: seeking improvement with promising teamwork tools and strategies. In: Henriksen KBJ, Keyes MA, et al., editor. Advances in patient safety: new directions and alternative approaches. Volume 3: performance and tools Rockville (MD): Agency for Healthcare Research and Quality (US); 2008. Available at: https://www.ncbi.nlm.nih.gov/books/NBK43683/ [PubMed] [Google Scholar]
- 6.Lorincz CY, Drazen E, Sokol PE, Neerukonda KV, Metzger J, Toepp MC, Maul L, Classen DC, Wynia MK. Research in Ambulatory Patient Safety 2000–2010: A 10-Year Review American Medical Association, Chicago IL: 2011. Available at: https://c.ymcdn.com/sites/npsf.site-ym.com/resource/resmgr/PDF/Research-in-Amb-Pat-Saf_AMAr.pdf [Google Scholar]
- 7.National Center for Health Statistics. National Ambulatory Medical Care Survey: 2015 State and National Summary Tables Centers for Disease Control and Prevention; 2015. https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2015_namcs_web_tables.pdf. [Google Scholar]
- 8.Singh H, Meyer AND, Thomas EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Quality & Safety 2014;23(9):727–731. doi: 10.1136/bmjqs-2013-002627 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hickner J, Graham DG, Elder NC, et al. Testing process errors and their harms and consequences reported from family medicine practices: a study of the American Academy of Family Physicians National Research Network. Quality and Safety in Health Care 2008;17(3):194–200. doi: 10.1136/qshc.2006.021915 [DOI] [PubMed] [Google Scholar]
- 10.Abramson EL, Bates DW, Jenter C, et al. Ambulatory prescribing errors among community-based providers in two states. Journal of the American Medical Informatics Association 2012;19(4):644–648. doi: 10.1136/amiajnl-2011-000345 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.2008 Update on Consumers’ Views of Patient Safety and Quality Information The Henry J. Kaiser Family Foundation. https://www.kff.org/health-reform/poll-finding/2008-update-on-consumers-views-of-patient-2/. Published October 1, 2008. Accessed April 25, 2018.
- 12.American College of Physicians. Patient Safety in the Office-Based Practice Setting American College of Physicians; 2017. https://www.acponline.org/acp_policy/policies/patient_safety_in_the_office_based_practice_setting_2017.pdf. Accessed April 25, 2018. [Google Scholar]
- 13.Kohn LT, Corrigan J, Donaldson MS. To Err Is Human: Building a Safer Health System Washington: National Academy Press; 2000. https://www.nap.edu/catalog/9728/to-err-is-human-building-a-safer-health-system. Accessed April 25, 2018. [PubMed] [Google Scholar]
- 14.Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: a New Health System for the 21st Century Washington, D.C.: National Academy press; 2001. [Google Scholar]
- 15.NCQA Analysis Shows PCMH Initiatives Growing Across the Country January 2016. https://www.ncqa.org/news/ncqa-analysis-shows-pcmh-initiatives-growing-across-the-country/. Accessed April 25, 2018.
- 16.American Academy of Family Physicians American Academy of Pediatrics, American College of Physicians, American Osteopathic Association. Joint principles of the patient-centered medical home https://www.aafp.org/dam/AAFP/documents/practice_management/pcmh/initiatives/PCMHJoint.pdf.
- 17.Morton S, Shih SC, Winther CH, Tinoco A, Kessler RS, Scholle SH. Health IT-Enabled Care Coordination: A National Survey of Patient-Centered Medical Home Clinicians. The Annals of Family Medicine 2015;13(3):250–256. doi: 10.1370/afm.1797 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Rural-Urban Continuum Codes. U.S. Department of Agriculture Economic Research Service https://www.ers.usda.gov/data-products/rural-urban-continuum-codes.aspx.
- 19.HRSA Data Downloads. Health Resources & Services Administration https://data.hrsa.gov/data/download
- 20.Cotton P. Patient-Centered Medical Home Evidence Increases With Time. Health Affairs Blog September 2018. 10.1377/hblog20180905.807827/full/. Accessed January 22, 2019. [DOI]
- 21.Taylor SL, Dy S, Foy R, et al. What context features might be important determinants of the effectiveness of patient safety practice interventions? BMJ Quality & Safety 2011;20(7):611–617. doi: 10.1136/bmjqs.2010.049379 [DOI] [PubMed] [Google Scholar]
- 22.Tanner C, Gans D, White J, Nath R, Pohl J. Electronic Health Records and Patient Safety: Co-occurrence of Early EHR Implementation with Patient Safety Practices in Primary Care Settings. Applied Clinical Informatics 2015;06(01):136–147. doi: 10.4338/aci-2014-11-ra-0099 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Tanielian T, Farmer C. The US Military Health System: Promoting Readiness and Providing Health Care. Health Affairs 2019; 38(8): 1259–1267. [DOI] [PubMed] [Google Scholar]