Abstract
Objective:
To examine perceptions of facilitators and barriers to quality measurement and improvement in palliative care programs and differences by professional and leadership roles.
Methods:
We surveyed team members in diverse US and Canadian palliative care programs using a validated survey addressing teamwork and communication and constructs for educational support and training, leadership, infrastructure, and prioritization for quality measurement and improvement. We defined key facilitators as constructs rated ≥4 (agree) and key barriers as those ≤3 (disagree) on 1 to 5 scales. We conducted multivariable linear regressions for associations between key facilitators and barriers and (1) professional and (2) leadership roles, controlling for key program and respondent factors and clustering by program.
Results:
We surveyed 103 respondents in 11 programs; 45.6% were physicians and 50% had leadership roles. Key facilitators across sites included teamwork, communication, the implementation climate (or environment), and program focus on quality improvement. Key barriers included educational support and incentives, particularly for quality measurement, and quality improvement infrastructure such as strategies, systems, and skilled staff. In multivariable analyses, perceptions did not differ by leadership role, but physicians and nurse practitioners/nurses/physician assistants rated most constructs statistically significantly more negatively than other team members, especially for quality improvement (6 of the 7 key constructs).
Conclusions:
Although participants rated quality improvement focus and environment highly, key barriers included lack of infrastructure, especially for quality measurement. Building on these facilitators and measuring and addressing these barriers might help programs enhance palliative care quality initiatives’ acceptability, particularly for physicians and nurses.
Keywords: palliative care, quality measurement, quality improvement, quality
Introduction
Given the aging of populations in North America, establishing and maintaining high-quality palliative care programs is essential to improving patient-centered care for people living with serious, life-limiting illness.1–3 To serve these populations, palliative care programs must incorporate quality measurement (QM) and quality improvement (QI) efforts as part of routine operations.1,4 Collective efforts promoting QM not only inform palliative care program QI efforts but also facilitate comparisons and collaborations between palliative care programs.5 National and international collaborations in palliative care QM and collection efforts deepen our understanding of the practice of palliative care and help standardize high-quality palliative care delivery.2
However, adoption of QM and QI practices in palliative care clinical settings remains suboptimal.1 For institutions with palliative care QM systems, data are often incomplete or of poor quality,2,6 and improvement initiatives are generally not integrated into palliative care interventions.7 The few existing studies examining barriers to and facilitators of QM and QI in palliative care are qualitative.5,8 Quantitative understanding of perceptions of barriers and facilitators to QM and QI could inform interventions to improve quality programs in palliative care and thus the care that patients receive.9–11 These barriers and facilitators may include organizational characteristics found to influence quality initiatives in nonpalliative care settings, such as leadership characteristics, teamwork, communication, attitudes toward quality, and the degree to which QM and QI are supported and prioritized.11–13 Perceptions of these barriers and facilitators, as well as views on care quality and culture in areas other than palliative care, can vary based on team member role (eg, physicians compared to nurses)14,15 and between leaders and frontline clinicians.16,17 These differences could inhibit opportunities for teamwork in quality initiatives and their success if perceptions of what may be effective differ by team members’ roles.
The goal of this study was to describe perceptions of facilitators and barriers to QM and QI processes among palliative care team members. We also compared perceptions by palliative care team professional roles (eg, physicians and nurses) and participation in leadership.
Methods
Study Design, Sample, and Data Collection
We conducted a cross-sectional survey of specialty, nonhospice palliative care programs in the United States and Canada. From February to November 2017, we recruited a convenience sample through palliative care collaboratives, quality networks, conferences, and coinvestigators. We aimed to recruit sites diverse in program size, services offered, presence of quality programs, and geography. The potential sample within each palliative care program included palliative care team members as defined by each site. Respondents included oncology and palliative care physicians, nurses, nurse practitioners, physician assistants, social workers, chaplains, pharmacists, administrators, and others. We required a minimum of 5 respondents from each program to complete the survey, which we administered either on paper or electronically (via Microsoft Word or PDF). After site completion of the survey, qualitatively trained members of the research team conducted discussion groups focused on content validity of the survey. Eight of the 11 sites participated in these discussion groups.
Measure
We used the “Measuring and Improving Quality in Palliative Care” survey18,19 assessing facilitators and barriers to palliative care program quality assessment and improvement, as perceived by palliative care team members. Prior manuscripts detailed development of this survey18 and evaluation of content validity and reliability.19 In brief, based on the adaptation of the Consolidated Framework for Implementation Research for complex interventions20 and review of the literature, we selected key domains most relevant to palliative care QM and QI: organizational (program) characteristics, characteristics and roles of team members, implementation processes, and intervention characteristics. We identified key constructs corresponding to each of these domains and adapted subscales to measure these constructs from published instruments and additional item sets (see Table 2 for constructs addressed in the survey).18 The validated survey (available upon request from the authors) uses items for each of these constructs to assess the domains of teamwork and communication, as well as QM and QI, with each including constructs for educational support and training, leadership, and prioritization. We refined the survey through pilot testing and cognitive interviews and evaluated the adapted subscales’ internal consistency reliability and content validity and variation among sites.19 The current manuscript describes administration of this validated survey at these sites to describe palliative care professionals’ perceived barriers and facilitators to QM and QI in their palliative care programs. The Johns Hopkins School of Public Health Institutional Review Board determined that this was not considered human subjects research.
Table 2.
Key Barriers and Facilitators for Quality Assessment and Improvement in Palliative Care, Participant-Level Results.a
| Domain | Construct | Meanb (SD) |
|---|---|---|
| Teamwork and Communication | Openness to ideas, viewpoints, questions and disagreement among palliative care team (Openness of Communication) | 4.1 (0.7) |
| Team members help each other out, work well together, treat each other with respect, emphasize teamwork (Perceptions of Teamwork) | 4.4 (0.7) | |
| Quality Measurement | Focus on educational support for quality measurement in palliative care | 2.3 (1.0) |
| Program recognition for quality measurement in palliative care | 2.8 (0.8) | |
| Program focus on rewards for quality measurement in palliative care | 1.5 (0.7) | |
| Personal accountability for quality measurement in palliative care | 2.4 (1.3) | |
| Quality Improvement | Program focus on quality improvement in palliative care | 4.5 (0.6) |
| Team and leaders value QI and understand and effectively use quality improvement methods (Implementation Climate) | 4.0 (0.6) | |
| Program focus on educational support for quality | 3.0 (0.9) | |
| Program focus on selection of staff for quality improvement | 2.6 (0.9) | |
| Strategies used for QI (information, skills training, encouragement by leadership) | 2.7 (0.9) | |
| Creating systems, reducing barriers, organizing, setting goals and benchmarking, customizing for QI | 2.7 (1.0) | |
| Methods to evaluate or reduce negative consequences of QI | 2.2 (1.1) |
Abbreviations: QI, quality improvement; SD, standard deviation.
n = 103.
All are on a 1–5 scale. Table includes key constructs with more positive (≥4) or negative (≤3) mean scores.
Variables
We rescaled all multi-item survey responses to a numerical scale from 1 to 5, where higher scores indicated positive responses. We calculated constructs (subscale scores) for QM and QI by averaging survey responses across all items included in the construct, as defined by the original published survey instrument methodology. These constructs were the dependent variables for our descriptive and multivariable analyses. We evaluated variation between sites to determine whether analyses could be combined across sites.
For the independent variable professional roles, we combined professional roles into 3 groups based on similarities in roles: attending physicians and physician fellows; nurse practitioners, clinical nurse specialists, registered nurses, and physician assistants; and others, including social workers, pharmacists, chaplains, and administrators. These groups served as the categorical independent variables for assessment of differences in survey responses according to team member role. For the independent variable leadership, we used respondents’ self-defined roles as leaders or nonleaders and also conducted specific analyses for program, departmental, or organizational leadership.
Data Analysis
We calculated internal consistency reliability using Cronbach’s α for all constructs (range: 0.63–0.87). We excluded one construct, Program focus on rewards for quality improvement, which did not meet a minimum criteria of 0.70.21 The unit of analysis was the individual palliative care team member. We calculated respondents’ perceptions of the quality of care provided by their programs with mean of survey items on their perceptions of programs’ adherence to established quality measures. We imputed missing data (0% for almost all items) and “does not apply” (<6% for almost all items) responses using hot deck imputation, in which missing values from 1 respondent are imputed from an individual with similar characteristics;22 for regressions, we excluded those with missing professional (n = 3) or leadership role (n = 6).
In descriptive analyses, we described respondent-identified key barriers and facilitators using survey constructs. We defined key barriers as constructs with negative assessments as defined by a mean score of ≤3 on the 1 to 5 scale and key facilitators as those with more positive assessments or ≥4 (agree) across sites. To examine differences in individual-level perceptions of key barriers and facilitators by the independent variables professional role and leadership, we conducted 2 separate sets of multivariable linear regressions, as most subscales were normally distributed. For the first model, the independent variable was the respondent’s professional role category (with the social worker/administrator/other group as the reference group), and for the second, the independent variable was leadership role. The dependent variables for both sets of models were the key constructs for barriers and facilitators for palliative care QM and QI. We accounted for clustering by site within the multivariable models using robust variance estimates.23 Multivariable regression models controlled for country (United States vs Canada) and the key other respondent characteristic we hypothesized might affect perceptions—the length of time working in the palliative care program.
We used Stata 14.1 (StataCorp LP, College Station, Texas) for all analyses.24 We set statistical significance at P < .05, and all tests were 2-tailed.
Results
Characteristics of Study Participants
A total of 103 study participants (range by site, 5–23) from 11 palliative care program sites in the United States (n = 5) and Canada (n = 6) completed the survey (overall response rate for participants: 84%). Program structures ranged from small, single-institution palliative care programs to multisite centers; one program included pediatrics and one provided pediatric palliative care exclusively. Across the 11 sites, 45.6% of respondents were physicians, 26.2% were registered nurses, nurse practitioners, or physician assistants, and the remainder were other staff members including social workers, chaplains, pharmacists, and administrators (Table 1). Fifty percent identified as currently serving in a leadership role in the program: 38.2% identified as part of palliative care program leadership, 16.7% in departmental leadership, and 20.6% in organizational leadership (many had more than one role). Most respondents (86.1%) had worked for the palliative care program for more than 1 year, and 67.7% worked more than 40 hours per week. Almost all (92.2%) had direct interaction with patients in daily activities. Respondents’ perception of their programs’ quality of care was high (mean = 4.4, standard deviation [SD] = 0.4).
Table 1.
Characteristics of Survey Respondents.
| % (n)a (Total, n = 103) |
||
|---|---|---|
| Professional role in palliative care team | ||
| Physicians | Physician attending | 38.8 (40) |
| Physician fellow | 6.8 (7) | |
| Nurse practitioners/registered nurses/physician assistants | Nurse practitioner or clinical nurse specialist | 14.6 (15) |
| Registered nurse | 11.7 (10) | |
| Physician assistant | 1.9 (2) | |
| Social workers/administrators/other | Social worker | 7.8 (8) |
| Administration/management | 6.8 (7) | |
| Other (including pharmacist, chaplain/pastoral care) | 7.8 (8) | |
| Primary location of clinical services provided | Outpatient clinic | 20.4 (21) |
| Inpatient consults | 37.9 (39) | |
| Home-based palliative care | 3.9 (4) | |
| Inpatient palliative care | 15.5 (16) | |
| Other | 3.9 (4) | |
| Multiple sites | 17.5 (18) | |
| Length of employment in palliative care program | <1 year | 13.6 (14) |
| 1–5 years | 43.7 (45) | |
| 6–10 years | 25.2 (26) | |
| 11 years or more | 15.5 (16) | |
| Hours per week in program | Less than 20 hours | 9.7 (10) |
| 20–39 hours | 22.3 (23) | |
| 40–59 hours | 67.0 (68) | |
| Direct interaction with patients | Yes | 92.2 (95) |
| No | 6.8 (7) | |
| Leadership roleb | Palliative care program leadership | 38.2 (39) |
| Departmental leadership | 16.7 (17) | |
| Health system level leadership | 20.6 (21) | |
| No leadership role | 49.0 (50) | |
| Location of program | Canada | 58.3 (60) |
| United States | 41.8 (43) |
Responses for several questions were missing for 1 to 2 respondents, so not all percentages add to 100%.
Respondents could have more than 1 leadership role.
Overall Barriers and Facilitators: Descriptive Analyses
Overall results for key barriers and facilitators to palliative care quality assessment and improvement are shown in Table 2, grouped by teamwork and communication, QM, and QI (all on a 5-point scale except for priorities). Results for these key barriers and facilitators were relatively consistent across sites: Given that only 2 (Creating systems and reducing barriers and methods to reduce or evaluate negative consequences of QI) had >1-point difference between the 10th and 90th percentile of sites and that almost all sites rated both these constructs negatively (≤3), we combined results across sites for analyses. Team members viewed teamwork and communication positively with participant mean scores of 4.1 (SD = 0.7) for Openness of Communication and 4.4 (SD = 0.7) for Perceptions of Teamwork. Respondents viewed constructs related to QM processes in their program more negatively; all constructs for QM had a mean score <4. Specifically, constructs for QM with participant mean scores ≤3 (in the negative range) included Focus on Educational Support for QM (mean = 2.3, SD = 1.0), Recognition for QM (mean = 2.8, SD = 0.8), Personal Accountability for QM (mean = 2.4, SD = 1.3), and Rewards for QM (mean = 1.5, SD = 0.7).
Respondents rated overall QI constructs positively, including Focus on QI (mean = 4.5, SD = 0.6) and implementation climate (Team leaders value QI and understand and effectively use QI methods; mean = 4.0, SD = 0.6). However, they tended to rate key constructs for QI implementation processes more negatively. Constructs with mean scores ≤3 included Strategies used for QI (mean 2.7, SD = 0.9), Creating systems and reducing barriers (mean = 2.7, SD = 1.0) for QI, Methods to reduce or evaluate negative consequences of QI (mean = 2.2, SD = 1.1), and Selection of staff for QI (mean = 2.6, SD = 0.9).
Respondents tended to rate the implementation environment for QI more positively compared to QM. These included Educational support (mean = 3.0, SD = 0.9 vs mean = 2.3, SD = 1.0 for improvement and measurement, respectively), and Personal accountability (mean = 3.7, SD = 1.0 vs mean = 2.4, SD = 1.3 for improvement and measurement, respectively).
Associations of Professional Roles and Leadership With Key Barriers and Facilitators: Multivariable Regression
Individuals with leadership roles did not have significantly different responses for any key barriers and facilitators compared to nonleaders in either simple or multivariable regression (results not shown). In the multivariable regression for professional roles, for the quality implementation environment, although overall all groups rated teamwork and communication openness positively with mean scores ≥4 and ratings for communication openness were similar, the nurse practitioners/nurses/physician assistant group scored teamwork significantly less positively than the social workers/administrators/other group (mean 4.4 vs 4.1 vs 4.5 for the 3 groups; physicians [not significant], nurse practitioners/nurses/physician assistants [P < .01] vs social workers/administrators/others, respectively). While no significant difference in scores was seen among team member groups for incentives, such as Recognition or Rewards for QM, physicians rated Educational support for QM (mean 1.9 vs 2.2 vs 2.7 for the 3 groups, respectively, P <.05 for physicians vs social workers/administrators/others) and QI (mean 2.8 vs 2.8 vs 3.6 for the 3 groups, respectively, P <.001 for physicians and <.05 for nurse practitioners/nurses/physician assistants vs social workers/administrators/others) significantly more negatively than the social workers/administrators/others group. Professional groups’ scores did not differ significantly for other key constructs in QM (Table 3).
Table 3.
Key Barriers and Facilitators for Quality Assessment and Improvement in Palliative Care by Professional Role.a
| Mean (SD) |
||||
|---|---|---|---|---|
| Domain | Construct (on 5-Point Scale) | Physicians | Nurse Practitioners/Nurses/Physician Assistants | Others |
| Teamwork and Communication | Openness of Communication | 4.1 (0.7) | 4.0 (0.9) | 4.2 (0.6) |
| Perceptions of teamwork | 4.4 (0.7) | 4.1 (0.8)b | 4.5 (0.5) | |
| Quality Measurement | Focus on educational support | 1.9 (0.8)c | 2.2 (1.0) | 2.8 (1.2) |
| Program recognition | 2.7 (1.0) | 2.6 (0.9) | 3.1 (1.1) | |
| Program focus on rewards | 1.3 (0.5) | 1.6 (0.7) | 1.7 (0.9) | |
| Personal accountability | 2.3 (1.3) | 2.4 (1.2) | 2.7 (1.6) | |
| Quality Improvement (QI) | Program focus on QI | 4.4 (0.6) | 4.4 (0.6) | 4.7 (0.4) |
| Implementation Climate | 3.9 (0.5)c | 3.9 (0.7)b | 4.3 (0.5) | |
| Program focus on educational support | 2.8 (0.8)c | 2.8 (0.9)b | 3.6 (0.9) | |
| Program focus on selection of staff | 2.6 (1.0)c | 2.6 (0.9)b | 3.3 (1.1) | |
| Strategies used for QI | 2.4 (0.7)c | 2.5 (0.8)b | 3.3 (1.0) | |
| Creating systems, reducing barriers, organizing, setting goals and benchmarking, customizing for QI | 2.5 (1.0)c | 2.6 (1.0)b | 3.4 (0.9) | |
| Methods to evaluate or reduce negative consequences of QI | 1.9 (1.1)c | 2.3 (1.2) | 2.9 (1.1) | |
Abbreviations: QI, quality improvement; SD, standard deviation.
n = 100. Note that 3 respondents did not note their professional role on the survey. Multivariable regression controlled for time spent in program and Canadian versus US program and accounted for clustering by site.
P < .05.
P < .01 compared to the others reference group.
In almost all comparisons, physicians and nurse practitioners/nurses/physician assistants rated the other key barriers to implementing QI significantly more negatively than the social workers/administrators/others group. The largest differences were for the specific QI processes: Strategies used for QI (mean 2.4 vs 2.5 vs 3.3 for the 3 groups respectively, P = .01 for physicians and P < .05 for nurse practitioners/nurses/physician assistants vs social workers/administrators/others), Creating systems and reducing barriers (mean 2.5 vs 2.6 vs 3.4 for the 3 groups, respectively, P <.01 for physicians and P <.05 for nurse practitioners/nurses/physician assistants vs social workers/administrators/others), and Methods for evaluating or reducing consequences of QI (mean 1.9 vs 2.3 vs 2.9 for the 3 groups, respectively, P <.01 for physicians vs social workers/administrators/others; Table 3).
Discussion
In a survey of key barriers and facilitators to QM and QI in palliative care, we found that palliative care team members across 11 diverse sites rated the following constructs as key facilitators: teamwork, communication, program focus on QI, and implementation climate (valuing QI and understanding and effectively using QI methods). For QM, key barriers included lack of program recognition, accountability, or rewards. For QI, key barriers included specific processes of not selecting staff for QI; strategies, systems, organizing, setting goals and benchmarking, and customizing; and methods to reduce or evaluate negative consequences of QI. In general, similar constructs were rated less positively for QM than for QI. Perceptions of barriers did not differ significantly by leadership role but did significantly differ by professional role for some constructs, with physicians tending particularly to rate constructs more negatively than other team members.
Discussion groups that we conducted with many of the programs after survey completion19 provided explanations for some of these findings. Respondents agreed that insufficient resources were dedicated to QM and QI and that infrastructure, support, and integration in hiring and management were lacking. Many palliative care programs reported not measuring quality and those that were had concerns about the meaningfulness of those efforts for seriously ill patients. Although the survey addressed quality initiatives as program-level issues, respondents often conceptualized quality in different ways: as intrinsic to their everyday work or the operations of their program; as related to program resources (eg, sufficient number and disciplines of team members); or as access to care. For our finding of no differences in perceptions by leadership role, many programs described palliative care quality initiatives as grassroots rather than initiated by leadership. In addition, respondents (including many in leadership) often felt they were not aware of all that was happening around quality initiatives and improvement in their programs.
Although respondents rated implementation climate and focus on QI highly across sites, constructs related to the specific processes of QM and QI were generally rated as in need of improvement. Recent initiatives have highlighted other key issues in QM in palliative care, such as the challenges of implementing QM efficiently through electronic medical records6 and of integrating patient-reported outcome data critical to palliative care quality, such as function or needs.25 Our results align with research in other settings, which has also found that lack of incentives and resources and engaging leadership are key improvement barriers26 and that communication, teamwork, organizational readiness, and other infrastructure are key facilitators.27
Our study has both strengths and limitations. Although we used a convenience sample to select programs, we sampled a diverse set of palliative care programs and a variety of professional roles. Our participant response rate was high within sites (>80%), which supports the representativeness of our sample and low nonresponse bias at the participant level. Item nonresponse rates were generally low. Although we were able to evaluate a large number of different barriers and facilitators using validated surveys, we were limited to previously developed subscales, which did not address all potential constructs relevant to palliative care. The relatively small sample also limited the scope of analysis by less common professional team roles. In addition, we grouped social workers, pharmacists, chaplains, and administrators together in the analysis. These team members may have distinct experiences and perceptions, which should be addressed in future research.
Conclusions
Our findings highlight key features of palliative care programs that may facilitate the success of quality initiatives, as well as areas in need of improvement. Although teamwork, communication, and the climate and focus on QI were perceived as strong across programs, QM infrastructure was perceived negatively overall as were specific processes for QI. Key areas noted for additional resources across programs were the need for particular staff with focus or strength in quality, greater educational support, and strategies and structure for QI. More attention is needed to making QM relevant to palliative care and integrating quality processes into palliative care programs, to improve care and outcomes for people with serious, life-limiting illnesses and their families.
Acknowledgments
The authors would like to acknowledge the Department of Supportive Care Medicine, Moffitt Cancer Center.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article. Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R21CA197362. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr Walling was supported by the Cambia Health Foundation Sojourns Scholar Leadership Award. Dr Isenberg’s work was supported in part by the Canadian Institutes of Health Research #146181.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- 1.Kamal AH, Hanson LC, Casarett DJ, et al. The quality imperative for palliative care. J Pain Symptom Manage 2015;49(2): 243–253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kamal AH, Harrison KL, Bakitas M, et al. Improving the quality of palliative care through national and regional collaboration efforts. Cancer Control 2015;22(4):396–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sommerbakk R, Haugen DF, Tjora A, Kaasa S, Hjermstad MJ. Barriers to and facilitators for implementing quality improvements in palliative care—results from a qualitative interview study in Norway. BMC Palliat Care 2016;15:61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Dy SM, Walling AM, Mack JW, et al. Evaluating the quality of supportive oncology using patient-reported data. J Onc Pract 2014;10(4):e223–230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Leemans K, Van den Block L, Vander Stichele R, Francke AL, Deliens L, Cohen J. How to implement quality indicators successfully in palliative care services: perceptions of team members about facilitators of and barriers to implementation. Supp Care Cancer 2015;23(12):3503–3511. [DOI] [PubMed] [Google Scholar]
- 6.Curtis JR, Sathitratanacheewin S, Starks H, et al. Using electronic health records for quality measurement and accountability in care of the seriously ill: opportunities and challenges. J Palliat Med 2018;21(S2):S52–S60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lau BD, Aslakson RA, Wilson RF, et al. Methods for improving the quality of palliative care delivery: a systematic review. Am J Hosp Pall Care 2014;31(2):202–210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Henry M, Hudson Scholle S, Briefer French J. Accountability for the quality of care provided to people with serious illness. J Palliat Med 2018;21(S2): S68–S73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Davies HT, Nutley SM, Mannion R. Organisational culture and quality of health care. Qual Health Care 2000;9(2):111–119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zoutman DE, Ford BD. Quality improvement in hospitals: barriers and facilitators. Int J Health Care Qual Assur 2017;30(1): 16–24. [DOI] [PubMed] [Google Scholar]
- 11.Ghaferi AA, Dimick JB. Importance of teamwork, communication and culture on failure-to-rescue in the elderly. Br J Surg 2016;103(2):e47–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rosen MA, DiazGranados D, Dietz AS, et al. Teamwork in healthcare: key discoveries enabling safer, high-quality care. Am Psychol 2018;73(4):433–450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Solberg LI, Asche SE, Margolis KL, Whitebird RR. Measuring an organization’s ability to manage change: the change process capability questionnaire and its use for improving depression care. Am J Med Qual 2008;23(3):193–200. [DOI] [PubMed] [Google Scholar]
- 14.Shannon SE, Mitchell PH, Cain KC. Patients, nurses, and physicians have differing views of quality of critical care. J Nurs Scholarsh 2002;34(2):173–179. [DOI] [PubMed] [Google Scholar]
- 15.Hamric AB, Blackhall LJ. Nurse-physician perspectives on the care of dying patients in intensive care units: collaboration, moral distress, and ethical climate. Crit Care Med 2007;35(2):422–429. [DOI] [PubMed] [Google Scholar]
- 16.Singer SJ, Falwell A, Gaba DM, Baker LC. Patient safety climate in US hospitals: variation by management level. Medical Care 2008;46(11):1149–1156. [DOI] [PubMed] [Google Scholar]
- 17.Parand A, Burnett S, Benn J, Pinto A, Iskander S, Vincent C. The disparity of frontline clinical staff and managers’ perceptions of a quality and patient safety initiative. J Eval Clin Pract 2011;17(6): 1184–1190. [DOI] [PubMed] [Google Scholar]
- 18.Dy SM, Al Hamayel NA, Hannum SM, et al. A survey to evaluate facilitators and barriers to quality measurement and improvement: adapting tools for implementation research in palliative care programs. J Pain Symptom Manage 2017;54(6):806–814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Dy SM, Sharma R, Kuchinad K, et al. Evaluation of the measuring and improving quality in palliative care survey. J Onc Pract 2018;14(12):e834–e843. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rojas Smith L, Ashok M, Dy SM, Wines RC, Teixeira-Poit S. Contextual Frameworks for Research on the Implementation of Complex System Interventions Rockville, MD: Agency for Healthcare Research and Quality (US); 2014. [PubMed] [Google Scholar]
- 21.Nunally JC, Bernstein IH. Psychometric Theory 3rd ed. New York, NY: McGraw-Hill; 1994. [Google Scholar]
- 22.Andridge RR, Little RJ. A review of hot deck imputation for survey non-response. Int Stat Rev 2010;78(1):40–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rogers WH. Regression standard errors in clustered samples. Stata Tech Bull 1993;13(1):19–23. [Google Scholar]
- 24.Stata statistical software: Release 14 [computer program] 2015.
- 25.National Academies of Sciences, Engineering, and Medicine. Implementing Quality Measures for Accountability in Community-Based Care for People With Serious Illness: Proceedings of a Workshop; 2018; Washington, DC. [PubMed] [Google Scholar]
- 26.Pannick S, Sevdalis N, Athanasiou T. Beyond clinical engagement: a pragmatic model for quality improvement interventions, aligning clinical and managerial priorities. BMJ Quality Safety 2016;25(9):716–725. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Kampstra NA, Zipfel N, van der Nat PB, Westert GP, van der Wees PJ, Groenewoud AS. Health outcomes measurement and organizational readiness support quality improvement: a systematic review. BMC Health Serv Res 2018;18(1):1005. [DOI] [PMC free article] [PubMed] [Google Scholar]