Experiences of Participants in a Collaborative to Develop Performance Measures for Hospice Care

Abstract

Background

There has been increasing attention paid to quality assessment in hospice as the industry has grown and diversified. In response, policymakers have called for standardized approaches to monitoring hospice quality. The experiences of a set of hospices involved with the National Association for Home Care & Hospice (NAHC) Quality Assessment and Performance Improvement Collaborative, which was designed to test the use of a standardized patient symptom assessment tool as an exemplar of efforts to standardize symptom assessment in hospice, were examined.

Methods

Transcripts of semistructured telephone interviews with 24 individuals from eight of the nine participating hospices, which were conducted in July–August 2007, were analyzed using the constant comparative method. Interview questions centered on the collaborative’s impact on the process of quality assessment at the hospices.

Findings

The collaborative activities influenced several hospices’ quality assessment processes, most beneficially by prompting greater attention to quality assessment processes, by promoting the adoption of quality assessment tools, and by creating a supportive community. Challenges included the limits of distance communication technology, participants’ misconceptions about data to be received, and potential lack of support and resources for quality assessment.

Conclusions

The experiences of the participating hospices in the NAHC collaborative are intended to inform the design of future interorganizational learning efforts to promote quality assessment initiatives within hospice settings. Future hospice collaboratives should use multiple methods of communication to build a close participant network and be clear about collaborative goals and participant expectations and about the reciprocal relationship of the collaborative and the participants.

There has been increasing attention to quality assessment in hospice as the number of hospices has more than doubled in the last 15 years.¹ Partly because of the growing diversity of organizations providing hospice, policymakers have called for standardized approaches to monitoring hospice quality. As of December 2008, hospices must perform routine quality assessment under the final Medicare Hospice Conditions of Participation.² To prepare for this requirement, nine hospices participated in a national collaborative project sponsored by the National Association for Home Care & Hospice (NAHC). Unlike most collaboratives, which have been used to facilitate quality improvement in organizations,^3,4 the NAHC collaborative was designed to develop and test quality measures that could be used in future quality assessment and improvement programs.

Previous studies of quality improvement collaboratives have focused on acute and chronic care sectors,^5,6 with only two studies examining the effectiveness of collaboratives within the hospice setting.^7,8 These initiatives focused primarily on improving pain and symptom management, and the broader effects and experiences of hospice collaboratives are largely unknown. It remains unclear as to why hospices join collaboratives and what features of collaboratives and of hospices help and hinder meeting objectives.

To explore this issue, we examined the experiences of participants in the NAHC collaborative, which was designed to test the use of a standardized patient symptom assessment tool, the Edmonton Symptom Assessment System (ESAS),⁹ as an exemplar of efforts to standardize symptom assessment in hospice. We report the outcome of this test elsewhere.¹⁰ Our research question in the present study was how participating hospices experienced the NAHC collaborative. We sought to understand why hospices joined the collaborative and their views about the impact of the collaborative on the process of quality assessment at their hospices. We also explored features of the collaborative and of the hospices that participants viewed as more or less effective in promoting changes in organizational practices concerning quality assessment. Understanding the experiences of hospices within such a collaborative can help inform the design of future interorganizational learning efforts to promote quality assessment initiatives within hospice settings.

Methods

COLLABORATIVE DESIGN AND PARTICIPANTS

In Fall 2006, a letter inviting participation in the collaborative was sent to a purposive sample of 11 NAHC-member hospices by NAHC’s vice president for hospice programs. The invitation stated that the purpose of the collaborative was to develop methods and performance measures for a hospice quality assessment and performance improvement program. Nine hospices chose to participate. The sample varied in ownership type, size, experience, and location. Participating hospices were given a manual explaining the purpose and procedures of the collaborative, the ESAS, the performance measures (to address the priority symptoms of pain, shortness of breath, and constipation), and the Excel-based data collection tool (to track and report symptom ratings).¹¹

For the duration of the collaborative—November 2006–August 2007—conference calls with collaborative organizers and hospices occurred every two to three weeks to obtain feedback, troubleshoot, and refine measures and the data collection tool. Throughout the collaborative, feedback on implementation was used to make modifications for further testing. At the end of the project, the collaborative summarized individual and aggregate data on symptom management for each hospice. The procedures of the collaborative have been described in detail elsewhere.¹⁰

PARTICIPANT INTERVIEWS

We collected data by recording semistructured telephone interviews with participants in July–August 2007. Interview questions centered on the collaborative’s impact on the process of quality assessment at the hospices. Specific topics were as follows:

1. How the ESAS was incorporated into hospices’ usual processes and plans for future quality assessment efforts

2. What worked or did not work in implementing the work of the collaborative

3. Organizational barriers and facilitators to participation

4. Strategies to overcome barriers

5. Suggestions for future collaborative efforts with hospices

We also collected data on participants’ roles in their hospices and in the collaborative and on their motivation for joining.

Of the 39 participants in the collaborative, 24 individuals from eight of the nine participating hospices agreed to be interviewed (response rate, 62%). Eleven individuals (46%) participated in a group interview (versus an individual interview) with one or two additional participants. The remaining hospice did not participate in interviews because of a possible conflict of interest. Of the 15 individuals who did not participate, 4 reported that they were not adequately involved to respond, and 1 was no longer employed at the hospice. The other 10 individuals were unable to be reached after multiple attempts. Nonparticipants did not differ from participants in terms of collaborative role or hospice characteristics.

DATA ANALYSIS

Digital recordings were transcribed by a professional transcriptionist and coded using Atlas.ti qualitative software (Scientific Software; Berlin). We performed line-by-line review of transcripts using the constant comparative method of qualitative data analysis.^12–14 Two authors [D.S.-G., E.C.] independently read the first two transcripts and assigned a descriptive phrase or code to key concepts. These authors then compared codes and developed a basic coding scheme. This process continued iteratively with subsequent transcripts. The code key was expanded and refined as new concepts were identified and/or concepts were consolidated into larger conceptual categories. Differences in coding were resolved through negotiated consensus.

The final code structure was reviewed by all authors, applied to all transcripts, and then summarized into the following four themes that characterized the data:

1. Reasons for joining the collaborative

2. Perceived impact of the collaborative

3. More and less helpful collaborative activities

4. Organizational characteristics perceived to influence impact of the collaborative

Findings

PARTICIPANT CHARACTERISTICS

As shown in Table 1 (page 40), the 24 participants included 6 administrators, 6 quality assessment directors or quality assessment nurses, 3 performance improvement coordinators, 4 team leaders or program supervisors, 2 staff nurses, 2 hospice presidents or chief executives, and 1 chaplain. Participants were involved with various activities of the collaborative, including team supervision, participation in conference calls, data entry, graphing and reviewing data, and staff training.

Table 1.

Hospice and Participant Characteristics^*

Hospice Characteristics (N = 9)
Ownership Type	n	%
Nonprofit	6	67
For-profit	3	33
Structure
HHA–based	3	33
Free–standing	3	33
Hospital-based	1	11
Hospital-affiliated	1	11
Corporate chain	1	11
Experience
New (2006)	1	11
Established 1974–2000	8	89
Location
Primarily rural area	2	22
Primarily urban area	3	33
Serve urban and rural areas	4	44
Participant Characteristics (N = 24)
Role in Hospice	n	%
Administrator	6	25
Quality assessment director/quality assessment nurse	6	25
Team leader/program supervisor	4	17
Performance improvement coordinator	3	13
Staff nurse	2	8
Hospice president/chief executive	2	8
Chaplain	1	4

^*HHA, home health agency.

REASONS FOR JOINING THE COLLABORATIVE

Participants described a range of reasons for joining the collaborative; one participant cited interest in ensuring high quality of care:

Just knowing that there is leadership and guidance out there trying to research how we can best serve the needs of the patients … [it is] nice to know that we are looking at overall quality of care for everybody regardless of payment source, and dignity and comfort is what hospice all about.^*

Another reason cited was that participants wanted to be on the cutting edge of hospice industry initiatives and viewed participation in the collaborative as a way of being involved in innovations in quality assessment:

We wanted to participate in a groundbreaking effort to try to quantify hospice and what we do, and those quality indicators have measurable outcomes to show what we do. [7]

Participants viewed participation as a way for their hospices to become more prepared for the future:

Our outlook generally is we like to be proactive. We like to have as much opportunity [as possible] to prepare for changes that are coming so we don’t have to have kneejerk reactions.

A common motive for participating was the desire to influence the hospice industry’s process of identifying, standardizing, and benchmarking quality indicators:

We are struggling with ways that we can improve our quality, and that was our hope, that we would be able to get a viable tool that would help our nurses standardize some of their care.

PERCEIVED IMPACT OF THE COLLABORATIVE

In several hospices, the collaborative activities influenced the hospice’s process of quality monitoring more generally. Participant views were diverse in terms of the nature and degree of the collaborative’s influence on the hospice. Participants from five hospices described substantial impact of the collaborative on their hospice’s approach to quality monitoring. In these hospices, participation prompted greater formality and attention to quality assessment, including adopting for routine practice the tools and protocols for patient assessment that were tested. In addition, participants in these hospices reported that the collaborative renewed and/or focused attention on quality, not just as a mandate, but as a value:

It is not just about dotting the i’s and crossing the t’s. It is not just about being in compliance. It is … are you documenting improvements in quality of life? Are you documenting improvements in comfort levels? Is [quality] really coming across?

In the remaining three hospices, participants reported more modest impacts on the general approach to quality monitoring. The participants indicated that although the collaborative made them think more about quality assessment issues, there was limited adoption of the tools or processes that they tested due to the lack of concrete data on the impact of their use on patient outcomes:

I don’t think that in the project we had enough interpretation of results to show [staff] that, “Gee, this really makes a difference in practice,” and so you can cut down on your visits or you can make more defined visits based on the tool.

This participant summarized what she would have liked to have gained from the collaborative, including final data and practice implications:

We got a little taste of what [the system] is going to be like, but … we need the next step to what other people are doing, and this works really well, we have had improvement in our scores because we are doing better … symptom assessment, and we don’t have that piece of it yet.

Another participant explained:

I think we are probably waiting more for results to come back before we would initiate any major changes … we are talking about 750 employees, nurses directly providing care, so any time we make changes we are very cautious … until we have some real proof that this is going to make a difference for our patients.

Another reason for nonadoption of the standardized instruments was that some hospices already had formal quality assessment policies and procedures in place and did not see a need for another approach:

I did see that some of the questions on the questionnaire were things that we were already doing in our regular nursing assessment… . So the actual assessment was good, [but] like I said, I felt we were already doing it.

MORE/LESS HELPFUL COLLABORATIVE ACTIVITIES

Participants noted collaborative activities that were helpful with implementing quality assessment activities, such as the networking opportunity through the conference calls. Those calls provided a forum to bring questions to the group and receive feedback, to share experiences and struggles, to receive group support, and to receive technical assistance and mentoring from the collaborative leaders and from more established hospices. The network was viewed as helpful to problem solving, learning across organizations, and feeling less alone through setbacks in making changes:

Listening to people from all over the nation and their struggles, which were the same as our struggles, was very comforting to me to say, “Okay, it is not just nurses from [name of hospice] that we can’t get buy-in from, it is all over,” and listening to what they had done with their staff that made a difference helped.

Some aspects of the collaborative activities were noted as being less helpful, for example, that calls were too lengthy and lacking in sufficient discussion to resolve complex issues:

We just didn’t get to do a lot … with that many people involved and then discussing individual issues that they had with the tool or how to document … I guess I just don’t feel like we were [able to] to come to a solidified conclusion of what we were doing and where to go from here… . We wanted some tangibles to come up with to change our practice and I’m not sure we ever got that far.

Some participants described the limitations of distance-learning technology in creating a feeling of partnership among hospices. Despite recognition that conference calls reduced travel costs and time, participants noted the limitations of this approach for learning and deep problem solving:

We were limited a lot by the fact that all of the meetings were on the telephone… . You don’t have the benefit of body language and eye contact… . I just find the telephone tough to have a real good dialogue about something.

Some participants reported that they would have liked to have had at least one in-person meeting; others suggested using technology such as Webcasts, Internet chat sessions, and video technology to communicate.

Participants expressed a keen interest in receiving project data. For example, reports that tracked symptom assessment data were reported as helpful because these results could be taken back to nursing staff to demonstrate the effects of quality assessment processes on patient care:

They have seen the benefit of [data tracking]. When we discuss it at care conferences, they can see subtle changes in the elevation if we are noticing shortness of breath, a [slow] incline over the past couple of weeks. We are catching on to that.”

Such reports could also highlight issues that needed attention and provide staff training:

If there is a big change in the total distress score, then I discuss it with the nurse, and make sure that she is aware.

Although the project data was viewed as helpful, some participants described wanting more data that benchmarked their quality against other hospices’ quality on the same measures:

At the beginning of this we were going to try to do data analysis based on the aggregate of the hospices, but then do an analysis which was going to be given to each individual hospice showing us where we stand… . [This] would be extremely helpful … [we] want the next step [to] make something a little bit more solidified.

Such comments revealed misperceptions about the objectives of the collaborative, which focused on developing and testing instruments rather than reporting data trends in performance. Paradoxically, participants felt that there was an inadequacy of data when the data from the project was generated by them; that is, some participants misunderstood that the amount of data they would receive was contingent on the amount of data they submitted. This misunderstanding about the purpose of the collaborative being to develop and test quality measures for future quality assessment and improvement programs was expressed by a participant:

At one point, probably about halfway or three-quarters through, the tool was changed, revised again, and at that point we lost all our trend data and that was a vital piece of information to let me know where my team is at with looking at all these symptoms.

This participant described the critical function of revision that is inherent to any testing process; however, the remark also highlights the misconception among some participants that the data might have been used for benchmarking, which was not the purpose of the collaborative.

ORGANIZATIONAL CHARACTERISTICS

Organizational characteristics were identified by participants as influencing the collaborative’s impact on participating hospices. Hospice administrators’ support of staff participating in the collaborative enabled staff to focus on the collaborative-related work, problem-solve, and maintain a positive attitude. A participant explained:

The administration very much wanted us to participate in this and so they were very supportive with everything that I needed to get done, and also supporting me with working with the nurses to have [the work of the Collaborative] done.

Some hospices had quality assessment resources, such as a quality department, manuals, and/or staff in-services, in place before the collaborative. These resources complemented those provided by the collaborative and helped to ensure that staff were educated about the importance of the collaborative-related quality assessment activities. Lack of resources that hindered the work related to the collaborative included staff shortages, absence of in-house technical support, no quality department, and lack of time to complete data entry, especially if the hospice used paper versus electronic documentation.

Discussion

The NAHC collaborative engaged nine hospices in a test of instruments and procedures for symptom assessment and data collection so that their experiences and ongoing feedback could help shape the development of quality measures and tools. Participants identified additional benefits of participating in the collaborative. For several hospices, the collaborative prompted greater formality and attention to quality assessment processes, promoted the adoption of quality assessment tools, and created a supportive community for problem-solving and organizational learning. These benefits seemed most apparent in hospices where there was senior management support for the improvement efforts and adequate resources to ensure implementation of piloted tools and procedures. These hospices viewed the new tools and procedures as complementing their current practices and as helpful for quality assessment. In contrast, other participants did not adopt the piloted tools and procedures, citing inadequate resources or redundant processes.

Although the overarching goal of the NAHC collaborative was met, the experiences of participants revealed critical aspects of the collaborative approach that could be helpful in future collaborative designs (Table 2, page 42). First, this collaborative experience demonstrated the limitations of relying solely on phone and e-mail communication for complex tasks such as developing and piloting quality assessment methods. Relationship building and ability to discuss complex issues are of particular importance in the hospice setting, where subject matter is both intense and sensitive. Use of multiple methods to build a closer network, such as videoconferencing, expanded in-person meetings, or site visits, and more one-on-one contact, would likely be beneficial and worth the investment of time and funds.

Table 2.

Recommendations for Collaboratives

Collaborative Aim	Recommendations to Work Toward Collaborative Aims
Build a close network.	■ Use multiple methods of communication, including in-person meetings  and site visits, videoconferencing, teleconferences, and e-mail.
Be clear about collaborative goals.	■ Initially explain and periodically reiterate collaborative goals.
	■ Explore and reset participants’ expectations as needed.
Be clear about participants’ and collaborative’s responsibilities.	■ Review participants’ individual responsibilities.
	■ Emphasize reciprocal nature of the participant-collaborative relationship.

Second, collaborative organizers should pay close attention to managing participants’ expectations about the data that they will receive. Some of our participants misconceived the purpose of the collaborative and the data. They expressed expectations that were more in line with a quality improvement collaborative, such as wanting more benchmarking data over a longer duration before implementing organizational changes. This issue highlights the importance of reiterating collaborative goals, which participants may not understand initially if they have their own goals in mind. An effective general approach would be to manage participants’ expectations by asking about, identifying, and resetting expectations as needed.

Finally, persons using the collaborative approach should recognize a paradox of collaboratives, which is that participants expect the group to produce the data but may not realize that they must contribute to do so. Again, some participants misunderstood how important it was for them to collect and submit data to the collaborative to receive aggregate data for bench-marking purposes. Therefore, although collaborative participants must function as a group for networking and learning purposes, clarity about individual contributions is important, and the reciprocal nature of this relationship should periodically be re-emphasized. The challenges of defining participants’ roles and expectations of the collaborative and of collecting data have been reported in other collaboratives—and have been cited as key to successful collaboratives.¹⁵

Some limitations to our findings should be noted. The collaborative developed and tested the quality measures in an exploratory context. Therefore, some of the collaborative activities duplicated hospices’ preexisting quality assessment processes, which may have affected participants’ time and willingness to participate. In addition, although 15 participants were unable to be interviewed, at least one individual from eight of the nine hospices was interviewed.

IMPLICATIONS

This article demonstrates how the intent of the NAHC collaborative actually played out in practice with hospice organizations. Our findings provide insight into implementation efforts and the discrepancy between organizers’ intentions and actual practices at the hospice organizations. Given the increasing use of collaboratives to foster uptake of best practices,^4,16 the reported implementation challenges are important for researchers, policymakers, and practitioners to consider, as anticipating potentially problematic issues in implementation is an important strategy for those attempting organizational change.

Quality assessment will continue to be a focus in hospice organizational planning, given hospices’ desire to provide quality care and meet the expectations of regulatory agencies. Recent health care reform legislation^17,18 requires publication of quality measures for hospices by October 2012 to enable reporting beginning in October 2013. The legislation represents major changes that hospices must integrate into their organizational practice. Despite some limitations, use of a collaborative approach to organizational learning can be an effective method to provide support to hospices to test new processes and to work towards organizational change related to quality assessment.