Can the UK 24-item family satisfaction in the intensive care unit questionnaire be used to evaluate quality improvement strategies aimed at improving family satisfaction with the ICU? A qualitative study

Abstract

Background

The experiences and satisfaction of family members of patients are important indicators of healthcare quality in the intensive care unit. The family satisfaction in the intensive care unit (FS-ICU-24) questionnaire, developed in Canada and now validated in the UK, is becoming the gold standard measure to evaluate family members’ satisfaction with the intensive care unit. To inform future use of the UK FS-ICU-24 to evaluate quality improvement strategies aimed at improving family satisfaction with the intensive care unit, we set out to explore the extent to which the 24-scored items and domains of the UK FS-ICU-24 reflect common suggestions and priorities for quality improvement self-reported as important to family members in the UK.

Methods

Two data sources were thematically analysed – (1) open-text responses from family members who completed the UK FS-ICU-24 in a large observational cohort study; (2) a set of quality improvement activities generated by patients, family members and staff through experience-based co-design in a mixed-methods’ intensive care unit quality improvement study. Summarised themes were then mapped to the 24-scored items and domains of the UK FS-ICU-24 to assess coverage by the UK FS-ICU-24.

Results

We found a good degree of coverage between the topics and themes identified as important to family members with the 24-scored items and domains of the UK FS-ICU-24.

Conclusion

Our study confirms the face validity of the UK FS-ICU-24 and indicates that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the intensive care unit is appropriate.

Introduction

Providing high-quality, person-centred care for the critically ill is challenging. Specifically, those aspects of quality, such as the humanity and equity of care, which involve communication and shared decision making may present challenges that are unique to critical care. Evaluating the experience of critical care is even more challenging – while some patients may be conscious during part of their stay and able to reflect on their own satisfaction with care, the majority are not. Furthermore, approximately one-fifth of patients do not survive their stay in the intensive care unit (ICU) and hospital. The relationship, therefore, between staff and family members (defined as persons who have a close familial, social or emotional relationship with the patient) is pivotal and, for these reasons, evaluating the experience of family members and their satisfaction with care for their loved one is an important measure of quality in the ICU.

The family satisfaction in the intensive care unit questionnaire (FS-ICU-24)^1,2 is a self-completed, 24-scored item questionnaire used to measure family members’ satisfaction with the ICU. The FS-ICU-24 also includes three questions, at the end, inviting open-text responses. Originally developed with family member input in Canada, it is rapidly becoming the gold-standard.¹ The FS-ICU-24 was initially validated internationally^3,4 and has now been validated in a large UK study.^2,5 This resulted in a UK adaptation (UK FS-ICU-24) as psychometric assessment indicated three distinct domains (rather than two) – satisfaction with care, satisfaction with information and satisfaction with decision making.⁶ While aimed at family members, many of the questions elicit information about the care of the patient. Improvements in family satisfaction are, therefore, likely to reflect in improvements in quality of care and, thus, family satisfaction could be an important outcome for evaluating quality improvement strategies in the ICU.

Although the UK FS-ICU-24 has been shown to detect important differences between ICUs in terms of family satisfaction,² as with all questionnaires, there may be issues around whether the items included fully capture the issues which matter most to family members about the quality of care of their loved one. In order to inform future use of the UK FS-ICU-24 to evaluate quality improvement strategies aimed at improving family satisfaction with the ICU, we set out to assess the extent to which the 24-scored items reflected common suggestions and priorities for quality improvement self-reported by family members in the UK. In order to do this, we compared the 24-scored items and domains of the UK FS-ICU-24 with themes identified from open-text comments collected as part of a large evaluation of family satisfaction with NHS critical care services, and with a set of quality improvement activities identified in a national ICU quality improvement study.

Methods

Data sources

Two family member-generated data sources, pertaining to quality improvement suggestions and priorities, were used. The first of these derived from the family reported experiences evaluation (FREE) study^2,5 and the second from a national experience-based co-design (EBCD) study.^7,8

Briefly, FREE was a National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme funded (reference: 11/2003/56) observational cohort study of family satisfaction with NHS critical care services in 20 representative ICUs.^2,5

Briefly, EBCD was an NIHR HS&DR Programme funded (reference: 10/1009/14) mixed-methods study in two ICUs involving co-design activities with patients (where able) and family members and staff to identify and implement priorities for quality improvement.^7,8

FREE study

Raw UK FS-ICU-24 data were extracted for 1855 adult (aged 18 years or over) family members (self-identified as next of kin) of patients staying more than 48 h for the three questions inviting open-text response in the UK FS-ICU-24. These questions are

• Do you have any suggestions on how to make care provided in the ICU better?

• Do you have any comments on things we did well?

• Please add any comments or suggestions that you feel may be helpful to the staff of this ICU.

Anticipating that the content of open-text responses would vary by overall family satisfaction score and to address representativeness,^9,10 stratified sampling was used to ensure that responses from the full spectrum of overall level of satisfaction were included. Family members, overall satisfaction scores on the UK FS-ICU-24, were stratified into five groups, from which samples of open-text responses (of 50 characters or more) were drawn for thematic analysis.

EBCD study

Thirty-eight improvement activities were extracted from the report of the EBCD study.^7,8 The improvement activities were synthesised through implementation of an accelerated version of EBCD. EBCD is a quality improvement approach which involves patients and family members in describing their experiences of care and then working in partnership with staff to design and implement service changes. This started with inductive analysis of national patient and family member recorded interviews to identity important ‘touchpoints’ or moments of interaction with the service where things could be improved which were then edited into a ‘trigger’ film. The trigger film was then shown at co-design workshops with former patients, family members and staff, from which top local improvement priority areas were identified. The improvement priorities were then taken forward by working groups of patients, family members and staff, which were formed to plan and implement specific improvements, resulting in the 38 improvement activities.

Thematic analysis

Based on a framework approach, first author SL (independent of both the FREE and EBCD studies), with supervision from co-author LL, undertook an inductive thematic analysis of the UK FS-ICU-24 open-text responses.¹¹ What is meant by inductive, in this case, is that analysis started from the family members’ own words in the open-text responses, looking for themes and patterns, rather than a deductive approach starting with the existing items and domains of the UK FS-ICU-24.

Open-text responses from each UK FS-ICU-24 respondent were read for immersion. An initial coding framework was developed to capture themes and developed further from reading the set of EBCD improvement activities (see above).⁷ The framework consisted of overarching topics, each containing several themes. The data from both data sources were imported into the qualitative analysis software package QSR NVivo version 10 and coded systematically to the framework. The framework was refined and updated during the coding period. After coding was completed, the responses in each of the codes were summarised, with illustrations from the raw data.

Summarised themes, of the suggestions and priorities for quality improvement strategies aimed at improving family satisfaction with the ICU, were then mapped to the 24-scored items of the UK FS-ICU-24 (see Supplementary File – Table 1) to assess coverage by the UK FS-ICU-24 (face validity) by the first author (SL) and discussed with co-authors to reach consensus.

Results

Sampling for thematic analysis

UK FS-ICU-24 responses from the 1855 family members were stratified into five groups by overall satisfaction score (0–100) at score intervals (0 to <20, 20 to <40, 40 to <60, 60 to <80, 80 to 100; Figure 1). Most responses were in Group 5 (63%) with overall satisfaction scores of 80 and above. Less than 1% of responses were in Group 1 with overall satisfaction scores below 20. A purposive sample was then selected, including all family members in Groups 1 (n = 11) and 2 (n = 49) and, from each of Groups 3, 4 and 5, n = 60 randomly selected family members. Random selection used a pseudorandom number generator. The open-text responses included in the thematic analysis derived from the 240 family members.

Figure 1.

Distribution of overall satisfaction score by Group for the 1855 family members.

Overall satisfaction score UK FS-ICU-24: Group 1, 0 to <20; Group 2, 20 to <40; Group 3, 40 to <60; Group 4, 60 to <80 and Group 5, 80 to 100.

Thematic analysis results

The final coding framework, capturing all the topics and themes emerging from both the 240 open-text responses from family members and from the set of 38 improvement activities, is presented in Table 1. The framework comprised six topics, five topics of which related to care in the ICU, each containing several themes.

Table 1.

Coding framework –Topics and themes emerging from thematic analysis of two family-member generated data sources.

Topic	Theme
The physical environment	Improving orientation and reducing boredom
	Importance of rest and sleep
	Quality of the waiting room
Care of patient and person	Knowing the person
	Preserving dignity
	Coping with hallucination and assisted ventilation
Care of family members	Improving contact with sick relatives
	Orientation for family members
	Emotional support from staff
	Privacy of grieving
Communication and decision making	Improving family contact with doctors
	Improving day-to-day communication
	Improving patient communication
Care of staff	Staff training
	Staff support
Hospital journey	Leaving intensive care unit
	Communication with other departments

Illustrations of the inductively derived thematic analysis of the 240 open-text responses, in relation to selected topics and themes identified, are set out below.

The physical environment

Importance of rest and sleep

Rest and sleep were considered important for recovery. Family members were sometimes critical of noise levels in the unit and attempts to move patients during the night. Bright lights were also highlighted as a barrier, and eye covers suggested.

(FSSD15—160) My mother asked for an eye cover so she could sleep in the bright rooms she had to ask it wasn't offered maybe it should be part of your introduction to the patient to discuss this to make them as comfortable as possible, as quickly as possible in an [intimidating] atmosphere of medics to patients.

Care of patient and person

Knowing the person

Patients in the ICU are often unconscious or unable to communicate, making it particularly difficult for healthcare professionals to learn information about them: information that makes them a person, rather than just another patient.¹² Staff who knew the patient and communicated with them as such were appreciated by relatives.

(FSSD17—109) Some of the nurses kept calling my mother (the patient) 'Dear' as in “C'mon Dear” or “Take this dear”! I felt she would have responded better by being called her name.

(FSSD16—127) Certain nurses took the time to really get to know my mum and to find out about her - this really showed just how much they cared.

Care of family members

Improving contact with sick relatives

Access to their relative was often an issue. Doors to ICU are locked to protect vulnerable patients and their families, and staff members control entry. Some family members commented that they were left waiting, an inconvenience repeated over many visits. In addition, this was a waste of precious visiting hours, which family members already found restrictive. Long waits to see a family member were additionally distressing if no information was given as to the reason for the delay.

(FSSD15—63) Keep visitors better informed when delays in being able to see family member. I had to wait 3hrs on ICU—I was convinced he was dying.

This also relates to the communication topic. Some respondents resented staff asking them to leave the bedside during visiting hours for personal cares or rounds, and some wished to be more involved in their relative’s physical care.

(FSSD15—9) More involvement of close family— some family who care for their relatives when they were at home would like to be involved with the daily care whilst in ITU … Patients in ITU would often find it comforting to have a family member helping with their care and being with them more.

Orientation for family members

Family members valued regular progress updates from staff. Although family members had many complex information needs, such as whether or not their loved one would ever regain their former level of function, they also had simpler information needs such as where to find refreshments, how to get subsidies for parking, how to stay updated outside the hospital and how to help physically care for their relative. Though these may seem minor, they cause extra strain on family members already dealing with distress. Having limited or no experience with the hospital system, family members may not know or might be too shy to ask about amenities such as free parking and would prefer to have the information volunteered.

(FSSD17—85) We were not told anything about parking my mother was in ICU for 4 days and we weren’t told anything about free parking.

(FSSD17—88) Would have liked more information about my husband’s condition. This was offered when I asked but not volunteered very often.

Communication and decision making

Improving contact with doctors

A common complaint from family member respondents was that they found it difficult to contact doctors, and thus were excluded from the decision-making process and frustrated by a lack of information about their loved one.

(FSSD15—65) I was never briefed by a doctor. I had no time to ask questions about her care or progress from doctors. This continued till she was discharged. Nurses answered questions where they could. I never felt involved in her care …

(FSSD15—79) Communication with ITU doctors could be improved. Ward rounds and decisions are made in the absence of relatives. Either too early in the morning or too late after visiting time.

Care of staff

Staff support

Other than to comment about their communication skills and attentiveness, respondents did not discuss staff in many other ways. Some respondents did describe their appreciation of the extremely hardworking staff and suggested improving working conditions and paying them more. Staff being busy was recognised as a reason for lack of attention.

Comparison with UK FS-ICU-24 items

Table 2 presents an overview of how we mapped the topics and themes, identified from the qualitative analysis, onto the 24-scored items and domains of the UK FS-ICU-24.

Table 2.

Topics, themes, results of UK FS-ICU-24 item-level comparisons and examples of quality improvement activities which may improve family satisfaction with the ICU.

Topic	Theme	UK FS-ICU-24 domain coverage			Examples of quality improvement activitiesa
		Satisfaction with care	Satisfaction with information	Satisfaction with decision making
Physical environment	Improving orientation and reducing boredom	Q12, Q13			Sourcing appropriate clocks Sourcing quieter bins Change telephone ring tone
	Importance of rest and sleep	Q12, Q13
	Quality of waiting room	Q13
Care of patient and person	Knowing the person	Q1, Q9, Q11	Q1, Q2, Q3, Q4		Use of a sign to indicate personal care is taking place Develop information for staff and patients on the impact and experience of hallucinations
	Preserving dignity	Q1, Q9
	Coping with hallucination and assisted ventilation	Q1, Q2, Q3, Q4, Q9, Q11
Care of family members	Improving contact with sick loved one	Q5, Q7			Promoting the involvement of families in personal care via team news Answering the ICU buzzer promptly during visiting hours Doctors to wear name badges Relatives information booklet to be updated with key contact numbers Orientation and ‘house rules’ booklet to be updated with key information
	Orientation for family members		Q2, Q4, Q5, Q6
	Emotional support from staff	Q5, Q6
	Privacy of grieving	Q5, Q6, Q7
Communication and decision making	Improving family contact with doctors		Q1, Q2, Q3, Q4, Q5, Q6	Q1, Q2, Q3, Q4	Discharge summary redesigned with patients and professionals from all teams involved Trialing set times for family meetings Training nurses in communication skills Identify suitable tablet computer applications to assist ventilated patients to communicate
	Improving day-to-day communication	Q5, Q7	Q1, Q2, Q3, Q4, Q5, Q6	Q1, Q2, Q3, Q4
	Improving patient communication	Q1
Care of staff	Staff training	Q2, Q3, Q4, Q8, Q9, Q10, Q11			Delirium working group to be created Nurses in charge to be reminded to offer support to juniors Nurses to be informed of support mechanisms via critical care news
	Staff support	Q1, Q5, Q7, Q8

Q denotes ‘question’ within the domain on the UK FS-ICU-24 (see Supplementary File – Table 1).

^aDerived from Locock et al.⁶

We found a good degree of coverage between the themes identified as important to family members, from our thematic analysis, with the 24-scored items and domains of the UK FS-ICU-24 – both in terms of specificity and in the number of questions relating to the topics – particularly for the ‘care of patient and person’, ‘care of family members’ and ‘communication and decision-making’ topics.

Discussion

Principal finding

Our study found that the topics and themes, identified as important by family members of ICU patients in the UK, are represented by the 24-scored items and domains of the UK FS-ICU-24, confirming its face validity and indicating that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the ICU is appropriate.

Strengths

A major strength of this study is the use of two data sources generated from individual family members with direct experience of ICU in the UK. It provides an inductively derived framework which can be used to underpin future quality improvement projects in the ICU, grounded in lived experience.

The topics and themes identified, through thematic analysis in our study, were similar to those identified in a qualitative study carried out in Canada,¹³ suggesting that family members in the UK may have experiences that resonate in other healthcare systems. However, to our knowledge, ours is the only study to directly map resultant topics and themes onto the items and domains of the UK FS-ICU-24.

Limitations

The list of EBCD improvement activities, used as a data source in our study, may not reflect an exhaustive list of all potentially relevant quality improvement activities. However, the thematic analysis of the open-text responses from the FREE study highlighted similar topics and themes. Whether, or not, these topics and themes reflect the suggestions and priorities in future studies is difficult to determine given the inherent variability in co-designed quality improvement efforts. There will always be limits as to how far in advance we can anticipate exactly what needs to be measured. In some cases, the mapping of topics and themes onto the 24-scored items and domains of the UK FS-ICU-24 may be imperfect due to the subjective nature of some of the 24-scored items of the UK FS-ICU-24.

The UK adaptation of the FS-ICU-24, as a questionnaire to assess family satisfaction with the ICU, has three conceptual domains which assess satisfaction with care, information and with decision making – all within the ICU. As may be expected, no questions are included to directly evaluate the wider ‘hospital journey’ (e.g. family members’ experience of transfer or discharge, appropriate step-down care on other wards, etc.). As this topic relates to care received after the ICU, it is unlikely that quality improvement activities solely conducted within the ICU will change these family members’ experiences. This topic may also be influenced by quality improvement activities conducted elsewhere in the hospital, noting that family members have many experiences in other parts of the hospital or community that they relate and deem relevant to the ICU.

Conclusion

Based on qualitative analysis, our study found that the topics and themes identified as important by family members of ICU patients in the UK were represented by the UK FS-ICU-24 questionnaire. These results indicate that its inclusion as an outcome measure for evaluating quality improvement strategies aimed at improving family satisfaction with the ICU is appropriate.