Measuring the Quality of Dying and Death in the Pediatric Intensive Care Setting: The Clinician PICU-QODD

Abstract

Context

In the pediatric intensive care setting, an accurate measure of the dying and death experience holds promise for illuminating how critical care nurses, physicians, and allied psychosocial staff can better manage end-of-life care for the benefit of children and their families, as well as the caregivers.

Objectives

To assess the reliability and validity of a clinician measure of the quality of dying and death (PICU-QODD-20) in the pediatric intensive care setting.

Methods

In a retrospective cohort study, five types of clinicians (primary nurse, bedside nurse, attending physician, and the psychosocial clinician and critical care fellow most involved in the case) were asked to complete a survey for each of the 94 children who died over a 12-month period in the pediatric intensive care units (PICUs) of two children’s hospitals in the northeast U.S. Analyses were conducted within type of clinician.

Results

In total, 300 surveys were completed by 159 clinicians. Standard item analyses and substantive review led to the selection of 20 items for inclusion in the PICU-QODD-20. Cronbach’s alpha for the PICU-QODD-20 ranged from 0.891 for bedside nurses to 0.959 for attending physicians. For each type of clinician, the PICU-QODD-20 was significantly correlated with the quality of end-of-life care and with meeting the family’s needs. In addition, when patient/family or team barriers were encountered, the PICU-QODD-20 score tended to be significantly lower than for cases in which the barrier was not encountered.

Conclusion

The PICU-QODD-20 shows promise as a valid and reliable measure of the quality of dying and death in pediatric intensive care.

Introduction

Over the last several decades, both researchers and practitioners have made a concerted effort to improve care at the end of life for adults (1–7) and, in the last decade, children (8–12). These efforts have begun to distinguish three separate but inter-connected constructs: quality of end-of-life care, quality of life at the end of life, and quality of the dying and death experience (13, 14). Quality of end-of-life care typically refers to an assessment of the actions taken by clinicians. In contrast, both quality of life at the end of life and quality of dying and death encompass the experience of the patient. Quality of life at the end of life refers to the functional status and fulfillment of needs essential to living in the face of terminal illness. Special tools to measure this construct among terminally ill adult patients have been developed (15–17). Quality of dying and death is a construct that focuses more specifically on the final stage of illness, just prior to death, and is an attempt to measure the degree to which a “good death” has been achieved. This paper focuses on the latter construct.

The quality of dying and death has received considerable attention in the adult intensive care setting. A tool developed and validated by Curtis and colleagues to measure the quality of dying and death among adults has been adapted for the intensive care setting (18–23). Regardless of the setting in which death occurred, Curtis defined the quality of dying and death for adults in terms of the degree to which “the preferences of the adult patient as reported by others after his or her death” were met (18). Central to this definition is the assumption that an adult has considered his or her death, and has preferences and ideas about his or her last few days and hours. One feature of this model is that it strongly privileges the experience of the patient, and relies upon the family and clinicians primarily as surrogate reporters on this outcome. Whether the preferences and/or needs of the family are met is relevant primarily to the extent that they are important to the patient, which is of course often the case.

The death of a child is qualitatively different than the death of an adult. Whereas the death of an adult may sometimes be understood as the natural conclusion of a long and completed life, this is, by definition, never the case for children. In addition, the death of a child, regardless of the cause or the location, often profoundly impacts the family for the rest of their lives (24). In the pediatric intensive care unit (PICU), the experience of dying and death varies according to the age of the child. On one end of the continuum, for example, are small children or babies who have never experienced or expressed preferences, and at the other end are adolescents or young adults who may have had detailed conversations with their family and caregivers about their hopes and expectations during the dying process. Since typically those who experience the death of a child most acutely in the pediatric setting are the parents and/or family of the child who is dying -- whose memories of their family’s and child’s experience will last forever -- we defined the quality of dying and death in the pediatric setting as the degree to which the hopes and priorities of the patient and/or the family for the process of dying and the moment of death are respected and met.

The pediatric Quality of Dying and Death questionnaire (QODD), therefore, takes a somewhat more inclusive and holistic approach than the adult QODD, because it ascribes intrinsic value to the experience of the family members, rather than merely the derivative value of this experience as reflected in the adult QODD. In addition to their role as reporters for the experiences of the patient, in the pediatric QODD the parents are asked to integrate and synthesize the known or imagined hopes and expectations of the patient as well as their own hopes and priorities into their responses to the survey items. This makes the pediatric model conceptually more complex, but may make it more empirically accurate and normatively defensible than the adult model.

These differences in the pediatric context necessitated the development of an instrument, specific to pediatric patients and their families. To do so, we began by seeking input from intensive care clinicians who care for dying children and their families and parents of children who had died in intensive care. Focus groups with clinicians and qualitative interviews with parents (29), together with data from a targeted literature review, provided valuable insight into the dying and death experience in PICUs. The themes generated from these data were compared and used to generate survey items for both clinician and parent measures of the quality of the dying and death experience. The development and assessment of the parent instrument will be reported on in a separate publication.

The availability of a valid and reliable clinician-completed measure of the quality of dying and death in the PICU is crucial. The logistical difficulties of obtaining data from parents and family members are evident in the low response rates that are characteristic of such research (25, 26). Unknown differences in the quality of the dying and death experience between parents who do, and parents who do not, complete a measure complicate the interpretation of parent-based ratings. Consequently, the availability of a validated measure for a set of readily accessible proxy respondents – the clinicians who care for children who die in the PICU – is critical to improving the quality of the death and dying experience in the PICU. Previous research on the quality of dying and death in the adult ICU setting found that attending physicians gave more favorable ratings than either nurses or residents (20). In addition, research on proxy respondents underscores the importance of assessing and understanding the views of each type of respondent (14, 27, 28). Therefore, this paper 1) reports on the development and testing of a pilot version of a clinician measure of the quality of the dying and death experience in the pediatric intensive care setting (the clinician PICU-QODD) and 2) assesses the performance, reliability, and construct validity of the measure by the disciplinary type of the clinician completing the tool.

Methods

Sample

Data were collected in the PICUs of two large children’s hospitals in the northeastern U.S. For a 12-month period beginning in summer 2008, for every death that occurred in each of those units, five types of clinicians who were involved in the care of that decedent during the last three days of life were asked to complete a self-administered survey. The five types of clinicians were: 1) the bedside nurse, i.e., the nurse who was present at the bedside at the time of the child’s death; 2) the child’s primary nurse; 3) the attending intensivist who directed the child’s care; 4) the critical care fellow most involved in the child’s care, and 5) a clinician, other than the physicians and the nurses, who provided psychosocial support to the family (e.g., social worker, psychologist, child life specialist or chaplain). For each death that occurred during the accrual period, a study Research Coordinator in the unit identified the appropriate clinicians and distributed the self-administered survey booklet to them. Consent to participate was obtained prior to the study from attending intensivists, critical care fellows, and psychosocial staff regularly assigned to the PICU who would likely care for one or more decedents during the 12-month accrual period. Given the size of the nursing staff, consent to participate was obtained from nurses only when they were involved in the care of a particular decedent and thus were eligible for the study. Human subjects review and approval for the study was obtained at the participating hospitals and at Education Development Center, Inc., the nonprofit research and development organization that was awarded the funding that supported the project.

Measures

PICU-QODD

The format of the PICU-QODD was modeled after the adult QODD (18). However, as noted in the introduction, the pediatric context is substantially different than the adult context and the definition adopted for the PICU-QODD emphasizes the hopes and priorities of the family rather than the preferences of the patient. Consequently, focus groups with PICU clinicians who have cared for children who died and qualitative interviews with parents of children who died in a PICU were conducted to incorporate their views in the development of the measure (29). The thirteen themes derived from the interviews with parents, the focus groups with clinicians (conducted separately for nurses, psychosocial staff, physicians) and extensive review of the literature on pediatric end-of-life care are listed in Table 1. Multiple survey items were generated to capture each theme, which resulted in 140 items for systematic evaluation of their relevance to the theme, answerability, and clarity. Based on these criteria, the investigators culled the pool of items to 60. Next, a small number of cognitive debriefing interviews were conducted with parents and with PICU clinicians to produce 43 items for the pilot version. Although the goal is for the PICU-QODD to be concise, the number of items was deliberately set high in the pilot instrument. The substantive redundancy was intended to allow us to significantly cut back on the number of items later, based on their psychometric performance.

Table 1.

Themes Generated from Focus Groups With Clinicians, Interviews with Parents, and Literature Review

Communication

Privacy and other PICU Environment Issues

Decision to Withdraw Life Support

Decisions about the Circumstances of the Death

Pain and Symptom Management

Emotional Needs of Child

Emotional Needs/Support of Family

Physical and Instrumental Needs of Family

Spirituality and Religion/Cultural Issues

Continuity/Coordination of Care

Acting like a Parent/Parent as Caretaker in PICU

Grief and Bereavement

The Death Event

Each item in the PICU-QODD described an aspect of care that is important to the quality of the dying and death experience for the child and their family. Clinicians evaluated each aspect of the family’s experience of their child’s dying and death on an 11-point scale from 0=terrible to 10=as good as it could be, under the circumstances. An initial total score was computed for each clinician by averaging across evaluation ratings. Consistent with Curtis et al. (18), if fewer than half of the ratings had a valid response, the total score was undefined. To construct a standardized score that ranged from 0 to 100, the original total score was divided by the range (10) and multiplied by 100. Higher scores indicate a more positive experience.

Additional Measures for Establishing Construct Validity

The survey instrument completed by the clinicians incorporated a number of additional measures for psychometric purposes. These measures include two single-item questions about the quality of end-of-life care and the quality of the dying moment (18), and the three-item Meeting Family Needs Scale (MFN), which assesses satisfaction with how well the needs of a family are met during an ICU stay in which a family member dies (30, 31). Responses were provided on an 11-point scale ranging from 0 (not at all satisfied) to 10 (very satisfied). The Cronbach alpha for the MFN was between .754 and .860 for all five types of clinicians.

In the same study of intensive care social workers (31) and nurses (30) in which the MFN was developed, a set of yes/no questions about potential barriers to providing care that would meet the family’s needs also was developed. From the original set of fourteen barriers, eight that would apply to all types of clinicians were selected for inclusion in the survey instrument. Five of the potential barriers were issues clinical staff might encounter in working with the patient and/or family (e.g., the family is angry, the family has unrealistic expectations of medical treatment) and three were issues about the team or system (e.g., conflict with others on the team, not enough staff/heavy patient load). We hypothesized that when any of these barriers were encountered, the quality of the dying and death experience would decrease. That is, the PICU-QODD score would be lower when clinicians reported encountering the barrier.

Medical Record Review

Medical record abstraction was completed for each decedent, primarily to obtain the age, gender, and racial background of the decedent and the nature of the illness.

Statistical Analyses

As described in the Introduction, clinicians are proxy respondents with respect to the quality of the dying and death experience for the child and their family. Research in the adult setting suggests that different types of clinicians have different views on the quality of that experience (20) and current recommendations for improving measurement in end-of-life research call on researchers to assess potential measures separately for each type of proxy respondent (14, 32, 33, 28). Therefore, we conducted all analyses separately for each type of clinician – attending physician, fellow, psychosocial staff, bedside nurse, and primary nurse. The unit of analysis is a decedent.

Standard item analyses were conducted to assess the performance of individual items as part of the PICU-QODD scale used in the pilot. The percentage of respondents who did not provide a valid response, the percentage who selected 0, the percentage who selected 10, the skewness of the item distribution, and the correlation of the item to the total score (when the item was deleted) were examined. Based on commonly used criteria (34, 18), items with more than 15% missing responses, more than 15% of responses at 0 or 10, and/or skewness >3 were considered candidates for deletion. In addition, given the deliberate redundancy included in the pilot, the substantive content of the item also was considered in the final decisions about which items to remove or retain. The result was a reduced instrument of 20 items, which we refer to as PICU-QODD-20.

The performance of the PICU-QODD-20 was assessed using more stringent criteria. We expected less than 5% of respondents to not have a valid score, to have a score of 0, or to have a score of 100. The overall distribution of the PICU-QODD-20 was expected to be normal with skewness less than 2.0.

Finally, the internal consistency of the PICU-QODD-20 was examined using Cronbach’s alpha. Cronbach’s alpha is designed for “single-level” rather than clustered data (35) but, as was described above, two types of clustering occur in these data (some clinicians reported on multiple decedents; some decedents were reported on by multiple clinicians of a particular type). To investigate the potential effects of clustering on internal consistency, Cronbach’s alpha was computed and compared for three samples: 1) all decedents without regard to any clustering, 2) a single decedent was randomly selected for each clinician who reported on more than one decedent, creating a reduced set of decedents in which each clinician is represented only once, and 3) all decedents but with a single averaged score for a decedent when there were multiple ratings from clinicians of a particular type (e.g., two psychosocial staff reported on the same decedent).

To examine construct validity of the reduced instrument, the correlation between PICU-QODD-20 and related constructs was examined. The quality of care at the end of life, the quality of the dying and death moment, and satisfaction with the extent to which family needs were met were expected to be significantly and positively correlated with the PICU-QODD-20 score. Similarly, the PICU-QODD-20 was expected to be lower when barriers were encountered. To account for the clustering of observations within clinician, multilevel models were used to calculate the correlations and to assess the difference in the PICU-QODD-20 when a barrier was encountered (36–39).

Results

Decedents and the Clinicians Who Cared for Them

During the 12-month accrual period, 94 children died in the two participating PICUs. Two-thirds of the decedents had a chronic disease or condition rather than an acute injury or illness occurring in a previously healthy child. Just under half of the decedents were female. The mean (SD) age was 7.3 (7.2) years, with an age range of 0–24. A few of the decedents were young adults rather than children. Typically, these individuals had a life-threatening condition for which they had received care at that hospital for many years. For one-quarter of the decedents, racial information was not available in the medical record. About one-third of the decedents with racial information were non-Caucasian and the other 44% were Caucasian. Overall, 44% of decedents were from Site A

Table 2 describes both the involvement of clinicians (as indicated by the distribution of surveys) in the care for these decedents during the last three days of life and the completion of surveys by those clinicians. As noted above, the study design called for five different types of clinicians to provide their views about the quality of dying and death for each decedent. Some decedents had more than one clinician of a particular type, so a total of 551 clinician surveys were distributed. Nearly half of the decedents had two or more psychosocial staff members involved in their care during the last three days of life. About 20% of decedents had two or more fellows and/or bedside nurses involved in their care, but less than 6% of decedents had two or more attending physicians and/or primary nurses involved in their care. For 18 decedents, surveys were distributed to three or four types of clinicians but not all five. For these decedents, either the clinician in a particular role declined to participate in the study prior to the accrual period or no clinician functioned in that role during the last three days of life.

Table 2.

Number of Surveys About a Decedent Distributed to and Completed by Type of Clinician

No. (%) decedents	Attending	Fellow	Psychosocial Staff	Bedside Nurse	Primary Nurse
Number (%) of decedent with
No survey distributed	0 (0%)	5 (5%)	12 (13%)	6 (6%)	2 (2%)
1 survey distributed	88 (94%)	69 (73%)	38 (40%)	71 (76%)	90 (96%)
2 or more surveys distributed	6 (6%)	20 (21%)	44 (47%)	17 (18%)	2 (2%)
Total # surveys distributed	100	109	142	106	94
Number (%) of decedents with
No survey completed	38 (40%)	50 (53%)	32 (34%)	41 (44%)	47 (50%)
1 survey completed	55 (59%)	37 (39%)	42 (45%)	48 (51%)	47 (50%)
2 or more surveys completed	1 (1%)	7 (7%)	20 (21%)	5 (5%)	0 (0%)
Total # surveys completed.	57	51	87	58	47
Response Rate:
Surveys completed/surveys distributed	57%	47%	61%	55%	50%

Of the 551 surveys that were distributed, 300 were completed and returned. At least half of decedents had a survey completed by a psychosocial staff member (66%), an attending (60%), a bedside nurse (56%), and/or a primary nurse (50%). Nearly half of decedents (47%) had a survey completed by a fellow. Four decedents did not have any clinician survey completed about them. There were no significant differences in the gender, age, race, or nature of the illness between decedents who had surveys completed by a particular type of clinician versus those decedents who did not. Note that there are no decedents for whom there is more than one survey completed by primary nurses, and only a small number of decedents with two or more surveys completed by attendings (one decedent), fellows (seven decedents), and/or bedside nurses (five decedents). For 20 decedents, two or more psychosocial staff members completed surveys about the same decedent.

Within type of clinician, the percentage of distributed surveys that were completed was 57% for attending physicians, 47% for fellows, 61% for psychosocial staff, 55% for bedside nurses, and 50% for primary nurses.

As is summarized in Table 3, 236 clinicians provided care during the last three days of life for one or more of the 94 decedents. Many of the attending physicians, critical care fellows, and psychosocial staff cared for more than one decedent during the accrual period. In contrast, nurses generally cared for only one decedent. Consequently, relatively few nurses (17% of bedside and 16% of primary) completed surveys about more than one decedent. But 67% of attending physicians, 55% of fellows, and 62% of psychosocial staff completed surveys about more than one decedent. As expected, attending physicians were most likely to be male, Caucasian, and have the most experience in medicine and in critical care.

Table 3.

Characteristics of Clinicians Completing the Surveys

	Attending	Fellow	Psychosocial Staff	Bedside	Primary
Number of clinicians who cared for a decedent and received at least one survey	38a	32 a	42	83 b	76 b
Median number of decedents cared for (range)	2 (1–10)	2 (1–9)	2 (1–20)	1 (1–3)	1 (1–3)
Number of clinicians who completed at least one survey (% of received survey)	24 (63%)	22 (69%)	26 (62%)	48 (58%)	39 (51%)
Median number of surveys completed (range)	2 (1–8)	2 (1–6)	2 (1–18)	1 (1–3)	1 (1–3)
% of clinicians who completed at least one survey who
% completed only one survey	33%	45%	38%	83%	84%
% completed 2–4 surveys	63%	41%	42%	17%	16%
% completed 5 or more surveys	4%	14%	20%	-	-
Number of clinicians who completed survey with demographic information (%)	21 (88%)	18 (82%)	22 (85%)	42 (88%)	32 (82%)
% Female	33%	50%	86%	95%	97%
% non-Caucasian	5%	22%	27%	10%	9%
Years in medicine
% 0–2	0%	0%	23%	24%	6%
% 3–10	38%	89%	36%	48%	68%
% 10+	62%	11%	41%	29%	26%
Years in critical care
% 0–2	5%	62%	36%	33%	22%
% 3–10	57%	39%	41%	48%	59%
% 10+	38%	0%	23%	19%	19%
% Any previous training in end-of-life care	95%	100%	95%	76%	81%

^aNote that three clinicians served as a critical care fellow and then an attending physician during the accrual period. So the total number of physicians involved in caring for the decedents was 67 (=38+32-3).

^bNurses could serve as either the bedside or the primary nurse for any given decedent. There were 32 nurses who cared for one decedent as a bedside nurse and another as a primary nurse. So the total number of nurses involved in caring for the decedents was 127 (=83+76-32).

Item Performance

About half of the 43 items in the pilot version of the PICU-QODD had fewer than 15% missing responses, although the percentage of missing responses across all items ranged from 5% to 44%. There was only one item with a rating of zero from any respondent.

Some items were more likely than other items to receive a response of 10. The percentage of responses of 10 ranged from 26% to 63% across the 43 items. Three quarters of the items had more than 40% of respondents rating that item as a 10. However, only 5% of respondents reported 10 for every item they completed in a survey, and just over one-quarter of respondents did not rate any item as a 10. Fellows were least likely to report 10s, with an average of 25% of items receiving a rating of 10. Psychosocial staff members were most likely to report 10s, with an average of 51% of items receiving a 10. Attending physicians on average rated 47% of items as 10s, bedside nurses 46%, and primary nurses 39%.

One possible explanation for the number of 10 responses is that some decedents received excellent care and thus warranted the highest rating on many items from all of the clinicians who reported on that decedent. That is, for a small set of decedents, all of the clinicians reporting on those decedents would rate many of the items as a 10. However, 65 decedents (69% of the 94 decedents) had at least one clinician who reported 10 for at least half of the items. So the use of the highest possible rating for a large number of items occurred for many decedents. Alternatively, a consistently high percentage of 10 responses from a small set of clinicians might suggest a social desirability or response set bias. However, the surveys on which 80% or more of the items had responses of 10 were completed by 49 (31%) different clinicians, including 17 doctors, 11 psychosocial staff, and 21 nurses. Despite the large number of 10 ratings for many items, the skew of only one item exceeded 3.0. Similarly, the correlations of each item with a total score based on the other 42 items were generally high, ranging from .427 to .990.

Although the criterion of 15% or fewer responses of 10 had to be relaxed, these analyses of the performance of each item within type of clinician were coupled with substantive considerations (e.g., retaining items to represent the range of themes) to select a smaller number of items. Overall, 23 items were dropped, leaving 20 items in the final PICU-QODD (PICU-QODD-20). Table 4 lists the items retained in the PICU-QODD-20.

Table 4.

Items Included in the PICU-QODD-20 by Domain

Pain and Symptom Management

1a. the child was free of pain

1b. the child was free of other troubling symptoms

1c. clinical staff responded quickly to parents’ concerns about their child’s symptoms

Communication Issues

1e. clinical staff gave parents information about their child in a way that they could understand

3d. clinical staff prepared parents for what might happen to their child

1m. clinical staff discovered and respected parents’ wishes and decisions

1g. clinical staff created an atmosphere in which parents felt comfortable asking questions about their child

Decisions to Withdraw Life Support

1p. clinical staff offered parents opportunities to discuss options about their child’s care with the healthcare team

1r. there were no conflicts between parents and the clinical staff about the best way to care for the child

Privacy and PICU Environment Issues

5a. clinical staff provided parents with privacy with their child near the end of their child’s life

Physical and Instrumental Needs of Family

1v. parents could easily meet their basic physical needs (accessible bathroom, showers, affordable meals, places to stay, parking, etc.)

Emotional Needs/Support of Family

1j. clinical staff demonstrated that they cared about the child as an individual

1k. clinical staff supported the parents emotionally

1w. clinical staff provided parents with opportunities to be near their child

Fulfilling the Parental Role

3b. clinical staff helped parents find ways to touch, hold, and/or connect with their child

Spirituality and Religious/Cultural Issues

1y. hospital clergy and chaplains were available

1z. staff discovered and respected the family’s spiritual and/or religious needs

Continuity/Coordination of Care

2b. nurses and doctors did a good job of passing information about the child onto the next shift or rotation

Grief and Bereavement

5d. clinical staff helped parents create memories (such as handprints, lockets of hair, photographs) of their child

5g. once the child died, his/her parents were allowed to stay with him/her for as long as they wanted

Performance of PICU-QODD-20

Once the measure was refined to a smaller, more workable set of items, the next step was to assess the performance of the measure as a whole. Table 5 summarizes the distribution and performance of the PICU-QODD-20 by type of clinician. Consistent with the large number of 10 responses for many items, the median and mean values are high, although the percentage of decedents with a score of 100 is low, ranging from 2% for fellows to 9% for attending. For attending physicians, psychosocial staff, and primary nurses, the minimum score was below 50. For fellows, bedside nurses, and primary nurses, 25% of decedents had PICU-QODD scores less than 80. The Cronbach alpha presented in the last row of Table 5 was calculated using all cases, not taking into account the clustering by clinician or decedent. We also calculated Cronbach alphas on two additional versions of the sample, as described above under Statistical Analysis, and these Cronbach alpha values were all within 0.03 of the values reported here.

Table 5.

Performance of the PICU-QODD-20 by Type of Clinician

	Attending (n=57)	Fellow (n=51)	Psychosocial Staff (n=86)	Bedside Nurse (n=58)	Primary Nurse (n=47)
Percentage missing	7%	2%	12%	5%	6%
Percentage with score of 0	0%	0%	0%	0%	0%
Percentage with score of 100	9%	2%	6%	5%	4%
Mean (SD) score	88.1 (11.4)	82.6 (9.7)	90.2 (9.8)	86.6 (10.0)	84.5 (12.4)
Skew	−1.6	−.2	−2.5	−.6	−1.2
Median score (50th percentile)	90.8	82.9	92.2	90.5	85.9
25th percentile	83.6	76.7	88.5	78.9	78.5
10th percentile	75.0	68.0	81.1	70.0	69.5
Minimum score	48.4	64.2	42.9	65.8	43.3
Cronbach’s alpha (all cases)	0.959	0.932	0.906	0.891	0.927

Construct Validity

To assess the validity of the PICU-QODD-20, the correlation between the PICU-QODD-20 and three constructs with which it should theoretically be associated were calculated. Table 6 presents these correlations within type of clinician. That is, the correlation in each cell is based on the PICU-QODD-20 and the measure in that row as reported by the type of clinician represented in the column. For all five clinician types, the total PICU-QODD-20 score was substantially and significantly related to a single-item report of the decedent’s quality of end-of-life care and to a validated measure of the extent to which family needs were met during the ICU stay. The association between the PICU-QODD-20 and a single-item report on the quality of the moment of death was significant only for the psychosocial staff and bedside nurses. For both attending physicians and primary nurses, the P-values for the association between the PICU-QODD-20 and the quality of the moment of death were less than 0.15, indicating a trend toward significance. These correlations may not have achieved statistical significance because one-third of the attending and primary nurses who reported being present at the bedside at the moment of death did not respond to the question about the quality of the moment of death.

Table 6.

Evidence for Construct Validity of the PICU-QODD-20

Correlationa of PICU-QODD-20 with	Attending	Fellow	Psychosocial Staff	Bedside Nurse	Primary Nurse
Quality of end of life care	0.552 b (n=52)	0.422 b (n=50)	0.391 b (n=76)	0.386 c (n=54)	0.557 c (n=44)
Quality of moment of death	0.217 (n=44)	0.144 (n=45)	0.665 b (n=65)	0.275 d (n=45)	0.271 (n=34)
Meeting family needs	0.797 b (n=52)	0.333 c (n=49)	0.780 b (n=76)	0.451 c (n=55)	0.610 b (n=44)

^aThe correlation in each cell is based on the PICU-QODD-20 and the measure in that row as reported by the type of clinician represented in the column. Correlations were computed in a mixed model to account for nesting of decedents within clinicians.

^bP ≤ 0.001.

^c0.001<P ≤ 0.01.

^d0.01<P ≤ 0.05.

Table 7 reports the change in the total PICU-QODD-20 score when a team or patient/family barrier was encountered. Overall, clinicians reported encountering the five family barriers more frequently, ranging from 8% to 34% of decedents, than the three team barriers, which were reported for only 5% to 10% of decedents. For attending physicians and psychosocial staff, the PICU-QODD-20 score was significantly lower when three of the five patient/family barriers were encountered. For fellows and primary nurses, only one of the five patient/family barriers lowered the PICU-QODD-20. For psychosocial staff, two of the team barriers lowered the PICU-QODD-20 by nearly 20%. For primary nurses, patient load significantly lowered the PICU-QODD-20 score. For attending physicians and fellows, the PICU-QODD-20 was significantly decreased when there was a lack of communication among clinical staff.

Table 7.

Change^a in the PICU-QODD-20 When Family or Team Barriers Were

Barrier (% yes)	Attending (n=57)	Fellow (n=51)	Psychosocial Staff (n=86)	Bedside Nurse (n=58)	Primary Nurse (n=47)
Patient/Family Barriers
patient too sick for interaction (34%)	−5.96b	NS	−4.86 b	−3.87 c	NS
family does not visit or call (20%)	−27.3 c	NS	−13.3c	NS	NS
the family is angry (15%)	NS	NS	−7.27 c	NS	NS
family has unrealistic expectations of medical treatment (17%)	NS	−3.86 b	NS	NS	−11.3 b
language difficulties (8%)	−10.4 b	NS	NS	NS	NS
Team Barriers
patient load (10%)	NS	NS	NS	NS	−13.5 c
conflict among clinicians (6%)	NS	NS	−18.0 c	NS	NS
lack of communication among clinical staff (5%)	−12.2 d	−5.4 d	−24.1 c	NS	NS

^aThe difference in the PICU-QODD-20 for decedents where the barrier was encountered versus those where it was not was computed in a mixed model to account for nesting of decedents within clinicians.

^b0.01<P ≤ 0.05.

^c0.001<P ≤ 0.01.

^d0.05<P ≤ 0.10.,

NS: P = 0.10.

Discussion

Many efforts to improve end-of-life care for adults have identified the development of outcome measures as a high priority (11, 33). In the pediatric intensive care setting, quality of dying and death is a more relevant construct than quality of life at the end of life because it focuses specifically on the endstage of illness, just prior to death. Children often arrive in the ICU comatose or so debilitated that the measurement of quality of life would make little sense. An accurate measure of the dying and death experience, however, will provide a metric for assessing the extent to which a “good death” has been achieved. In addition, a valid and reliable measure of the quality of dying and death holds promise for illuminating how critical care nurses, physicians, and allied psychosocial staff can better manage the dying period to benefit the children and their families, as well as the caregivers.

The PICU-QODD-20 shows promise as a valid and reliable measure of the quality of dying and death in the pediatric intensive care setting. The 20 items retained in the PICU-QODD-20 cover the full range of themes relevant to dying in the PICU as identified through focus groups with PICU clinicians, parents of children who died in the PICU, and a systematic literature review. Despite a response label that may have skewed the individual items towards high ratings, the overall PICU-QODD-20 had favorable measurement properties with no floor effects, limited ceiling effects, and limited missing scores for all five types of clinicians. The Cronbach alpha was 0.90 or better for all clinicians except bedside nurses (0.891). The PICU-QODD-20 was negatively associated with at least two of eight potential barriers for all clinicians except bedside nurses and, for all types of clinicians, was positively associated with both the quality of end-of-life care and meeting family needs. In sum, these data suggest that the PICU-QODD-20 may be a valid and reliable outcome measure of the quality of dying and death in the PICU setting.

Although much work on improving end-of-life care for adults calls for providing care and support for the family of the decedent, the family as the unit of care is essential in the pediatric setting. Nearly half of the children who die are neonates, infants, or children so young they have not developed and are unable to communicate their own preferences (11). In the intensive care setting, even older children are often comatose and thus unable to communicate their preferences and/or participate in decision making. In addition, the death of a child is so traumatic and has such long-lasting consequences for a family that it is critical for parents to integrate their own hopes and priorities with those of other members of their family and those of their dying child. Consequently, the definition of the quality of dying and death in the PICU takes into account the “hopes and priorities of the patient and the family.” Fifteen of the 20 items in the final version of the clinician PICU-QODD-20 relate directly to the experience of the family rather than the decedent.

Given the importance of caring for the family in the pediatric setting, obtaining the views of parents and other family members is essential to assessing any measure of the quality of dying and death in the PICU. Future studies must compare clinician and parent/family ratings. However, the importance of clinicians as proxy raters is heightened given the typically low levels of response from bereaved parents in nearly all pediatric end-of-life research (25). Given the importance of outcome measurement to quality improvement efforts, the use of clinicians as proxy raters will help move forward efforts to improve the quality of dying and death and end-of-life care in the pediatric intensive care setting. In addition, the availability of validated clinician reports of the quality of dying and death will provide an opportunity to examine potential non-responder bias among parent reports. That is, comparison of clinician reports of the quality of dying and death for parents who do and do not respond to a survey request may illuminate potential response bias among parents (44).

Another important advantage of clinicians as raters is their ability to evaluate the dying and death experience in light of their experience with a range of other deaths. Fortunately, parents rarely experience this traumatic event more than once and do not necessarily have knowledge of what a high quality dying and death experience and good end-of-life care could be. Parents of children who die may be subject to a paradox similar to the one that occurs for patients in pain. Patients in pain, when questioned, sometimes indicate that they are very satisfied with their care even though their pain is inadequately treated because they do not realize that good care would have likely alleviated their pain (45). Parents may likewise be prone to believing that the quality of the dying and death experience was good because they have no reference point for what a “good death” may involve.

In retrospect, in the 20 items that constitute the PICU-QODD-20, the label used for a rating of 10 – “as good as it could be under the circumstances”— inadvertently provided a relative rather than an absolute anchor for high ratings. The use of “As good as it could be, under the circumstances” was meant to be sensitive to the parents’ grief (both parent and clinician instruments used the same anchors), but may have produced overly positive ratings. Clinicians may have considered how the circumstances of a case affected the family’s experience rather than evaluating the experience of the family in relationship to an ideal dying and death experience. Thus, cases in which circumstances interfered with obtaining the highest quality dying and death experience may have received high ratings when, compared with an idealized vision of the highest quality dying and death experience, the high rating was not warranted. In the final version of the PICU-QODD-20, the label for the score of 10 will be changed to make explicit that the comparison is an ideal vision. In the adult QODD, Curtis et al. used the labels “a terrible experience” for a score of 0 and “a near perfect experience” for a score of 10. For clinicians, this labeling, or perhaps even “a perfect experience” for a score of 10, may be appropriate in the pediatric version.

One limitation of this pilot study is that too few cases were available within type of clinician to assess the dimensionality of the PICU-QODD-20. As noted in Table 4, the items span ten domains and the use of subscales may be appropriate. Another limitation is the use of only two hospitals in one area of the country. Finally, very few parent/family surveys were completed (n=17, 28% of 61 parents who were eligible and could be contacted), thus limiting our ability to compare parent and clinician ratings. Of the 94 decedents, 22 parents were not eligible to be contacted because of language (n=9), suspected child abuse (n=6), or hospital risk management concerns (n=7). In addition, contact information for 11 parents was not valid.

Further data collection efforts in this study, and others, will continue the process of establishing the PICU-QODD-20 as a valid and reliable measure of the quality of dying and death in pediatric intensive care settings. The availability of a validated outcome measure will help drive efforts to improve the quality of dying and death and the quality of end-of-life care in this setting going forward.