Development and validation of a novel informational booklet for pediatric long-term ventilation decision support

Abstract

Objectives:

To provide accessible, uniform, comprehensive, and balanced information to families deciding whether to initiate long-term ventilation (LTV) for their child, we sought to develop and validate a novel informational resource.

Methods:

The Ottawa Decision Support Framework was followed. Previous interviews with 44 lay and 15 professional stakeholders and published literature provided content for a booklet. Iterative versions were cognitive tested with 6 parents facing decisions and 5 pediatric intensivists. Ten parents facing decisions evaluated the booklet using the Preparation for Decision Making Scale and reported their decisional conflict, which was juxtaposed to the conflict of 21 parents who did not read it, using the Decisional Conflict Scale Twelve home ventilation program directors evaluated the booklet’s clinical sensibility and sensitivity, using a self-designed six-item questionnaire. Data presented using summary statistics.

Results:

The illustrated booklet (6^th-grade reading level) has 9 topical sections on chronic respiratory failure and invasive and noninvasive LTV, including the option to forgo LTV. Ten parents who read the booklet rated it as helping “Quite a bit” or more on all items of the Preparation for Decision Making Scale and had seemingly less decisional conflict than 21 parents who did not. Twelve directors rated it highly for clinical sensibility and sensitivity.

Conclusions:

The LTV booklet was rigorously developed and favorably evaluated. It offers a resource to improve patient/family knowledge, supplement shared decision-making, and reduce decisional conflict around LTV decisions. Future studies should validate it in other settings and further study its effectiveness.

Introduction

Each year, hundreds of children develop chronic respiratory failure (CRF) due to complex chronic conditions or sequela of severe critical illness.¹ Consequently, their families must decide whether to commit them to long-term ventilation (LTV), often via tracheostomy. Not doing so can lead to dyspnea, poor growth, reoccurring or serious lung illnesses, and even death. Although LTV can relieve these symptoms and be life-prolonging, it is not curative, does not mitigate other comorbidities or life-limiting conditions that these children commonly have, and confers its own risks for morbidity and mortality. Furthermore, LTV often requires constant, complex care, which can lead to physical, emotional, financial, and other burdens for the family.^2,3,4,5 Therefore, forgoing LTV is morally permissible in some situations. In order to make informed decisions that are aligned with personal values and realistic goals, families must be apprised of LTV’s potential benefits, risks, and burdens for their child and family and be patiently guided through the decision-making process. However, evidence shows that the provision of information on LTV can be untimely, inconsistent, and/or insufficient.^6,7,8,9 Moreover, some providers question its appropriateness for some children with life-threatening conditions or profound neurodevelopmental deficits.^10,11,12

Given the shortcomings and variability of provider counseling and the complexity and gravity of these decisions, we created a caregiver-informed booklet to ensure that the provision of information about LTV is accessible, uniform, comprehensive, and balanced. We describe here the structured process used to develop, validate, and pilot test this novel informational booklet about LTV for children.

Methods

Conceptually, we followed the Ottawa Decision Support Framework, which broadly involves three steps: 1) Needs assessment – assess patient and professional determinants of decisions to identify decision support needs; 2) Booklet development – develop decision support tailored to those needs; and 3) Booklet evaluation – evaluate the decision making process and outcomes.¹³ This evidence-based framework has been used in numerous decision tools and interventions. Our goal was to create an information and decision support tool that discusses invasive LTV via tracheostomy and non-invasive LTV, addresses topics pertinent to both child and family in the short- and long-terms (including sensitive topics), and is applicable in both inpatient and outpatient settings. The Columbia University Irving Medical Center Institutional Review Board approved this study.

Needs Assessment

Over an eighteen month year period, we interviewed three stakeholder groups to gain their perspectives on families’ decisional needs: 1) parents with a child with CRF who had faced a decision around LTV (n=16); 2) parents with a child with CRF who were facing a decision (n=28); and 3) directors of pediatric home ventilation programs (n=15). We focused on children with medical complexity for whom CRF would be long-term or life long, presuming that their families would have greater and more complex decisional needs. The methods and findings of these interviews have been published elsewhere.^14,15 Interviewees were told that the information gathered would hopefully be used to help future families, but not that an information and decision support resource was the eventual goal. Concurrently, we performed a comprehensive review of relevant published and on-line literature to corroborate and supplement our interview findings. Taken together, the following domains were deemed of importance: nature of CRF and LTV; positive and negative impact of LTV on child and family; potential complications of LTV; and alternatives to LTV. Although extremely important to decision-making, the heterogeneity of children with CRF precluded including prognosis with or without LTV as a domain.¹⁶

In order to ascertain their degree of decision-making uncertainty, 21 out of 28 parents who were facing a decision (17 regarding invasive ventilation; 4 regarding noninvasive) also completed the traditional, statement format Decisional Conflict Scale (DCS).¹⁷ This scale has 16 items in 5 response categories. Scores range from 0 (no decisional conflict) to 100 (extremely high decisional conflict). Scores >37.5 are associated with decisional delay or uncertainty (21). Scores ≤37.5 have been interpreted differently by researchers—<25 categorized as low and 25-37.5 as moderate decisional conflict¹⁸ or, alternatively, 1-25 mild and >25 severe conflict.¹⁹ The scale’s test-retest correlations and Cronbach alpha coefficients (ie, measures of reliability) exceed 0.78.¹⁷ All parents completed the DCS before tracheotomy or, in the case of noninvasive LTV, before discharge.

Booklet Development

A printed format was deemed apt for the tool, because families are familiar with healthcare informational materials (eg, brochures), not all families have access to technology/internet, and previous printed pediatric and critical care materials have been shown to be effective.^20,21 One investigator (JDE) compiled content relevant to the domains into an initial draft of a booklet, with the assistance of all coauthors in areas of their specific expertise. Following recommendations for health-related information and appreciating that lay readers have a range of literacy, we targeted a 6^th grade reading level or lower.²² The booklet explicitly encourages families to ask questions and discuss issues with their medical team. It encourages families, if possible, to discuss its contents with their child in a developmentally-appropriate way. Families who are considering noninvasive LTV are encouraged to read the sections on invasive LTV because a notable proportion of such children ultimately cannot be supported noninvasively²³, necessitating their families to face a decision regarding invasive LTV. Professionally drawn illustrations and a glossary explaining terms used in the booklet were included. The draft was initially revised three times with the guidance of senior investigators (HBP, JW, JEN, RLM).

We then performed cognitive testing with parents facing the decision around LTV in our pediatric intensive care units (PICU) and with pediatric intensivists to ensure the booklet adequately communicated information with clarity, face validity, and content validity. Six parents and 5 intensivists were recruited using critical case purposeful sampling. Seeking to recruit parents with a range of health literacy, all potential participants were first discussed with the PICU attending and social worker before approaching them. Over a four-month period, 8 English-speaking parents/parental sets were identified. Six were approached, consented, and agreed to participate (4 considering invasive ventilation; 2 considering noninvasive ventilation); 2 declined because they felt uncomfortable participating in research. A copy of the booklet was given to each participant who was given several days to read it. An investigator (JE) then used cognitive response and “think-aloud” techniques to test content and language for comprehensibility and clarity.²⁴ Participants were asked to explain every section of the booklet in his or her own words and respond to questions about specific aspects of the content. Probing questions were used to delve further into any item identified as unclear or not matching the interviewer’s interpretation, as well as examine how well the booklet held the participant’s attention and if it was too long. Notes were taken by the interviewer during each interview. The booklet was edited in an iterative process after each successive interview until 2 parents and 2 intensivists deemed it clear and useful without need for further changes.

Booklet Evaluation

Over a six-month period, all English-speaking parents facing the decision regarding LTV for their child in our PICUs were approached about their willingness to read the booklet, share their impressions, complete 2 inventories related to their decision-making and the booklet, and participate in cognitive testing. Thirteen parents/parental sets were identified and approached; 3 parents/parental sets declined citing the stress of the situation or unwillingness to participate in research involving their child. Ten parents (8 considering invasive LTV; 2 noninvasive LTV) consented and participated. Demographic information was collected from these parents. After reading the booklet and before tracheotomy or, in the case of noninvasive LTV, discharge, parents completed the Preparation for Decision Making Scale and the DCS. The former is a 10-item scale that assesses a person’s “perception of how useful a decision aid or other decision support intervention is in preparing the respondent to communicate with their practitioner at a consultation visit and making a health decision.”²⁵ We removed the scale’s tenth item because it did not pertain to the clinical situation. The scale’s test-retest correlation is 0.94 and Cronbach alpha coefficients range from 0.92-0.96.²⁶ An investigator (JDE) performed the cognitive testing and took notes on this and the parents’ impression of the booklet. Cognitive testing for this cohort was more focused and resulted in only minor edits in the booklet.

In addition, we asked the same cohort of interviewed directors of home ventilator programs to evaluate the booklet for clinical sensibility (material is appropriate for CRF and typical settings, ie, has clarity, face validity, and content validity)²⁷ and emotional sensitivity (material is sensitive to families’ distress, compassionate). Fourteen directors (one of the original cohort was unavailable) were invited to read the booklet and anonymously provide ratings on six evaluatory items using a five-point scale (1 being the lowest; 5 being the highest). They were also invited to provide any free text comments. Only minor edits were made in light of their comments.

Lastly, the booklet was professionally translated into Spanish.

Analysis

Readability was assessed using Flesch Reading Ease and Flesch-Kincaid Grade Level measures. Median and interquartile ranges (IQR) and means and standard deviations (SD) were used to present the results of all the scales/questionnaire. Composite scores were not calculated for the Preparation for Decision Making Scale because of the lack of the tenth item. DCS composite and subcategory scores of parents that did and did not use the booklet in their decision-making were presented. Stata 16 (StataCorp LP, College Station, TX) was used for all analyses.

Results

Both English and Spanish versions of the booklet, “When a child may use a ventilator for a long time”, can be found at https://www.pediatrics.columbia.edu/research/labs-and-teams/edwards-team/select-research-projects and on OPENPediatrics (an open-access medical e-learning platform) at https://learn.openpediatrics.org/learn/course/internal/view/elearning/4870/Long-Term-Ventilation-Information-and-Decision-Support-Booklet. It is comprised of 9 topical sections (Table 1) with 7 illustrations. Flesch Reading Ease was 73.2 (fairly easy to read), and Flesch-Kincaid Grade Level was 6.9.

Table 1.

Topics addressed in informational booklet on long-term ventilation in children

What is chronic respiratory failure?

How can a vent help my child? What can a vent do and not do?

What other equipment might my child use?

For a child in the hospital, how long does it take to get home with a vent?

How does using a vent at home affect the child and family?

What is long-term invasive ventilation?

What is long-term noninvasive ventilation?

Are there any other options?

How to make a decision about using a vent for a long time?

Parents overwhelmingly described the booklet as clear, informative, and helpful and did not believe it was too long. No parent expressed that the booklet altered the choice she or he thought they would make before reading it, rather they reported that it made them more confident, educated, and prepared to ask in-depth questions of their medical team. Four shared that they appreciated that it was printed on paper because they could highlight things they found important and make notes on it. On average, parents who read the booklet rated it as helping “Quite a bit” or more (median and mean scores ≥4, with 5 being the highest possible) on all items of the Preparation for Decision Making Scale (Table 2). Parents who read the booklet had lower total and subcategory scores on the DCS (potentially implying less decisional conflict/uncertainty) than parents who did not (median score 13 [IQR 8–28] v. 34 (IQR 23–42), respectively,see Table 3). Median responses for each DCS item can be found in the E-Table 1. The demographics of parents of the 31 parents that took the DCS are found in the E-Table 2.

Table 2.

Preparation for Decision Making Scale responses of 10 parents who received information booklet about long-term ventilation

Scale items	Mean (SD)	Median (IQR)
Did this educational material …
Help you recognize that a decision needs to be made?	4 (0.67)	4 (4–4)
Prepare you to make a better decision?	4.1 (0.88)	4 (4–5)
Help you think about the pros and cons of each option?	4.4 (0.84)	5 (4–5)
Help you think about which pros and cons are most important?	4.4 (0.7)	4.5 (4–5)
Help you know that the decision depends on what matters most to you?	4.4 (0.52)	4 (4–5)
Help you organize you own thoughts about the decision?	4.4 (0.7)	4.5 (4–5)
Help you think about how involved you want to be in this decision?	4 (1.25)	4 (4–5)
Help you identify questions you want to ask your doctor?	4.3 (0.67)	4 (4–5)
Prepare you to talk to your doctor about what matters most to you?	4.3 (0.95)	4.5 (4–5)

1=Not at all; 2=A little; 3=somewhat; 4=Quite a bit; 5=A great deal

IQR, interquartile range; SD, standard deviation

Table 3.

Decisional Conflict Scale subcategory scores for parents who did and did not receive long-term ventilation information booklet

Subcategory scores	Parents who did not receive booklet (n=21)		Parents who received booklet (n=10)
	Mean (SD)	Median (IQR)	Mean (SD)	Median (IQR)
Total scorea	35.3 (18.7)	34 (23–42)	16.6 (12.5)	13 (8–28)
Uncertainty subscoreb	45.6 (19.1)	42 (33–58)	23.3 (18.8)	17 (8–42)
Informed subscorec	34.1 (22.2)	33 (25–42)	18.3 (15.1)	21 (8–25)
Values clarity subscored	28.2 (24.6)	25 (8–42)	17.5 (17.3)	13 (8–25)
Support subscoree	30.6 (23.2)	33 (17–33)	13.3 (11.9)	17 (0–25)
Effective decision subscoref	37.5 (18.6)	38 (25–44)	11.9 (11.9)	6 (0–25)

^aScores range from 0 (no decisional conflict) to 100 (extremely high decisional conflict)

^bScores range from 0 (feels extremely certain about best choice) to 100 (feels extremely uncertain about best choice)

^cScores range from 0 (feels extremely informed) to 100 (feels extremely uninformed)

^dScores range from 0 (feels extremely clear about personal values for benefits & risks/side effects to 100 (feels extremely unclear about personal values)

^eScores range from 0 (feels extremely supported in decision making) to 100 (feels unsupported in decision making)

^fScores range from 0 (good decision) to 100 (bad decision)

IQR, interquartile range; SD, standard deviation

Twelve (86%) of 14 invited directors read and evaluated the booklet for clinical sensibility and emotional sensitivity. In all aspects, these directors rated it highly (median score of 5, with 5 being the highest possible, for all evaluatory items; see Table 4). A sample of their comments, edited only for clarity, can be found in E-Table 3.

Table 4.

Clinical sensibility and emotional sensitivity of long-term ventilation informational booklet as rated by 12 directors of pediatric home ventilator programs

Booklet quality	Mean (SD)	Median (IQR)
Clear and understandable to families	4.67 (0.62)	5 (4.5–5)
Includes important information	4.83 (0.37)	5 (5–5)
Does not omit important information	4.42 (0.95)	5 (4–5)
Only includes important information	4.33 (0.85)	5 (3.5–5)
Sensitive and compassionate	4.83 (0.37)	5 (5–5)
Helps prepare families for decisions	4.75 (0.43)	5 (4.5–5)

1=Lowest rating; 5=Highest rating

IQR, interquartile range; SD, standard deviation

Discussion

The decision whether or not to initiate LTV for a child is difficult and consequential, as LTV impacts nearly all aspects of the child’s and family’s lives. At the same time, forgoing LTV may mean the child’s underlying condition takes its natural, potentially terminal, course. Counseling families on this decision is difficult for providers because of the gamut of complex information that must be conveyed to families with a range of health literacy and values, the lack of prognostic information, the need to discuss sensitive topics, their own explicit and implicit biases, and unrealistic expectations of all parties.^14,28 These challenges leave families less able to participate in shared decision-making, without proper anticipatory guidance, and at risk for decisional distress and regret.^19,29,30 In addition, initiation of LTV and tracheotomy were the most commonly identified clinical risk factors for adverse psychological outcomes in families following the discharge of their child from the PICU.⁷ Therefore, commentators have called for clearer, broader, unbiased, timelier discussions of the burdens, risks, and benefits of LTV and standardized decision-making processes, including the use of written resources.^7,8,9,31,32

In response, we developed a novel informational booklet in order to provide accessible, uniform, comprehensive, balanced information on LTV and that could be used by providers and families to guide discussion of options as part of a larger shared decision-making conversation. This booklet draws content from interviews with lay and professional stakeholders and from existing literature. It was rigorously developed using evidence-based methods and was highly favorably evaluated by these stakeholders. In addition, our data suggests that using the booklet may decrease decisional conflict in parents who face a decision regarding LTV for their child.

Others have created informational or decision-support resources for parents facing end-of-life^33,34 or tracheostomy^35,36 decisions for their child. A parent organization posted clips of interviews of two families with children with spinal muscular atrophy—one who elected to use invasive LTV and one who did not.³⁷ This booklet complements and goes beyond these resources by providing greater in-depth information on invasive and noninvasive LTV and addressing issues pertinent to both the child and family.

Informational materials for patients/surrogates have been shown to improve decision quality—ie, improve knowledge, accuracy of perceptions of risk/outcomes, and congruence between values and decisions, as well as reduce decisional conflict and passivity—and have been endorsed as adjuncts to provider counseling.^20,21 Similarly, we believe this booklet can supplement provider counseling and should be given to families early in their information-gathering and/or decision-making processes. Even when families believe there is no “decision” to be made, it can help ensure uniform information and anticipatory guidance are offered. It may be especially useful in settings where there are limited institutional resources, such as formal home ventilation programs or organized decision-making processes³², to assist families during their decision-making. It also can be helpful in educating providers-in-training about what information should be conveyed to families.

While this booklet is a novel and potentially important resource, the booklet and this study do have limitations. In general, there is limited evidence that such interventions impact other outcomes beyond knowledge and decisional conflict (eg, time to decision, post-ICU stress).^18,38 Moreover, substantial practice changes are required before informational resources are incorporated into routine decision-making.³⁹ Like other such informational resources, this booklet is meant for a broad audience (and, in this case, with heterogeneous conditions) and thus lacks personalization.⁴⁰ Regarding this resource, it was validated in a critical care setting. So while it was written to be useful in other settings (eg, neonatal ICUs, outpatient clinics) and institutions and the surveyed directors suggest this is the case, its generalizability has not been confirmed. Using purposeful sampling for recruitment of parents for cognitive testing may have introduced selection bias, and parents and directors may have given answers they thought the investigators wanted to hear. While we included patients in our needs assessment interviews, none were included our cognitive testing or evaluation phases. Also, the validity and reliability of the Preparation for Decision Making Scale may be different than previously reported values, as an item was deleted without reevaluating its psychometric properties. Given that we could not be sure that groups that did and did not receive the booklet were analogous, we could not statistically compare their DCS. Thus, additional investigation is needed to evaluate the booklet’s effectiveness on decisional conflict and other outcomes associated with its use. Finally, it is possible that providing some people with this degree of information on LTV’s risks and burdens or providing it in this format may increase their decisional conflict and uncertainty.

Using an evidence-based framework, we assessed the decisional needs of families facing whether to initiate LTV for their child with CRF, rigorously developed an information and decision support booklet for these families, evaluated it with stakeholders, and pilot tested its effectiveness in lowering decisional conflict. We believe this booklet can help mitigate shortcomings and variability in provider counseling by providing accessible, uniform, comprehensive, and balanced information about LTV. In the future, the booklet can be validated in other settings, further studied for its effectiveness, adapted for the internet, and made customizable so that individual institutions can tailor it to reflect their own local healthcare system (eg, with discharge processes).

Abbreviations:

CRF

chronic respiratory failure

DCS

Decisional Conflict Scale

IQR

interquartile range

LTV

long-term ventilation

PICU

pediatric intensive care unit

SD

standard deviation