Potential Acceptability of a Pediatric Ventilator Management Computer Protocol

Abstract

Objective:

To examine issues regarding the granularity (size/scale) and potential acceptability of recommendations in a ventilator management protocol for children with Pediatric Acute Respiratory Distress Syndrome (ARDS)

Design:

Survey/questionnaire

Setting:

The 8 pediatric intensive care units in the Collaborative Pediatric Critical Care Research Network (CPCCRN)

Participants

122 physicians (Attendings and Fellows)

Interventions:

none

Measurements and Main results:

We used an online questionnaire to examine attitudes, and assessed recommendations with 50 clinical scenarios. Overall 80% of scenario recommendations were accepted. Acceptance did not vary by provider characteristics but did vary by ventilator mode (HFOV 83%, PRVC 82%, PC 75%, p=.002) and specific parameter adjusted (ranging from 88% for PIP and 86% for FiO2 changes, to 69% for PEEP changes). Acceptance did not vary based on child size/age. There was a preference for smaller PEEP changes but no clear granularity preference for other parameters.

Conclusions:

While overall acceptance rate for scenarios was good, there was little consensus regarding the size/scale of ventilator setting changes for children with Pediatric ARDS. An acceptable protocol could support robust evaluation of ventilator management strategies. Further studies are needed to determine if adherence to an explicit protocol leads to better outcomes.

Introduction

Ventilator management for children with Acute Lung Injury (ALI) and Acute Respiratory Distress Syndrome (ARDS), now known collectively as Pediatric ARDS or PARDS (1), varies between institutions and between pediatric intensivists (2–4). Clinicians treating adults generally accept NIH/NHLBI ARDS network (ARDSNet) ventilator protocols with improved outcomes (5–7) but protocol implementation is not yet widespread (8,9). Few ventilator protocols exist for pediatrics although studies of ARDs in children (10–12) have used protocols described as similar to ARDSNet protocols. Pediatric strategies are predominantly based on findings from adult studies (13) but there are differences in ventilator management practices (14) between adult and pediatric intensive care units (ICUs) and patient differences (15) that may need to be accommodated in a pediatric ventilation protocol.

Objective

The primary purpose of this study was to examine issues related to granularity (size/scale) and potential acceptability of recommendations in a protocol for ventilator management in children with ARDS. A secondary objective was to inform future refinements of the exemplar protocol, but the study did not explicitly test the exemplar protocol rules.

Ventilator Protocol

Protocols to standardize care can help remove confounders from ventilator practice for rigorous trials of lung protective strategies. The Pediatric Acute Lung Injury Consensus Conference, an international panel of experts, recommended using explicit protocols and definitions to guide research in mechanical ventilation and Pediatric ARDS (1). Explicit protocols support research reproducibility, and clinical practice based on best available evidence, by disambiguating guidelines. When the majority of the care process becomes reliable and predictable, expert clinicians can focus on the situations that are critical or unusual (16). Explicit protocols can be paper based or implemented within a computer system. Computer-based clinical decision support (CDS) tools can support consistent protocol navigation, capture data at each decision point, and monitor protocol adherence (3). Whether paper based or electronic, protocols need to be acceptable to clinicians if they are to be widely used (17). Clinicians using protocols retain the responsibility to assess applicability of protocol recommendations for each clinical situation (16). In this study they were able to decline protocol recommendations in each patient scenario.

The pediatric ventilator protocol that served as an exemplar for conventional mechanical ventilation was adapted from ARDSnet protocols and adult CDS tools, based on expert opinion from pediatric intensivists (3) from the Collaborative Pediatric Critical Care Research Network (CPCCRN) and Pediatric Acute Lung Injury and Sepsis Investigators (PALISI), in an iterative process over several years. The HFOV protocol was based on a protocol developed in adults (18) where increasing amplitude rather than decreasing frequency is promoted as the lung protective strategy in situations of significant acidosis. The exemplar protocol is detailed and explicit with separate rule sets for differing ventilator modes: high frequency oscillatory ventilation (HFOV); and conventional pressure control (PC), pressure regulated volume control (PRVC), and volume control/assist control (VC/AC) ventilator modes. The strategy used in the exemplar rules is to identify the patient’s physiological status and make recommendations aiming to keep ventilator settings in line with lung-protective strategies. Rules take the form of if-then statements, such as: IF PRVC mode and pH >7.45 and Tidal Volume > 6 ml/kg and PIP <30 cmH20 THEN decrease Tidal Volume by 1 ml/kg. Multiple recommendations may be generated each time ventilator management is evaluated, with changes to inspired oxygen (FiO2), ventilator rate, and/or ventilator pressures.

Theoretical Framework

The Unified Theory of Acceptance and Use of Technology (UTAUT) (19) informed our study design. The UTAUT integrates elements from multiple behavioral and motivational models and is a widely used framework for evaluating health technology (20). Behavioral intent, or statements suggestive of a willingness to use the technology, reliably predict actual technology use. Four constructs are seen to influence behavioral intent: performance expectancy (will the technology be helpful in performing a specified task), effort expectancy (ease of use), social influence (belief that peers, supervisors, or others think you should use the technology), and facilitating conditions (resources and support). Each of these constructs can be influenced or modified by participant characteristics such as age, gender, or experience (19,20). Performance expectancy may also be influenced by the “fit” between the technology and the tasks to be accomplished (21,22). For this study, fit was operationally defined as perceived appropriateness of recommendations given a clinical scenario, and measured as participant acceptance of the scenario recommendation.

Materials and Methods

After review and approval by the University of Utah Institutional Review Board (IRB) and site IRBs, we invited 192 intensive care attending physicians and fellows in the NIH/NICHD Collaborative Pediatric Critical Care Research Network (CPCCRN) (23) to participate in an online survey from Dec 2012 through Feb 2013. The survey was delivered via Checkbox survey software and housed at the CPCCRN Data Coordinating Center at the University of Utah. We asked questions about specific ventilator management practices. Attitudes and perceptions about computer protocols were assessed with Likert-type questions (1 = strongly disagree to 5 = strongly agree) modified from the UTAUT questionnaire. We replaced generic phrases in the UTAUT questionnaire (“the system”) to ask specifically about computer protocols. The UTAUT questionnaire is widely used and well-validated in multiple clinical contexts, and was designed to accommodate the type of modifications we made (19,24).

This survey was the second of a two-part study. Aim 1, conducted just prior to this survey, analyzed mechanical ventilation practice in routine clinical care, in the CPCCRN intensive care units. The aim 1 analysis (25) included examining usual care data in subsets that correspond to the physiological states (IF statements) in the exemplar protocol. This allowed us to design focused scenarios for this survey, in areas where it appeared that there was a lack of consensus among critical care experts. We created 50 scenarios that spanned PC, PRVC, and HFOV ventilator modes, the more frequently used modes of ventilation in the CPCCRN sites at the time of this study. Scenarios explored issues of recommendation granularity (size/scale) and acceptability. For example, the adult ARDS protocol made inspired oxygen concentration changes in increments of 10%, but our experts had initially suggested that changes in increments of 5% may be more acceptable for pediatric patients; while examination of data from CPCCRN sites showed clinical practice almost evenly split between inspired O2 changes of 5% and changes of 10%. We similarly explored the size of PIP and PEEP recommendations. We focused scenarios on areas where clinical practice seemed to have highest variability, and areas where specific recommendations might be seen as “controversial” by some clinicians such as changes to PEEP versus FiO2. To explore granularity, many scenarios recommended changes that were larger or smaller than the recommendations currently in the exemplar protocol. Clinician collaborators (critical care RNs and MDs) reviewed the scenarios for clarity, readability, and clinical plausibility.

With 50 scenarios, we were concerned about possible respondent fatigue; a phenomenon in which participants become tired of the survey task and attention and motivation drop toward later sections of the questionnaire (26). We evaluated this using 2 approaches. We created 6 survey versions with identical scenarios that were presented in different sequences; participants were randomly assigned to a survey version. We evaluated for differences in response patterns based on survey version (scenario order). In addition, we interspersed a few scenarios that had sub-optimal but still plausible recommendations, on the premise that attentive participants would likely decline those recommendations.

Differences in mean recommendation acceptance rate, among groups defined by participant demographics and recommendation content characteristics, were analyzed using analysis of variance (ANOVA). Pearson correlation coefficient was used to test for a relationship between overall acceptance rate and participant age. Independent t-tests were used to test for differences in acceptance rate between small and large recommended changes. We used generalized linear models to assess for interactions between child size and other factors; acceptance rate was used as the dependent variable and categorical variables as fixed factors. Analyses were performed by statisticians at the CPCCRN Data Coordinating Center using SAS software, version 9.4 (SAS Institute).

Results

Participant description

Of 192 invited physicians, 140 (71.8%) responded to at least some questions and 122 (63.5%) responded to the entire survey (all questions and scenarios). Only completed responses were used for statistical analysis, as per CPCCRN statistical analysis procedures. Incomplete responses did not differ significantly from the completed responses. Demographics (Table 1) showed a broad range of clinician characteristics. Males and females were equally represented. Mean age was 40.5 years (SD, 10.1 years). Participants were mostly white (70%) and not Hispanic or Latino (87%), which appears to reflect the physician characteristics in the participating hospitals. The participants were experienced (61% Attending, 40% Fellow) physicians with ICU experience ranging from less than 1 year to more than 10 years.

Table 1.

Acceptance rate by demographic characteristic

Characteristic	Sample description n (%)	Recommendation Acceptance rate (%)
		Mean (SD)	p value*
Gender			0.33
Female	53 (43%)	78 (14)
Male	68 (56%)	82 (12)
Unknown	1 (1%)	77
Ethnicity			0.59
Hispanic or Latino	8 (7%)	81 (10)
Not Hispanic or Latino	106 (87%)	80 (14)
Unknown/prefer not answer	8 (7%)	75 (14)
Race			0.64
White	85 (70%)	81 (12)
Asian	27 (22%	78 (17)
Other or multiple	3 (2%)	85 (1)
Unknown/ no answer	7 (6%)	76 (15)
Professional Role			0.26
Attending	75 (61%)	81 (13)
Fellow	47 (39%)	78 (15)
Years ICU experience			0.90
Less than 1	16 (13%)	78 (16)
1 to 3	29 (24%)	79 (14)
4 to 10	33 (27%)	80 (13)
More than 10	44 (36%)	81 (13)
	Mean (SD)	Overall acceptance
Age (years) n=109	40.5 (10.1)	80 (13)	0.21**

Overall total n participants = 122.

^*p values, except where noted, are from an ANOVA test.

^**p value is based on the Pearson correlation coefficient.

Recommendation Acceptance

Overall, 80% of the simulated protocol recommendations were accepted (Table 1). Recommendation acceptance rate was not associated with clinician demographic or professional characteristics including physician experience. The acceptance rate did not vary by CPCCRN clinical site (p = .72). Acceptance rate did not vary by the order in which scenarios were presented (survey version) (p=.25), and 89% of participants declined the sub-optimal but plausible recommendations, suggesting that participants remained engaged throughout the survey.

Acceptance rate did vary by ventilator mode and by the specific parameter adjusted (Table 2). PEEP change recommendations had the lowest acceptance (69%), whereas changes to FiO2 were accepted at a higher rate (86%), t=7.2, p<.001. This is consistent with what we had observed in observation of clinical data. HFOV settings were seldom changed in routine clinical practice, but survey responses showed 78% acceptance for amplitude change recommendations and 74% acceptance for changes to frequency, suggesting that clinicians may be willing to consider changes if prompted.

Table 2.

Acceptance rate by recommendation characteristic

Recommendation Characteristic	Acceptance rate % accepted Mean (SD)	p-value
Ventilator Mode		0.002
HFOV	83 (15)
Pressure Control	75 (15)
PRVC	82 (10)
Parameter changed in the recommendation		< 0.001
FiO2	86 (10)
Frequency (HFOV mode)	74 (19)
Amplitude (HFOV mode)	78 (10)
Ventilator rate or tidal volume	82 (12)
PEEP	69 (12)
PIP	88 (10)

Recommendation Granularity

Although there appeared to be a slight preference for smaller changes (Table 3), the differences were not statistically significant for smaller (5% adjustment) versus larger (10% adjustment) changes to FiO2 (p=.25) or PIP (2 cmH20 vs 4 cmH20, p=.26). There was a preference for smaller (2 cmH20 vs 4 cmH20) changes to PEEP (p<.001).

Table 3.

Granularity evaluations

Parameter	Acceptance Small change	Acceptance Large change	Effect of granularity on acceptance (p)*
FiO2 (5 vs 10)	90%	87%	0.25
PEEP (2 vs 4)	72%	53%	<.001
PIP (2 vs 4)	90%	85%	0.26

^*p values are from a two-sided t-test.

We created parallel scenarios with a smaller (younger) child and a larger (older) child, to see if child size/age affected recommendation acceptance rate. Across all scenarios and recommendation types, there was no difference in acceptance rate for recommendations for a smaller/younger child (79.6% accepted) or larger/older child (80.2% accepted). We found no interactions between child size and any other factors (recommendation granularity, ventilator mode, parameter adjusted) in terms of influence on recommendation acceptance rate.

Ventilator Management

General settings

We asked how often ventilator management should be evaluated and settings potentially changed, for a child with ALI or ARDS who has been stabilized. The most common responses were every 4 hours (39%) or every 2 hours (36%), but responses also included every 1 hour and “very frequently”, as well as longer time frames (every 6 hours, every 8 hours). Some responded that timing was variable or “it depends” (6%).

We asked which body weight parameter was chosen, when ventilating to a targeted volume per kilogram. PICU admission weight (48%) and predicted body weight (36%) were the most common choices. The ARDSnet guidelines and the adult computer protocol from which the pediatric protocol was derived use predicted body weight (26).

Oxygenation Index

High frequency oscillatory ventilation (HFOV) mode is often used for severe oxygenation problems (2, 28). The Oxygenation Index (OI) calculation reflects oxygenation dysfunctions, with higher scores indicating more severe illness (14, 29). We asked if there was a particular OI score at which physicians would choose, or strongly consider, switching from conventional to HFOV ventilation. The most common response was OI = 20 with most participants indicating a value between 20 and 30 (Figure 1).

Figure 1.

Oxygenation Index threshold at which physicians would consider moving patients to HFOV mode.

Volume Control/Assist Control (VC/AC) Ventilation

Preliminary data suggested that VC/AC mode was seldom used in the CPCCRN PICUs. However, 87 survey participants (72%) reported that this mode was used in their PICU. Further, most indicated that if a child was on VC/AC ventilation they would leave the child on that mode, whether this was a young/small child (n = 87, 72%) or a larger/older child (n = 92, 76%).

We asked about pressures in VC/AC mode; specifically, about plateau pressure and peak inspiratory pressure (PIP). The majority (108, 91%) of participants said they knew how to measure plateau pressure for a child on VC/AC ventilation, and more than half said they were “likely” or “very likely” to measure plateau pressure (Table 4). Responses were mixed regarding whether the participant would be more inclined to use PIP or plateau pressure for decision making about ventilator management.

Table 4.

Pressures in VC/AC mode

Question	n(%)
Do you know how to measure plateau pressure for a child on VC/AC ventilation?
Yes	108 (91%)
No	11 (9%)
How likely are you to measure plateau pressure for a child on VC/AC ventilation?
Very unlikely	11 (9%)
Unlikely	9 (7%)
Neutral	30 (25%)
Likely	46 (38%)
Very likely	25 (21%)
Would you be inclined to make decisions about the child’s ventilator management based on PIP or plateau pressure?
I would use PIP exclusively	5 (4%)
I would use plateau pressure exclusively	1 (1%)
I would use PIP more often than plateau pressure	65 (54%)
I would use plateau pressure more often than PIP	30 (25%)
I would use PIP and plateau pressure about equally	18 (15%)
I would use a different parameter (not PIP or plateau pressure)	2 (2%)

Perceptions about computer protocols

The responses to attitude and perception questions were moderately favorable toward computer protocols (Table 5). The majority of participants indicated they could use a computer protocol with no assistance (42, 35%) or with only help files (46, 38%). Nearly a quarter of participants, however (29, 24%), responded that they would like help. Most participants reported that their organizations would support computer protocol usage.

Table 5.

Perceptions about Computer Protocols

Category/Question Scored 1=strongly disagree to 5 = strongly agree	Mean (SD)
Attitude
Using a computer protocol for ventilator management is a good idea	3.7 (0.9)
Using a computer protocol for ventilator management is a bad idea	2.2 (0.9)
Social influence
My organization would support the use of a computer protocol	3.6 (0.8)
People who are important to me think that I should use a computer protocol	2.7 (0.9)
My peers will discourage me from using a computer protocol	2.5 (0.9)
Effort expectancy
It would be easy for me to become skillful at using a computer protocol	4.2 (0.7)
Performance expectancy
Using a computer protocol would enable me to accomplish tasks more quickly	3.6 (0.8)
Using a computer protocol would increase my productivity	3.3 (0.9)
I would find a computer protocol useful	3.7 (0.8)
Facilitating conditions
There are sufficient resources in my organization to support using a computer protocol	3.6 (1.0)
I have the knowledge necessary to use a computer protocol	3.8 (0.9)
Anxiety
I feel apprehensive about using a computer protocol	2.8 (1.0)
Using a computer protocol is somewhat intimidating to me	2.0 (0.9)
I hesitate to use a computer protocol for fear of making mistakes	2.3 (1.0)

Discussion

Physician perceptions about recommendations may influence protocol adherence by respiratory therapists and nurses (30, 31). Responses to attitude questions and reasonably high scenario recommmendation acceptance rates suggest that pediatric intensivists may be willing to consider using protocols to manage mechanical ventilation for pediatric patients with ARDS. This study examined apects of a protocol where clinical practice was variable or where protocol recommendations might be seen by some clinicians as controversial, including recommendations that were at adult scale (larger than the proposed protocol), and overall recommendation acceptance was still 80% . It seems logical to assume that protocol recommendations that are similar to existing clinical practices would have even higher acceptance rates. Therefore we believe that compliance with a pediatric MV protocol could potentially be at least as high as the acceptance rate for the scenarios in this study. However, actual practice doesn’t always match stated intent, and intensivists seem to have not yet reached consensus on certain aspects of MV, such as when to initiate HFOV, or on the approach for pressure changes.

We had anticipated that ventilator setting changes might need to be small to achieve an acceptable pediatric ventilator protocol. However, we found no strong consensus, other than preference for smaller changes to PEEP. Smaller PEEP changes than were examined here may result in different acceptance rates, but the values selected for this study were felt to be plausible given what we had observed in the clinical data. There was disagreement about OI thresholds, and little consensus around management of the VC/AC mode or what parameter to use to monitor patients who receive this mode of MV (Table 4). If few people actually measure Plateau Pressure, it makes it more difficult to design an MV protocol that includes Volume Control mode; and similarly challenges studies that are based on metrics like transpulmonary pressures at PEEP. The lack of strong consensus probably contributed to the variability in care practices observed in previous studies (2–5) and continues to present a barrier to the design of a pediatric MV protocol. Efforts to design an acceptable protocol should continue, as challenging current beliefs is the first step in gaining consensus and standardizing practice. The pediatric MV protocol that guided this study continues to be refined but has not been formally validated against clinically important outcomes such as ventilator-free days or mortality. Its actual benefits and compliance rate are unknown, and validation studies are needed (1,14).

There were limitations to the study. Participant physicians were limited to those in the pediatric intensive care units in the CPCCRN network; responses by other groups of critical care physicians or clinical sites could vary in ways not appreciated here. We assessed attitudes and perceptions about computer protocols in general. If questions had been addressed toward a specific protocol or clinical problem set, differences in either direction may have been more apparent. We focused on a specific exemplar, and findings for other MV protocols might be different. Only an intentionally targeted sampling of recommendations were evaluated.

Strengths of the study included an adequate number of participants for statistical analyses, and participation from sites across the country. Other strengths were general correspondence with previous studies and with clinical care practices, use of a protocol that was designed around best available evidence, and use of a validated theory-based survey to evaluate attitudes.

The lack of consensus reinforces the need for an agreed upon protocol for MV management, and for measurement of protocol compliance. At least for the conduct of trials, an explicit protocol could reduce variable ventilator management as a major confounder. Even after clinicians agree on an approach to MV, implementation can be challenging, though. CDS tools, despite documented benefits in consistent protocol navigation and ease of tracking compliance, face pragmatic, organizational, technical, and regulatory implementation challenges (17). The CDS tool that was the exemplar for this study contains features that have been shown to be important for implementation, including explicit recommendations provided at the time and location of decision-making, and documentation of reasons for not following recommendations (which can lead to rule refinement). Better integration into clinical workflow and feedback to clinicians are likely to also be important for successful protocol implementation (32).

Evaluating issues such as the extent to which computer-based ventilator protocols are deployable across multiple PICUs and how best to provide the tool to bedside clinicians (i.e, web based, through a dedicated computer at the bedside, or other means) are important next steps for this research. Integration of CDS into complex and changing pediatric intensive care units will require further study. This study focused on the content of rules, rather than implementation issues. Those will be evaluated in future work examining how well the protocol can be deployed to various PICUs and actual acceptability of the recommendations.

Conclusions

Although pediatric intensivists have philosophically embraced lung protective ventilation strategies, ventilator management continues to vary substantially. An accepted ventilator management protocol might encourage less variability and more lung protective decisions. It could also lead to robust evaluation of ventilator management practices, with less confounding of mechanical ventilation trials than under our current approach. However, a randomized, controlled trial is needed to determine if adherence to such a protocol leads to a better outcome.