Implementation and Evaluation of the OWLL Intervention to Improve the Quality and Safety of Pediatric Dental Sedation: A Mixed-Methods Approach

Abstract

Background:

The OWLL (Open-Wide Learning Lab) intervention was developed using Human-Centered Design to improve the quality and safety of pediatric dental sedation. The intervention includes a patient-facing informational sedation brochure and video, and an enhanced set of clinical sedation records. Here, we report on the process evaluation 6 months after implementation in daily clinical practice.

Methods:

This study was conducted at the outpatient pediatric dental clinics of 2 large US academic dental institutions. We used a mixed-methods design to assess the fidelity, acceptability, appropriateness, and feasibility of the OWLL intervention. Quantitative data were gathered through chart reviews, while qualitative data were collected through surveys and interviews with providers and patients.

Results:

During implementation, the intervention was used for 81 pediatric sedation patients across sites. Overall, the intervention was well-received by both clinicians and patients. Chart reviews indicated that sedation records were consistently utilized, though occasional gaps in data completeness were observed. Providers welcomed the intervention, finding it compatible with their daily routines and easy to use. However, some challenges were reported, primarily related to time constraints and the need to adjust clinical encounters for effective intervention integration. Parents appreciated the sedation brochure and video, describing them as a valuable and positive addition to the care process.

Conclusion:

The findings suggest that the intervention was used and is a welcomed addition, with the potential for broader application in clinical settings to further improve patient care. Future efforts should examine the intervention’s impact on patient safety outcomes across diverse clinical settings.

Patient safety in dentistry is a critical aspect of delivering high-quality oral health care.^1,2 Published studies have documented that dental providers may unintentionally make errors that can result in adverse events and patient harm.^3–5 The dental profession is beginning to adopt a more proactive approach to identifying and mitigating potential risks and adverse events during dental procedures to enhance patient outcomes.^2,6–8 Central to delivering safe and high-quality care is the patient experience.^9,10 Negative experiences often stem from unclear communication, unmet expectations, and a lack of rapport between providers and patients.^11,12 When patients are well-informed, supported, and valued, they are more likely to engage in their care and adhere to instructions and safety measures, resulting in more favorable health outcomes.¹³

In the pediatric dental literature, there is limited understanding of the experiences of children and their parents during dental sedation and how it may impact patient safety (In this article, the term “parents” is used to refer to both parents and legal guardians).^10,14 Available research on dental sedation often focuses on the provider’s success in completing treatment, often overlooking the perspective of patients and their parents.^10,14,15 By minimizing discomfort and fear, sedation aims to ensure a more comfortable and positive experience for pediatric patients.¹⁶ This not only helps in managing immediate anxiety but also contributes to better long-term oral health by fostering a more positive attitude towards dental care. Therefore, addressing the patient experience is crucial, as it significantly impacts the safety and quality of dental care and patients’ attitudes towards future care.

We used a human-centered design approach to develop the OWLL (Open-Wide Learning Laboratory) intervention to improve the quality and safety of pediatric dental sedations.¹⁷ The multicomponent OWLL intervention includes: (i) a parent-facing brochure, “Pediatric Dental Care with Conscious Sedation: What-to-Expect.” Going beyond a simple checklist of what to expect during their child’s sedation procedure, the brochure was tailored to highlight safety messages. Available in English and Spanish, the brochure details what to expect before, during, and after the sedation appointment; (ii) a patient educational video, “Beni’s Sedation”: A brief, child-friendly video that guides patients and parents through the sedation experience. Using a stuffed animal, Beni the bear, the video provides a social story and simulates each step of the visit, from health check to recovery. Available in English and Spanish, the video can be accessed through a QR code on the brochure. Parents are directed to view the video with their child before the sedation appointment; and (iii) an enhanced set of sedation records for the dental staff for documenting patient and sedation information, including behavioral data. The redesigned sedation forms include a digital preoperative sedation form with questions about patients’ temperament, behavior patterns, and attitudes toward dental visits, a paper-based intraoperative sedation form with dedicated sections for patient behavior, and a digital postdischarge form to capture postdischarge sedation-related adverse events (Appendix A, http://links.lww.com/JPS/A775 and B, http://links.lww.com/JPS/A776).

In this study, we aimed to determine whether the intervention could be implemented in clinical practice, whether it was perceived as useful and meaningful by the providers and patients, and whether we could collect patient behavioral outcomes data to inform a future efficacy study.

METHODS

Study Design

We used a mixed-methods design to evaluate the fidelity, acceptability, appropriateness, and feasibility of the OWLL intervention.

Implementation

We implemented the OWLL intervention across the graduate pediatric dental clinics of 2 academic dental institutions (Site #1 and Site #2). The implementation period lasted 6 months at Site #1 (August 2023 to January 2024) and 4 months at Site #2 (October 2023 to January 2024). These sites participated in earlier stages of the project, and the clinical teams provided input and feedback at all stages of development.¹⁷ Before the rollout of the intervention, we conducted 2 training sessions, one at each site, to train the pediatric dental residents and faculty on the use of the intervention. The intervention was then rolled out for all sedation patients during the implementation phase. Inclusion in the evaluation phase was limited to patient cases that co-occurred with all stages of the implementation. Patients with planned sedations scheduled before implementation, who therefore missed presedation intervention stages, were excluded.

Implementation Outcomes

We defined the implementation outcomes in line with Proctor et al¹⁸ conceptualization of implementation science research outcomes and Carroll et al¹⁹ conceptual framework for implementation fidelity. Intervention fidelity was analyzed using mixed methods by (i) examining adherence rates through comprehensive chart reviews and (ii) analyzing providers’ focus groups and parent interviews. The intervention acceptability, appropriateness, and feasibility were analyzed using qualitative methods. Table 1 describes the implementation outcomes evaluated in this study.

TABLE 1.

Implementation Outcomes

Outcome	Description	Data sources	Data types
Fidelity	The degree to which the OWLL intervention was implemented as described originally	Patient charts data Parent interviews Providers’ focus groups Providers’ survey	Quantitative Qualitative
Acceptability	The perception among providers and parents that the OWLL intervention is agreeable, palatable, or satisfactory	Providers’ survey Providers’ focus groups Parent interviews	Qualitative
Appropriateness (or usefulness)	Perceived fit, relevance, or compatibility of the OWLL intervention for the daily practice	Providers’ survey Providers’ focus groups	Qualitative
Feasibility	The extent to which the OWLL intervention can be successfully used within the dental clinic and its suitability for everyday use	Providers’ survey Providers’ focus groups	Qualitative

Behavioral Outcomes and Sedation Effectiveness Data

During each sedation, providers documented patient behavior in the intraoperative sedation records, categorizing it as: (i) crying but not moving, (ii) moving but not crying, or (iii) struggling (crying and moving). If the patient was quiet, no box was checked. At the end of the visit, the overall patient behavior was rated as: (i) excellent, (ii) very good (some limited crying), (iii) good (treatment interrupted but eventually completed), (iv) poor (only partial treatment completed), or (v) aborted (no treatment rendered). Sedation effectiveness was also recorded (Appendix B, http://links.lww.com/JPS/A776). All behavioral and sedation effectiveness data were extracted through chart reviews.

Data Collection and Analysis

Quantitative Data

We conducted a retrospective analysis of pediatric dental sedations completed at both sites. Patient charts and sedation records were accessed and reviewed. Two team members, one at each site, reviewed the charts. Relevant data were retrieved, captured, and managed using the REDCap electronic data capture tool.²⁰ All items and sections of the sedation forms were reviewed to assess data completeness. Singular items on the forms were considered completed when they were checked or answered. Sections containing multiple items were deemed completed if at least one item in the section was checked. For instance, the rapport-building section includes 5 predefined questions; if the provider documented an answer to at least one question, the section was considered completed. In addition, reviewers collected self-reported data on whether providers reviewed the brochure with parents, as documented in the preoperative sedation record, along with patient behavior and sedation effectiveness.

Qualitative Data

We conducted 2 focus groups with pediatric dental providers at each site (Site #1, n=17; Site #2, n=20). Each session, facilitated by one qualitative expert and one clinician, lasted an hour and included both structured and open-ended questions. In addition, an anonymous survey was emailed to the same providers at both sites to assess the intervention’s fit within clinical workflows, reception, and practicality. Adapted from the Intervention Appropriateness Measure (IAM), the survey asked providers to rate statements on a Likert scale from 1 (strongly agree) to 5 (strongly disagree), with 3 indicating neutrality.^21,22 We also conducted semistructured interviews with a random sample of parents of patients who had undergone dental sedation during the implementation period at both sites (Site #1, n=5; Site #2, n=2). The virtual interviews, conducted by a qualitative expert, covered the intervention’s acceptability, benefits, challenges, communication with providers, and improvement suggestions. Each interview lasted 15 to 20 minutes, was audio-recorded, and analyzed using thematic analysis.

RESULTS

Implementation and Patient Demographics

The OWLL intervention was used for 81 sedation patients during the study period (Site #1; n=48, Site #2; n=33). Patient demographics varied between the sites. At Site #1, the mean patient age was 8 years (SD=2.53), and sedations were exclusively oral. At Site #2, the mean age was 5 years (SD=2.0), and both oral (n=15) and intramuscular sedations (n=18) were performed. Full demographic details are provided in Table 2.

TABLE 2.

Demographics and Characteristics of the Patients Who Were Exposed to the OWLL Intervention During the Implementation Phase

	Values
	Site #1 n=48, n (%)	Site #2 n=33, n (%)
Age, y (mean±SD)	8±2.53	5±2.0
Age, y
Min	4	3
Max	14	10
Sex
Female	24 (50)	12 (36)
Male	23 (48)	21 (64)
Other	1 (2)	0
Ethnicity
Hispanic or Latino	13 (27)	18 (55)
Not Hispanic or Latino	34 (71)	10 (30)
Missing	1 (2)	5 (15)
Race
Black or African American	3 (6)	1 (3)
White	3 (6)	19 (58)
Asian	13 (27)	0
American Indian and Alaskan Native	0	0
Native Hawaiian or Other Pacific Islander	0	0
Other	29 (61)	8 (24)
Missing	0	5 (15)
Insurance*
Public	25 (52)	25 (76)
Private	27 (56)	5 (15)
None/self-pay	0	3 (9)

^* Site #1 patients had a combination of both private/public insurance.

Implementation Outcomes

Fidelity (Was the Intervention Delivered as Intended?)

Both self-report and chart reviews indicated strong fidelity for the educational brochure. Providers reported discussing the brochure with parents in 76% of sedation cases at Site #1 and 75% at Site #2. Focus groups revealed that providers used the brochure as a checklist to ensure parents understood critical sedation information. Fidelity was lower for the sedation video, which providers inconsistently referenced during communications with parents due to time pressures.

Documentation fidelity was high for the intraoperative sedation form, which was completed diligently at both sites. However, adherence to the preoperative sedation form was lower (Table 3). In focus groups, providers noted that the intraoperative form fit well into their workflow, but questioned the clinical relevance of some preoperative form items, leading to inconsistent usage. Postdischarge follow-up calls were successfully documented in all instances where parents could be reached, but this contact occurred in less than half of the sedation appointments (Site #1: n=18, Site #2: n=15).

TABLE 3.

Adherence Rates for the Preoperative and Intraoperative Sedation Forms: Percentage of Forms With Sections Completed at Each Site

Forms	Sections	Percentage of forms with sections completed
		Site#1 (N=48)	Site#2 (N=33)
Preoperative sedation form	Medical history and review of systems*	77	42
	Airway assessment*	75	100
	ASA classification*	75	97
	Indication for medical consultation*	71	94
	Behavioral Frankl scale*	79	91
	Patient temperament †	73	82
	Rapport building questions †	52	70
	Other questions (ability to take x-rays and oral medications…) †	71	94
Intraoperative sedation form	Patient assessment †	88	100
	Drug dosage calculations †	85	100
	Sedation agent administration †	83	100
	Patient preparation †	77	100
	Patient monitoring †	85	100
	Patient discharge †	79	100

^*Singular items.

^†Sections.

Acceptability (How Satisfied Were Providers and Parents?)

Survey data showed that the brochure was highly acceptable to providers (Site #1, Mean=1.5; Site #2, Mean=1.4), who strongly agreed with using the brochure in daily practice (Mode=1 at both sites). Beni sedation video and the preoperative sedation form were less enthusiastically received, with more variability in provider ratings (Table 4).

TABLE 4.

Summary Provider Ratings for the Intervention Components Using a 5-Point Likert Scale (1 = Strongly Agree, 2 = Agree, 3 = Neither Agree nor Disagree, 4 = Disagree, 5 = Strongly Disagree). Lower Mean Scores Indicate Higher Levels of Agreement, While Higher Scores Reflect More Disagreement.

Intervention component	Site #1	Site #2
	Mean (SD)	Mean (SD)
What-to-expect brochure
Welcomed	1.5 (0.6)	1.4 (0.5)
Fits daily practice	2.0 (0.6)	1.7 (0.8)
Easy to use	1.8 (0.9)	1.5 (0.5)
Beni sedation video
Welcome use	2.0 (1.0)	2.2 (1.1)
Fits daily practice	2.3 (1.0)	2.2 (1.1)
Easy to use	2.2 (1.1)	1.8 (0.6)
Preoperative sedation form
Welcomed	2.0 (0.9)	2.1 (0.9)
Fits daily practice	2.3 (1.0)	2.1 (0.8)
Easy to use	2.5 (1.1)	2.0 (0.4)
Intraoperative sedation form
Welcomed	2.1 (0.8)	1.5 (0.7)
Fits daily practice	2.1 (0.8)	1.6 (0.7)
Easy to use	2.1 (0.8)	1.7 (0.7)
Postdischarge sedation form
Welcomed use	2.2 (0.7)	1.6 (0.5)
Fits daily practice	2.4 (0.8)	1.7 (0.5)
Easy to use	2.4 (0.8)	1.6 (0.5)

In focus groups, providers had a unanimous positive reception of the brochure and expressed an interest in sharing it with outside clinics. However, challenges were noted, such as remembering to use the brochure consistently and uncertainty about its impact on parents and patients. Providers appreciated Beni sedation video but struggled to find time to discuss or refer to it. Table 5 highlights key findings from the focus groups.

TABLE 5.

Key Findings From Providers’ Focus Groups

Intervention component	Questions	Key findings
The “What-To-Expect” brochure	Why do you think it is important to use this material? What was challenging about using this material? How did it influence your practice (organization…)? Have you noticed an impact on patient care/ practice? What areas of your practice were impacted? Did the leaflet impact the quality of your discussion about the sedation appointment with the patient/parent? Would you like to introduce the leaflet in a place that does not have it?	Providers found the leaflet valuable because it reinforced information that parents may forget from the appointment. It served as a useful reminder and enhanced recall with visual cues. The main difficulties included remembering to use the leaflet and integrating it smoothly into the clinic workflow. There are some uncertainties about its impact on parents and how they utilize it afterward. The providers use the leaflet as a checklist to guide their discussion with parents/patients. There is interest in introducing the leaflet to additional areas where it is currently not used
Beni’s sedation video	Did you reference the video to the patient? Why or why not? How do you view the Beni video as a part of this project?	Providers valued the concept but struggled to find time to show or discuss the video. The video was not always referenced when distributing brochures. Providers desire a more inclusive video that uses terms like “caregivers” instead of “mothers/fathers,” less mother-centered language, and reduces anxiety by avoiding messages about parents leaving the operating room, for example
The preoperative sedation form	Did the tool assist you in the difficult discussions with parents/patients about capturing behaviors that might be problematic? Did it help to have an artifact to help converse with the parents about their child’s behavior? How confident did you feel in asking the behavioral questions? Did the information collected actually help with the sedation?	Providers found that the form did not significantly contribute to meaningful discussions with parents. They stated that parents found the questions on this form confusing or difficult to answer. Providers lacked confidence in asking the rapport-building questions as they felt the questions were sometimes not age-friendly (for example, what makes you happy?) …Providers did not always refer to this form and preferred using progress notes
The intraoperative sedation form	Does the form capture all the necessary information required for intraoperative sedation, or are there additional details that should be included?	The form effectively captures all necessary information required for intraoperative sedation, ensuring comprehensive documentation of key details
The postdischarge sedation form	Did you appreciate the digital format of the postoperative form? Was it easier for the faculty to approve the documentation? What were the benefits of the standardization that you noticed while filling out the form? What are the challenges?	Participants appreciated the digital format of the postdischarge form for its ease of use and found that the standardization of postdischarge events facilitated easier approval of the documentation by faculty. No specific challenges related to the standardization were described

From the parents’ perspective, the intervention materials were seen as integrated, nondisruptive components of their care. While they did not single out the brochure as a memorable item, they described it and other resources as helpful additions that complemented their primary interactions with the clinical team.

Appropriateness (Was the Intervention a Good Fit?)

Providers rated the brochure as well-suited to their clinical practice (Site #1, Mean=2.0; Site #2, Mean=1.7). The fit of other components, such as the Beni sedation video and the preoperative form, was rated as moderate with greater variability (Table 4).

In focus groups, providers stressed the need for a more inclusive video, suggesting terms like “caregivers” instead of “mothers” or “fathers” to acknowledge all families. They also recommended avoiding language about parents leaving the treatment room to reduce anxiety and create a more reassuring environment for children and families.

Feasibility (Could it be Successfully Used in this Setting?)

Providers at both sites found the brochure easy to use (Site #1: Mean=1.8; Site #2: Mean=1.5). The intraoperative and postdischarge sedation forms were also deemed feasible, particularly at Site #2. The primary feasibility challenge, identified in focus groups, was the time required to effectively integrate the Beni sedation video. Providers also noted hesitation in asking certain rapport-building questions, feeling they were inapplicable to younger children. Conversely, providers appreciated the digital format of the postoperative sedation form for its convenience and ease of use, and the standardized format for documenting postdischarge events streamlined the process and facilitated faculty approval.

Behavioral Outcomes and Sedation Effectiveness Data

Chart review data indicated that providers consistently tracked and documented patient behavior in most sedations. Patients showed signs of distress, agitation, and struggle in 39% of cases at Site #1 and 45% at Site #2. At Site #1, treatment was interrupted but completed in 15% of sedations and partially completed in 12%. At Site #2, treatment was interrupted but completed in 34% of sedations and partially completed in 3%. No treatments were aborted due to behavior at either site. Sedation was deemed ineffective in 17% of cases at Site #1 and 6% at Site #2. Patient behavioral data were missing in 23% of sedations at Site #1 and 15% at Site #2. Table 6 details patient behavior and sedation effectiveness data.

TABLE 6.

Patient Behavior and Sedation Effectiveness Data

	Site#1 (N=48) (%)	Site#2 (N=33) (%)
Patient's overall behavior
Excellent, no crying or movement	17 (35)	8 (24)
Very good, some limited crying	7 (15)	8 (24)
Good, treatment interrupted but completed	7 (15)	11 (34)
Poor, only partial treatment completed	6 (12)	1 (3)
Aborted, no treatment rendered	0	0
Not mentioned	11 (23)	5 (15)
Sedation overall effectiveness
Ineffective	8 (17)	2 (6)
Effective	19 (39)	19 (58)
Very effective	21 (44)	8 (24)
Overly sedated	0	0
Not mentioned	0	4 (12)

DISCUSSION

The findings of this evaluation indicate that the OWLL intervention has been utilized and positively received by providers and parents. Its implementation in the current settings suggests that the intervention could be adapted for broader application across various clinical environments. The human-centered design approach used during intervention design, implementation, and evaluation helped cultivate a collaborative partnership among researchers, providers, and patients.^23,24 The positive outcomes observed at both sites underscore the importance of this collaborative atmosphere, which is essential for fostering meaningful changes in clinical practice. Engaging key stakeholders promotes a sense of ownership and investment in the intervention. This approach encourages stakeholders to voice their perspectives, share insights, and contribute to the refinement of the intervention, thereby enhancing its relevance and applicability.²⁴ This partnership supports a culture of continuous feedback and improvement, allowing for ongoing adjustments that reflect the dynamic nature of clinical environments. As providers, parents, and patients engage in this iterative process, their experiences and suggestions can inform future intervention adaptations, further strengthening its implementation and effectiveness. Overall, the human-centered design approach not only enhances the intervention’s alignment with the needs of its users but also lays a strong foundation for sustained collaboration and innovation in health care delivery. Although well-received at both sites, implementation outcomes were more favorable at Site #2 than at Site #1. Several factors may explain this difference. At Site #1, the transition to a new electronic health record (EHR) System occurred around implementation, introducing additional complexity and potential workflow disruptions. In addition, Site #1 experienced changes in leadership around the implementation phase. In contrast, Site #2 benefited from a stable leadership structure and a consistent presence of on-the-ground champions who had been involved with the intervention from the beginning. Their ongoing presence reinforced the intervention’s objectives and encouraged consistent usage, resulting in a more favorable reception.

This evaluation showed that the redesigned sedation records could improve providers’ ability to track and document patient behavior during sedation. The real-time behavioral tracking section could help clinicians and researchers understand patient responses to sedation, identify trends, and adjust sedation practices. Mood and behavior fluctuations are common during sedation and affect both patient experience and provider well-being.^25,26 In our evaluation, 39% of patients at Site #1 and 45% at Site #2 exhibited signs of distress, agitation, or struggle during sedation, aligning with prior findings. A survey of pediatric dentists in the United States reported that 50% of respondents reported patient agitation during pediatric dental sedation.²⁷ Moreover, retrospective chart reviews of pediatric dental sedations have identified patient agitation in 48% of sedation cases.¹⁴ Negative patient behaviors can prevent the completion of dental procedures, leading to delayed care, repeated sedation visits, and traumatic experiences for patients, families, and providers. This evaluation also found that 12% of sedations at Site #1 resulted in only partial treatment rendered due to patient behavior. Detailed documentation of patient behavior is crucial for improving pediatric dental sedation, reducing stress, and fostering a positive care environment.

The preoperative sedation form demonstrated low adherence rates. During follow-up discussions, providers acknowledged inconsistencies in the use of the preoperative form. They expressed that some of the questions included in the form offered a limited perceived benefit. When providers do not see a clear benefit, their engagement may decline, leading to reduced use. Although providers were involved from design through implementation, the low adherence suggests a disconnect between initial input and practical utility. This highlights the need for ongoing feedback and refinement to ensure the form is used consistently and meaningfully in real-world settings.

The findings of this study may not be fully generalizable, as the research was conducted in only 2 academic dental settings. In addition, the potential influence of the Hawthorne effect should be acknowledged; participants may have adjusted their behavior due to the awareness of being observed or involved in an intervention, possibly enhancing implementation outcomes. Nevertheless, this study lays a robust foundation for future research aimed at assessing the OWLL intervention’s impact on patient safety outcomes across different clinical settings. Future work will focus on key metrics, including adverse events, patient anxiety, treatment outcomes and satisfaction. A broader approach across varied contexts will help develop standardized guidelines, improve care quality, and identify best practices.

CONCLUSION

Understanding the patient experience is key to advancing patient-centered care and improving the quality and safety of care. The findings indicate that the intervention was well-received and holds potential for wider application to enhance patient care across different clinical settings. Future research should explore its impact on patient safety outcomes in diverse environments.

Supplementary Material

pts-22-261-s001.docx ^{(249.2KB, docx)}

pts-22-261-s002.docx ^{(718KB, docx)}