Virtual Teach-To-Goal adaptive learning of inhaler technique for inpatients with Asthma or COPD

Abstract

BACKGROUND

Asthma and Chronic Obstructive Pulmonary Disease (COPD) result in over one-million hospitalizations annually. Most hospitalized patients misuse respiratory inhalers. This misuse can be corrected with in-person education; however, this strategy is resource intensive and skills wane quickly after discharge.

OBJECTIVE

To develop and pilot a virtual Teach-To-Goal (V-TTG) inhaler skill training module, using innovative adaptive learning technology.

METHODS

Eligible adults with asthma or COPD completed a V-TTG metered dose inhaler session with tailored rounds of narrated demonstration and self-assessments. The primary outcome was the proportion of participants with inhaler misuse post versus pre V-TTG; secondary analyses tested mastery, self-efficacy, and perceived versus actual inhaler skills. Analyses were tested with McNemar’s chi-square (p<0.05).

RESULTS

Among 90 enrolled participants, the majority were African American (94%), female (62%), and had asthma (68%), with a mean age of 48 years. Among those completing both pre- and post V-TTG (n=83), misuse was significantly lower post versus pre V-TTG (24% vs 83%, p<0.001). Mastery and confidence both improved significantly (46% vs. 7%. p<0.001; 83% vs. 67%, p<0.001) post versus pre V-TTG. After V-TTG, there was greater congruence between perceived versus actual inhaler skills (p<0.01). No differences were seen in subgroup analyses for age, health literacy level, or diagnosis.

CONCLUSIONS

This study is the first to demonstrate the efficacy of adaptive V-TTG learning to teach inhaler technique. V-TTG improved most participants’ technique to an acceptable level reached mastery for half, and also increased self-efficacy and actualized skill. V-TTG has potential to improve healthcare across care transitions.

Clinical Trial Registration

Clinicaltrials.gov#NCT02278237

INTRODUCTION

Asthma and Chronic Obstructive Pulmonary Disease (COPD) are two of the most common pulmonary disorders, and together result in over a million hospitalizations annually.^1–4 Current national financial penalties for COPD readmissions emphasize the growing urgency to reduce preventable readmissions.^5–8 Many hospital (re)admissions are thought to be potentially preventable as effective treatments exist to treat and control patient symptoms related to asthma and COPD,^9–12 However, despite these treatments’ efficacy, many patients suffer from exacerbations of their asthma or COPD requiring elevated levels of care.^13–16

A critical component of reducing preventable readmissions, is improving patients’ self-management, through increased knowledge, skills, and self-efficacy.^17–23 For instance, Teach-to-Goal (TTG) is an in-person educational strategy that reduces inhaler misuse.^18,24,28,29 Previous studies demonstrate that TTG is associated with reduced acute care.^18,28 TTG’s strengths include that it is patient-centered and tailored to the number of rounds of inhaler assessment and demonstration each patient requires. However, TTG requires provider time and training, may lack fidelity when attempting to implement through large scale interventions, and lacks portability for at-home refresher education.²⁸ Further, previous studies show that one TTG session is insufficient for long term retention.²⁸ Effective self-management skills require repeated assessment and education,^7,25 yet healthcare professionals are often under-trained or lack time to provide this needed education.^30,31 Further, patients need multiple sessions to learn and retain the skills.²⁸ Therefore, the use of self-directed adaptive learning technology to provide effective training for respiratory inhalers could improve standardization of, and access to, this critical education.

To address the existing barriers to access that clinicians and patients face for having access to effective, consistent, repeated patient training, we developed a virtual Teach-To-Goal (V-TTG) learning session to teach one critical self-management skill, inhaler technique. V-TTG uses adaptive learning technology to provide patient-tailored learning sessions that are modeled after the in-person TTG sessions. V-TTG allows for standardization of the inhaler technique instruction, which enhances fidelity. V-TTG can be accessed on any desktop or hand-held device, which increases its reach to patients outside of the health care system. Importantly, like TTG, V-TTG provides a tailored learning session that includes assessments and demonstration, repeated as needed for optimal patient learning. Before implementing V-TTG on a large scale, though, it was necessary to test its initial efficacy and validate the V-TTG learning session’s ability to improve knowledge, efficacy and skill. Therefore, the primary hypothesis is that hospitalized adult patients with asthma or COPD would be significantly less likely to misuse their metered-dose inhaler (MDI) after completing the inpatient V-TTG session.

MATERIALS AND METHODS

This pre/post study evaluates the efficacy of an interactive virtual educational strategy, V-TTG, that uses adaptive learning technology to customize the session to teach correct metered-dose inhaler technique to hospitalized adult patients with asthma or COPD. The self-assessment items were validated through iterative testing to determine whether they provided learning and/or assessment value. Trained research staff screened the electronic medical records of hospitalized patients on weekdays to identify patients admitted to the inpatient medicine service with asthma or COPD. The study was approved by the University of Chicago Institutional Review Board (IRB12-1844).

Patients were eligible for enrollment if they were 18 years or older, hospitalized on the general medicine service, with a physician diagnosis of asthma or COPD, with planned use of MDI after discharge, inpatient primary team assent, and written informed consent. Patients were excluded from the study if they were unable to provide consent (e.g., cognitive impairment or limited English proficiency), hospitalized in the intensive care unit, or previously enrolled in the study. A research assistant (RA) consented, enrolled, and completed the participant assessments before and after the V-TTG education.

Intervention: Virtual Teach To Goal Video Module

Participants completed the interactive V-TTG session comprised of cycles of demonstration and adaptive self-assessment (Figures 1 and 2) through short-answer questions (eFigure 1 and 2). The V-TTG is a self-contained platform developed with videos produced by Click to Play Media® (Berkeley, CA, USA) presented on an adaptive learning platform developed in partnership with, and hosted by, Smart Sparrow® (Sydney, NSW, Australia). V-TTG consists of a pre-assessment series of multiple choice and true/false questions about proper inhaler technique, followed by a narrated video demonstration. Participants are then presented with a post-assessment series of the same short-answer questions with the addition of an applied skill question using a video with incorrect technique shown. If participants failed to answer any of post-education questions correctly, they were prompted to re-watch the narrated demonstration and repeat the post-assessment up to three times in order to customize the learning session to participants’ needs. Upon completion of the post-education questions, additional information was provided on use of the MDI without a spacer.

Figure 1.

Conceptual Model of V-TTG

Figure 2.

V-TTG video module Screen Captures 1A: Opening screen; 1B: Demonstrator during the video of the correct inhaler technique; 1C: Example question (both pre/post). If answer is incorrect in post-test rounds 1 and 2, prompts to retake the question after reviewing the demonstration; 1D: If the question remains incorrect on the last round, the correct answer is provided.

Participants with incorrect answers on the final round were provided with the correct answer via written text on the screen.

Validation of V-TTG self-assessment items

The V-TTG pre and post assessment item domains are supported by Bloom’s learning theory and include: cognitive (knowledge), psychomotor (applied skills, e.g., identifying incorrect technique on a video-question), and affective (attitude: self-efficacy) domains.³² The validation portion of the study consisted of three rounds, with 90 patients enrolled. Participants were enrolled in a round until a total of 30 had completed the study, and then the next round was open for enrollment, until 90 total participants had enrolled in total. Each round had ongoing mid-point evaluations to continuously assess the items for validity, clarity, and utility (eFigure 2).

To test for the validity and utility of the self-assessment questions, we evaluated the proportion of participants that answered each question correctly on the pre and post demonstration assessments. Reasons for elimination or revision of any item included identification of results that indicated participant confusion of the item, overlapping content with another question, lack of significance, or questions that were determined to be too easy. For example, if the statistical significance of an item did not change after viewing the inhaler video demonstration portion of the module, most items were not included moving forward. This was because it was determined that participants either adequately understood the tested topic prior to the video demonstration and therefore did not require further education, the question was not written clearly, or the question was not effectively addressed in the demonstration video. The exception to eliminating items based on non-significant testing pre and post viewing the video demonstration was if the items contained perceived knowledge related to an actual skill. These questions remained in the V-TTG module to allow for comparison of perceived skill knowledge versus actual skill technique when observed by the RA.

Data Collection

Participants self-reported socio-demographic information including age, race, ethnicity, gender, address, education level, and smoking history via an interviewer-administered survey. In addition, participants answered questions about their utilization of healthcare services related to their asthma and/or COPD diagnosis. Health literacy was assessed using the Short Test of Functional Health Literacy in Adults (S-TOFHLA).³³ Vision was assessed using the Snellen screening card and sufficient vision was defined as better than 20/50 in at least one eye.³⁴

The primary outcome was MDI inhaler technique with spacer before and after V-TTG, as observed and assessed by a trained RA and scored using a previously validated 12-step checklist.^24,28 Participant inhaler technique skill was assessed by the RA before and after the participant completed the V-TTG session. The primary outcome of post vs. pre V-TTG inhaler technique efficacy was based on a definition of misuse previously published, categorizing misuse as less than 75 percent of steps (<10/12 steps) executed correctly. ^24,28 Because the misuse cutoff was determined as a conservative, clinically significant cutoff, further sensitivity analyses to determine efficacy were completed using complete mastery (12/12 steps correct), evaluating technique as a continuous variable, and adjusting the misuse cutoff by +/− 1 step (<11/12 and <9/12).

Statistical analysis

Descriptive statistics included means, medians, standard deviations, and proportions. T-tests were used to test for differences in means. McNemar’s exact test was used for paired comparisons to assess the primary outcome, the differences in misuse before and after completing VME, for the pre/post paired mastery comparisons, and for the validation of the assessment questions. Chi-square tests were used for non-paired comparisons. A sensitivity analysis was completed by testing complete mastery (perfect technique), +/− 1 additional item (misuse cutoff at <9 and <11 respectively), and using technique as a continuous variable, to ensure that efficacy was not presumed based simply on the previously published definition of misuse. A two-tailed p-value of less than 0.05 was deemed statistically significant. Computations were performed using STATA version 14.

RESULTS

Between November 2014 and October 2015, of 683 patients screened, 401 were ineligible, and 282 were eligible for the study; among those eligible, 90 participants were enrolled and 83 completed the study (Figure 3). Reasons for exclusion included being transferred to the intensive care unit or surgery (n=13), having been a prior participant (n=9), already having received inhaler education (n=19), being too ill or otherwise unable (e.g., near-blindness) to participate (n=5), not being able to obtain the clinician’s verbal assent (n=333), or did not meet other eligibility criteria (n=22). Reasons for non-enrollment when eligible included being discharged prior to approach (n=170), refusal to participate (n=21), or withdrawing consent prior to initiating the study (n=1). Reasons for not completing the study (n=7) included the participant being discharged before the study could be completed (n=5) or declining to continue the study (n=2).

Figure 3.

Flow diagram of screening, enrollment, and participation.

Demographic and descriptive data (Table 1)

Table 1.

Participant characteristics

	All Participants (n=90)	Asthma (n=61)	COPD * (n=29)	p-value
Age (Average [SD†])	47.9 [14.1]	42.5 [13.1]	59.2 [8.5]	<0.001
Female (n, %)	56 (62.2%)	39 (63.9%)	17 (58.6%)	0.6
African American (n, %)	85 (94.4%)	57 (93.4%)	28 (96.6%)	>0.999
High School: some or graduate (n, %)	42 (51.2%)	28 (48.3%)	14 (58.3%)	0.4
Sufficient Vision (n, %)	79 (88.8%)	55 (91.7%)	24 (82.8%)	0.2
Less than Adequate Health Literacy (n, %)	6 (16.7%)	4 (16.0%)	2 (18.2%)	>0.999
Sees Healthcare Provider (n, %)	49 (59.0)	31 (54.4)	18 (69.2)	0.2
Hospitalized in the last 12 months (n, %)	59 (65.6)	36 (59.0)	23 (79.3)	0.06
Near fatal (ICU‡ stay and/or intubation in lifetime) (n, %)	48 (53.3)	33 (54.1)	15 (51.7)	0.8

Note: Eight participants did not provide education data; only 36 participants completed the Short Test of Functional Health Literacy in Adults (STOFHLA), asthma (n=25), COPD (n=11) either due to insufficient vision or declining to complete the tool; 7 participants did not provide information about seeing a healthcare provider.

^*COPD, Chronic Obstructive Lung Disease

^†SD, Standard Deviation

^‡ICU, Intensive Care Unit

The mean age of participants who completed the study was 48 years and a majority of the participants were African American (94%), female (62%), and had asthma (68%) (Table 1). In general, participants with asthma and COPD had similar characteristics, participants with COPD were more likely to be older than participants with asthma (mean age: 59 vs. 43 years; p<0.001). No demographic differences were found between those completing and not completing the study. (Data not shown).

Pre V-TTG baseline respiratory inhaler misuse

Pre V-TTG baseline misuse was high for all enrolled participants, with 83% misusing metered dose inhalers (MDIs); the mean score was 6.7 steps correct, and only 7% had mastery. Baseline misuse rates for asthma and COPD patients were similar (87% and 79%, respectively). Performance for each step is shown in Table 2. Participants had the lowest baseline performance on attaching inhaler to the spacer (step 3: 73% missed), when breathing out fully and away from the device (steps 4 and 5: 76% and 78% missed, respectively). Likewise, only 3 steps had high performance (90% or greater getting step correct), these included: removing the cap (step 1: 100% correct), putting the device into their mouth (step 3: 94% correct), and removing the device from their mouth (step 10: 90% correct).

Table 2.

Percent of patients demonstrating correct inhaler technique by step (n=83) pre-and post- Video Module Education (VME).

Inhaler Steps Correct		Pre-VME*	Post-VME	p-value
1	Removes cap of inhaler and spacer	100%	100%	>0.999
2	Shakes inhaler up and down	57%	87%	<0.001
3	Attaches inhaler to back of spacer	27%	98%	<0.001
4	Breathes OUT fully	24%	81%	<0.001
5	When breathing out fully (step #4), does so away from spacer/MDI†	22%	80%	<0.001
6	Puts spacer mouthpiece or MCI mouthpiece (if not using spacer) into mouth, closes lips around mouthpiece	94%	98%	0.5
7	Activates inhaler by pressing down on canister 1 time	74%	89%	0.02
8	Breathes IN SLOWLY, filling lungs with medicine. No whistle should be heard	28%	82%	<0.001
9	Holds breath for at least 5 seconds (with or without spacer in mouth)	39%	86%	<0.001
10	Removes spacer/MDI from mouth before breathing normally	90%	95%	0.2
11	Breathes normally for at least 30–60 seconds	43%	76%	<0.001
12	Repeats sequence for second puff	67%	87%	0.004

P-values significant (<0.05) for all specific steps except “1: removes cap,” “6: places mouth,” and “10: removes mouth.”

^*VME, Video Module Education

^†MDI, Metered-Dose Inhaler

Post V-TTG respiratory inhaler misuse

Among the 83 participants with complete pre and post data, post V-TTG misuse decreased to 24% from 83% pre V-TTG (p<0.001) and complete mastery was achieved by 46% of patients compared to 7% pre V-TTG (p<0.001) [Figure 4]. Further sensitivity analyses demonstrated that adjusting the misuse cutoff by +/− 1 point on the checklist did not significantly change the results. If the misuse cut off increased to <11/12 steps or <9/12 steps, then pre vs. post V-TTG proportion of misuse was 91% vs. 34% (p<0.001) or 80% vs. 18% (p<0.001). Finally, using the score as a continuous variable, significance remained between pre and post V-TTG scores: post V-TTG mean score was 10.6 steps correct, significantly improved from the baseline of 6.7 steps correct pre V-TTG (p<0.001). In subgroup analyses, no differences in post V-TTG misuse were found for subgroup analyses by age, race, diagnosis or health literacy level, using any of the three misuse cutoffs (score <9, <10, <11; p>0.05) or by mastery (score=12; p>0.05), though sample sizes were low. The only difference in post V-TTG scores was level of health literacy (mean score low versus adequate health literacy: 9.7 vs. 11.2, p=0.02).

Figure 4.

Pre vs. Post V-TTG proportion of misuse with the original published cutoff (A: <10/12 score), two variations of misuse cutoff with +/− step (B: <9/12; C: <11/12); and perfect technique (mastery 12/12 score) in 83 participants completing both assessments.

Post V-TTG step-by-step comparisons

Post V-TTG, all steps improved (Table 2) so that at least 75% of participants performed each step correctly; this was a significant improvement (p<0.05) for all steps except those that were the highest performing steps pre V-TTG (1, 6, 10) due to ceiling effect (pre V-TTG started at 90% or greater; p>0.05). The lowest performing step post V-TTG was step 11, breathing normally for 30–60 seconds, with only 76% of participants performing this step correctly.

V-TTG self-assessment item validation

For the V-TTG knowledge questions, around half or more of the participants answered each of the six questions correctly prior to V-TTG education. After education however, the proportion of participants answering the questions correctly improved for all of the knowledge questions, with significant differences seen for all but one question (Table 3). This increase was consistently seen in each of the three validation rounds. With regard to the skills domain, four of the V-TTG knowledge questions (Qs 2, 4, 5, 6) tested self-perceived knowledge of associated inhaler technique skill (Figure 5). Of these four questions, none showed significant congruence between knowledge and skill prior to VME (p<0.01). After VME, there was greater congruence between participant knowledge and skills, with only one question (Q5, p<0.01) continuing to have discordance.

Table 3.

Pre vs. post V-TTG inhaler technique

Final #	Pilot #	Video Module Question	Domaina K/A/S	Pre n/N (%)	Post n/N (%)	p-value
Q1	Q1	How strongly do you agree: I am confident that I know how to use my rescue inhaler correctly	A	54/81 (67%)	66/80 (83%)	0.01
Q2	Q5	T/F: You should breathe in with rapid, shallow breaths	K	40/75 (53%)	57/64 (89%)	<0.01
Q3	Q6	Please number the steps in the correct order from first to last	K	39/78 (50%)	60/77 (78%)	<0.01
Q4	Q8	T/F: When you are told to take ‘2 puffs’ of your rescue MDI (inhaler) that means pumping the inhaler twice in a row	K	42/74 (57%)	68/76 (89%)	<0.01
Q5	Q9	T/F: Before you pump the MDI (inhaler) you should empty your lungs by taking a deep breath in	K	66/75 (88%)	75/75 (100%)	<0.01
Q6	Q10	T/F: If you hear the spacer make a whistle sound when breathing in to fill your lungs with the medicine, this means you are breathing in correctly	K	15/20 (75%)	21/23 (91%)	0.1
Q7	Q11	From the video that you just watched, please choose the step that was incorrect	S	-	52/76 (68%)	-

^aK/A/S: Knowledge, Attitude, Skill

Figure 5.

Assessment of inhaler technique skills vs. knowledge

With regard to attitude assessment, confidence in inhaler technique increased after education. Prior to education, 67% of participants agreed or strongly agreed with the statement, “I am confident that I know how to use my rescue MDI (inhaler) correctly,” increasing to 83% after education (p<0.01). (Figure 5)

DISCUSSION

This study is the first to demonstrate the efficacy of a self-directed adaptive virtual Teach-To-Goal learning tool to teach inhaler technique. Among hospitalized patients with asthma or COPD, V-TTG not only improved most participants’ technique to an acceptable level, but moved the bar to “mastery” for almost half of the participants. These results remained stable when adjusting the misuse cutoff score and when using the score as a continuous variable. Almost all of the individual steps improved, with the only limitation being a ceiling effect for a handful of steps where participants scored 90% or higher pre-VME. Further, participants’ confidence in their inhaler skills improved, as did congruence between their perceived knowledge of, and actual performance of, respiratory inhaler skills.

With respect to improving hospitalized patients’ MDI technique, this study extends the literature on educational interventions for hospital-based patients with asthma or COPD. Previous studies in hospitalized adults with asthma or COPD studied the efficacy of brief verbal instructions (BI) compared to an in-person TTG strategy.^18,24,28 Those studies demonstrate improved efficacy of TTG over BI in the hospital setting; however, both strategies demonstrate waning technique adherence by one month post-discharge.²⁸ Therefore, a strategy first used in the hospital and then repeated at home for reinforcement of the education is needed. For instance, hospitalized patients could first use V-TTG during admission, then receive instructions to repeat the learning session at home after discharge.

V-TTG may be an effective at-home strategy to teach and reinforce inhaler skills since it is efficacious at reducing inhaler misuse in the hospital setting. Further studies are needed to determine whether patients will be willing and able to use post-discharge V-TTG and to determine if at-home V-TTG improves retention of learned inhaler skills and whether this in turn leads to improved health outcomes.

This study also extends the evidence related to understanding the difference between patients’ perceived and actual knowledge of inhaler skills. Not only did participants’ confidence improve after the V-TTG session compared to before the education, the congruence between perceived and actual knowledge of how to use the inhaler improved. This indicates that virtual demonstrations with self-assessments can help illuminate the difference between participants’ self-perception that they have correct technique and actually being able to demonstrate correct technique. For instance, the majority of patients correctly answer “false” prior to viewing the video demonstration to a question related to breathing in with “rapid, shallow breaths” when using their MDI. However, their observed skill prior to completing the V-TTG session was usually incorrect. After the V-TTG session was completed, participants still answered the self-assessment item correctly; however they now demonstrated the concordant correct technique. These results illustrate that simply testing knowledge without testing actual skill may be insufficient to understand patient self-management skill learning needs.

There may also be practical advantages of using V-TTG within the hospital setting. In-person teaching requires significant resources including personnel time and costs. Further, ongoing education and training for those providing the intervention will be needed to initiate and sustain high-quality assurance of the intervention, incurring additional time and financial burdens. Research is needed to compare V-TTG directly to in-person inpatient education to determine if V-TTG is an adequate substitution for the current in-person standard. Should the V-TTG intervention prove non-inferior to in-person TTG, V-TTG may provide a more financially sustainable strategy while also providing greater fidelity to the educational intervention.

Therefore, these initial results demonstrating V-TTG as an effective educational strategy are very promising. However, there are limitations of this study, primarily related to generalizability and understanding the full potential of the intervention beyond the hospital setting. The study was performed at a single urban academic medical center and was only tested among hospitalized patients. Larger, multi-center studies are needed to examine increased generalizability and scalability, and to determine whether at-home VME leads to longer term retention of proper inhaler technique. Additionally, the study enrolled a predominantly high-risk, urban minority population with frequent emergency department visits and hospitalizations. Whether the improvement demonstrated in this study can be extended to other populations and settings is not clear. In addition, this particular education program only taught MDI technique with spacer. Not all patients use spacers with their MDI devices because they are not prescribed, are not available, or are not affordable. Further, numerous inhaler devices exist and require their own unique steps for correct technique. Future work should evaluate whether a similar strategy is effective for patient education to teach inhaler technique across diverse types of devices. Finally, should future studies demonstrate a comparative advantage of V-TTG to in-person TTG, cost-effectiveness studies will be required.

In conclusion, the study found that the innovative V-TTG adaptive learning strategy is an effective tool to teach MDI technique to hospitalized patients with asthma or COPD. After inhaler teaching with V-TTG, almost all participants demonstrated improved inhaler technique with reduced rates of inhaler misuse, and nearly half had complete mastery, demonstrating the potential efficacy of this learning tool. Larger, multi-institution comparative studies are necessary to examine the relative effectiveness of this V-TTG video module education in comparison to other inhaler education approaches, including in-person TTG demonstration, in the hospital setting. Ultimately, studies that test the full potential of V-TTG to be used across care transitions, such as at home after discharge and/or in outpatient settings, for longer term retention and improved health outcomes, are needed to understand the full potential of V-TTG to support long-term retention of inhaler technique skills and to improve patient health outcomes.

Highlights Box.

1. What is already known about this topic: Patients frequently misuse their rescue and controller devices that provide needed medications. In-person Teach-To-Goal sessions improve patients’ inhaler technique, but skills wane within 30-days. A portable, effective learning strategy is needed to refresh skills.

2. What does this article add to our knowledge: This article demonstrates that virtual Teach-To-Goal, a novel adaptive learning strategy, is effective at teaching patients inhaler skills and may help transform patient education by providing a portable, effective strategy for skill acquisition and retention.

3. How does this study impact current management guidelines: Current guidelines recommend assessing and teaching inhaler technique at all health care encounters, however lack of provider time and/or training is prohibitive. This virtual Teach-To-Goal strategy may be employed to achieve the guideline-recommended care.

Abbreviations

BI

brief verbal instructions

COPD

chronic obstructive pulmonary disease

MDI

metered-dose inhaler

RA

research assistant

TTG

teach-to-goal

V-TTG

virtual teach-to-goal