Development and pretesting of an electronic learning module to train health care professionals on the use of the Pediatric Respiratory Assessment Measure to assess acute asthma severity

Abstract

BACKGROUND:

Severity-specific guidelines based on the Pediatric Respiratory Assessment Measure (PRAM), a validated clinical score, reduce pediatric asthma hospitalization rates.

OBJECTIVE:

To develop, pretest the educational value of and revise an electronic learning module to train health care professionals on the use of the PRAM.

METHODS:

The respiratory efforts of 32 children with acute asthma were videotaped and pulmonary auscultation was recorded. A pilot module, composed of a tutorial and 18 clinical cases, was developed in French and English. Health care professionals completed the module and provided feedback. The performance of participants, case quality and difficulty, and learning curve were assessed using the Rasch test; quantitative and qualitative feedback served to revise the module.

RESULTS:

Seventy-two participants (19 physicians, 22 nurses, four respiratory therapists and 27 health care trainees) with a balanced distribution across self-declared expertise (26% beginner, 35% competent and 39% expert) were included. The accuracy of experts was superior to beginners (OR 1.79, 1.15 and 2.79, respectively). Overall performance significantly improved between the first and latter half of cases (P<0.001). Participants assessed the module to be clear (96%), relevant (98%), realistic (94%) and useful (99%) to learn the PRAM. The qualitative/quantitative analysis led to the deletion of three cases, modification of remaining cases to further enhance quality and reordering within three levels of difficulty.

DISCUSSION:

Using rigorous educational methods, an electronic module was developed to teach health care professionals on use of the PRAM score. Using the back-translation technique, both French and English versions were developed and validated simultaneously. The pilot module comprised a tutorial and three case-scenario sections, and was tested on a target audience of physicians, nurses, respiratory therapists and medical trainees.

CONCLUSION:

The final electronic learning module met the clarity and quality requirements of a good teaching tool, with a demonstrated learning effect and high appreciation by health care professionals. Available in French and English, it is offered to facilitate implementation of PRAM-based acute pediatric asthma guidelines.

Evidence-based guidelines for the management of acute pediatric asthma are severity specific (1). In children, the major obstacle to guideline implementation is the assessment of severity. In fact, due to age and/or severity of the exacerbation, nearly all preschoolers and 48% of school-age children cannot reproducibly cooperate with spirometry in the emergency department (2). The Pediatric Respiratory Assessment Measure (PRAM), a 12-point clinical score, has emerged as a valid tool for assessing acute asthma severity in children one to 17 years of age. It exhibits all the requirements of a good clinical score including internal consistency, inter-rater reliability, discrimination and responsiveness to change (3,4). In two independent pediatric cohort studies, severity-specific guidelines based on the PRAM reduced hospital admission by 20% (5) and length of stay in the emergency department by approximately 45 min (5,6). Given its rapid uptake in several academic institutions and multicentre clinical trials (7–10), there is an increasing demand for an efficient means to offer rapid, effective and large-scale training of health care professionals to use the PRAM.

In settings that have implemented the PRAM, learning methods have primarily relied on oral presentations followed by bedside teaching by local champions (Personal communication, R Zemek, Children’s Hospital of Eastern Ontario, Ottawa, Ontario; DW Johnson, Alberta Children’s Hospital, Calgary, Alberta; and D McGillivray, Montreal Children’s Hospital, Montreal, Quebec). Four main difficulties are encountered with this traditional approach, namely: reaching all health care professionals who require training; teaching a score with three sensory (audio, visual and tactile) modalities; ensuring a uniform teaching experience in terms of asthma severity and case difficulty; and assessing acquired competencies. Because the target audience comprises various acute care health professionals (physicians, nurses, respiratory therapists, paramedics and health care trainees) working different shifts, an asynchronous teaching model in which learners could undergo the training anywhere at any time appeared to be advantageous over repeated around-the-clock presentations. Several interactive electronic learning modules to teach cardiac auscultation (11,12) and musculoskeletal physical examination (13) have successfully overcome the challenge of teaching audio, visual and tactile elements, and providing a uniform learning experience (14). Such electronic learning modules have proven to be effective at improving knowledge, problem solving skills and change in practice, thus confirming the acquired competencies (15–17). Consequently, an electronic learning module appears to be the best way to overcome the identified difficulties in teaching the PRAM to a target audience.

The purpose of the present study was to develop, pretest the educational value of and, based on received feedback, revise an electronic learning module to teach health care professionals and trainees on the use of the PRAM score to assess the severity of an acute pediatric asthma exacerbation.

METHODS

The study was approved by the Institutional Review Board of the Centre Hospitalier Universitaire (CHU) Sainte-Justine (Montreal, Quebec) and Our Lady’s Children’s Hospital in Ireland. Approval for the former was recognized by the ethics board of the other participating Canadian institutions, in which members and research personnel of Paediatric Emergency Research Canada (PERC) network worked. Informed consent for the audio-visual material acquisition phase was obtained from parents (or legal guardians), and assent from children seven years of age and older. Informed consent was obtained from all participants in the pretest phase.

Audio-visual material acquisition and processing

The 12-point PRAM has been decribed in detail elsewhere (4,18). Briefly, it consists of five elements: a visual sign (suprasternal retraction); a tactile sign elicited by palpation (contraction of scalene muscles); two audio signs documented on auscultation (air entry and wheezing); and oxygen saturation measured using pulse oximetry. Between February and August 2010, participants in the asthma clinic, emergency department, hospital wards and intensive care unit of the CHU Sainte-Justine were prospectively recruited to cover the entire spectrum of acute asthma severity. Children were eligible if they were 18 months to 17 years of age, presented with asthma, and (and/or their parents) had adequate English or French language skills. Patients with impending respiratory failure or other chronic diseases were excluded.

The participants’ suprasternal region was photographed and videotaped using a high-resolution video camera (HDC-TM700, Panasonic, USA), while respiratory sounds were audiotaped using an electronic stethoscope (Bluetooth electronic 3200, 3M, USA) at eight auscultation areas on each patient. A medical illustrator provided the drawings for the scalene and adjoining muscle positions, and the front and back of a child’s thorax. Under the direction of the authors, an audio-visual technician edited and standardized all audio and video recordings. Audio recordings were filtered to remove extraneous noise. The video recordings were edited for clarity. The framing of the photograph and video to exclude, or the addition of a blurry band to cover, the children’s eyes enhanced confidentiality. When indicated, some original recordings were deleted and replaced with higher-quality recordings recorded at other auscultation sites or in other patients.

Elaboration of the pilot educational module

An electronic tutorial explaining the rating of each criterion and the interpretation of the PRAM score was created using PowerPoint (Microsoft Corporation, USA). Eighteen unique clinical cases were elaborated, each comprised of a brief medical history, the visual and audio recordings, and the detailed proposed response. An independent group comprised of two pediatric emergency physicians and a pediatric respiratory therapist provided feedback regarding the structure of the tutorial and clinical cases. Three authors (FM, MM and SG) independently reviewed the proposed correct response of each of the 18 clinical cases and the classification into one of three levels of difficulty; discordance was resolved by consensus. The cases were then grouped in three sections of increasing difficulty, each with specific learning objectives. The French tutorial and cases were translated into English by a professional translator and back-translated to French by another translator; the final French version was compared with the original to ensure concordance. The French and English pilot modules available on DVD were composed of four sections: a tutorial and 18 original clinical cases grouped into three levels of difficulty (each level containing six original cases and two mixed cases, in which the audio records of one case and the video of the other were exchanged).

Pretesting the pilot educational module

Between August 2010 and November 2011, physicians, nurses, respiratory therapists and medical trainees of the CHU Sainte-Justine, the PERC network in Canada and Our Lady’s Children’s Hospital in Ireland were invited to participate in pretesting of the module’s psychometric properties in the language of their choice. Baseline demographics, self-reported comfort with the chest examination of a child with acute asthma, and previous experience with the PRAM and with e-learning were documented. Each participant individually completed the tutorial and the three case scenario sections. After each case, participants recorded their rating of each of the five PRAM criteria, the total PRAM score, the overall severity classification, their rating of the case difficulty, any technical problems and other comments including suggested corrections to the proposed correct response. After each section, participants completed a brief questionnaire evaluating the time taken for completion, attainment of stated objectives, content clarity, relevance, adequacy, usefulness, case realism, global appreciation and general recommendations. Participants participated in a draw or received $10 compensation.

Statistical analysis

The sample of 60 participants was conservatively selected to achieve 95% confidence on a two-tailed test that no item calibration was more than 1 logit away from its stable value on the Rasch test; between 27 and 61 participants were required assuming a balanced distribution of participants skills (beginner, competent and expert) (19).

Using standard statistics, the length of completion of the module, the module appreciation on a four-point scale of 0 (strongly disagree) to 3 (strongly agree), and the distribution of correct responses was reported. In combination with other qualitative and quantitative feedback, the polytomous partial credit Rasch model served to explore participants’ expertise and assess the case quality and difficulty (20). The Rasch model is a standard measurement model used in education to estimate person expertise and item difficulty on the same scale. Briefly, the probability of a correct PRAM score was modelled as a logistic function of the difference between the ability of participants to correctly rate the case and the difficulty of each case. In other words, individuals’ ability could be eliminated during the statistical estimation of the case difficulty (item parameter) and vice versa. After ascertaining concordance with the model expectations, the Rasch analysis provided statistics related to case difficulty (item), participants (person) and case-participant (person-item) interaction.

Participants’ expertise was confirmed by their PRAM score rating accuracy (as a binary outcome) and analyzed using a generalized estimating equation model, with case order as a time variable (using both a linear and quadratic effect) and expertise as a factor. The performance within each expertise level was analyzed using the Rasch Pearson separation index and presented as the group mean position and SD on the person-item location distribution, negative values indicating that the participants’ expertise was lower than the case difficulty.

The quality of each clinical case was scrutinized using the Rasch model fit, the percentage of correct response for each PRAM criterion and for the total PRAM score across each expertise level, and participants’ qualitative feedback. These served to identify modifications (editing, deletion and substitution) to the audio and/or video recordings and/or the proposed correct response. The difficulty level of each case was assessed using three constructs, namely: the participant’s assessment of the case difficulty; the percentage of correct PRAM score across each expertise level and the item position on the person-item location distribution; and a negative position reflected an easy case (ie, that the mean expertise of all participants was greater than the case difficulty and a positive position represented a difficult case).

The overall learning effect in the PRAM score rating accuracy was examined by the group difference in person location (or Pearson separation index) between the first and latter half of the retained quality cases using a Student’s unpaired t test, adjusting for case difficulty. Unfortunately, the two mixed cases per section could not be analyzed according to PRAM criterion due to nonindependence of criteria or by the total PRAM score due to the absence of a comparator because each mixed case pertained to two original cases. Statistical analyses were conducted using SPSS version 20.0 (IBM Corporation, USA) and RUMM 2020 (Rumm Laboratory); P<0.05 was considered to be statistically significant.

RESULTS

Acquisition of audio-visual material

Thirty-two children 18 months to 15 years of age were enrolled at the CHU Sainte-Justine. Most children were preschoolers (88%), Caucasian (69%) and male (56%), with a preponderance of children with moderate asthma (PRAM score 4 to 7 [44%]), and with a fair representation of those with severe airway obstruction (PRAM score 9 to 12 [13%]). Adequate representation of age groups, ethnicity, sex and asthma severity in the development of clinical cases was ensured.

Pretesting of psychometric properties

Seventy-two individuals were enrolled to pretest the pilot module, including 45 health care professionals and 27 medical trainees with a similar distribution of expertise levels (Table 1). One expert who completed only four cases was removed from the analysis. Forty participants (56%) completed the French and 32 (44%) completed the English versions. The median (25th percentile, 75th percentile) completion time of the module was 109 min (92, 143); the tutorial required 24 min (18, 30) and each section took 35 min (27, 47), 27 min (21, 35) and 24 min (17, 34) to complete for the novice, intermediate and advanced sections, respectively. The overall appreciation of the pilot module was excellent: 89% to 100% of participants gave positive feedback to the attainment of stated objectives, content clarity, relevance, adequacy, usefulness, case realism, global appreciation and general recommendations (Table 2).

TABLE 1.

Characteristics of participants

	n (%)
Health care professionals*, n	72
Physicians	19 (26)
Pediatrician	7 (10)
Pediatric emergency physician	9 (12)
Pediatric intensivist	1 (1)
Other	2 (3)
Nurses	22 (31)
Pediatric ward	4 (6)
Pediatric emergency department	8 (11)
Other	10 (14)
Respiratory therapist/physiotherapist	4 (6)
Trainees	27 (37)
Fellow	4 (5)
Interns	8 (11)
Clerk	15 (21)
Female sex	54 (75)
Years of practice (n=72)
0 to 9	47 (66)
10 to 19	12 (17)
20 to 29	5 (7)
≥30	7 (10)
Past e-learning experience	37 (51)
Comfort with pediatric chest examination
Novice	19 (26)
Intermediate	36 (50)
Advanced	17 (24)
Previous experience with PRAM use†
Yes	19 (26)
Expertise‡
Beginner	19 (26)
Competent	25 (35)
Expert	28 (39)

^*Participants were recruited through the Pediatric Emergency Research Canada network (n=26), at the CHU Sainte-Justine (Montreal, Quebec) (n=38) and at Our Lady’s Children’s Hospital (Ireland) (n=8);

^†Participants who reported having previously used the Pediatric Respiratory Assessment Measure (PRAM) in practice; those with previous experience reported having used the PRAM for a median (25th percentile, 75th percentile) of 27 (12, 36) months;

^‡Based on the self-declared level of comfort with the chest examination of a child with acute asthma and their previous experience with the PRAM, participants’ expertise was operationally defined as ‘beginner’ in those rating themselves as such, ‘competent’ for those reporting an intermediate comfort level and no experience with the PRAM, and ‘expert’ for those either with an intermediate comfort level and previous experience with the PRAM or those rating their skills as advanced

TABLE 2.

Per cent agreement regarding the qualitative assessment of each module according to participants

Questions	Section
	Tutorial	Novice	Intermediate	Advanced
The objectives stated in the module were met	69 (100)	69 (97)	66 (97)	66 (97)
The contents of the module were clear	69 (99)	69 (97)	66 (92)	66 (92)
The explanations were relevant	69 (100)	66 (98)	64 (97)	64 (97)
The amount of information presented in the module was adequate	69 (99)	69 (100)	66 (98)	65 (98)
The level of the information presented in the module was adequate	69 (99)	69 (99)	65 (97)	65 (98)
The case scenarios were realistic	58 (98)	68 (93)	66 (89)	65 (91)
The feedback provided was useful and clear	64 (98)	66 (100)	65 (94)	64 (97)
The module contributed to a better understanding of the PRAM score	67 (100)	69 (100)	66 (97)	66 (97)

Data presented as n (%). The novice section pertained to cases originally classified as easy; the intermediate section, to cases originally classified as moderate; and advanced to cases originally classified as difficult

Overall, the percentage of correct responses for each of the 18 original cases varied from 22% to 76% among experts (Figure 1). When a variation of one point around the correct PRAM was accepted to account for interobserver variability, the overall case performance increased from 54% to 77% among experts. Experts’ overall accuracy was significantly better than that of beginners (OR 1.79, 1.15 and 2.79, respectively) and showed a trend toward being superior to competent participants (OR 1.18, 0.75 and 1.83, respectively), adjusting for case order. Taking into account the case difficulty level, the group performance in PRAM rating skill improved significantly on the person-item location between the first half and latter half of cases (−0.17±0.33 versus 0.25±0.41; P<0.001), attesting to a learning effect.

Figure 1).

Percentage of correct responses on the y-axis across participants’ expertise labelled as novice (dotted line), competent (intermediate) individuals (long dash) and experts (advanced) (solid line). In the pilot module, six cases were originally rated as novice (N1-N6), intermediate (I1-I6) and advanced (A1-A6) and were presented in the order listed on the x-axis

The percentage of correct responses across expertise levels, participants’ assessment of case difficulty and case position on the Rasch test item location provided complementary, yet sometimes conflicting, information on the case difficulty (Table 3). With the exception of several cases in the advanced section rated as less challenging, participants’ assessment of the case difficulty level was generally concordant with the original classification. The item location on the Rasch test suggested some degree of misclassification of difficulty across all sections. In fact, the overall difficulty level of cases was slightly superior to the mean participant expertise, with a negative mean (± SD) person position of −0.34±0.219. The qualitative feedback identified the case quality as the source of most of conflicting difficulty ratings and prompted minor or major modifications to each case.

TABLE 3.

Psychometric properties and classification of the 18 original clinical cases

Cases	Proportion of accurate response by participants’ self-declared expertise, %			Participants’ assessment of the case difficulty, %			Rasch test*			Modifications made†	Classification after modifications		Final classification
							Item location	Interpretation	P
	Beginner	Competent	Expert	Easy	Moderate	Difficult					Expert 1	Expert 2
Novice
N1	33	40	46	87	11	1	1.01	Difficult	0.63	Minor	Easy	Moderate	Difficult‡
N2	59	52	58	78	21	1	1.18	Difficult	0.93	Minor	Easy	Easy	Easy
N3	41	36	55	64	33	3	−1.26	Easy	0.03	Deleted	Moderate	Easy	–
N4	47	76	59	85	15	0	−0.44	Moderate	0.32	Minor	Easy	Easy	Easy
N5	37	46	64	69	27	5	−0.98	Moderate	0.1	Minor	Easy	Easy	Easy
N6	53	39	76	74	24	2	2.19	Difficult	0.93	Major	Moderate	Moderate	Moderate
Intermediate
I1	11	8	22	13	75	11	−1.35	Easy	0.52	Major	Difficult	Difficult	Difficult
I2	33	23	33	6	84	10	−1.1	Easy	0.52	Minor	Moderate	Moderate	Difficult‡
I3	26	38	44	16	73	11	1.53	Difficult	0.46	Major	Easy	Easy	Moderate§
I4	16	32	37	15	79	6	0.63	Moderate	0.02	Deleted	Difficult	Moderate	–
I5	42	44	63	20	77	3	0.37	Moderate	0.89	Minor	Moderate	Easy	Moderate§
I6	47	46	64	17	82	2	1.75	Difficult	0.34	Minor	Moderate	Moderate	Difficult‡
Advanced
A1	21	38	38	5	23	72	−2.1	Easy	0.83	Major	Moderate	Moderate	Moderate
A2	47	58	72	7	38	56	−0.66	Moderate	0.25	Minor	Easy	Easy	Easy
A3	50	63	68	9	50	41	2.48	Difficult	0.46	Major	Easy	Easy	Moderate§
A4	63	56	42	13	46	41	−2.59	Easy	0.18	Major	Easy	Easy	Easy
A5	37	54	67	10	20	69	−0.63	Moderate	0.67	Deleted	Difficult	Moderate	–
A6	42	54	63	17	47	37	−0.04	Moderate	0.59	Major	Difficult	Moderate	Difficult

^*The position of a case on the Person-item location distribution reflects the difficulty of the case in relation to the mean expertise of participants; a negative position indicates an novice case where the mean expertise of participants was greater than the case difficulty, a positive position denotes an advanced case whose difficulty level was superior than the mean participants’ skill level, while the position of intermediate cases was closed to 0. The cases were ranked in order from the easiest (most negative position) to the most difficult (most positive position);

^†A modification is considered to be: minor when an error in the proposed correction was rectified or changes were made within an audio or video recording to increase intrarecording homogeneity, with no impact on the clinical case’s proposed correction; it was considered to be major when audio or video recordings were moved, deleted or replaced, often affecting the clinical case proposed correction.

^‡Upgraded to ‘difficult’ because of subtleties and pitfalls in the interpretation of clinical signs in complex clinical situations, meeting the objectives of the advanced module;

^§Upgraded to ‘moderate’ because of subtleties in the recognition of indrawing and/or pitfalls in the interpretation of asymmetric findings, meeting the objectives of the intermediate module

Elaboration of the final module

Three of the 18 original cases were removed, namely, N3 and I4, which failed to meet the Rasch model assumptions, and A5, whose quality of recording was reported as poor and could not be satisfactorily improved. Whenever competent or expert participants failed a criterion, the confusing section of audio or video recordings in remaining cases were identified; all underwent modifications to maximize quality and to eliminate confusing recordings while making every effort to maintain the original integrity of the case. Of the 15 remaining cases, 10 received minor modifications, that is, changes within an audio or video recording to increase intrarecording homogeneity by deleting confusing breath cycles (eg, wheezing heard in only one of several breaths) or video sections (eg, difficulty to distinguish indrawing from swallowing in a moving child), with no impact on the clinical case’s proposed answer; alternatively, a correction in the proposed response was accepted. Five cases underwent major modifications in which poor-quality audio or video recordings were deleted and replaced, affecting the proposed answer. Importantly, to minimize the confusion reported by many participants between inspiratory and expiratory wheezing in a rapidly breathing child, all audio recordings were cut to four respiratory cycles and edited to start on inspiration. Several video recordings were edited for clarity and their length limited to 15 s.

Recognizing the potential impact of these modifications on the final difficulty level, two authors (MM and SG) independently verified the quality and assessed the difficulty of revised cases after all corrections had been made. All decisions were reviewed by a third author (FMD) in light of the objectives specified a priori for the novice, intermediate or advanced sections. These objectives specified that the novice level depicted cases with straightforward criterion rating, the intermediate level dealt with asymmetric findings, difficulties in the assessment of suprasternal retraction and/or pitfalls in PRAM scoring, and the advanced level presented complex cases with subtleties and technicalities in the interpretation of clinical signs. The final module contains five cases in each difficulty level.

DISCUSSION

Using rigorous educational methods, we developed an electronic module to teach health care professionals about the use of the PRAM score. Using the back-translation technique, both French and English versions were developed and validated simultaneously. The pilot module was comprised of a tutorial and three case scenario sections and was tested on the target audience of physicians, nurses, respiratory therapists and medical trainees. It was rated highly for all criteria including, clarity, realism, relevance and usefulness. As expected, the performance of participants for adequately rating the PRAM varied significantly across expertise levels, with experts performing significantly better than beginners. In light of the performance of expert participants and the apparent misclassification of, and discordance in, the difficulty level of several cases, each case was scrutinized and the quality systematically improved based on qualitative and quantitative feedback. The final e-learning module, composed of a 42-slide tutorial and 15 clinical cases divided in three levels of difficulty, is offered as a training tool to assist in the implementation of PRAM-based acute asthma treatment guidelines in children.

One of the challenges in module conception was simultaneously teaching five PRAM criteria, each dependent on one of three different sensory modalities: audio, visual and tactile. An additional challenge was the acquisition of high-quality and realistic audio-visual recordings. While recordings of respiratory sounds were commercially available, they were usually provided as single examples of given sounds and recorded in different patients; these could not be combined into a realistic case with the eight auscultation areas due to marked variation in respiratory rate, air entry pitch and wheezing pattern. The use of an electronic stethoscope to record breath sounds in acutely ill children with asthma was a salient feature of our module, as were the photograph of the neck and video recordings of suprasternal indrawing in naturally moving sick children. The realism of the cases was agreed on by most participants. The palpation of the contraction of scalene muscles, the only tactile element of the PRAM, was demonstrated in the tutorial by an animated anatomical illustration, instruction around self-palpation during deep inspiration and a demonstration video on a patient. Participants who correctly identified the appropriate palpation area using a click-mouse on the module screen were provided with the findings (presence or absence of scalene contraction) relevant to the specific case scenario. While the pretest of the pilot module could not confirm the ability to correctly identify the presence or absence of scalene contraction, participants learned the correct palpation area. The pilot module was recognized by participants as meeting the stated objectives. It was highly rated for its clarity, appropriateness of content and feedback, realism of case scenarios and relevance, and was considered to be a useful means of learning to use the PRAM. Importantly, as reported in other electronic modules, we observed a statistically significant improvement in the performance across all participants between the initial and the latter one-half of the cases, confirming that the module provided the expected learning experience.

Two important benefits of the pretesting were the external assessment of case quality and difficulty. As expected, experts performed better than those with self-declared beginners’ skills, but not significantly different from that of competent individuals, although the difference approached significance; this was true regardless of whether it was assessed by the person position on the Rasch or the percentage of correct response, suggesting similar competence between nurses experienced in the PRAM and physicians who constituted the expert group and that of nurses and residents untrained in the PRAM who generally constituted the competent group. On average, less than two-thirds of experts and competent individuals accurately rated the exact PRAM score as suggested in the proposed answer. The suboptimal performance may be accounted for, in part, by the expected inter-rater reliability previously documented at 0.72, which would explain a variation of a point above or below the correct response; in fact, the mean performance increased to 77% when a variation of one point around the correct PRAM was accepted. Participants’ performance also varied widely across cases. The Rasch analysis enabled dissociation of the effect of individual skills from that of the case characteristics. We suspect that variation in the case quality interfered with the assessment of case difficulty. The distinction between the latter two was significantly aided by participants’ comments.

The quantitative and qualitative feedback led us to make at least minor changes to all cases to reduce nonhomogeneity within the same audio or video recording, remove unclear sounds or video section, and enhance the distinction between inspiration and expiration. While every effort was made to ensure the fidelity of the revised to the original case, some modifications to the PRAM score were inevitable. All cases were reassessed for quality and the accuracy of the PRAM score was independently reviewed and met the approval of two experts. While we did not retest the final module with a new group of participants, we firmly believe the modifications would only have improved the cases’ quality and heightened their learning experience; this will be confirmed in a future study to examine, using similar techniques, the performance characteristics of the final module in consenting health care professionals accessing the online module.

We acknowledge the following strengths and limitations. Consistent with other well-accepted asthma tools (21,22), the module was created in an iterative fashion using a rigorous bilingual development process. The feedback from independent experts, followed by pretesting of the module’s psychometric properties in a large sample of the target audience of health professions, with an evaluative design using qualitative and quantitative feedback, and appropriate statistics to dissociate individuals’ expertise from case quality and difficulty represent significant strengths of this educational project. The case selection with a majority of preschoolers reflects reality because most children presenting with an acute asthma exacerbation are <6 years of age (23,24). Because the pilot module was available as a PowerPoint presentation with participants’ answers recorded on paper, it was possible to verify the correct answer slide before recording their PRAM assessment; thus, we cannot exclude the possibility of an overestimation of performance. Some participants encountered technical difficulty playing certain video or audio recordings on computers with different operating systems and suboptimal video or audio card, resulting in a small proportion (5%) of missing data. Because the use of mixed cases proved useless, we would recommend selecting one case per difficulty level and offer them as pre- and post-test to ascertain the efficacy of different training methods (ie, online versus conventional teaching). Our study was not designed to test the participants’ PRAM accuracy in the real-life setting or to test a potential shorter version. However, the realism of clinical cases, a key feature underlined by participants, has been known to enhance acceptance, effective acquisition skills, retention and satisfaction by the learner; the final module should prove superior to the pilot (14,25,26). The final module will be made available on the University of Montreal (Montreal, Quebec) website (www.childasthmatools.umontreal.ca/PRAM) to facilitate training at the user’s most convenient time and place as well as just-in-time learning, that is, serving accessible resource when and where the learner needs it (14,27,28).

CONCLUSION

We developed and validated an electronic learning module, rated by target end-users to be a clear, relevant, realistic and appreciated tool to assess the severity of an asthma exacerbation using the validated PRAM score. It met the essential psychometric properties of a good educational tool – namely, relevance and quality – and demonstrates an important training effect. We are offering this module in French and English versions at www.childasthmatools.umontreal.ca for health care professionals to learn the PRAM and facilitate implementation of PRAM-based pediatric acute asthma guidelines.