Examining perception and actual knowledge change among learners in a standardized burn course

The incidence of burn injuries in low- and middle-income countries continues to be high, and represents an important health problem in Africa. Given the significant proportion of individuals who survive burn trauma, the need for health care systems that can manage these complex injuries is great, especially in these settings. This article describes the creation, development and ongoing evaluation of a burn training program, created in conjunction with the Canadian Network for International Surgery, for parts of east Africa.

Abstract

BACKGROUND:

Essential Burn Management (EBM) is a burn training program created for East Africa and aims to meet the needs of low- and middle-income countries. The authors present a report on objective testing of change in knowledge, with pre and post tests, and comparison of this testing with the self perception of knowledge gained to explore course utility.

OBJECTIVE:

To evaluate the ability of EBM to improve knowledge in burn care among course participants; and to explore whether participants’ self-perception of knowledge gained is comparable with their actual change in knowledge.

METHODS:

Twenty health care providers from a variety of disciplines participated in EBM and completed the pre and post course burn care knowledge test. Participants also self-rated knowledge in burn care both pre and post course. All tests and self-rated assessments were anonymous and consent was obtained. Paired t tests were conducted on pre and post test scores. Improvements in pre-post scores according to module or topic area were examined to determine the proportion correct, and then analyzed using Wilcoxon signed-rank tests. Module scores pre-post course were compared with individuals’ self-rating of knowledge both before and after the course for that particular module.

RESULTS:

Pre-post course tests reflected an increase in knowledge. An increase in self-rated knowledge was matched with a significant increase in module test scores for primary survey, burn resuscitation, wound management, infection control and electrical injury, but not inhalation injury and compartment syndrome modules.

CONCLUSION:

Findings support a combination of self-report and objective pre-post testing to evaluate courses designed to teach burn management.

Burns are an important public health problem in African and other low-/middle-income countries (LMICs). Africa represents a significant proportion of global burn injury, having the second highest rate of fatal burns worldwide, and is responsible for 15% of global fire-related deaths (1). While the number of burn injuries in high-income countries are decreasing, numbers remain high in LMICs. Nearly 100% of fire-related deaths occur in LMICs, highlighting the fatal consequences of burn injuries in this environment (2,3). While many individuals succumb to burn injuries, a large majority survive. These burn survivors are in need of a health care system that can fully manage this complex problem (4,5). One of the ways to ensure that health care systems can manage the complex needs of burn survivors is through education. Essential Burn Management (EBM) is a burn training program created for East Africa and aims to meet the needs of LMICs. EBM was created in 2005 in conjunction with the Canadian Network for International Surgery (CNIS). The present article highlights the development and ongoing evaluation of this course. We previously reported on whether the course met local needs, and participants’ satisfaction with the course (6).We now report on objective testing of change in knowledge, with pre and post testing, and comparison of this testing with the self-perception of knowledge gained to explore the utility of the course. This is the first time that this evaluation of a burn course for LMIC is being reported in the literature.

EBM in East Africa: Study location and course development

Over the past five years, EBM was created and piloted in Ethiopia and Tanzania to address the important need for standardized burn care in a LMIC setting. Course content was based on standard burn teaching for Canadian medical students and residents, and modified through discussion with course participants and burn surgeons in Jimma (Ethiopia) and Dar Es Salaam (Tanzania). Course content and development details are available elsewhere (6). The present article provides findings from the second iteration of EBM that was taught in Dar Es Salaam. Dar es Salaam is Tanzania’s largest urban economic centre, with a population of 2.5 million (7). The setting of the course was at Muhimbili National Hospital, which serves as the de facto regional burn unit for Dar Es Salaam.

Brief course description and evaluation methodology

EBM is taught over a three-day period and comprises several components ranging from seminar instruction, skill stations using models, and simulation and interoperative modules (6). The overall course objective of EBM is “to provide the knowledge base, technical skills and rationale to create effective and competent burn teams in low-resource centres” (See Appendix A. More EBM course details available from CNIS on request). While the EBM course includes some surgical techniques and information, it was designed to educate all members of the multidisciplinary burn care team. The teaching of harvesting skin grafts portion of the course was held in the operating room with only the surgeons and direct surgical staff. During this operating room portion of the course, the overall intra- and postoperative care of the patient was discussed with the surgeons. During the classroom portion of the course, the rest of the burn team was introduced to the practice of grafting and what to expect, as well as discussions about postoperative care.

The complete evaluation of EBM uses three tools: the learner course evaluation questionnaire; the learner pre-post course test; and the facilitator and faculty post-course evaluation meeting. The present article presents findings from the learner pre-post course test.

The learner pre-post course test

The learner pre-post course test was distributed and completed at two time periods: following the introduction of EBM before course commencement, as well as after completion of the course just before the awarding of course completion certificates. The test questions were created by course instructors and examined for content and face validity by a modified Delphi technique, and consensus among the three burn care specialist instructors. A series of questions were vetted to ensure they reflected course content, wording and appropriateness, and were agreed on by consensus. The test consisted of 21 multiple choice questions, each with four possible response options. Test questions were written in English and were the same both pre and post, although the order of the questions was changed between the tests. Test questions reflected seven different topics or course modules: primary survey, inhalation injury, burn resuscitation, compartment syndrome, wound management, infection control and electrical injury. All learners provided their consent to using gathered information for research purposes. Pre-post tests were completed and submitted anonymously, but assigned numbers allowed them to be paired. Paired t tests were conducted on pre- and post-test scores (means). Participants were aware of their test scores immediately after completing the test. In addition to assessing a change in burn knowledge or test score, participants also self-rated their knowledge and comfort level in core subject topics before and after the course (6). Improvements in pre-post scores according to module or topic area were examined to determine the proportion correct, and then analyzed with Wilcoxon signed-rank tests. Effect sizes or Cohen’s d values were also calculated using standard deviations as the samples were dependent (8). Cohen’s d is a way to show the standardized difference between two means. Module scores pre-post course were compared with individuals’ self-rating of knowledge both before and after the course for that particular module.

RESULTS

Twenty-one learners completed the course, and 20 completed both the pre- and post-course test. Complete details on sample characteristics have been published elsewhere (6). In summary, learners were multidisciplinary and spanned nursing (n=11), physiotherapy (n=1), medicine (n=6), occupational therapy (n=1) and nutrition (n=1), and included participants from both adult and pediatric burn care units. The average age of course participants was 37 years, and >70% worked in a burn unit. The majority of respondents (55.6%) had <3 years of experience working with burn patients. The mean pre-course score was 12.75, compared with 18.45 post course. Pre-post course tests reflected a statistically significant increase in knowledge among participants that completed both tests (Table 1). Nineteen individuals had an improved score, with one individual maintaining the same score pre and post test. In addition, significantly improved scores were found for primary assessment, burn resuscitation, wound management, infection control, and electrical injury (Table 2). The module with the greatest improvement in score was wound management. The module with the least amount of improvement was inhalation injury (see Table 2 for effect sizes/Cohen’s d). Self-rated knowledge scores are presented in Table 3. An increase in self-rated knowledge was matched with a significant increase in module test scores for primary survey, burn resuscitation, wound management, infection control and electrical injury, but not inhalation injury and compartment syndrome modules.

TABLE 1.

Pre-post test scores

	n	Score		P*
		Pre	Post
Test score	20	12.75±3.43	18.45±2.72	<0.0001

Data presented as mean ± SD unless otherwise indicated.

^*Paired t test

TABLE 2.

Pre-post scores according to course module analyzed using Wilcoxon signed-rank tests

Course module	n	Questions per module, n	Score, % correct		d*	P
			Pre	Post
Primary survey	20	3	66.30±28.70	88.20±24.96	0.81	0.025
Inhalation injury	20	2	92.50±18.31	90.00±20.52	−0.13	0.655
Burn resuscitation	20	5	66.00±27.61	88.00±16.42	0.97	0.007
Compartment syndrome	20	2	77.50±34.32	90.00±20.52	0.44	0.132
Wound management	20	3	48.05±27.64	92.00±14.87	1.98	0.002
Infection control	20	1	65.00±48.94	100.00±.000	1.01	0.008
Electrical injury	20	4	56.25±31.28	81.25±25.49	0.88	0.015

Data presented as mean ± SD unless otherwise indicated.

^*Cohen’s d used to show the standardized difference between two means. d=0.2 is considered to be a ‘small’ effect size, 0.5 a ‘medium’ effect size and 0.8 a ‘large’ effect size (8)

TABLE 3.

Self-rated knowledge

Question	Poor	Fair	Good	Very good	Excellent
My knowledge of primary survey before the workshop was:	3 (15.0)	5 (25.0)	10 (50.0)	1 (5.0)	1 (5.0)
My knowledge of primary survey after the workshop is:	0 (0)	0 (0)	3 (15.0)	11 (55.0)	6 (30.0)
My knowledge of inhalation injury before the workshop was:	1 (5.0)	13 (65.0)	4 (20.0)	2 (10.0)	0 (0)
My knowledge of inhalation injury after the workshop is:	0 (0)	0 (0)	2 (10.0)	12 (60.0)	6 (30.0)
My knowledge of burn resuscitation before the workshop was:	3 (15.0)	2 (10.0)	13 (65.0)	2 (10.0)	0 (0)
My knowledge of burn resuscitation after the workshop is:	0 (0)	0 (0)	3 (15.0)	7 (35.0)	10 (50.0)
My knowledge of compartment syndrome before the workshop was:	6 (30.0)	3 (15.0)	10 (50.0)	1 (5.0)	0 (0)
My knowledge of compartment syndrome after the workshop is:	0 (0)	0 (0)	5 (25.0)	9 (45.0)	6 (30.0)
My knowledge of wound management before the workshop was:	0 (0)	6 (31.6)	11 (57.9)	1 (5.3)	1 (5.3)
My knowledge of wound management after the workshop is:	0 (0)	0 (0)	2 (10.0)	10 (50.0)	8 (40.0)
My knowledge of infection control before the workshop was:	0 (0)	6 (30.0)	12 (60.0)	2 (10.0)	0 (0)
My knowledge of infection control after the workshop is:	0 (0)	0 (0)	2 (10.0)	11 (55.0)	7 (35.0)
My knowledge of electrical injury before the workshop was:	3 (15.0)	9 (45.0)	6 (30.0)	2 (10.0)	0 (0)
My knowledge of electrical injury after the workshop is:	0 (0)	0 (0)	4 (20.0)	6 (30.0)	10 (50.0)

Data presented as n (%)

DISCUSSION

The evaluation of the EBM course examined the strengths, limitations and observations as noted by EBM course participants. As reported previously (6), participants had an increase in self-rated knowledge in a variety of areas related to burn care. While self-rated knowledge is an important component, an increase in self-rated knowledge did not correlate with the actual results on the test (Table 4). This discrepancy was especially evident with inhalation injury, in which there was a perceived knowledge gain, but a decrease in test score. Interestingly, the biggest test score gain occurred in the wound management module, which showed the lowest increase in self-perceived gain. It is encouraging that test scores demonstrated an overall increase in actual knowledge post course. For example, while significant improvements were found in primary survey, burn resuscitation and wound management scores, when compared with self-rated knowledge of each module area, only approximately 10% of individuals believed that their knowledge in the area was very good or excellent before EBM, compared with 85% to 90% of respondents at the end of the course. Similarly, significant improvements were found in the proportion who correctly answered the infection control question, which also reflected a large improvement in perceived knowledge. While only 10% of individuals rated their knowledge as good or excellent before the course, following the course, 90% of respondents rated their level of knowledge as good or excellent, and 100% of individuals answered the question on infection control correctly. While this is encouraging, there was only one question pertaining to infection control on the test; therefore, caution must be used in interpretation of this finding. Finally, questions related to electrical injury had a significant improvement in proportion of questions answered correctly; however, 20% of individuals answered at least one-half of the post-course questions incorrectly. Conversely, individual’s self-reported knowledge of electrical injury post-test reflected confidence, with 80% of individuals rating their knowledge as very good or excellent, and no individuals rating their knowledge as fair or poor. While there is a disconnect between actual and perceived knowledge on some modules, it may be that asking about overall knowledge on a subject area may not directly be comparable to a few questions on a test. These findings confirm that a combination of self-report and objective pre-post testing is needed to evaluate the effectiveness of any course designed to teach burn management. In addition, identifying which areas have a smaller increase in correct responses allows us to re-evaluate the course content and presentation, helping refine it to ensure that the presentation is better understood by the participant.

TABLE 4.

Mean increase in self-rated knowledge and test scores

Course module	Self-rated knowledge*	Self-rated knowledge†	Test score†
Primary survey	1.55	31.00	22.10
Inhalation injury	1.75	35.00	−2.50
Burn resuscitation	1.60	32.00	22.0
Compartment syndrome	1.65	33.00	12.50
Wound management	1.40	28.00	43.95
Infection control	1.45	29.00	35.00
Electrical injury	1.80	36.00	25.00

^*Five-point Likert-type scale, in which 1 = poor, 2 = fair, 3 = good, 4 = very good and 5 = excellent;

^†Percent change

While the present study addressed a gap in the literature and was the first to evaluate both actual and self-rated knowledge gain in the area of burn care following an international educational course (ie, EBM), it is important to mention potential limitations. First, the pre-post study design used an individual’s pre score to measure knowledge change. While it is encouraging to see a direct increase in test score following the EBM course, it is not possible to assess the longevity of this knowledge change. Future plans are underway to examine the retention of knowledge among EBM participants over a longer time period to assess knowledge change over time. Finally, three individuals were not fluent in English. This factor may have impacted test scores and self-rated knowledge assessments for these three individuals. As a result, CNIS is considering translating course materials into Swahili and other native languages for future course sessions. Throughout the training and testing sessions there was an English-Swahili translator available.

In summary, it was possible to demonstrate an increase in burn care knowledge as assessed by the EBM pre and post course test. Not only individual’s perceptions of their self-rated knowledge but also actual test scores improved. While this change in knowledge among course participants is encouraging, future research examining the retention of this information is needed, as well as future evaluations of EBM on additional individuals and multidisciplinary teams. Our future plans include expanding the question bank to increase the number of available questions for each subject and also to increase the number of students to improve the power to show differences between pre and post course tests.

Future work also involves developing a burn registry at this site to gather more data with hopes of implementing a burn prevention component.

APPENDIX A

Objectives (1): To obtain a competent knowledge level concerning the initial assessment of the trauma patient, the overall understanding of burns and the importance of team management of burns. To recognize the features of an airway burn and know how to respond. To develop trauma skills including airway management, venous access, sterile techniques in handling burn wounds and dressing application. To gain competence in the management of minor burns.

Objectives (2): To competently assess and manage circulatory needs and intravenous resuscitation in the burn patient. To recognize compartment syndrome and know how to respond. To understand the unique elements of chemical burns and be able to manage these. To know the issues of burn wound management and controlling infections. To develop skills of escharotomy, skin grafting and correct maintenance of joint positions to avoid later disabilities. To gain competence in the management of major burns.

Objectives (3): To obtain knowledge of the special considerations of electrical burns. To know the issues in pain management and nutritional needs of the burn patient. To understand the need for physiotherapy in burn patient care. To know the standard requirements for patient transfer. To solidify the knowledge gained at this course for practical, competent usage as part of a team to improve burn care in the hospital where you work. The objective of the course is to create Burn Teams in under-resourced centres where everyone knows their role in the management and care of the burn patient and understands the roles and responsibilities of the other team members.

• Skill session 1 objective: To be able to prepare for an emergency endotracheal intubation. To be able to perform or assist in an adult intubation.

• Skill session 2 objective: To be able to set up and establish, under sterile conditions, an IV infusion and secure it in place. To be able to set up a surgical tray and equipment for an intravenous cutdown and to perform or assist at the procedure.

• Skill session 3 objective: To be able to remove and replace burn wound dressings according to universal standards of sterility.

• Skill session 4 objective: To be able to set up a surgical tray for and perform or assist at an escharotomy in order to relieve a compartment syndrome. To be able to debride a burn wound of necrotic tissue.

• Skill session 5 objective: To be able to harvest a split-thickness skin graft by using both a dermatome and a Humby knife. To be able to prepare the graft, apply it to the wound and properly fix it in place. To properly care for the donor site.

• Skill session 6 objective: To be able to position at-risk joints in the correct position and to maintain this position through appropriate splinting.