Multimodal learning for emergency department triage implementation: experiences from Papua New Guinea during the COVID-19 pandemic

Summary

Background

Triage implementation in resource-limited emergency departments (EDs) has traditionally relied on intensive in-person training. This study sought to evaluate the impact of a novel digital-based learning strategy focused on the Interagency Integrated Triage Tool, a three-tier triage instrument recommended by the World Health Organization.

Methods

A mixed methods study utilising pre-post intervention methods was conducted in two EDs in Papua New Guinea. The primary outcome was the mean change in knowledge before and after completion of a voluntary, multimodal training program, primarily delivered through a digital learning platform accessible via smartphone. Secondary outcomes included the change in confidence to perform selected clinical tasks, and acceptability of the learning methods.

Findings

Among 136 eligible ED staff, 91 (66.9%) completed the digital learning program. The mean knowledge score on the post-training exam was 87.5% (SD 10.4), a mean increase of 12.9% (95% CI 10.7–15.1%, p < 0.0001) from the pre-training exam. There were statistically significant improvements in confidence for 13 of 15 clinical tasks, including undertaking a triage assessment and identifying an unwell patient.

In an evaluation survey, 100% of 30 respondents agreed or strongly agreed the online learning platform was easy to access, use and navigate, and that the digital teaching methods were appropriate for their learning needs. In qualitative feedback, respondents reported that limited internet access and a lack of dedicated training time were barriers to participation.

Interpretation

The use of digital learning to support triage implementation in resource-limited EDs is feasible and effective when accompanied by in-person mentoring. Adequate internet access is an essential pre-requisite.

Funding

Development of the Kumul Helt Skul learning platform was undertaken as part of the Clinical Support Program (Phase II), facilitated by Johnstaff International Development on behalf of the Australian Government Department of Foreign Affairs and Trade through the PNG-Australia Partnership. RM is supported by a National Health and Medical Research Council Postgraduate Scholarship and a Monash Graduate Excellence Scholarship, while PC is supported by a Medical Research Future Fund Practitioner Fellowship. Funders had no role in study design, results analysis or manuscript preparation.

Research in context.

Evidence before this study

Emergency department (ED) triage aims to identify patients with urgent care needs. World Health Organization guidance for the management of COVID-19 recommends that EDs use a standardised triage system, such as the Interagency Integrated Triage Tool (IITT). To date, the IITT has been implemented in a range of low- and middle-income country (LMIC) settings, including Papua New Guinea (PNG) in the Western Pacific.

There is a paucity of evidence to guide the implementation of triage systems in resource-constrained EDs. In settings where there is limited local experience with triage, the introduction of new tools has often required intensive in-person training delivered by external personnel. This study sought to explore the efficacy and acceptability of a novel, multimodal approach to triage implementation in two PNG EDs, including the use of a smartphone-compatible digital learning platform purpose-designed for low bandwidth environments.

Added value of this study

This study has verified that emergency care clinicians in PNG are receptive to the use of digital learning tools to support triage implementation. Specifically, it has demonstrated that electronic learning is effective in improving knowledge and confidence, when coupled with in-person mentoring and embedded in a broader ED development program.

Predictably, the study has also highlighted the essential pre-requisites for the use of digital learning tools in LMIC ED settings, including adequate internet access and protected training time. Efforts to increase engagement with the learning management system through culturally-specific iconography, micro-learning approaches and offline accessibility appear to have been effective.

Implications of all the available evidence

This study provides further evidence that the IITT is highly regarded by emergency care clinicians in PNG. Moreover, it offers ‘proof of concept’ for the use of digital learning tools to support IITT implementation in LMIC EDs, when instituted alongside a clinical redesign and mentoring process, and applied in the context of a mature development partnership.

The implications are significant. For instance, the use of digital learning methods potentially reduces the requirement for in-person training, including the reliance on international clinicians to deliver didactic education ‘on the ground’. This is likely to represent a more sustainable strategy for triage implementation and up-skilling, and will help promote ‘train the trainer’ approaches.

When read alongside the existing literature regarding triage in resource-limited settings, this study suggests that blended approaches to IITT training are acceptable, feasible and effective. Based on this evidence, there would be value in producing a range of globally relevant, open access IITT training resources, including digital learning tools, that could be utilised by LMIC EDs.

Introduction

Throughout the pandemic, emergency departments (EDs) have played a critical role in the screening, assessment and management of patients with suspected and confirmed COVID-19.^1,2 The World Health Organization (WHO) recommends a systematic approach to the reception of these patients, including use of a “standardised triage tool”. The Interagency Integrated Triage Tool (IITT), a three-tier, colour-coded system developed by the WHO, International Committee of the Red Cross and Médecins Sans Frontières, is specifically listed in WHO guidance as a suitable triage instrument.^3,4 Limited data suggest the IITT can identify ED patients with urgent care needs, including those with COVID-19.5, 6, 7

As with the rest of the world, the Pacific region has been heavily impacted by the pandemic.8, 9, 10, 11 Surge events have highlighted the importance of screening and triage at the point of hospital entry, and the value of efficient streaming and patient flow in the ED.^2,8,12, 13, 14 As a consequence, emergency clinicians in Pacific Island Countries and Territories have been called upon to rapidly implement new processes, requiring dynamic and innovative approaches to staff training and change management.^{2,8,10,12,13,15,16}

Previous efforts to implement triage tools in Pacific EDs have relied on in-person training and peer mentoring, often delivered by ED clinicians from high-income countries.^17,18 This reflects that alternate education strategies, such as remote and digital (or electronic) learning, can be challenging to apply in low- and middle-income country (LMIC) settings.^19,20 Recognised barriers include poor access to smartphones and computers, the prohibitive cost of data and lack of familiarity with online learning management systems.19, 20, 21 Globally, there is limited data on the efficacy of different approaches to triage training, especially on the impact and feasibility of digital learning in LMICs.²²

In the context of pandemic travel restrictions and recommendations for physical distancing, a novel, multimodal approach to triage training was piloted in two urban EDs in Papua New Guinea (PNG). This study sought to evaluate the impact of the learning strategy as well as the acceptability of the IITT.

Methods

Setting and participants

This mixed-methods study was conducted at ANGAU Memorial Hospital (AMH) in Lae and Port Moresby General Hospital (PMGH) in the nation's capital. These hospitals are the largest in the country, and act as referral facilities for surrounding provinces. As in other parts of PNG, there is a significant burden of both communicable and non-communicable disease in Lae and Port Moresby.23, 24, 25 The cities were also heavily impacted by COVID-19 surges throughout 2021.^7,26

AMH and PMGH EDs each receive approximately 120 patients per day, and are staffed by a mixture of emergency physicians, registrars (specialists in training) and nurses. AMH also employs health extension officers and community health workers, clinicians with skill sets analogous to clinical officers and nurse aids, respectively, in other LMICs. Challenges for the EDs include suboptimal infrastructure, access block and understaffing. Prior to this project, both departments were utilising the Australasian Triage Scale, but triage assessment and patient flow processes were not systematised or consistently applied.

As part of the AMH redevelopment, funded by the Australian Government Department of Foreign Affairs and Trade in partnership with the Government of Papua New Guinea, a Clinical Support Program (CSP) was established to enhance clinical care at AMH and PMGH. Through this initiative, ED staff at both sites identified that improved emergency care processes were required. The urgency of this work was highlighted by recurrent waves of COVID-19, which seriously impacted emergency care capacity and health services across the country.²⁶

All AMH and PMGH ED staff members were eligible to participate. Non-ED staff who enrolled in the training program were excluded from the analysis. All participants were required to provide explicit consent to contribute data to the study.

Intervention

Following a comprehensive needs assessment, local staff worked collaboratively with Australian clinicians to identify learning objectives and priorities for ED systems improvement. A Technical Advisory Group was appointed by the AMH Executive to lead this process. Decisions were made to focus on triage, patient flow and data management, and the IITT was chosen as the preferred triage instrument. The tool has previously been implemented in two PNG EDs, with high levels of acceptability among local clinicians.¹⁸

A multimodal change management strategy was devised, with a central focus on digital learning. Educational content was delivered using a web-based, smartphone-compatible learning management system purpose designed for low-resource settings. This was developed by a specialist digital learning provider, Catalpa (Perth, Western Australia), based on a platform used in other LMIC settings.²⁷ Scoping was performed to ensure compatibility with the commonest web browser and smartphone platforms used by local clinicians, and to test accessibility on local mobile networks.

A micro-learning pedagogical approach was utilised, with learning materials delivered through short modules and simple audio-visual materials. Content development occurred through a remote co-design process involving regular video conferences between local clinicians, Australian emergency care advisors and Catalpa staff.²⁸ All imagery, iconography and animation was bespoke, and aimed to accurately reflect the Melanesian context (see examples in Appendix 1). Platform functionality included embedded assessment tools, resource folders (which acted as a repository for reference materials), discussion boards and offline accessibility. For this application, the learning management system was branded ‘Kumul Helt Skul’ (KHS) and made available at https://kumulheltskul.com. Kumul is the Tok Pisin word for Bird of Paradise, a national symbol in PNG.

Learning materials were collated on KHS as the ‘Essential Emergency Care Systems Training Program’, a comprehensive training package structured around 10 discrete courses (Appendix 1). The first five courses were primarily focussed on triage and flow, and the remaining on related concepts such as documentation, patient assessment, equipment and infection control. Consistent with micro-learning principles, content for each course was divided into several lessons. The system was designed to allow participants to access the materials offline and ‘on-demand’, and they were able to complete as few or as many lessons as they wished in a single engagement. All courses were accompanied by relevant implementation resources, including IITT flowcharts for patient assessment and streaming. Selected reference materials were included in a purpose-designed Quick Reference Guide, which was provided to all ED staff in hard copy (Appendix 2).

In the setting of a major COVID-19 surge in mid-2021, a small number of Australian emergency nurses were deployed in pairs to AMH and PMGH to assist with ED operations. A total of six specialist emergency nurses contributed to this program between July and September (see timeline at Appendix 3). These clinicians were able to supplement the digital learning content with regular in-person review tutorials (including scenario-based activities and question and answer sessions) aligned with the courses on KHS. They also mentored local staff in relation to the new system, supporting AMH and PMGH clinicians to perform triage assessments and implement the new flow procedures. Additionally, several video conference sessions with Australian emergency physicians were held to troubleshoot IITT implementation issues. These educational opportunities were designed to reinforce content delivered by KHS, and to address any site-specific challenges.

At both sites, the 10 KHS courses were released gradually over a two-month period (Appendix 3), and clinicians were encouraged to complete them contemporaneously. A clinical redesign process was conducted in parallel, leveraging previous work in Pacific EDs.^17,18 This involved designation of three streams of care (resuscitation, acute and fast track), demarcated using coloured signs and floor tape. Selected diagrams and photos depicting the changes are available in Appendix 4. The IITT was implemented at the end of the training period, alongside new processes for patient flow, documentation and data management (Appendix 3). Participation in the training program was voluntary and free of charge, and staff provided explicit consent to contribute data to the evaluation.

Evaluation objectives and instruments

The primary objective of the evaluation was to assess the change in triage and emergency care knowledge through participation in the training program. Secondary objectives were to assess the change in clinician confidence regarding triage and emergency care skills, and perceptions of the learning methods and IITT.

Evaluation utilised pre-post intervention methods. Pre-training data was obtained from an exam embedded in course one of the program (Appendix 5). Part A of the exam included a series of quantitative, multiple-choice questions related to triage and emergency care knowledge. Part B comprised a series of quantitative questions related to confidence in performing selected triage and emergency care skills. These utilised a four-point Likert scale from strongly disagree to strongly agree.

The post-intervention component comprised two data sources. First, a post-course exam embedded in course ten replicated Parts A and B of the pre-training survey (Appendix 5). As with the pre-intervention version, Part A was focussed on knowledge and Part B on confidence. Second, a voluntary, anonymous, online evaluation survey (Part C) was housed on the Qualtrics platform (Qualtrics, Provo, Utah, USA). All ED staff were invited to participate. As with Part B, quantitative questions regarding perceptions of the triage system and learning methods utilised a four-point Likert scale from strongly disagree to strongly agree. The survey instrument (Appendix 6) was developed based on a brief questionnaire used to evaluate IITT implementation in another setting.¹⁸ Qualitative data were captured through a series of questions related to acceptability of the learning methods and triage process.

Outcomes, analysis and ethics

The primary outcome was the average change in Part A scores between the pre-training and post-training exams, expressed as a mean and standard deviation. Statistical significance, assessed using a paired t-test, was defined as a p-value of <0.05. The Shapiro–Wilk test was applied to confirm normality.

For the secondary outcomes, the proportion of respondents that agreed or strongly agreed with statements regarding their clinical confidence (Part B) as well as the triage system and learning methods (Part C) were summarised using frequencies and percentages. Differences in the pre- and post-training responses for Part B were compared using McNemar's chi-square test for categorical variables.²⁹ The exact binomial version of this test was applied for variables with low frequencies. Statistical significance was again defined as a p-value of <0.05. Qualitative data were assessed using inductive thematic analysis.

All quantitative data were initially exported from KHS and Qualtrics into Microsoft Excel, with statistical analysis undertaken in Stata v17 (StataCorp, College Station, Texas, USA). Ethics approval was provided by Monash University Human Research Ethics Committee (references 28000/29524), with subsequent endorsement from AMH and PMGH executive leadership and the PNG Medical Research Advisory Committee (references 22.44/22.45).

Results

Participants

The sample for the study is summarised in Fig. 1, and participant demographics are detailed in Table 1. Among a total of 136 ED staff across both sites, 124 (91.2%) accessed KHS and 114 (83.8%) commenced the training program by undertaking the first course.

Fig. 1.

Study participants. ∗Includes all participants who completed the first course on Kumul Helt Skul. Further data on this group, stratified by those who did and did not complete the training program, is provided in Table 2. †One additional clinician completed the program but did not provide consent to participate in the study. ‡Includes respondents who did not complete the training program.

Table 1.

Study participants.

		Commenced training programa	Completed training programb	Completed evaluation surveyc
Total participants		114 (100%)	91 (100%)	30 (100%)
Facility	AMH	25 (21.9%)	14 (15.4%)	8 (26.7%)
	PMGH	89 (78.1%)	77 (84.6%)	22 (73.3%)
Gender	Male	45 (39.5%)	37 (40.7%)	13 (43.3%)
	Female	69 (60.5%)	54 (59.3%)	16 (53.3%)
	Other/Not specified	0 (0.0%)	0 (0.0%)	1 (3.3%)
Role	Doctor	34 (29.8%)	26 (28.6%)	11 (36.7%)
	Health extension officer	3 (2.6%)	2 (2.2%)	2 (6.7%)
	Nurse	71 (62.3%)	60 (65.9%)	15 (50.0%)
	Community health worker	3 (2.6%)	1 (1.1%)	1 (3.3%)
	Administrator or clerk	3 (2.6%)	2 (2.2%)	1 (3.3%)

All data are expressed as frequencies, with percentages (within each column) in parentheses.

^aIncludes all participants who completed course one. Table 2 provides further detail on this group, stratified by program completion.

^bSample for primary outcome and secondary outcomes related to confidence (Parts A and B).

^cSample for other secondary outcomes (Part C).

Overall, 91 (66.9%) completed the entire program. As described in Fig. 1 and Table 2, completion rates were higher among staff at PMGH (p = 0.001). There were no other statistically significant differences in demographics between those who did and did not complete the training program (Table 2). The majority of KHS user engagements occurred via smartphone (Appendix 7).

Table 2.

Study participants stratified by training program completion.

		Completed training programa	Did not complete training programb	p-value
Total participants		91 (100%)	23 (100%)	N/A
Facility	AMH	14 (15.4%)	11 (47.8%)	0.001
	PMGH	77 (84.6%)	12 (52.2%)
Gender	Male	37 (40.7%)	8 (34.8%)	0.61
	Female	54 (59.3%)	15 (65.2%)
Role	Doctor	26 (26.6%)	8 (34.8%)	0.11❡
	Health extension officer	2 (2.2%)	1 (4.25%)
	Nurse	60 (65.9%)	11 (47.8%)
	Community health worker	1 (1.1%)	2 (8.7%)
	Administrator or clerk	2 (2.2%)	1 (4.35%)
Agec	Less than 40	77 (84.6%)	17 (73.9%)	0.38❡
	40 or above	11 (12.1%)	5 (21.7%)
	Age not specified	3 (3.3%)	1 (4.4%)

All data are expressed as frequencies, with percentages (within each column) in parentheses. The statistical significance of differences between the groups (expressed as a p-value) was determined using Pearson's chi-square test, except where low frequencies were observed. In these instances, marked with a pilcrow (^❡), Fisher's exact test was utilised.

^aIncludes participants who completed all ten courses and provided consent to participate in the study. See Fig. 1 for further information.

^bIncludes participants who completed the first course of the program. See Fig. 1 for further information.

^cAge data captured on Kumul Helt Skul were recorded in 5-year brackets based on year of birth. For the purposes of this analysis, participant ages were approximated based on the mid-point of each age bracket.

Primary outcome: knowledge (Part A)

Clinicians who completed both the pre-training and post-training exam, and provided consent to participate in the study, comprised the sample for the primary outcome (Fig. 1). Among 91 participants, 77 (84.6%) were from PMGH and 14 (15.4%) were from AMH. A majority (60, 65.9%) were nurses (Table 1).

The mean score on Part A of the pre-training exam was 74.6% (SD 10.7). The mean score on Part A of the post-training exam was 87.5% (SD 10.4). Overall, there was a mean improvement in knowledge scores of 12.9% (95% CI 10.7–15.1%, p < 0.0001).

Secondary outcome: confidence (Part B)

The same sample contributed data for the secondary outcomes related to confidence, which were examined in Part B of the pre- and post-training exams. Staff confidence to perform the specified list of emergency care tasks before and after completion of the training package is summarised in Fig. 2 as well as the table that appears in Appendix 8.

Fig. 2.

A: Confidence (pre-training).

For 13 of 15 tasks, ≥98.0% of participants agreed or strongly agreed that they were confident to perform the activity by the end of the program. Lower levels of confidence were reported for defibrillation and nitrous oxide use, with 78 (85.7%) and 69 (75.8%) respectively agreeing or strongly agreeing.

Owing to an unexpectedly high level of baseline confidence, statistical significance of the change in confidence was determined by comparing the proportion of respondents that strongly agreed with the various statements before and after completion of the training program. There were statistically significant improvements for 13 of 15 clinical tasks (Appendix 8). For instance, the proportion of respondents that strongly agreed they were confident to perform a triage assessment increased from 51.6% to 71.4% (p = 0.0004). As evidenced in Fig. 2, this pattern was observed for each of the clinical tasks, but did not achieve statistical significance in the case of defibrillation and nitrous oxide use.

Secondary outcome: acceptability of the learning methods (Part C)

As described in Fig. 1, secondary outcomes related to acceptability of the learning process and the triage system relied on data from a smaller sample of ED clinicians (ie, those who completed the evaluation survey). The proportion of participants that agreed or strongly agreed with a range of statements regarding the learning methods are summarised in Fig. 3A. Responses were overwhelmingly positive.

Fig. 3.

A: Acceptability of the learning methods.

In terms of qualitative feedback, participants were asked about barriers to engagement and how the learning platform could be improved (Appendix 6). Three key themes emerged from the available data: difficulties accessing the internet (through WiFi or mobile data) and a compatible smartphone were barriers to engagement; a lack of protected training time at work acted as a disincentive; and the addition of further content and more interactive elements would add to the value of the platform (Box 1). Overall, feedback was extremely positive, but emphasised the need for hospitals to facilitate program completion by ensuring adequate internet access (through WiFi or subsidised mobile data) and allocating training time within rostered duty hours.

Box 1. Indicative quotes from qualitative data.

• Acceptability of the learning methods

  • How could the learning platform (Kumul Helt Skul) be improved?

    • “By being more accessible and inclusive of other health workers who may wish to participate and be more interactive”
    • “Learning platform is great, looking forward to learning more using the platform.”
    • “More short videos would make long and difficult [explanations] much easier to understand.”
  • What were the most significant barriers to you engaging with the learning platform (Kumul Helt Skul)?

    • “[WiFi] and timing during shifts to complete the courses.”
    • “Accessing the internet using expensive Digicel data”
    • “The training itself was excellent, the only problem was data to go online.”
• Acceptability of the triage system

  • How do you think the triage and flow system could be improved?

    • “Triaging system is great however we have a very limited space as well as staff and the patient overflowing is too overwhelming.”
    • “Employing more staff and having much needed resources available at all times.”
    • “By putting up more [colour] coded signs [and] doing more awareness to public at the triage deck what levels we categorize sick patients with and what [colour] represents each level.”
  • Would you like to make any other comments about the new triage and flow system or the training process?

    • “The new triage system has really helped with patient flow, reduce workload on staff and minimise unnecessary resource use. What I want improved to make it more effective is: 1. Involvement of non-clinical staff in training and implementation; 2. Improve staff manpower; 3. Enough space and facilities.”
    • “Blessed to have learnt it and to implement it. I really hope we make a difference and I hope it helps us save more lives. Would really like to see more changes be communicated to everyone and not only the leaders as we are all implementing this now.”
    • “I am very excited in learning the new triage and flow systems because it has helped me a lot as a new nursing officer working in emergency department.”

Secondary outcome: acceptability of the triage system (Part C)

Fig. 3B summarises participant responses in relation to the acceptability of the triage system. All participants agreed or strongly agreed that the new triage system helps identify and prioritise the most urgent patients; the triage process is easy to follow and implementation of the triage and flow system has improved patient and staff safety in the ED.

With respect to qualitative feedback, participants were invited to comment on how the triage and flow system could be improved (Appendix 6). Several key themes emerged: understaffing limited the effectiveness of the new system; overcrowding and limited physical space were major barriers to effective patient flow; and further education to non-clinical staff and patients would assist with implementation. Feedback on the IITT was generally very positive, and most suggestions for improvement reflected longstanding challenges for the EDs, including under-resourcing and lack of an effective hospital information management system. Indicative quotes are provided in Box 1.

Discussion

This mixed methods study assessed the impact and acceptability of a novel, multimodal approach to triage implementation in two resource-limited EDs in PNG. The intervention proved effective, with a modest but statistically significant improvement in knowledge as a result of the training program. Confidence to perform a range of clinical tasks also improved, with the exception of defibrillation and nitrous oxide use. Clinicians expressed strong support for the digital learning methods as well as the triage system.

COVID-19 has driven enormous change in global emergency care collaboration, and opened up new possibilities for remote and digital education.²¹ This program is an example of a pandemic ‘silver lining’, in that it catalysed the development of an innovative approach to training and a successful collaboration between local and Australian clinicians. The findings of this study suggest that digital learning tools are acceptable to ED staff in Port Moresby and Lae, despite being relatively novel in the Pacific context.²⁰ Similar to other settings, clinicians appear to support blended teaching methods, incorporating both online and in-person training.30, 31, 32

In LMIC EDs, as with high-resource contexts, the application of digital learning has several advantages over in-person training.³⁰ First, learning resources can be accessed on-demand, enabling learners to undertake training at their convenience. Scheduling in-person training for ED staff can be challenging because of shift work, and the need to ensure adequate clinical coverage at all times. Second, the transient and dynamic nature of the ED workforce means that there is a constant need to train new staff. The existence of an enduring learning platform means that new clinicians can be efficiently trained and credentialed. Third, the approach may be cost effective in a setting where the alternative option is to fund external facilitators to deliver didactic, in-person training.

The results of this study suggest that some skills benefit from in-person training. For instance, participant confidence to perform defibrillation and use nitrous oxide did not increase to a statistically significant extent. These more practical tasks may be better learned using ‘hands on’ approaches, including small-group workshops and in-situ simulation. This is an important finding, because a recent review of emergency care literature has identified limited evidence in this area.³⁰ Digital education may still have a role as a learning adjunct, particularly in helping to prime clinicians for future practical training.

The use of digital learning in healthcare is not new, but application in resource-limited ED settings has been relatively limited.^19,20,32 A range of recent initiatives suggest that online learning concepts are gaining traction. For instance, multimodal training including e-learning modules is now being utilised in relation to disaster management, and Médecins Sans Frontières has developed a learning platform (Tembo) for a range of clinical applications, including emergency care.^33,34 The WHO, through its OpenWHO website, is also producing a variety of modules for acute care clinicians, as well as online case studies linked to its Basic Emergency Care course.^32,35 The evidence provided by this study adds to global experience in this area.

There are several factors that are likely to have contributed to the acceptability of this digital learning application. First, content was developed in partnership with PNG clinicians (through the project's Technical Advisory Group, appointed by the AMH Executive) to ensure relevance and applicability to the local context. Consistent with this objective, all imagery on KHS was distinctly Melanesian, and animation voiceovers were narrated by local ED staff. Similar approaches have been successfully used in other Pacific settings.²⁸ Second, the ability to support the digital learning process with in-person review sessions and peer mentoring may have helped to address the needs of staff with diverse learning styles. It also provided a valuable opportunity to tailor KHS material to local processes. Third, although participation in the training was voluntary, it was linked to a major change management strategy in the ED. Evaluation data suggest that staff were motivated to engage in KHS in advance of the implementation of the IITT and new patient flow processes.

Importantly, this multimodal learning program was undertaken in the context of a longstanding partnership between Australian and PNG emergency care clinicians.^36,37 Recent research has demonstrated the importance of regional collaboration in effective emergency care development, and the value of enduring mentoring relationships between Pacific and global colleagues.^16,38, 39, 40 Long-term partnerships are essential to building trust and cross-cultural understanding, and can provide a foundation for the type of training program described here.

As identified in the qualitative data, the major barrier to the use of digital learning tools in resource-limited settings is internet access. In PNG, mobile data is expensive and network coverage is incomplete. For this program, the LMS was designed to include offline accessibility, such that clinicians could download all KHS content (for instance, at the hospital when connected to WiFi) and then access the learning resources without requiring a data connection. Despite this functionality, some participants still experienced difficulties accessing the platform.

Limitations of this study include its restriction to one Pacific Island Country (PNG), the relatively small sample size and the sub-optimal response rate for Part C. Additionally, the pre-post design applied to Part A may have been subject to bias, in that participants responding to the post-training exam may have ‘learned’ the questions from the pre-training version (even though the correct answers were not provided at the time). Similarly, recall bias is a potential limitation of Part B, because participants completing the post-training exam may have been unable to remember their baseline level of confidence. There is also an inherent power imbalance between local clinicians and international advisors, and this may have influenced responses to the survey component (Part C) of the evaluation. That said, the primary outcome was an objective assessment of knowledge and should not have been susceptible to this form of bias.

Notwithstanding these issues, this study provides strong endorsement of a multimodal approach to triage training and implementation in a resource-limited environment. It adds to a broader body of evidence about the value of digital learning tools in LMICs, and the role of information communication technology in advancing global emergency care partnerships and development.¹⁸ Further research in this area might consider the cost-effectiveness of different approaches to triage training, and the applicability in different cultural contexts.

Conclusion

The use of digital learning to facilitate triage implementation in resource-limited EDs is feasible and effective when coupled with peer mentoring and embedded in a broader development program. Effort is required to address access barriers, including internet accessibility and a lack of dedicated training time. Irrespective of the implementation approach, PNG ED clinicians consider the IITT to be a safe and effective triage tool.

Contributors

RM, SB and CB were primarily responsible for development of the multimodal training program, with support and input from DP, WS and SK through the project's Technical Advisory Group. JPM and TC contributed learning content for several modules. AE provided advice regarding educational technology and the learning management system, while SK designed the monitoring and evaluation framework for the project as a whole. RM and SB compiled data from Qualtrics and Kumul Helt Skul respectively. RM was primarily responsible for study design and statistical analysis, with input from GOR and PC. All AMH and PMGH clinical staff listed as co-authors contributed to roll-out of the training program and triage system. RM generated the first draft of this manuscript, which was reviewed by all co-authors.

Data sharing statement

Study protocols and de-identified data that underpin these findings may be available (up to 24 months after publication) to investigators whose proposed use of data has been approved by an independent review board, subject to any restrictions imposed by relevant research and ethics committees (including the PNG Medical Research Advisory Committee), funders (the Australian Government) and health services (Port Moresby General Hospital and ANGAU Memorial Hospital). Requests should be made in writing to the corresponding author, Port Moresby General Hospital and ANGAU Memorial Hospital.

Declaration of interests

RM, SB, JPM, TC, AE and CB declare that they received consulting fees for their contributions to the Clinical Support Program.