Enhancing medical training in conflict zones and remote areas through innovation: introducing the Canadian Virtual Medical University Initiative

Summary

Background

The WHO projects a global shortage of 4.3 million physicians by 2030, with the largest deficits in developing and conflict-affected regions. Our aim is to train competent physicians rapidly and affordably through remote education programs.

Methods

We developed an online medical training curriculum with four levels, focusing on different aspects of human body systems using a competency-based, student-centered approach. This study evaluates the first three levels; level four (internship) is outside this scope. The 105 medical students from eight Afghan universities were randomly assigned to nine groups. The curriculum includes Entrustable Professional Activities (EPA) for the cardiovascular system: level 1 covers basic medical sciences, level 2 pathology and basic clinical skills, and level 3 full clinical competencies. EPAs were delivered asynchronously online via Lecturio, CyberPatient, and Zoom. The 30-day intervention included 4 h of weekly online classes for formative assessment, collaborative learning, and evaluation, supervised by medical faculty members. Virtual pre- and post-intervention evaluations used multiple-choice questions and objective structured clinical examination (OSCE). We also conducted a satisfaction survey and open interview forum. Data triangulation from observations, surveys, and interviews validated curriculum effectiveness. The benchmarking method assessed cost-effectiveness.

Findings

Pre- and post-intervention analysis showed a significant increase in clinical competencies and knowledge acquisition (P < 0.0001). The CyberPatient intervention improved clinical competency quality (P < 0.0001) and shortened decision-making time (P < 0.001). Cost analysis revealed that a virtual medical university would be 95% more cost-effective than traditional medical education.

Interpretation

Integrating virtual technology with modern curriculum concepts in pre-internship years can effectively address healthcare training gaps and enhance education quality for healthcare workers at a low cost.

Funding

Provided by CanHealth International. A UBC spin-off not-for-profit organization.

Research in context.

Evidence before this study

The study utilized PRISMA-ScR guidelines and a systematic literature search from 2000 to 2024, including PubMed, Medline, Scopus, Web of Science, Embase, and IEEE Xplore, to identify relevant articles on establishing virtual universities in medicine.

We conducted a database search utilizing the following keywords:

Virtual education, online universities, open and distance education, technological innovations, personalized learning experiences, educational technology, open-access ethos, virtual tutoring, course design, learner-centered approach, user feedback and competency-based medical education.

The World Health Organization predicts a global deficit of 4.3 million physicians by 2030, with the most severe impacts in low-income nations. The Association of American Medical Colleges anticipates a shortage of up to 124,000 physicians in the United States by 2034. Establishing new medical schools is a substantial undertaking, costing $300-$700 million and taking 5–10 years. The World Federation for Medical Education advocates diversifying educational methodologies, such as leveraging simulation techniques and virtual patient avatars, to advance medical training in conflict-affected regions. While modern education systems are exploring alternative teaching and learning strategies, a comprehensive pre-internship online medical education system remains unrealized.

Added value of this study

The added value of this study is that it provides an innovative and complete pre-internship online medical education system using state-of-the-art technological advances and modern pedagogical approaches such as student-centred competency-based medical education. The study also demonstrates the feasibility and effectiveness of this virtual medical training model, implemented through the Canadian Virtual Medical University Initiative (CVMUI). It provides scientific evidence on the enhancement of knowledge and clinical competencies of medical students. The reduction in time required to perform clinical tasks using this approach shows that students trained with this method will be more efficient in patient care and positively impact the healthcare system. The study shows that this approach can address the global shortage of well-trained physicians, particularly in conflict zones and remote areas. Student feedback regarding the virtual curriculum was highly positive, and the estimated financial analysis demonstrated substantial cost reductions compared to conventional medical education approaches. Overall, the research indicates that the CVMUI model represents a scalable, adaptable approach to cultural and local needs and a financially viable one for mitigating the global shortage of physicians, particularly for developing nations and conflict zones.

Implications of all the available evidence

The findings suggest that virtual medical education is possible, and it can revolutionize training and improve access to quality education for all, regardless of geographic location or political or religious barriers. The study highlights the shortage of doctors worldwide and the need for innovative solutions, especially in low-income countries and conflict zones. It emphasizes the cost-effective and accessible nature of online medical education, including virtual simulation methods, as a potential solution. The CVMUI model presented here will open new opportunities for innovative research. Future research can be focused on long-term outcomes, expanding to other health fields, evaluating virtual education in challenging settings and others.

Introduction

The World Health Organization (WHO) predicts a global shortage of 4.3 million physicians by 2030, with the worst impacts in low-income countries,¹ but recent reports suggest the shortage could reach 6.4 million.² In the US, the Association of American Medical Colleges (AAMC) projects a shortage of up to 124,000 physicians by 2034.³ The ratio of doctors per 10,000 population varies significantly, from 36 in the USA to 2.8 in Afghanistan, not accounting for factors like migration and educational restrictions.⁴

Building a medical school costs $300–$700 million and takes 5–10 years, with annual operating costs of $30–$100 million.⁵ To meet the 2030 demand, an immediate $4 trillion investment is required. Innovative, cost-effective solutions are crucial, such as delivering pre-internship medical education online, followed by practical internships. While theoretical education online is feasible, clinical competencies are more challenging, though simulation methods can help.

The World Federation for Medical Education (WFME) advocates for diverse educational techniques, including simulation methods using standardized patients (SP), mannequins, and other simulation accessories to obtain clinical competencies.⁶ The use of physical simulations for clinical competencies is routine in most universities. It also increases costs and hinders accessibility due to timing and location constraints, particularly in resource-poor countries. For instance, conducting an Objective Structured Clinical Examination (OSCE) to assess practical clinical competencies at a UK medical school costs approximately £355 per student,⁷ with similar costs reported for the OSCEs at the University of British Columbia (UBC). In this research, Ferahmand et al. demonstrated that online simulation using virtual patient avatars is as effective and about 90% less costly than when standardized patients were used.⁸

In the pursuit of advancing medical education within war-stricken regions, de Boer et al. have pioneered the “on-the-fly” methodology. This innovative strategy encompasses developing, deploying, and validating novel educational technologies.⁹ Joury's “Syrian Smiles” initiative has been instrumental in modernizing dental education amidst the adversities faced in Syria.¹⁰ Alfakhry has highlighted the prevalent challenges, such as antiquated policies, resource scarcity, and suboptimal curriculum design.¹¹ In Sudan, collaborative alliances, virtual engagement with the diaspora, and online learning have been proposed to maintain medical education during armed conflicts.¹² Incorporating “conflict medicine” into curricula is crucial for addressing war-related health issues.¹³ Alrashdi et al. have observed a global trend in dental schools adapting their curricula through integrating technology and hybrid teaching methods in response to the necessity for social distancing and enhanced safety protocols.¹⁴ A study by Joury E et al. examined the effects of prolonged war on dental students in 12 Arabic-speaking countries. The research revealed that students in war-affected countries faced wider challenges, including attendance difficulties and limited teaching time flexibility. The study suggests the need for support for these students, including the creation of online dental education programs.¹⁵ Nonetheless, there remains an imperative need for versatile virtual and Indigenous patient simulators that are apt for a variety of settings, including areas of conflict. Therefore, innovative solutions for delivering and assessing clinical competencies are needed everywhere and are especially useful in developing nations, where students often face educational barriers due to financial constraints, societal conflicts, and cultural or religious norms.^16,17 Modern education systems are experimenting with various alternative teaching and learning strategies.¹⁸ However, a fully pre-internship online medical education system has yet to be realized.¹⁹

To enquire about the experiences of other researchers in establishing open and virtual universities, we conducted a scoping review using PRISMA-ScR guidelines from 2000 to 2024 in relevant databases. The findings revealed that universities have successfully implemented virtual course delivery models, which have been well-received by the global academic community.20, 21, 22, 23, 24, 25 These studies emphasized the importance of proper infrastructure, efficient academic staff, curriculum design tailored to students' levels, and providing academic and technical support. However, these virtual universities were generally broad in scope, with a limited focus on the medical sciences. Given the practical and procedural nature of fields like medicine, they require a distinct educational approach compared to purely online learning.²⁶ Consequently, to effectively educate students in these disciplines, we should leverage the highest level of technology to foster a student-centred learning atmosphere. Establishing such specialized virtual universities has been challenging, yet it is crucial considering the pressing need for medical professionals.

We believe revolutionary curriculum reform is required to change the current medical education system into a fully online delivery system. The proposed reform of the medical training system will be essential for enhancing the capacity of the health workforce globally and is essential for developing nations and conflict zones. Therefore, an innovative virtual medical university concept with asynchronous knowledge and practical clinical competency delivery using technological advances could revolutionize medical education in these regions. The Canadian Virtual Medical University Initiative (CVMUI) has the potential to establish a virtual medical university using technological advances and modern pedagogical concepts, such as a competency-based, student-centred curriculum. It presents a viable solution to address the shortage of skilled medical professionals and the challenges exacerbated by financial and educational barriers. Such an institution would provide flexible, accessible, and affordable medical education for everyone without discrimination, aligned with the global right to education. It would have a multiplier effect, empowering students and faculty while cost-effectively responding to societal needs. We hypothesize that asynchronous delivery and implementation of the CVMUI pre-internship virtual medical curriculum will be as effective and less costly than traditional in-person medical education (TME). Our research objectives are three-fold: (1) to assess the implementation of CVMUI's competency-based, student-centred curriculum concept using state-of-the-art technological advances in a technologically challenged environment, (2) to assess the effect of CVMUI's virtual, competency-based, student-centred curriculum on pre-internship medical students and (3) to estimate the cost-effectiveness of CVMUI versus traditional in-person medical education using benchmarking methods.

Methods

Curriculum development

Before rolling out our observational study, we designed a competency-based, student-centred curriculum using modern pedagogical concepts. We selected state-of-the-art technologies for online delivery, creating a suitable virtual environment for this study. The curriculum encompasses theoretical and practical knowledge of all human body systems; however, cardiovascular content was chosen for this experiment (Appendix p1-5). This study used three methods, namely observation, survey, and open discussion/interview, to evaluate the study's outcome. The results were triangulated to ensure internal validity. Additionally, a cost analysis was conducted to assess the cost-effectiveness of the proposed innovative method of medical education. This project spanned a 12-month period from its initial conception to implementation, from June 4, 2023, to June 15, 2024. Nine months were dedicated to the curriculum design and creation of Entrustable Professional Activities (EPAs), two months to study planning, and one month to educational intervention.

Study design

This study employed an explanatory mixed-methods approach in which qualitative data was leveraged to enhance the validity of the quantitative findings.

Quantitative section

Sample and setting

Ad hoc power analysis using G∗Power software determined that to detect a moderate primary effect size of 0.33 with 84% statistical power (β = 0.17) and a two-tailed test at α = 0.05, a minimum of 29 participants were required per study level. On February 12, 2024, 120 Afghan medical students officially studying at one of Afghanistan's eight government-run public medical schools were included by inclusion criteria. Each university introduced 15 students and 3 faculty members to CVMUI. Volunteer students and faculty signed consent forms and took an English proficiency test organized by their university. After interviews by the CVMUI education committee and suitability tests from 120, only 105 students entered the study. Exclusion criteria included students' time commitment to self-directed study, participation in weekly activities, and previous involvement in similar cardiovascular system courses, workshops, or hospital work. The 105 selected students were assigned to three levels of the CVMUI competency-based curriculum: first- and second-year students were enrolled in level 1 (n = 43); third- and fourth-year students in level 2 (n = 31); and fifth and sixth-year students in level 3 (n = 46). Subsequently, we randomly assigned students from eight medical universities at each level into three groups (nine groups in total) in order to homogenise the professional activity and ensure that students from the same university were not placed in one group. Three trained faculty members from Afghanistan and one from Canada, with a student-instructor ratio of no more than 10:1, monitored and supported these groups.

Pre-intervention training and testing

All medical students and faculty participating in the study received training on teaching methodology and technical competency on Lecturio and CyberPatient platforms, 5 working days, and 4 h of virtual classroom practice (Appendix p6-7). Students at three levels underwent an initial assessment (pre-test) of their cardiovascular system knowledge. This assessment included 100 multiple choice questions (MCQs) specific for each of the 3 levels, for a total of 300 MCQs to assess. Levels 2 and 3 pre-tests also included clinical skill competency assessments via virtual OSCEs, with two virtual testing substations for level 2 (patient history-taking and physical examination) and 7 substations for level 3 (history-taking, physical examinations, diagnosis management, treatment, documentation, and follow-ups). Level 3 had an additional written component that included ten open-ended written questions. Each pre-test totaled 100 points. Knowledge tests were administered using OnlineExamMaker, and CyberPatient was used for online OSCE assessments (Fig. 1).

Fig. 1.

Illustration of the experimental design for the observational study. MCQ—Multiple-choice questions, OSCE—Objective Structured Clinical Examination The figure's colour density serves as an indicator of the level of difficulty associated with the training process and examination.

Intervention

The intervention lasted 4 weeks. Levels 1–3 used the Lecturio online platform for theoretical cardiovascular topics. Clinical competency activities were not delivered to level 1. Levels 2 and 3 used CyberPatient for clinical competencies and Lecturio. CyberPatient access was limited to history-taking and physical examination for level 2, and level 3 had full access. Weekly 4-h online Zoom classes facilitated discussion, mentor guidance, assignment finalization, self-assessment, and formative assessment. WhatsApp groups offered additional support. Mentors and group leaders facilitated communication. Activities were recorded and archived for verification. Weekly formative assessments were conducted using Lecturio's question bank. In addition, 2–3 CyberPatient cases per week were provided to students to develop their clinical competencies. Both Lecturio and CyberPatient have AI-empowered tutoring systems that analyze students' actions, provide instant feedback based on their performance, and guide them to better performance. Zoom classes with video capabilities were also used to provide an opportunity for students for in-person assistance by faculty and classmates. University officials observed classes for transparency. Students received free access to all platforms in the program and were reimbursed for acceptable Internet access (see Fig. 1).

Post-intervention assessment

At the end of the four-week program, students with a formative assessment score above 80% (calculated from faculty assessments representing 30% of scores and knowledge tests representing 70%) were eligible to take the final exam. We designed this process to assess competence in performing interventions using the simulator, adhering to the guidelines outlined in AMEE Guide No. 82.²⁷ The post-test assessment was similar with the pre-test with 300 MCQs, clinical skill assessments via OSCE for levels 2 and 3, and additional written components for level 3. The post-test totalled 100 points for each level, with knowledge tests on OnlineExamMaker and OSCE assessments on CyberPatient.

Data collection

The post-assessment was conducted two days after the final session, with results stored on the server for analysis. Different pre- and post-test questions from Lecturio's question bank were used, each exam level consisting of 100 questions. Validity and reliability were confirmed with Cronbach's alpha values of 0.86 for level 1, 0.83 for level 2, and 0.81 for level 3. OSCE assessments included two cardiovascular cases with varied station numbers across levels. Performance was documented within specified times for accurate skill assessment. Scores were standardized for comparison. Reliability and validity were measured using Cronbach's alpha, yielding 0.84 for level 2 and 0.81 for level 3.

Data analysis

Data were analyzed using SPSS v26 and PRISM v8 software with a statistical significance level of 0.05. To assess the normality of the data distribution, we conducted Shapiro–Wilk statistical tests and examined histograms of the data. The statistical analyses revealed that the data from the knowledge tests at all three levels exhibited a normal distribution, enabling the use of paired t-tests. Similarly, the performance test data at level 2 displayed a normal distribution, warranting the application of paired t-tests. However, the performance test data at level 3 did not conform to a normal distribution, necessitating the employment of the Wilcoxon matched-pairs signed rank test. Reliability was assessed using the intraclass correlation coefficient (ICC) and Cronbach's alpha. Additionally, multiple linear regression analysis was employed to investigate the effectiveness of the educational intervention.

Qualitative section

Surveys and programme evaluation

Students participated in weekly Zoom classroom self-assessments and formative evaluations to measure the intervention's impact. Following the 4-weeks intervention, they completed questionnaires (Appendix p8-14) evaluating knowledge acquisition, course satisfaction, content quality, and faculty satisfaction. The survey captured student assessments of faculty, course, and program; peer evaluations of faculty, students, and the program; and external evaluations by the accreditation body (Ministry of Higher Education of Afghanistan). The surveys developed demonstrated strong reliability with a Cronbach's alpha exceeding 0.8.

Open forum discussion and interviews with stakeholders

A video and audio-enabled open discussion was held on April 23, 2024, involving 65 stakeholders from Afghanistan universities and Canadian Virtual University Project participants from Ontario and British Columbia. Discussion sessions and semi-structured interviews were conducted with participants from the CVMUI educational program. Stakeholders included students, faculty, deans, university presidents, and representatives of the Ministry of Higher Education of Afghanistan. The Data Management Committee developed the agenda for the 90-min session, which included three facilitators and 65 stakeholders. Two researchers independently analyzed the data using Braun & Clarke's thematic content analysis,²⁸ yielding 128 primary codes, consolidated into 55 codes, and organized into two overarching themes with nine subthemes. The agenda included key discussion points: the quality of medical education in this virtual approach, the success of the project, challenges encountered and proposed solutions, areas for improvement, and overall stakeholder acceptance. The discussion aimed to evaluate the research outcomes of the CVMUI, including curriculum validity, reliability, and the use of technology in distance medical education (Appendix p48-51).

Data triangulation

An explanatory sequential design integrated observation, surveys, and open forum discussion/and interview methodology to comprehensively assess the effectiveness of improving the quality and speed of the virtual medical training provided during the 4-week intervention. Observational data provided context on behaviours, interactions, and environmental factors. Structured surveys quantified these observations to evaluate the statistical significance of the results. Lastly, a virtual open forum discussion and in-depth interviews explored qualitative aspects of the intervention, allowing participants to share their perspectives. This multi-method approach helped to address individual method limitations and supported the validity and reliability of the research.

Cost-effectiveness assessment

This study conducted a comprehensive cost-effectiveness analysis to compare the CVMUI and TME models. Data collection involved calculating the one-month expenses for CVMUI in Afghanistan and extrapolating these to annual costs for 100 and 400 students while benchmarking TME expenses from existing literature. Cost estimation included detailed budgeting for CVMUI, covering platform costs, internet, faculty, staff, student services, course development, miscellaneous expenses, and infrastructure and operational costs for TME, such as construction, equipment, staffing, and compliance. An exponential growth factor analysis assessed cost variations as student numbers doubled, using independent two-sample t-tests for cumulative cost comparison. The cost-effectiveness ratio (CER) was calculated using the formula CER = Total Cost/Effectiveness, with effectiveness determined by pre- and post–experiment knowledge and OSCE levels. Finally, the percentage difference in cost-effectiveness between TME and CVMUI was calculated to highlight the comparative effectiveness of each model (Appendix p51-55).

Ethics statement

This project was approved under the University of British Columbia application H18-01932-A007. All participating students, instructors, and administrators signed consent forms.

Role of the funding source

This project received in-kind support from Lecturio ($360 USD per user, totalling $46 440.00 USD) and CyberPatient ($220 USD per user, totalling $28 380.00 USD). CanHealth International, a Canadian non-profit organization, raised an additional $ 60,000.00 USD to fund this project. This funding was mainly used to support all students, provide suitable Internet accessibility, and compensate faculty participants from Afghanistan. All faculty members from Canada and Argentina participated voluntarily (Appendix p51-55). It is important to note that none of the companies mentioned participated in analyzing the data, which was sourced exclusively from the stated platforms and integrated into the analysis unequivocally. CyberPatient and Lecturio provided no financial compensation.

Results

Descriptive

The mean ages of students at the first, second, and third levels were 18.34 ± 0.23, 20.7 ± 0.45, and 23.6 ± 0.14 respectively. The study's findings revealed that in the initial knowledge test for level 1, 31 of the 43 students participated, yielding a participation rate of 72.1%. Progressing to level 2, the participation rate increased to 83.87%, with 26 of the 31 students continuing. Finally, in level 3, 32 of the 46 students completed the study, resulting in a participation rate of 69.56%. The study revealed that in the OSCE exam, 45.16% of the students (14 out of 31) completed Level 2, while 58.7% (27 out of 46) finished Level 3. Post-knowledge test retention was higher, with 53.84% for Level 2 and a significant 84.37% for Level 3. The attrition from the knowledge test to the OSCE exam is attributed to the failure to meet the required 80% quorum.

Knowledge assessment (MCQ score)

The knowledge assessment by MCQ scores indicated a normal distribution across all three levels. There was a significant rise in mean scores from the pre-test to the post-test, showing notable knowledge enhancement after the intervention. The standard deviation values also increased, suggesting greater score variability without affecting the intervention's effectiveness. The P-values for all three levels were statistically significant at P < 0.0001, indicating meaningful differences between pre-and post-test scores within each difficulty level (Table 1 and Fig. 2). This demonstrates the effectiveness of the CVMUI curriculum delivered by Lecturio.

Table 1.

Comparative analysis of knowledge test results using paired T-test.

Variables for theoretical competence	Students (number)	Mean	Standard deviation	Mean difference	P-value
Pre-level 1	31	21.41	8.64	25.07	0.0001
Post-level 1	31	46.48	16.05
Pre-level 2	26	22.61	8.50	33.42	0.0001
Post-level 2	26	56.03	12.45
Pre-level 3	32	26.46	8.45	34.22	0.0001
Post-level 3	32	60.68	15.03

Fig. 2.

Clustered bar graph of mean knowledge scores.

Clinical competency assessment

OSCE test scores from training using the CyberPatient online simulator showed remarkable performance improvement. Pre- and post-test assessments at two difficulty levels revealed a significant (P < 0.0001) increase in mean scores, highlighting the intervention's effectiveness. The consistent improvement with significant P-values indicates that the CVMUI curriculum's clinical competency component, delivered via CyberPatient, was highly effective (Table 2 and Fig. 3).

Table 2.

Performance outcomes of the OSCE test.

Variable for clinical competence	Student participants (% of total participants)	Test	Mean	St dev	Mean difference	P-value
Pre-Level 2	14 (54%)	Wilcoxon matched-pairs signed rank test	19.90	10.26	53.6	0.0001
Post-Level 2	14 (54%)		73.50	16.36
Pre-Level 3	27 (85%)	Paired t test	32.43	14.06	41.38	<0.0001
Post-Level 3	27 (85%)		73.81	21.14

Fig. 3.

Bar graph comparing OSCE test scores.

OSCE completion times

Comparing OSCE completion times (OSCE CT) as a measure of curriculum effectiveness for levels 2 and 3 revealed a significant (P < 0.0001) decrease after the intervention for both levels (Table 3 and Fig. 4). This decrease indicates the positive impact of online simulation on students' speed of clinical decision-making.

Table 3.

OSCE completion times.

Level and timing	Student participants (% of total participants)	Test	OSCE completion times			P-value
			Mean (minutes)	St dev	Mean difference
Level 2
Pre-test	14 (54%)	Wilcoxon matched-pairs signed rank test	100.8	13.5	66.5	0.0001
Post-test	14 (54%)		33.5	14.3
Level 3
Pre-test	27 (85%)	Paired t-test	116.4	25.6	96.5	<0.0001
Post-test	27 (85%)		19.9	10.4

Fig. 4.

OSCE Completion times.

Multiple regression analysis

The multiple regression analysis using data from levels 2 and 3 separately demonstrated a positive correlation between predicted and observed values, with statistically significant results (P < 0.0006). This suggests a strong systematic association between predicted and observed variables. Furthermore, the multiple regression analysis of the entire study (both levels 2 and 3 indicated high predictive power with a multiple R-value of 0.9359 and an R-squared of 0.8759 (P < 0.02), confirming a strong positive correlation with the best-fit line between predicted and observed values (Fig. 5).

Fig. 5.

Scatter plot of actual versus predicted values.

OSCE substations

For level 2, Objective Structured Clinical Examination (OSCE) substations, including patient history-taking and physical examination substations, showed significant (P < 0.001) improvement post-intervention. For level 3, OSCE substations (history-taking, physical examinations, diagnosis management, treatment, documentation, and follow-ups) also showed significant (P < 0.0001) improvements, with slightly less significance in the follow-up substation (P < 0.03) (Fig. 6 and Table 4). Detailed results are provided in Appendix p15-31.

Fig. 6.

OSCE substation scores.

Table 4.

The results of OSCE substations in Level 2 and 3.

Variable OSCE substations
Level 2	Pre		Post		Mean difference	P-value
	Mean	St	Mean	St
History tacking	46	5.1	74.78	6.54	28.78	0.001
Physical exam	46.4	5.5	72.85	3.9	26.45	0.002

Variable OSCE substations
Level 3	Pre		Post		Mean difference	P-value
	Mean	St	Mean	St
History-taking	48.23	8.746	82.60	12.35	34.37	<0.0001
Physical examination	36.79	23.68	74.07	38.97	37.28	<0.0001
Diagnosis	44.32	13.65	89.67	16.10	45.35	<0.0001
Management	29.83	18.36	81.38	31.46	51.55	<0.0001
Treatment	24.44	20.41	76.78	29.79	52.34	<0.0001
Follow-up	20.60	17.31	37.67	48.72	17.07	0.0389
Documentation	31.91	15.47	69.78	26.26	37.87	<0.0001

Note: The study included 14 students in Level 2 and 27 in Level 3. Accordingly, a Wilcoxon matched-pairs signed rank test was used for the Level 2 analysis, and a paired t-test was applied for the Level 3 analysis.

Cofactor analysis

A Pearson correlation test analyzed cofactors affecting performance. For level 1, no significant correlation was found. For level 2, familiarity with computers and Eid celebrations had positive correlations (0.439 and 0.452, respectively), while university final exams had a strong negative correlation (−0.632). For level 3, final exams had a very strong negative correlation (−0.705), suggesting better performance without these exams (Appendix p32). This suggests that students could have done better if they had not simultaneously taken the university final exam.

Survey results

A survey on the CVMUI curriculum showed that 98% of students rated the content, relevance, and instructional quality positively, with 72% rating it excellent, 17% very good, and 8% good. The course was praised for its use of technology, which enhanced the learning experience. Some areas for improvement were noted, with certain aspects rated satisfactory or poor. These areas included Internet accessibility, technology preparedness, English language proficiency and sustainability. The survey highlighted the benefits of Lecturio and CyberPatient platforms for enhancing understanding, clinical confidence, and skill development. Detailed survey results are in Appendix p33-40.

Open forum discussion and interviews

An open forum discussion on April 23, 2024, involved stakeholders from eight Afghan and two Canadian provinces. Key discussion points included (1) universal satisfaction whereby stakeholders expressed satisfaction and moreover the desire to integrate the virtual medical training program permanently into their medical curricula; (2) for integration, they proposed a curriculum committee under the Ministry of Higher Education of Afghanistan to oversee the systematic integration of CyberPatient and Lecturio platforms to address gaps in medical training in Afghanistan, and (3) recognition of challenges including the description of technical difficulties and scheduling conflicts (e.g., how to schedule exams during cultural observation of Ramadan) were acknowledged, with plans to mitigate these challenges in the future.

Data triangulation

The triangulation of observation, survey, and open discussion data confirmed the experiment's internal validity and reliability. The results support the effectiveness of online clinical skills acquisition. Table 5 and Appendix p41-51 provide further details for the triangulation analysis.

Table 5.

Key outcome measures.

Key measures	Observation	Survey data			Open forum discussion and interviews (Appendix p52-55)
	Responses	Student satisfaction (Appendix p8-14)		Faculty	Students	Deans
Clinical knowledge at baseline	24	Knowledge gain: Excellent Very good Good Satisfactory Poor	52% 30% 16% 2% 0%	It promotes reflective thinking in the student. Clinical case presentation is based on student needs rather than solely on patient availability. It offers equivalent learning opportunities for different students.	The clinical case can be repeatedly practiced, affording the student continuous opportunities to hone their operational skills. This curriculum integrates prior knowledge and new knowledge. Immediate feedback with a corrective approach is provided on overall performance. .	It translates theoretical medical knowledge into practical clinical knowledge The program was designed to support an inclusive approach, catering to learners from beginner to expert levels. The simulator's focus was on developing technical skills, complemented by weekly sessions integrating practical and theoretical knowledge. The simulator effectively enables trainees to repeatedly practice and acquire biographical skills, but less emphasis was placed on communication skills and patient empathy.
Intervention effect on clinical knowledge	54
Clinical skills baseline	26	Clinical competency gain: Excellent Very good Good Satisfactory Poor	55% 27% 12% 6% 0%	Strongly positive comments: The clinical case can be repeatedly practiced, affording the student continuous opportunities to hone their operational skills.	Strongly positive comments: It translates theoretical medical knowledge into practical clinical knowledge.	Strongly positive comments: Clinical case presentation is based on student needs rather than solely on patient availability.
Effect of intervention	74
Intervention impact	Clinical decision-making improved 80%	Overall program rate: Excellent Very good Good Satisfactory Poor	72% 17% 8% 3% 0%	Positive Comments: It will make the clinical placement, and clinical exams much simpler, makes the students ready for safer clinical encounter Modifying and systematically altering the general medical curriculum of Afghanistan with the goal of digitalization Leveraging the university/institute's social networks to familiarize as many instructors and students as possible with the CVMUI curriculum Exploring new technology-driven methods and platforms for education Orienting the student to the subject matter Instructing on lesson planning for students Selecting educational resources and clinical scenarios aligned with established learning objectives Activating students' prior knowledge and experiences pertaining to the virtual clinical case through online discussions Delineating the educational goals of the scenario before engaging with it Specifying the student's responsibilities regarding the virtual patient Familiarizing students with the virtual clinical environment, case, and their assigned roles Facilitating the establishment of personal learning goals for the virtual clinical case Introducing the topic of the virtual patient scenario	Strongly positive Comments: Make me more competent and ready for clinical rotations and interaction with patients. We were not aware of how technology can help us in our studies. Now we know. Not applying this program to our schools will keep Afghan medical students in a disadvantage	Strongly positive Comments: We need this program to be integrated to our national curriculum The institutional leadership leverages their influence and authority to implement and integrate the program into the existing curriculum and offerings of the educational institution Deployment of the resources and facilities of the university to establish the program

Cost-efficiency analysis

Cost analysis (presented in Appendix p51-55) showed that the cost of online medical education, based on the data obtained in this experimentation, is about $24 Million per year for a university with 400 students. The infrastructure cost for online medical education is about $5 million (considering the 20 years of amortization, the infrastructure cost is $250K per year) and will be negligible. The benchmark average operating cost for TME with the same number of students is $65 million. The average infrastructure cost for TME is $425 Million (considering the 20 years of amortization), which will be about $33 Million annually. Therefore, the annual cost of TME for 400 students per year is $98 million and for CVMUI, $6.250 million. The cost difference between TME and CVMUI is about $92 million. We plotted these values in an exponential growth mode (Fig. 7), showing that with the increased number of students, the cost of TME and the cost difference grow exponentially while the cost of CVMUI is negligible.

Fig. 7.

Exponential growth model for TME and CVMUI, including the cost difference.

In the next stages of our cost analysis, we calculated the Cost-effectiveness ratio (CER) based on the effect and cost results obtained in this experiment. The CER for TME = 416,000, and the CER for CVMUI = 19,000. The difference between TME and CVMUI was ≈397,000. Calculation of the percent difference relative to TEM shows that CVMUI is 95% more cost-effective than TME.

Discussion

The results of this study indicate that a virtual medical training model, as implemented through the Canadian Virtual Medical University Initiative (CVMUI), is both feasible and effective in enhancing the knowledge base and clinical competencies of medical students. This approach leverages technological advancements and a competency-based, student-centred curriculum to address the global shortage of well-trained physicians, particularly in conflict zones and remote areas.

One of the most significant findings was the substantial improvement in students' knowledge acquisition and clinical competencies. Asynchronous knowledge delivery is not new; successful online education for theoretical subjects is widely accepted.²⁹ Similarly, granting degrees for online education has precedent worldwide.^30,31 While theoretical delivery under CVMUI using Lecturio is established,³² acquiring practical medical competencies historically believed only possible through in-person clinical experience³³ is challenged. Patient availability, a perennial issue,³⁴ has been supplemented by simulators like standardized patients and mannequins,³⁵ which are successful yet costly.³⁵ Previous studies support physical and online simulation efficacy.³⁶ This study uniquely demonstrates CVMUI's efficacy in competency-based, student-centred education, effectively covering theoretical and practical medical education. In this study, clinical competency, delivered by CyberPatient, supports effective learning of clinical competencies online in a virtual simulation environment. Online simulation's advantages, such as accessibility and cost-effectiveness,⁸ make it ideal for standardized medical education globally, overcoming logistical and ethical constraints.

Notably, this study found Level 2 and 3 students required significantly less time to perform clinical tasks (P < 0.001), reducing cognitive load and indicating confidence in clinical settings. This aligns with Andersen et al.'s findings on simulation reducing resident cognitive load.³⁷ Pusic et al. further suggest chronometry enhances medical procedure learning.³⁸ Moreover, CyberPatient modules improved Level 3 students' scores and reduced time, underscoring its dual benefit in performance time and effect. Novice medical students often lack self-confidence until later in training, hindered by limited patient exposure.³⁹ Asynchronous delivery of clinical competencies through virtual simulation environments mitigates this, allowing safe practice and rapid experience accumulation.⁴⁰

Student satisfaction with the virtual curriculum was notably high (98%). The flexibility and accessibility of the online platforms were particularly appreciated, which is crucial for students in resource-limited settings.^16,17 The positive feedback also highlights the importance of integrating interactive and engaging educational tools like Lecturio and CyberPatient to maintain student interest and motivation.^41,42

The qualitative data from open discussions provided valuable insights into the benefits and challenges of the virtual education model. Increased accessibility to quality education and learning at one's own pace are major advantages. However, challenges such as the need for reliable internet access and the initial adjustment period for students new to virtual learning were also noted. These issues underscore the importance of providing adequate technical support and resources to implement virtual education programs successfully.^33,43 The other important challenge was the sustainability of virtual programmes in developing nations such as Afghanistan. In addition to the research objectives, the survey and open discussion/interview analysis strongly recommend meaningful integration of technological advances such as CyberPatient and Lecturio into their traditional curricula to make a hybrid medical education program. As others have mentioned, this recommendation points toward reforming existing medical schools into a hybrid program.^43,44 The cost efficiency of virtual programmes, such as CVMUI, is the best factor for addressing sustainability concerns in developing nations.⁴⁵

The cost analysis revealed significant savings compared to traditional medical education models. Reducing infrastructure and operational costs makes the virtual model a financially viable option, especially for expanding access to medical education in developing countries.¹⁶ This cost-efficiency, combined with the program's demonstrated effectiveness, supports the CVMUI model's scalability to other regions facing similar challenges.¹⁷

The rationale for choosing Afghanistan is as follows. First, recent news releases, World Bank and UN statistics^4,46 consider Afghanistan's healthcare system a disaster for the foreseeable future. Second, stopping Afghan girls from medical school in Afghanistan called for immediate support and a reasonable solution to satisfy the rules passed by the Afghan government and bring the girls back to school. The CVMU initiative satisfied the religious and traditional rules for girls' education and supported female students' re-entry into the classroom. Third, given the limited technological and infrastructure capabilities, it was reasonable to assume that the success of this project in Afghanistan could be replicated in countries with similar or better technological conditions. Therefore, this situation allowed us to meticulously investigate the effects of infrastructure such as low-band Internet and the incorporation of complex technologies such as Lecturio and CyberPatient into the curriculum.

This project faced significant limitations, including unstable internet infrastructure and inadequate technological equipment like desktop computers. Although we anticipated technological challenges in developing countries like Afghanistan, this project highlighted the depth of the issue and ways to manage it. To address this, we facilitated numerous meetings to enhance interaction among decision-makers from Afghan universities. The program aimed to bring Afghan female students back to school; however, due to the current admission policies of Afghan universities, the project included only male participants. According to our agreement with Afghanistan's Ministry of Higher Education, all students can attend CVMU upon successfully completing this project, with female medical students given priority in the first year. Other factors, such as the holy month of Ramadan and final exams, also limited the project, as detailed in the co-factor analysis section. The Ministry of Higher Education also requested mentors from Afghan universities, some of whom had less proficient English skills than anticipated. This will be considered in future plans.

This study demonstrated that although infrastructure challenges, students and faculty's unfamiliarity with technology, cultural factors, and conflicts stemming from war can impact results, the proposed virtual training approach was able to overcome these obstacles. Despite infrastructural problems in Afghanistan, the researchers provided effective training and addressed participants' lack of familiarity through initial orientation sessions. Moreover, studies conducted in war-torn regions such as Syria^10,11 and Sudan¹² have recommended virtual training as a viable solution. Therefore, this educational innovation has the potential to surmount the aforementioned issues and deliver appropriate instruction in these areas where the need for such an approach is compelling. However, the active participation of policymakers is crucial for the successful implementation of this project.

In conclusion, the results of this study proved the successful implementation of CVMUI's competency-based, student-centred curriculum concept using state-of-the-art technological advances in a technologically challenged environment; it demonstrated the efficacy of CVMUI's virtual curriculum on pre-internship medical students; and provided a cost-effectiveness estimate of CVMUI versus traditional in-person medical education using benchmarking methods. The CVMUI model presents an innovative and promising solution to the global physician shortage. This approach can facilitate high-quality training for all medical students, particularly those in underserved and conflict-affected areas, by harnessing the power of modern online training technology and innovative educational strategies. We plan future research focusing on long-term outcomes and the potential to scale this model further to other health disciplines, such as nursing and allied health professionals, dentistry, pharmacy, etc. In the immediate term, we plan to perform a similar study with other countries in the region, such as Somalia, Yemen, Ethiopia, Sudan and Uzbekistan. Also, future research can build upon the present findings by including a control group to enable more rigorous comparisons. Additionally, longitudinal analyses of learning outcomes through repeated measurements could shed light on the long-term efficacy of the intervention. Importantly, evaluating student performance in challenging circumstances, such as crises following the implementation of this approach, may provide valuable insights into its real-world applicability and effectiveness.

Contributors

Karim Qayumi PhD- Experimental design, supervision, Seyedeh Toktam Masoumian Hosseini PhD- Data management committee chair.

Mohsen Masoumian Hosseini PhD- Data analysis, management.

Asadullah Nejat MD- Organization, Instruction.

Mohibullah Salih MD- Instruction.

Mammodullah Azimi MD- Instruction.

Sharif Forqani MD- Organization Instruction.

Banaras Akbar MD/MS- Instruction.

Ghulam Farooq MD- Instruction.

Najibullah Shafaq MD- Organization Instruction.

Hussain Rustampoor MD-Instruction.

Nasrin Oryakhil MD- Data analysis, management.

Masoud Rahmani MD-Data analysis, management.

Masood Noora MD- Organizer, Data analysis, management.

Mohammad Nasir Jallah MD-Organization, Instruction.

Asmatullah Naebkhil MD-Organization, Instruction.

Zubaida Anwari Zhwak MD- Organization, Instruction.

Sohaila Aziz MD- Organization, Instruction.

Farid Ahmad Omar MD- Organization, Instruction.

Ahmad Mustafa Rahimi MD- Instruction.

Parwin Mansuri MD- Organization Instruction.

Sumaira Yaftali MD- Instruction.

Nilofar Sadiq MD- Instruction.

Jahed Payman MD- Instruction.

Amanullah Arifzai MD-Instruction.

Mohammed Azim Azimee MD- Instruction.

Somaya Waqef MD- Instruction.

Stefan Wisbauer MPA- Organization.

Joffre Guzmán-Laguna MD-Instruction.

Alberto Ferreres MD-Instruction.

The data was accessed and verified by Dr. Karim Qayumi and Dr. Seyedeh Toktam Masoumian Hosseini for the study. All authors not only read and approved the content of the primary and revised manuscript but also participated in its preparation.

Data sharing statement

The educational intervention data of this study is presented in the appendix. Additionally, further information regarding the project data will be made available to researchers upon reasonable request from the corresponding author.

Declaration of interests

The authors have no competing interests to declare.