Evaluating the Use of Supplemental Training Technologies in Dermatology Education

Abstract

Physician assistants (PAs) are licensed to evaluate, diagnose, and treat dermatologic skin conditions. Data show that medical students have less than optimal dermatology diagnostic abilities. Although no known data exists for PA students, similar medical school and PA school training methods highlight a need for improved dermatology education in medical and PA programs. This project explored the use of perceptual and adaptive learning modules (PALMs) that target pattern recognition skills with PA students to hopefully improve PA knowledge of dermatology concepts.

Background

There are no known studies on dermatology training in PA programs. The data on medical school training suggest that about 50% of US medical schools provide 10 or less hours of dermatology training, while about 8% of US medical schools require no instruction in dermatology.^1,2 One study by Ulman et al demonstrates medical students are not proficient in diagnosis and treatment of dermatologic disease despite about 18 hours of training, and the authors of this study suggests evaluation of dermatology curriculum nationwide.¹ While our PA program provides roughly 29 hours of didactic education in dermatology, mastery of dermatology concepts cannot be achieved without practice.³ There is a discrepancy between learned knowledge in medical education and application of that knowledge during clinical practice, notably in medical skills that require recognition of clinical patterns, such as dermatology.³ Traditional teaching methods are thought to lack in the training of perceptual learning, defined as “experience-induced changes in the way perceivers extract information.”⁴ Whereas most instruction emphasizes explicit declarative and procedural learning, other crucial components of expertise, including pattern recognition, fluency, and clinical intuition involve different learning systems and advance through more implicit and interactive learning experiences.^3,4 Diagnostic expertise in medical learners advances through classification episodes that incorporate a range of instances that encompass normal and pathological variations across relevant categories. In traditional medical education in dermatology, as in other domains, these aspects of learning occur somewhat unsystematically through recurrent clinical experiences or exposures to patients with dermatology complaints.^5,6 Gaining this experience may take large amounts of time that is not afforded in conventional didactic dermatology education models.²

One resource proven to bridge this gap in medical education is an online supplemental technology resource developed by Insight Learning Technology, Inc. called Perceptual and Adaptive Learning Modules (PALMs).^3,7 PALMs were created as a technology resource to help students increase mastery of medical skills through pattern recognition. They are intended to supplement didactic training by providing practice examples that enable optimization of rate and retention of performance in specific medical skill areas.^3,7 PALMs have provided significant improvement in medical school education and medical resident training in the areas of echocardiogram interpretation, histopathology, electrocardiogram interpretation, and dermatology.^3,6,8,9 This supplemental platform, however, has never been studied in dermatology training specifically in PA education.

Briefly, PALMs target perceptual learning – changes in the pick up of information that occur in a given domain as a result of practice or experience.⁴ They incorporate a number of principles of learning, including spacing and interleaving, and systematic variation of exemplars, to accelerate expert pattern recognition skills. The adaptive components of PALMs pair advanced adaptive algorithms with perceptual learning, such that the spacing and recurrence of each learning category is based on each learner’s accuracy and speed with exemplars of that category in ways that tend to optimize the efficiency and durability of learning. The adaptive elements in PALMs also track learning of each category to mastery criteria that include both accuracy and fluency.^3,10 Currently, there are five dermatology PALMS allowing students to practice lesion morphology, lesion distribution, lesion configuration, lesion identification (skin cancer), and lesion diagnosis (Figure 1). In four out of five dermatology PALMs, students are presented with an image that requires them to choose a diagnosis or description (Figure 2, left). The exception is the “Lesion Identification: Skin Cancer” PALM, where the user is presented with two images, a regular image and a dermoscopic image, and the software then requires them to choose a diagnosis using both pictures (Figure 2, right). For all modules, after the student selects an answer choice, the software displays the correct response. The PALMs software adapts to the student user by adjusting presented cases based on the learner’s performance, thus making it more customized for each individual student learner. This adaption also allows the student to improve on areas of weak understanding and improve proficiency of the dermatology concepts listed in Figure 1.^3,6,8,9 Because PALMs have never been studied in dermatology training of PA students, this project evaluated the use of PALMs in PA education.

Figure 1:

Dermatology PALMs and Topics Covered in Each PALM (https://med.insightlt.com/site/, Insight Learning Technology, Inc. ©2021)

Figure 2:

(left) Example of “Lesion Diagnosis” PALM (right) Example of “Lesion Identification: Skin Cancer” PALM with one regular and one dermoscopic image (images: https://med.insightlt.com/site/, Insight Learning Technology, Inc. ©2021)

Research questions

1. Will use of supplemental training technologies in dermatology education improve PA student knowledge and diagnostic accuracy of dermatology concepts?

2. Will use of supplemental training technologies in dermatology education improve PA student perceptions of their dermatology knowledge and skill on clinical rotations?

Research methods

Group and Sample Size

This study was completed at a PA program from a single university. All students from the Class of 2020 (68 students), Class of 2021 (69 students), and Class of 2022 (70 students) used the PALMs educational intervention during the study period, though each class was in a different stage of their education (Figure 3).

Figure 3:

Current dermatology curriculum in our PA program (white boxes) and stage of education for Classes of 2020, 2021, and 2022 at the time of the educational intervention (navy boxes)

Methods

Students are typically trained in dermatology at different times in the curriculum (Figure 3). The traditional lecture-based and lab-based education was maintained in this study, totaling almost 29 hours of instruction. This instruction includes training in Physical Diagnosis of Skin, Hair, and Nails (3 hours) along with a Clinical Medicine dermatology module (21.5 hours) covering dermatologic disease states. Students also participate in a hands-on skills lab where they practice excisions, punch biopsies, shave biopsies, and dermoscopy. Lastly, all students attend dermatology review lectures during their clinical education year at an “End of Rotation” (EOR) training session (Figure 3).

For the educational intervention, each cohort was required to complete dermatology PALMs specific to their level of education within two to four weeks of completing their lecture series or lecture-based module (Figure 4). For each dermatology PALM, the technology platform was set so students completed a pretest, the PALMs training module, then a posttest. Students were then asked to voluntarily complete a survey regarding their experiences using PALMs in dermatology training.

Figure 4:

Required dermatology PALMs and PA student assessments by cohort

Data Collection and Statistical Analysis

Institutional Review Board (IRB) approval was obtained. IBM SPSS statistical software was used for all analyses. Perception survey responses and dermatology PALMs pretest and posttest scores were collected (Figure 4). Data were de-identified and aggregated by the following groups: Class of 2020, Class of 2021, and Class of 2022. Paired-sample t-tests were used to compare pretest and posttest scores for each group to look for changes before and after dermatology PALMs use, specifically looking at accuracy (percentage of images correctly identified) and fluency (percentage of images correctly interpreted within a response time of 15 seconds or less). Statistical significance was set at p<0.05.

In addition to evaluating PA student knowledge, we assessed PA student perceptions of PALMs using a 5-point Likert scale perception survey. Responses ranged from “strongly disagree” to “strongly agree;” for the purposes of this project, responses of “agree” and “strongly agree” were combined and noted as “respondents in agreement.” Overall survey response rates were 59.94% for the Class of 2020, 92.75% for the Class of 2021, and 88.57% for the Class of 2022.

Results

Student Performance

Class of 2020

The Class of 2020 was in their seventh semester of PA education at the time of the study (“PA Year 3 Spring,” Figure 3). This cohort was required to complete the “Lesion Diagnosis” and “Lesion Identification: Skin Cancer” modules. Students performed an average of 195 and 291 practice cases, respectively, for each module and it took students an average of 21.9 minutes and 44.2 minutes, respectively, to complete each module. For both the “Lesion Diagnosis” and “Lesion Identification: Skin Cancer” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 5).

Figure 5:

Class of (CO) 2020 student performance results by PALM online module

Class of 2021

The Class of 2021 was in their third semester of PA education at the time of the study (“PA Year 1 Fall,” Figure 3). This cohort was required to complete the “Lesion Morphology,” “Lesion Configuration,” and “Lesion Diagnosis” modules. Students performed an average of 102, 75, and 202 practice cases, respectively, for each module and it took students an average of 11.9 minutes, 8.9 minutes, and 26.6 minutes, respectively, to complete each module. For the “Lesion Morphology,” “Lesion Configuration,” and “Lesion Diagnosis” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 6).

Figure 6:

Class of (CO) 2021 student performance results by PALM online module

Class of 2022

The Class of 2022 was in their first semester of PA education at the time of the study (“PA Year 1 Spring,” Figure 3). This cohort was required to complete the “Lesion Morphology” and “Lesion Configuration” modules. Students performed an average of 105 and 85 practice cases, respectively, for each module; it took students an average of 11.5 minutes and 11.1 minutes, respectively, to complete each module. For both the “Lesion Morphology” and “Lesion Configuration” modules, there was statistically significant improvement (p<0.05) of pretest to posttest scores for both accuracy and fluency (Figure 7).

Figure 7:

Class of (CO) 2022 student performance results by PALM online module

Student Perceptions

More than 80% of respondents from all three cohorts agreed that utilizing the PALMs improved their overall understanding of dermatology. More than 60% of respondents from all three cohorts agreed that utilizing PALMs improved their ability to interpret skin lesions and their accuracy in diagnosing dermatology conditions. Lastly, more than 70% of respondents from all three cohorts agreed that “online dermatology modules (PALMs) should be added to the curriculum of medical education programs for Physician Assistants.” (Table 1)

Table 1:

PA Student perception survey results; respondents in agreement per cohort (Classes of (CO) 2020, 2021, and 2022)

PALMs Student Perception Survey	% of Student Respondents in Agreement (CO2020)	% of Student Respondents in Agreement (CO2021)	% of Student Respondents in Agreement (CO2022)
I feel the online dermatology modules (PALMs) improved my overall understanding of dermatology.	80.55%	85.94%	96.78%
I feel the online dermatology modules (PALMs) helped me to correctly identify abnormal skin lesions.	72.23%	79.36%	--*
I feel the online dermatology modules (PALMs) helped me improve my ability to interpret skin lesions.	69.45%	76.19%	93.55%
I feel the online dermatology modules (PALMs) helped me improve the rate at which I can interpret skin lesions.	58.33%	78.13%	85.48%
I feel the online dermatology modules (PALMs) will improve my ability to care for patients with dermatologic complaints.	55.56%	64.07%	80.32%
I feel the online dermatology modules (PALMs) will improve my accuracy in diagnosing dermatologic conditions.	61.11%	67.19%	75.81%
I feel the online dermatology modules (PALMs) have improved my confidence in assessing skin conditions.	48.57%	64.06%	77.42%
I feel online dermatology modules (PALMs) should be added to the curriculum of medical education programs for Physician Assistants.	72.22%	81.25%	96.77%

^*This question was not asked of the Class of 2022 since it was not applicable to the PALMs they completed.

Discussion

Research Question #1: Will use of supplemental training technologies in dermatology education improve PA student knowledge and diagnostic accuracy of dermatology concepts?

For all three cohorts, the use of supplemental PALMs improved student knowledge in dermatology concepts and diagnoses. These findings suggest that PALMs supplemental modules are a method to provide students with multiple practice examples that enable students to improve their knowledge base in dermatology. Because PALMs are housed in online modules, this supplemental training is an efficient way for students to practice concepts before seeing patients. This PA student performance improvement aligns with research using PALMs for training of medical students and medical residents.^6,8,9

The gains seen in these data came from relatively brief learning interventions. With the mastery criteria used in the PALMs tested here, completion times averaged under 20 minutes, with some being completed in an average of about 11 minutes. Despite this brevity, the learning gains were robust. These results are consistent with earlier work in indicating that PALMs address missing components of learning, such as pattern recognition, that are not much advanced by traditional instructional methods. It would be hard to believe that 11 minutes or 21 minutes of additional classroom time would have produced these results, after so many hours of time had been devoted to declarative and procedural instruction. If PALMs address missing elements of learning, it is encouraging to see that systematic interventions targeting perceptual learning can begin to exert their effects in a short time, given appropriate methods.

Lastly, the pretest levels did not vary greatly by cohort. This is another indicator that the PALMs are addressing aspects of learning that are not well-addressed by conventional instruction. If solely having a bit more conventional instruction would take care of what the PALMs are addressing, then one would expect large pretest differences for more advanced students.

Research Question #2: Will use of supplemental training technologies in dermatology education improve PA student perceptions of their dermatology knowledge and skill on clinical rotations?

Results of the perception surveys note overwhelmingly positive feedback from use of dermatology PALMs in all three cohorts for all questions asked (Table 1). It is important to highlight that respondents agreed that use of dermatology PALMs improved their overall understanding of dermatology, improved their ability to interpret skin lesions, and improved their ability to diagnose dermatology. These results support that dermatology PALMs improved PA student perceptions of their dermatology knowledge and skills. Lastly, a majority of student respondents from all three cohorts felt these dermatology PALMs should be added to PA curriculum, which captures an overall level of satisfaction with the modules and emphasizes student willingness to continue using these modules in PA education.

Instructor Implementation Perspective

Combining e-learning modules with traditional medical education methods has been shown to produce a more time-efficient and effective learning system than traditional learning methods alone.¹¹ As a dermatology educator, this supplemental online training platform was simple to implement into our PA program curriculum. The average completion time for each module was about 19 minutes (ranging from 8 minutes to 44 minutes), which made adding these modules achievable in a challenging and full curriculum. The adaptive nature of these modules allow individualized learning for all students and real-time feedback as they practiced, which was noted to be helpful for all students no matter their areas of strengths or weaknesses.

It should also be highlighted that these modules have images of patients with all six Fitzpatrick skin phototypes, allowing students to practice dermatology concepts on patients with variable skin pigmentation. Lastly, it is important to note the recognition that the importance of perceptual learning does not detract from the importance of declarative and procedural instruction in traditional medical education. PALMs address aspects of learning, involving different cognitive systems, that are difficult to address with traditional instruction; however, we do not feel these interventions replace the important aspects of conventional instruction used in dermatology education. In fact, the synergy of PALMs as additions to learning with more conventional elements may be particularly powerful in their long-term consequences for medical practice.

Limitations and Future Research

The dermatology PALMs modules were not inclusive of every dermatology lesion or abnormality students may encounter, therefore limiting students’ ability to practice all types of dermatology cases.

The findings may not be generalizable since this was a single site study; conducting this research at multiple PA programs would improve the validity. Future research should be considered to note the impact of dermatology PALMs on PANCE (Physician Assistant National Certifying Examination) board performance and/or clinical rotation performance. It would also be beneficial to study the use of dermatology PALMs in practicing PAs.

Conclusion

The combination of improved student performance data and positive student perception study data supports the continued use of dermatology PALMs in the dermatology training of PA students at our PA program.

Author bios:

Mallory M. Aycock, MPA, PA-C is a graduate of Medical College of Georgia in Augusta, GA. She has practiced both in Dermatology and Surgery. Mallory is currently a Clinical Assistant Professor at Mercer University in the Department of Physician Assistant Studies. She has indicated no relationships to disclose relating to the content of this article.

Craig D. Marker, PhD received his PhD and MS in clinical psychology from Rosalind Franklin University and a postdoctoral respecialization in quantitative methods from the University of Virginia. Dr. Marker is Professor, Chair, and Program Director of Clinical Psychology at Mercer University.

Philip J. Kellman, PhD received his PhD from the University of Pennsylvania. He is Distinguished Professor of Psychology, Adjunct Professor of Surgery, and the current Cognitive Area Chair in the Department of Psychology at the University of California, Los Angeles.