Short- and Long-Term Evaluation of General Practitioners’ Competences After a Training in Melanoma Diagnosis: Refresher Training Sessions May Be Needed

Abstract

General practitioners (GPs) are first-line clinicians in melanoma diagnosis. It is, therefore, important to ensure that they maintain their melanoma diagnostic accuracy over time. The objective of this study was to assess the short- and long-term competences of GPs after a training session in naked-eye melanoma diagnosis. An interventional prospective study was conducted whereby, over a 6-month period, GPs attended a 1-h melanoma diagnostic training session. To assess their acquired competences, GPs were asked to fill in a questionnaire on basic melanoma knowledge and to evaluate 10 clinical images of pigmented skin lesions prior to training, immediately after and 1 year later. In total, 89 GPs completed the questionnaire prior and immediately after training. As expected, the number of GPs who appropriately managed $\ge$ 50% of the melanoma cases increased after training (P < 0.001). One year after training, only 27 (30%) of the 89 GPs completed the questionnaire. This number of participants was too low to obtain significant figures but the GPs’ mean overall score of appropriately managed clinical cases was much lower than in the immediate post-test. In conclusion, although this short training improved the GPs’ diagnostic accuracy and management of melanoma in the short-term, participating GPs do not seem to have maintained these competences in the long-term. Further studies are needed to assess whether refresher training sessions are able to sustain acquired diagnostic and management skills.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13187-021-02063-6.

Introduction

Over the last two decades, general practitioners (GPs) have been recognized to play a key role in early melanoma diagnosis [1, 2]. Early detection is essential to reduce morbidity and mortality of patients with melanoma [3]. However, melanoma diagnostic abilities of GPs in Europe have been demonstrated to be suboptimal [4–6].

In order to help GPs deal with suspicious skin lesions, melanoma diagnostic training programs have been developed. Initially, these programs trained GPs to recognize melanoma clinically (with the naked eye) [7]. Later, programs teaching diagnosis using dermoscopy became more widespread [8, 9]. However, due to the long training period required to become competent, dermoscopy is underused for the diagnosis of melanoma in general practice [4, 10, 11]. Therefore, it is essential, at the very least, to train GPs in naked-eye recognition of melanoma.

Most training programs teaching naked-eye melanoma recognition have been shown to improve GPs’ short-term diagnostic accuracy and management in a training setting (on photographs of skin lesions) [12–17]. Nonetheless, it is also important to ensure that the competences acquired by GPs are maintained over time [18–20]. Only four studies were found in which long-term ($\ge$ 6 months) accuracy of GPs in melanoma diagnosis using clinical examination alone was evaluated [21–24]. The results of these studies were heterogeneous. One study showed a drop of GPs’ competences at 6 months post-training [21] and a second study showed a decrease over an 8-month period [22]. However, a major French study conducted in real-life demonstrated a positive long-term impact of their training with a decrease in the incidence of very thick melanomas (Breslow thickness ≥ 3 mm) over a 3-year period after a screening campaign performed by participating GPs [23]. Finally, a recent case–control study assessing 1662 referrals to dermatologists reported better quality of referrals by GPs trained in clinical melanoma diagnosis than by untrained GPs over a 10-month period [24]. Unfortunately, these two latter studies poorly evaluated the keys to the long-term sustainability of the acquired competences.

The aim of our prospective interventional study was to determine the short- and long-term competences of GPs after a 1-h training session, designed by our study team, on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

Material and Methods

Study Design

This prospective interventional study took place over an 18-month period from November 2015 to May 2017 and was approved by the Ethical Committee of the UCLouvain, Belgium. A 1-h training session in melanoma diagnosis was integrated into the annual mandatory medical education program of eight groups of Continuing Medical Education from the five French-speaking districts of Belgium (Hainaut, Brabant-Wallon, Province du Luxembourg, Province de Liège, and Bruxelles).

Training Program

A 1-h training session, designed by our study team primarily targeting GPs, was common to all participants. The session included the teaching of three widely used naked-eye melanoma diagnostic tools (the ABCD rule [25], “pattern recognition” [26], and the “ugly duckling” sign [27]) and instructions in basic knowledge on melanomas and pigmented skin lesions (PSL) such as risk factors of melanoma, preferred locations of melanoma according to gender, normal/abnormal evolution of PSL according to the patient’s age, and characteristics of the most common benign PSL (seborrheic keratosis, typical nevus, blue nevus, congenital nevus, halo nevus, actinic lentigo, and subungual hematoma). Management of PSL was also taught during the training sessions. To illustrate the diagnostic tools, a large number of clinical photographs of melanomas and benign lesions were shown.

Data Collection

All the study data was collected using REDCap® (Research Electronic Data Capture, Vanderbilt University, USA) tools hosted at the Cliniques universitaires Saint-Luc in Brussels (Belgium) [28, 29].

Prior to training, the GPs completed a three-section questionnaire. This questionnaire was the result of a consent between two GPs (C.D. and S.L.) and a dermatologist (I.T.). Section A of the questionnaire collected socio-demographic and medical practice data, Section B investigated the GPs’ usual melanoma diagnostic practices, and Section C evaluated their basic melanoma knowledge and their diagnostic accuracy and management of PSL.

Section A—Characteristics of the Study Population

Participating GPs were asked about their gender, age, workplace (urban, suburban, or rural area), work practice (solo, pair, medical group, or local network practice), and the number of patients seen per year.

Section B—Melanoma Diagnostic Practices

To explore the way in which GPs diagnose melanoma and manage their patients in daily medical practice, ten multiple-choice questions were formulated.

Section C—Basic Melanoma Knowledge and PSL Diagnostic Accuracy and Management

To assess the efficacy of our melanoma training session, GPs completed this section of the questionnaire before training, immediately after training, and 1 year later. The details of Section A and B are described in a previous publication [6].

For this third section, participants were randomly subdivided into two groups, with 45 GPs in group A and 44 GPs in group B. Two sets (A and B) of similar questions were created covering each essential training topic. Each group of GPs received one of the two sets to fill in prior to and immediately after training. The questionnaires completed prior to training were collected by the organizing team before the start of the training session. The sets were switched between groups at 1-year post-training. To avoid recall bias, the questions on the same topic were similar but not identical (there was no perfect parallelism between the diagnoses and/or management proposed within these similar questions). This 1-year questionnaire was sent by return mail to the GPs.

The section C questionnaire was divided into two parts and all the questions and answers were peer-reviewed and validated by six dermatologists. The first part consisted of five multiple-choice questions that explored basic melanoma knowledge (see Appendix 1). GPs were asked about the preferred locations of melanoma according to gender, criteria for urgent melanoma management, characteristics of high-risk melanoma patients, and patient follow-up. In the second part, GPs were asked to assess 10 clinical photographs of PSL and their clinical history (see Appendix 2). Every clinical history included the patient’s age, gender, localization, and evolution (new onset or old lesion, change in shape or size) of the lesion. For each lesion, GPs were asked to select the correct diagnosis or the appropriate patient management from among five proposals. It is important to note that the clinical photographs used in the questionnaires were similar but not identical to those presented at the training session.

Statistical Analysis

The statistical analysis was performed using the SAS software (SAS© 9.4, SAS Institute Inc., USA). Descriptive statistics were used to summarize the results considering absolute numbers, percentages, and 95% confidence intervals for the three periods of assessment. In addition, the GPs’ average number of correct answers to the 10 clinical cases was summarized using mean ± standard deviation and the GP’s responses to clinical cases corresponding to melanomas were also grouped according to a threshold of 50% correct answers. McNemar’s test for binary matched-pairs data was used to test for variations between section C of the questionnaires prior to and immediately after training with regard to the number of GPs who correctly identified or gave the appropriate management for the clinical cases. All tests were considered to be significant for P value < 0.05. Statistical inference was not made for the comparison with the 1-year questionnaire because of the low number of GPs who responded to this questionnaire and the subsequent correction to be made on the type I error in case of multiple comparisons.

Results

As part of their mandatory continuing medical education, 89 GPs participated in the training and completed the entire questionnaire prior to training. These 89 GPs also filled in Section C (basic melanoma knowledge and PSL diagnostic accuracy and management) immediately after training. Despite several reminders, only 27 GPs (30%) completed Section C of the questionnaire and returned it by mail 1 year after training.

Section A—Characteristics of the Study Population

The GPs’ characteristics are summarized in Table 1. The overall average age of the 89 participating GPs was 45.6 years (median = 45 years) with a M/F sex ratio of 0.98. One year after training, 40% of the responding GPs were under 35 years of age and 70% of them were female.

Table 1.

Characteristics of the study population

Characteristics of the general practitioners	Pre- and post-training n (%)	At 1 year n (%)
Total	89 (100)	27 (100)
Gender
Female	45 (51)	19 (70)
Male	44 (49)	8 (30)
Age
25–35 years	29 (33)	11 (41)
36–45 years	15 (17)	5 (19)
46–55 years	10 (11)	4 (15)
56–65 years	20 (22)	5 (19)
> 65 years	15 (17)	2 (7)
Workplace
Urban area	41 (46)	11 (41)
Suburban area	29 (33)	8 (30)
Rural area	19 (21)	8 (30)
Work practice
Solo	36 (40)	12 (44)
Pair	20 (22)	9 (33)
Medical group	31 (35)	4 (15)
Local network practice	2 (2)	2 (7)
Number of patients seen per year
< 500 patients	3 (3)	1 (4)
500–1200 patients	9 (10)	3 (11)
1200–2500 patients	21 (24)	8 (30)
2500–4400 patients	34 (40)	10 (37)
> 4400 patients	19 (22)	4 (15)

Section B—Melanoma Diagnostic Practices

The results of this section according to the GPs’ socio-demographic background and practice type were examined and discussed in a previously published scientific paper [6].

Section C—Basic Melanoma Knowledge and PSL Diagnostic Accuracy and Management

The data collected before the training are available in the aforementioned study [6]. Short- and long-term results are the subjects of the present study.

Short-Term Competences

Eighty-nine GPs (45 in group A and 44 in group B) responded to Section C of the questionnaire prior to and immediately after training. Table 2 shows the GPs’ correct responses to the basic melanoma knowledge questions, and Table 3 shows the short-term evolution of the GPs’ PSL diagnostic accuracy and management. Comparison of data from the basic melanoma knowledge questionnaire shows a significant improvement in the immediate post-training outcomes of group B for lentigo maligna management much greater than for group A, and the reverse (a greater improvement in the immediate post-training outcomes for group A than for group B) for the question on characteristics of high-risk melanoma patients. It should also be noted that pre-training results differed between the two groups for melanoma patient follow-up and characteristics of high-risk melanoma patients.

Table 2.

Short-term evolution of the GPs’ melanoma basic knowledge

Group	A GPs = 45					B GPs = 44
Theme addressed by the question	Pre-training		Immediate post-training		P value	Pre-training		Immediate post-training		P value
	n (%)	95% CI	n (%)	95% CI		n (%)	95% CI	n (%)	95% CI
Preferred locations of melanoma according to gender	16 (35.6)	[21.6; 49.5]	44 (97.8)	[93.5; 100.0]	< 0.001*	18 (40.9)	[26.4; 55.4]	44 (100)	[100.0; 100.0]	< 0.001*
Lentigo maligna management	12 (26.7)	[13.7; 39.6]	15 (33.3)	[19.6; 47.1]	0.513	10 (22.7)	[10.3; 35.1]	30 (68.2)	[54.4; 81.9]	< 0.001*
Criteria for urgent management of suspected melanoma	11 (24.4)	[11.9; 37.0]	36 (80.0)	[68.3; 91.7]	< 0.001*	13 (29.5)	[16.1; 43.0]	37 (84.1)	[73.3; 94.9]	< 0.001*
Characteristics of high-risk melanoma patients	13 (28.9)	[15.6; 42.1]	36 (80.0)	[68.3; 91.7]	< 0.001*	18 (40.9)	[26.4; 55.4]	32 (72.7)	[59.6; 85.9]	0.002*
Melanoma patient follow-up	7 (15.6)	[5.0; 26.1]	15 (33.3)	[19.6; 47.1]	0.021*	22 (50.0)	[35.2; 64.8]	27 (61.4)	[47.0; 75.8]	0.132

GPs general practitioners, 95% CI 95% confidence interval. *Statistically significant P values

Table 3.

Short-term evolution of the general practitioners’ appropriate diagnostic accuracy and management

Key: 95% CI 95% confidence interval, GPs general practitioners, SSM superficial spreading melanoma, BCC basal cell carcinoma, UQ unquestioned. *Statistically significant P values

In addition, Table 4 illustrates the number of GPs who appropriately managed $\ge$ 50% of the melanoma cases, i.e., at least one melanoma among the 2 melanomas of the 10 clinical cases in question set A and at least two of the three melanomas (including the lentigo maligna of the ear) presented among the 10 clinical cases in set B. As the number of melanomas differed in the two sets (A and B) of the questionnaire, we chose to use $\ge$ 50% rather than absolute scores to present the results. This number doubled from 19 (42.2; CI =[27.8; 56.7]) GPs to 39 (86.7%; CI =[76.7; 96.6]) GPs in group A (P < 0.001) and increased from 21 (47.7%; CI =[33.0; 62.5]) GPs to 36 (81.8%; CI =[70.4; 93.2]) GPs in group B (P < 0.001). However, only a few benign lesions were better recognized as harmless and not requiring referral to the dermatologist (Table 3).

Table 4.

The evolution of the general practitioners’ appropriately managed melanomas over time

	$\ge$50% appropriately managed melanomas						P value
	Baseline Prior to training		Short-term Immediate post-training		Long-term 1-year post-training		Baseline- Short-term	Short-term- Long-term
	n (%)	95% CI	n (%)	95% CI	n (%)	95% CI
Group A (GPs = 45)	19 (42.2)	[27.8; 56.7]	39 (86.7)	[76.7; 96.6]			< 0.001*
Group B (GPs = 44)	21 (47.7)	[33.0; 62.5]	36 (81.8)	[70.4; 93.2]			< 0.001*
Group A (GPs = 18)	8 (44.4)	[21.5; 67.4]	17 (94.4)	[83.9; 100.0]	9 (50.0)	[26.9; 73.1]		NA (Sample size too small)
Group B (GPs = 9)	4 (44.4)	[12.0; 76.9]	8 (88.9)	[68.4; 100.0]	5 (55.6)	[23.1; 88.0]

GPs general practitioners, 95% CI 95% confidence interval, NA not applicable. *Statistically significant P values

Long-Term Competences

Of the 89 GPs, only 27 (30%) completed the 1-year questionnaire. In total, 18 GPs of group A filled in question set B and question set A was answered by nine GPs of group B. Table 5 shows the long-term evolution of these 27 GPs’ competences in melanoma and PSL diagnosis and management.

Table 5.

Short-term and long-term evolution of 27 general practitioners

Group A Set A—> Set A—> Set B GPs = 18
Melanoma basic knowledge
Theme addressed by the question			Pre-training			Immediate post-training			At 1-year post-training
			n (%)	95% CI		n (%)		95% CI	n (%)	95% CI
Preferred locations of melanoma according to gender			5 (27.8)	[7.1; 48.5]		18 (100.0)		[100.0; 100.0]	13 (72.2)	[51.5; 92.9]
Lentigo maligna management			6 (33.3)	[11.6; 55.1]		4 (22.2)		[3.0; 41.4]	5 (27.8)	[7.1; 48.5]
Criteria for urgent management of suspected melanoma			4 (22.2)	[3.0; 41.4]		15 (83.3)		[66.1; 100.0]	6 (33.3)	[11.6; 55.1]
Characteristics of high-risk melanoma patients			5 (27.8)	[7.1; 48.5]		15 (83.3)		[66.1; 100.0]	6 (33.3)	[11.6; 55.1]
Melanoma patient follow-up			4 (22.2)	[3.0; 41.4]		9 (50.0)		[26.9; 73.1]	14 (77.8)	[58.6; 97.0]
PSL diagnostic accuracy and management
N°	Diagnosis	Pre-training		Immediate post-training			N°	Diagnosis	At 1-year post-training
		n (%)	95% CI	n (%)	95% CI				n (%) 95% CI
A1	Actinic lentigo	5 (27.8)	[7.1; 48.5]	6 (33.3)	[11.6; 55.1]		B10	Lentigo maligna	8 (44.4)		[21.5; 67.4]
A2	SSM with nodular component	2 (11.1)	[0.0; 25.6]	14 (77.8)	[58.6; 97.0]		B2	SSM with nodular component	8 (44.4)		[21.5; 67.4]
A3	Angioma	3 (16.7)	[0.0; 33.9]	9 (50.0)	[26.9; 73.1]		B3	Seborrheic keratosis	4 (22.2)		[3.0; 41.4]
A4	SSM	7 (38.9)	[16.4; 61.4]	14 (77.8)	[58.6; 97.0]		B4	SSM	9 (50.0)		[26.9; 73.1]
A5	Large typical nevus	6 (33.3)	[11.6; 55.1]	9 (50.0)	[26.9; 73.1]		B8	Congenital nevus	1 (5.6)		[0.0; 16.1]
A6	Pigmented BCC	2 (11.1)	[0.0; 25.6]	4 (22.2)	[3.0; 41.4]		B6	Pigmented BCC	3 (16.7)		[0.0; 33.9]
A7	Atypical nevus	2 (11.1)	[0.0; 25.6]	3 (16.7)	[0.0; 33.9]		B5	Atypical nevus	8 (44.4)		[21.5; 67.4]
A8	Plantar nevus	3 (16.7)	[0.0; 33.9]	9 (50.0)	[26.9; 73.1]		B1	Blue nevus	12 (66.7)		[44.9; 88.4]
A9	Subungual hematoma	0 (0.0)	[0.0; 0.0]	9 (50.0)	[26.9; 73.1]		B9	Benign melanonychia	3 (16.7)		[0.0; 33.9]
A10	Spitz nevus	9 (50.0)	[26.9; 73.1]	14 (77.8)	[58.6; 97.0]		B7	Pigmented Spitz (Reed) nevus	12 (66.7)		[44.9; 88.4]
Mean overall score (/10) ± SD		2.2 ± 0.99		5.1 ± 1.76					3.8 ± 1.52
Group B Set B—> Set B—> Set A GPs = 9
Melanoma basic knowledge
Theme addressed by the question			Pre-training			Immediate post-training			At 1-year post-training
			n (%)	95% CI		n (%)		95% CI	n (%)	95% CI
Preferred locations of melanoma according to gender			3 (33.3)	[2.5; 64.1]		9 (100.0)		[100.0; 100.0]	3 (33.3)	[2.5; 64.1]
Lentigo maligna management			3 (33.3)	[2.5; 64.1]		5 (55.6)		[23.1; 88.0]	6 (66.7)	[35.9; 97.5]
Criteria for urgent management of suspected melanoma			4 (44.4)	[12.0; 76.9]		9 (100.0)		[100.0; 100.0]	2 (22.2)	[0.0; 49.4]
Characteristics of high-risk melanoma patients			5 (55.6)	[23.1; 88.0]		6 (66.7)		[35.9; 97.5]	5 (55.6)	[23.1; 88.0]
Melanoma patient follow-up			5 (55.6)	[23.1; 88.0]		6 (66.7)		[35.9; 97.5]	2 (22.2)	[0.0; 49.4]
PSL diagnostic accuracy and management
N°	Diagnosis	Pre-training		Immediate post-training			N°	Diagnosis	At 1-year post-training
		n (%)	95% CI	n (%)	95% CI				n (%)		95% CI
B10	Lentigo maligna	4 (44.4)	[12.0; 76.9]	7 (77.8)	[50.6; 100.0]		A1	Actinic lentigo	2 (22.2)		[0.0; 49.4]
B2	SSM with nodular component	5 (55.6)	[23.1; 88.0]	6 (66.7)	[35.9; 97.5]		A2	SSM with nodular component	2 (22.2)		[0.0; 49.4]
B3	Seborrheic keratosis	2 (22.2)	[0.0; 49.4]	3 (33.3)	[2.5; 64.1]		A3	Angioma	0 (0.0)		[0.0;0.0]
B4	SSM	3 (33.3)	[2.5; 64.1]	9 (100.0)	[100.0; 100.0]		A4	SSM	4 (44.4)		[12.0; 76.9]
B8	Congenital nevus	1 (11.1)	[0.0; 31.6]	4 (44.4)	[12.0; 76.9]		A5	Large typical nevus	6 (66.7)		[35.9; 97.5]
B6	Pigmented BCC	3 (33.3)	[2.5; 64.1]	5 (55.6)	[23.1; 88.0]		A6	Pigmented BCC	0 (0.0)		[0.0; 0.0]
B5	Atypical nevus	3 (33.3)	[2.5; 64.1]	4 (44.4)	[12.0; 76.9]		A7	Atypical nevus	3 (33.3)		[2.5; 64.1]
B1	Blue nevus (hand)	3 (33.3)	[2.5; 64.1]	8 (88.9)	[68.4; 100.0]		A8	Plantar nevus	2 (22.2)		[0.0; 49.4]
B9	Benign melanonychia	2 (22.2)	[0.0; 49.4]	2 (22.2)	[0.0; 49.4]		A9	Subungual hematoma	1 (11.1)		[0.0; 31.6]
B7	Pigmented Spitz (Reed) nevus	7 (77.8)	[50.6; 100.0]	5 (55.6)	[23.1; 88.0]		A10	Spitz nevus	3 (33.3)		[2.5; 64.1]
Mean overall score (/10) ± SD		3.7 ± 2.00		5.9 ± 1.83					2.6 ± 2.24

GPs general practitioners, PSL pigmented skin lesions, N° case number, 95% CI 95% confidence interval, SSM superficial spreading melanoma, BCC basal cell carcinoma, SD standard deviation

Similar clinical cases that address the same training topic in both sets of questions are presented side by side in this table. For example, the GPs’ results for the Spitz nevus, clinical case n°10 in set A, are presented alongside the GPs’ results for the Reed nevus, clinical case n°7 in set B

One year after training, the GPs’ correct responses to the basic melanoma knowledge questionnaire were maintained only for one question and increased for another question in group A. Unfortunately, this was not the case for the last three questions. In group B, the 1-year results were similar or even worse than the pre-training results except for one question. In the second part of the questionnaire, the number of correctly managed clinical cases significantly decreased in the long-term. In group A, the GPs’ mean overall score immediately after training of 5.1 ± 1.76 out of 10 correctly managed clinical cases dropped to 3.8 ± 1.52 at 1-year post-training. For the nine GPs in group B, their immediate post-training score of 5.9 ± 1.83 out of 10 correctly managed cases fell drastically to 2.6 ± 2.24 at 1-year post-training. This score was even lower than their pre-training score of 3.7 ± 2.00. Except for the blue nevus, benign lesions were already relatively unrecognized immediately after training and were still difficult for GPs to diagnose in the long-term. Furthermore, of the 18 GPs in group A, 17 (94.4%; CI =[83.9; 100.0]) appropriately managed $\ge$ 50% melanomas immediately after training and only 9 (50.0%; CI =[26.9; 73.1] GPs did so in the long-term (Table 4). Unfortunately, the same trend was observed in group B with 8 out of 9 GPs (88.9%; CI =[68.4; 100.0]) who properly managed $\ge$ 50% melanoma cases in the short-term, but only 5 (55.6%; CI =[23.1; 88.0]) who did so in the long-term.

Discussion

The aim of our prospective interventional study was to determine the short- and long-term competences of GPs after a 1-h training session on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

Short-Term Competences

Results of the short-term evaluations revealed a significant increase in the immediate post-training assessment on most topics in the melanoma knowledge questionnaire. The greater improvements in the immediate post-training results of group B for lentigo maligna management, and group A for characteristics of high-risk melanoma patients, are most likely due to the non-identical but similar questions on the same training topic presented to both groups and mistakenly considered of equal difficulty by the authors. The difference in the pre-test results between the two groups for the questions on melanoma patient follow-up and characteristics of high-risk melanoma patients is probably also due to more difficult questions in set A than in set B.

The significant increase in the immediate post-training results of the PSL diagnostic accuracy and management are consistent with the main goal of the training session which was to teach melanoma diagnosis and management. However, most benign lesions were not diagnosed as such, and were referred to the dermatologist for further assessment or excision. Therefore, it is likely that the training did not focus sufficiently on the diagnosis of benign lesions and other differential diagnoses of melanoma, such as pigmented basal cell carcinoma. These PSL were still very poorly recognized after training which could result in over-diagnosis of melanoma. While over-diagnosis could lead to psychological stress for the patient, unnecessary referrals to dermatologist, and increased treatment costs [30], it would be in patients’ best interests for GPs not to underdiagnose melanoma, which is an aggressive cancer except when diagnosed at an early stage.

Long-Term Competences

Only 27 GPs (30%) responded to the 1-year questionnaire. Our main finding was that the competences they had acquired immediately after the training were not maintained over time.

Our results are in line with other studies and suggest that the long-term loss of acquired competences after training programs for GPs is common. Figure 1 shows the evolution of the GPs’ mean overall scores in diagnosing and managing skin lesions after nine different training programs in melanoma diagnosis. Overall, these graphs illustrate that GPs’ competences tend to improve for up to 6 months after training. However, beyond this limit, a gradual loss of competences occurs, despite the availability of refresher training material after six of the training sessions. The refresher training material varied between the different programs, and included unlimited e-learning access [31], self-assessment paper-based training sessions [32, 33], a second face-to-face training session at three months post-training [17], access to teledermatology feedback services [34], the provision of PDF-files of the course [35], and a triage algorithm for dermoscopic assessment of pigmented skin lesions [36]. Our study was one of the three training programs that did not provide refresher material [37–39].

Fig. 1.

a Short- and long-term evolution of general practitioners’ (GPs’) mean overall score in diagnosing and managing skin lesions reported in nine studies on training in melanoma diagnosis. b Distribution of the corresponding learning-curve slopes ($\mathrm{\Delta }\mathbf{S}\mathbf{c}\mathbf{o}\mathbf{r}\mathbf{e}/\mathrm{\Delta }\mathbf{T}\mathbf{i}\mathbf{m}\mathbf{e})$ over time. The graphs make no distinction between sessions that trained GPs in naked-eye recognition of skin lesions and/or diagnosis using dermoscopy. However, we only included studies that specified the short- and long-term scores of the GPs’ competences in the period after the educational program. The learning-curve slope evaluates the improvement or deterioration in the GPs’ competences over a period of time (expressed in months). Figure 1b shows that the competences tend to improve when measured up to 6 months after training, while competences measured at 6 months and beyond tend to decrease. In addition, the availability of refresher training material had no impact on the sustainability of the GPs’ competences over the long-term. To note that Badertscher et al. [33] and Augustsson et al. [34] used the median score instead of the mean test score, which could be a limitation in the comparison of these studies since the median and the mean are not always equal

There are several possible reasons for this long-term decline in the GPs’ competences, such as lack of daily opportunities to use the skills and knowledge obtained during training, lack of time in clinical practice to evaluate skin lesions, lack of confidence in their diagnostic abilities, the absence of prompt dermatologist’s feedback on the lesions assessed, and a gradual decline in the GPs’ motivation for self-training. Similarly, the low response rate (30%) observed in our study 1 year after training may also reflect a lack of time and loss of motivation for training in melanoma diagnosis. The low response rate of GPs may also possibly reveal a lack of confidence in their ability to assess skin lesions and fear of being judged for not having maintained the acquired skills.

Robinson et al. [36] and Marra et al. [24] proposed that refresher training might help physicians retain accuracy in melanoma diagnosis. However, they did not specify the type and form of these refresher activities. However, as demonstrated in Fig. 1, the long-term competences decreased in all studies, both in the training programs with additional refresher material and in those without it. We speculate that this decrease may be related to a progressive decline in the motivation and interest of GPs in skills they do not regularly use. In this respect, Marra et al. [24] encouraged the organization of “face-to-face” refresher training sessions at regular intervals in order to maintain appropriate management of skin lesions. This idea could be extended to interactive e-learning sessions, which have increasingly widespread since the COVID-19 pandemic in 2020. Finally, the ideal frequency of such refresher training sessions should be established. Figure 1 shows that the threshold between improvement and deterioration in competences was 6 months after training. This suggests that refresher sessions should take place at a regular frequency of at least once every 6 months.

In view of all these findings, we would recommend organizing some short interactive e-learning refresher sessions every three to 6 months after a training session. These sessions could consist of “real-life doctor-patient scenarios and problem solving” and should also include some valuable take-home messages that are easy to apply in the doctor’s office. As GPs have little time to devote to training, these sessions should not exceed 15 to 20 min. We are convinced that these refresher sessions would motivate GPs to maintain an interest in the field and encourage them to apply the competences they have acquired more regularly in their clinical practice.

Strengths and Limitations

Due to its geographic distribution in the five French-speaking districts of Belgium, the distribution of ages and gender, and the proportion of urban and rural GPs, the study population was representative of Belgium’s population of French-speaking GPs. As this study was conducted in the context of the GPs’ mandatory continuing medical education program, it is reasonable to assume that their inclusion was not significantly determined by their initial interest in melanoma diagnosis. Although the 10 clinical cases and five multiple-choice questions used in the two evaluation sets (A and B) were similar but not identical, we were able to avoid recall bias at 1-year post-training by switching the two sets of questionnaires between groups.

This study also has some limitations. Firstly, the 1-year questionnaire was sent to the GPs by return mail, which resulted in a significant drop-out of the participating GPs. Since only two of the four annual continuing medical education courses are mandatory, conducting the 12-month evaluation at one of these meetings, 1 year after the training, would have statistically reached 50% of the participants. In this context, we chose to send the questionnaire by mail in the hope of reaching more GPs. Unfortunately, this was not the case. As only 27 GPs completed the questionnaire, we could not test any significant inference on the outcomes of the three evaluation periods. Only descriptive statistics were obtained.

Secondly, the two groups of GPs who evaluated question sets A and B were constituted on the basis of their membership in the participating continuing medical education groups. Therefore, we cannot exclude socio-demographic differences between groups that could have influenced the pre- and post-training results. In addition, the process of switching the two evaluation sets (A and B) between groups made us realize that the questions in set A were more difficult than those in set B. This may also have influenced the results of the two groups.

Finally, this study assessed the GPs’ diagnostic and management skills solely in a training setting. When measured on photographs of skin lesions, the competences do not always reflect the GPs’ performance (changes in practice pattern) in their daily clinical practice. To further determine whether the measured competences reflect the GPs’ performance in daily practice, it would have been necessary to assess the number of melanomas the participants had diagnosed or referred to dermatology during the 12-month period between the two assessments.

Conclusion

This study measured the accuracy of GPs in the diagnose and management of PSL in a training setting after a 1-h training session on naked-eye melanoma diagnosis. Although the GPs’ competences improved in the short-term, they significantly decreased 1 year later. This is in line with other studies which showed that the skills acquired in training improved for up to 6 months, before rapidly regressing thereafter. To maintain and improve the GPs’ competences over a longer period, an essential aid may be the organization of timely refresher training sessions at regular intervals. However, further studies are needed to assess whether these refresher sessions are able to prevent the loss of the diagnostic and management skills that GPs acquire during trainings.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contribution

E. Harkemanne: writing of original draft, visualization, and conceptualization. C. Duyver and S. Leconte: investigation, resources, and methodology. K. Sawadogo: formal analysis and validation. M. Baeck: reviewing and editing. I. Tromme: investigation, methodology, reviewing and editing, and supervision.

Data Availability

The data collected and the material used to carry out this study are available upon request from the corresponding author.

Declarations

Ethics Approval

The study was approved by the Ethical Committee of the Catholic University of Louvain in Brussels, Belgium.

Consent to Participate

All participating general practitioners gave their consent to the use of their data collected for the purpose of this study. The clinical images of skin lesions shown to the participants during the training and used for evaluation of their competences were photographs from the database of images of patients from the Dermatology department of the Cliniques universitaires Saint-Luc, Brussels (Belgium). The patients of the Dermatology department gave their written personal consent to the use of their anonymous images for education and clinical studies.

Consent for Publication

All participating general practitioners and patients of the Dermatology department gave their consent for publication of their anonymized data.

Competing Interests

The authors declare that they have no conflict of interest.