Abstract
Objectives To assess whether adding a novel computerised diagnostic tool, the MoleMate system (SIAscopy with primary care scoring algorithm), to current best practice results in more appropriate referrals of suspicious pigmented lesions to secondary care, and to assess its impact on clinicians and patients.
Design Randomised controlled trial.
Setting 15 general practices in eastern England.
Participants 1297 adults with pigmented skin lesions not immediately diagnosed as benign.
Interventions Patients were assessed by trained primary care clinicians using best practice (clinical history, naked eye examination, seven point checklist) either alone (control group) or with the MoleMate system (intervention group).
Main outcome measures Appropriateness of referral, defined as the proportion of referred lesions that were biopsied or monitored. Secondary outcomes related to the clinicians (diagnostic performance, confidence, learning effects) and patients (satisfaction, anxiety). Economic evaluation, diagnostic performance of the seven point checklist, and five year follow-up of melanoma incidence were also secondary outcomes and will be reported later.
Results 1297 participants with 1580 lesions were randomised: 643 participants with 788 lesions to the intervention group and 654 participants with 792 lesions to the control group. The appropriateness of referral did not differ significantly between the intervention or control groups: 56.8% (130/229) v 64.5% (111/172); difference −8.1% (95% confidence interval −18.0% to 1.8%). The proportion of benign lesions appropriately managed in primary care did not differ (intervention 99.6% v control 99.2%, P=0.46), neither did the percentage agreement with an expert decision to biopsy or monitor (intervention 98.5% v control 95.7%, P=0.26). The percentage agreement with expert assessment that the lesion was benign was significantly lower with MoleMate (intervention 84.4% v control 90.6%, P<0.001), and a higher proportion of lesions were referred (intervention 29.8% v control 22.4%, P=0.001). Thirty six histologically confirmed melanomas were diagnosed: 18/18 were appropriately referred in the intervention group and 17/18 in the control group. Clinicians in both groups were confident, and there was no evidence of learning effects, and therefore contamination, between groups. Patients in the intervention group ranked their consultations higher for thoroughness and reassuring care, although anxiety scores were similar between the groups.
Conclusions We found no evidence that the MoleMate system improved appropriateness of referral. The systematic application of best practice guidelines alone was more accurate than the MoleMate system, and both performed better than reports of current practice. Therefore the systematic application of best practice guidelines (including the seven point checklist) should be the paradigm for management of suspicious skin lesions in primary care.
Trial registration Current Controlled Trials ISRCTN79932379.
Introduction
Differentiating melanomas from other pigmented skin lesions in primary care is challenging.1 Worldwide the incidence of melanoma is increasing faster than any other cancer, with an approximate doubling of rates every 10-20 years in countries with white populations.2 In the United Kingdom the incidence of melanoma has quadrupled over the past 40 years; data from Cancer Research UK for 2008 reported 11 770 new cases and 2070 deaths. Early detection is critical in reducing mortality and morbidity from melanoma, as stage 1 disease has five year survival rates of over 95% compared with 10-20% for stage 4 disease.3 As pigmented lesions are commonly presented in primary care consultations, general practitioners need to be able to reassure people with benign lesions and rapidly refer those with suspicious lesions. In 2003 the UK All-Party Parliamentary Group on Skin reported that 95% of lesions referred to a UK specialist were benign4; furthermore, a recent UK study showed that general practitioners recognised only 66.7% of skin malignancies.5 This difficulty in distinguishing benign from potentially malignant lesions is consistent with international evidence that general practitioners can be as sensitive but less specific than dermatologists at diagnosing melanoma.6 The appropriate referral of patients to secondary care has important clinical, safety, quality, and economic ramifications, not only for general practitioners working in the UK’s “gatekeeper” system but also globally.7 Therefore, novel interventions to improve the accuracy of identification of suspicious pigmented lesions have a potentially important role, especially in primary care, where such management initially occurs.
Interventions to improve general practitioners’ diagnostic performance and efficiency of referral have included the use of checklists and educational and technical approaches. The seven point checklist8 has been widely evaluated and revised,9 and although it has never been tested in a primary care trial, was recommended for use by all primary care professionals in the assessment of pigmented skin lesions by the 2005 English National Institute for Health and Clinical Excellence (NICE) guidelines on referral for suspected cancer.10 Evidence about the usefulness of brief educational approaches such as face to face11 and internet12 training courses is equivocal. Technical approaches have included the use of dermoscopy13 and digital monitoring; a recent Australian study in primary care found that the combination of these techniques could increase general practitioners’ sensitivity for the diagnosis of melanoma and thus significantly reduce the proportion of benign lesions excised, but learning these techniques took considerable time and was completed by only 62% of the trial doctors.14
The MoleMate system (see acknowledgments for manufacturer details) is a computerised diagnostic tool that applies the innovative technology of Spectrophotometric Intracutaneous Analysis (SIAscopy). It is easy and quick to learn and use.15 This non-invasive scanning technique uses light reflected from the skin in the visible and infrared spectra to produce images of the epidermal and dermal melanin and vasculature and the collagen content of the papillary dermis within the lesion. Patterns within these images indicate histopathological features consistent with melanoma and are highly predictive of melanoma in the experimental setting16 and in secondary care when applied in a scoring algorithm.17 The original secondary care algorithm was refined for primary care use to account for the higher prevalence of seborrhoeic keratoses and haemangiomas seen in this setting, and the increasing prevalence of these lesions with age (fig 1).18 The primary care scoring algorithm was integrated with a hand held SIAscopy scanner to create the MoleMate system (fig 2). We determined whether use of the MoleMate system in UK general practice would result in more appropriate referrals of suspicious pigmented lesions to secondary care than current best practice. We hypothesised that the system would increase the proportion of appropriate referrals without increasing the total number of referrals.
Fig 1 Primary care scoring algorithm17
Fig 2 Screenshot of MoleMate system. MedX Health (http://simsys-molemate.com/simsys-molemate/siascopy/)
Methods
We carried out a prospective, randomised open trial with pragmatic ascertainment of an endpoint, in 15 general practices in eastern England. The protocol for the MoleMate UK Trial has been published elsewhere.19
Participants
All general practice team members identified potential participants. Adults were eligible for enrolment if they were aged 18 or over and had a suspicious pigmented lesion. For the purpose of the trial the definition of a suspicious pigmented lesion was any lesion presented by a patient, or opportunistically seen by a family doctor or practice nurse, that could not immediately be diagnosed as benign and about which the patient could not be reassured. We excluded patients who were unable to give informed consent or were considered inappropriate to include by their family doctor. Potentially eligible patients were internally referred within the general practice to attend a trial appointment within one week.
Procedures
Lead clinicians
In each practice we trained two lead clinicians (a total of 28 general practitioners and two nurse practitioners) in consent procedures for the trial, data collection, and best practice assessment. They also learned to use and interpret the MoleMate system by completing a two hour training CD-ROM to identify relevant SIAscopic features of various pigmented skin lesions; this has been shown to significantly improve the ability of general practitioners to interpret SIAscopic images.15 At the trial appointment the lead clinician confirmed eligibility of the patients and obtained consent. We did not recruit practices already using a MoleMate system, and we excluded general practitioners with known dermatological expertise from being lead clinicians—that is, current hospital practitioners, clinical assistants in dermatology, and general practitioners with a specialist interest in dermatology.
Randomisation
The lead clinician randomised participants to either the best practice (control) group or the MoleMate (intervention) group on the basis of a block randomisation method, using computer generated, randomly permuted blocks of size 2, 4, and 6, established by the trial statistician (ATP). Sets of numbered, sealed envelopes were prepared, with the order of the sequences verified at completion of the trial (ATP). Randomisation was stratified by lead clinician and patient’s age (≤45 years, ≥46 years) to account for a potential differential effect on referral owing to the inclusion of age in the intervention system’s primary care scoring algorithm.
Best practice (control) group
Lesions of participants allocated to the control group were clinically assessed according to the Cambridge University Hospitals NHS Foundation Trust guidelines. This included taking a clinical history and naked eye examination.8 9
MoleMate (intervention) group
Lesions of participants allocated to the intervention group were clinically assessed as for the control group. Lead clinicians then used the MoleMate system to support their assessment and management of the lesion.
For all lesions, lead clinicians then decided whether to refer patients through the fast track skin cancer pathway or to manage them in primary care.
Reference standards
We recorded a reference standard final diagnosis for all lesions in the trial. For referred lesions we defined this as expert opinion by a histologist or dermatologist, and for non-referred lesions as review by two other dermatology experts of the recorded clinical history and examination, a digital photograph, and MoleMate image where available. For lesions where there was uncertainty based on this initial review (for example, poor quality digital photograph), the lead clinician collected these data on a second occasion 3-6 months later. All participants with non-referred lesions were also offered a follow-up consultation with the lead clinician 3-6 months later to collect these data, including a second photograph, for review by the dermatology experts to identify change over time and to confirm a benign diagnosis.20
Outcomes
The primary outcome was the appropriateness of referral, defined as the proportion of referred lesions that secondary care experts decided to biopsy or monitor; it was a measure of the diagnostic accuracy of the general practitioner with or without the aid of the MoleMate system.
Secondary outcomes included those related to the clinicians (diagnostic performance, confidence, learning effects) and patients (satisfaction, anxiety). We assessed the diagnostic performance of the lead clinician—namely, the proportion of benign lesions appropriately managed in primary care, the percentage agreement with the expert decision to biopsy or monitor (sensitivity), and the percentage agreement with the expert assessment that the lesion was benign (specificity)—using data from all lesions in the trial (histology result or expert clinical diagnosis). We assessed the confidence and attitudes of the lead clinician towards the intervention two weeks after the trial was set-up and at trial completion using a modified measure (1-7 scale) based on the theory of planned behaviour.21 We measured potential contamination in the comparison group due to the learning effects of the lead clinician by comparing differences in the appropriateness and volume of referrals between groups for the first 10 intervention consultations of each lead clinician’s data collection, when contamination would be minimal (naive period), with these differences for the remaining consultations (potentially contaminated period).
We measured patients’ satisfaction using dimensions of care items from EUROPEP (a 23 item validated and internationally standardised measure of patients’ evaluations of care in general practice).22 Patients’ anxiety was measured by questionnaire, including the Spielberger state trait anxiety inventory23 and a modified cancer worry scale,24 completed within one week and at three months after the consultation with the lead clinician.
Other secondary outcomes
Other secondary outcomes in the trial protocol included: examination of the association between the index of suspicion scale and the seven point checklist, and their predictiveness of lesion outcomes (the index of suspicion scale was not found to have clinical utility and so was omitted from analysis, whereas the predictiveness of lesion outcome using the seven point checklist will be reported separately); economic evaluation (this will be reported separately as it used the denominator of person rather than lesion); and five year follow-up of melanoma incidence, which is being undertaken in collaboration with the Eastern Cancer Registration and Information Centre and will report in due course.
Statistical analysis
We reviewed sample size estimates at a planned, blinded interim analysis. This confirmed a recruitment rate of 6-8 participants for each practice each month, and a lesion referral rate of 28%. We also confirmed a mean cluster size among the referred sample of 1.06, and that the dermatology experts chose to obtain a biopsy or to monitor 55% of the referred lesions in the combined arm data.19 The primary and secondary care clinicians (general practitioners, dermatologists, plastic surgeon) agreed by consensus that a 15% reduction in referrals to the fast track skin cancer clinic would be the minimum clinically important difference to justify such a significant change in practice. Therefore a sample size of 400 referred lesions from about 380 patients, with overall 1450 study lesions from 1150 randomised participants, would be required to detect an increase of 15% in the rate of biopsy or monitoring, with 80% power at the 5% level of significance.
The intention to treat population comprised all lesions from all randomised patients. Analysis of the primary outcome was based on all lesions referred to secondary care. We compared the proportion of referred lesions that were biopsied or monitored between randomised groups principally using Donner’s test for clustered proportions through a linear mixed effects model (using R with the nlme package) with patient as a random term. To examine whether the results were sensitive to clustering of lesions within patients, we additionally reported a planned secondary analysis that was unadjusted for clustering using the χ2 test. The same approach was used to compare the proportion of benign lesions appropriately managed in primary care, the percentage agreement with the expert decision to biopsy or monitor, and the percentage agreement with the expert assessment that the lesion was benign, and the volume of referrals (defined as the proportion of lesions from randomised participants that were referred to secondary care). For the principal analysis (adjusted for clustering), multi-lesion clusters were insufficient to provide a comparison of the percentage agreement with the expert decision to biopsy or monitor.
As the proportion of benign lesions appropriately managed in primary care and the percentage agreement with the expert decision to biopsy or monitor had extreme proportions with large denominators, for the clinician diagnostic performance analysis we used the exact confidence interval and corresponding P value from Wilson’s score method.25 We report the analyses unadjusted for clustering because, compared with the cluster adjusted results, the estimated differences in proportions are the same, with negligibly wider and therefore conservative confidence intervals and with the same conclusions. In each randomised group we summarised the histological and expert clinical diagnosis of melanoma as a rate.
We used a mean score and 95% confidence interval to summarise confidence and attitudes towards the MoleMate system. Assessment for contamination through learning effects was undertaken by testing for a difference in the intervention effect for the primary outcome across naive and potentially contaminated periods. Using a t test we compared patients’ measures (anxiety, worry about cancer, satisfaction with the consultation): analyses were unadjusted cross sectional comparisons between groups. Change over time was summarised, reporting means at baseline and mean changes between time points. We carried out analyses using SPSS version 17 and R version 2.12.0.
Results
Participants were recruited from March 2008 to May 2010 and data collection was completed in September 2010. Overall, 1580 lesions on 1297 participants from 15 general practices (range 45-151 patients per practice) were randomly assigned to study groups (fig 3). Four participants (total seven lesions) withdrew from the control group after randomisation. With the exception of these four participants, all other randomised patients were included in the intention to treat analyses for the primary outcome. The groups were well matched on baseline characteristics (table 1).
Fig 3 Flow of participants through study
Table 1.
Baseline characteristics of 1293 participants with suspicious pigmented skin lesions allocated to best practice (control) group or MoleMate (intervention) group.* Values are numbers (percentages) of participants unless stated otherwise
| Characteristics | Control group (n=650) | Intervention group (n=643) | Total (n=1293) |
|---|---|---|---|
| Mean (SD) age (years) | 44.8 (16.9) | 44.5 (16.7) | 44.6 (16.8) |
| Age group: | |||
| ≤45 | 358 (55.1) | 355 (55.2) | 713 (55.1) |
| ≥46 | 292 (44.9) | 288 (44.8) | 580 (44.9) |
| Men | 235 (36.2) | 230 (35.8) | 465 (36.0) |
| Women | 415 (63.8) | 413 (64.2) | 828 (64.0) |
| Ethnicity: | |||
| White† | 608 (93.5) | 606 (94.2) | 1214 (93.9) |
| Mixed, Asian, Black, Chinese, other | 25 (3.9) | 20 (3.2) | 45 (3.5) |
| Missing | 17 (2.6) | 17 (2.6) | 34 (2.6) |
| No of lesions assessed: | |||
| 1 | 535 (82.3) | 516 (80.2) | 1051 (81.3) |
| 2 | 97 (14.9) | 109 (17.0) | 206 (15.9) |
| 3 | 16 (2.5) | 18 (2.8) | 34 (2.6) |
| 4 | 2 (0.3) | 0 (0.0) | 2 (0.2) |
| Occupation: | |||
| Employed | 289 (44.5) | 292 (45.4) | 581 (44.9) |
| Looking after home or family | 29 (4.5) | 30 (4.7) | 59 (4.6) |
| Unemployed | 13 (2.0) | 7 (1.1) | 20 (1.5) |
| Retired | 91 (14.0) | 85 (13.2) | 176 (13.6) |
| Full time education | 27 (4.2) | 31 (4.8) | 58 (4.5) |
| Long term sickness | 7 (1.1) | 6 (0.9) | 13 (1.0) |
| Other | 7 (1.1) | 11 (1.7) | 18 (1.4) |
| Missing | 187 (28.8) | 181 (28.1) | 368 (28.5) |
| Education: | |||
| No qualifications | 47 (7.2) | 46 (7.2) | 93 (7.2) |
| GCSE or similar | 64 (9.8) | 76 (11.8) | 140 (10.8) |
| GCE A level or similar | 56 (8.6) | 54 (8.4) | 110 (8.5) |
| Higher education or similar | 58 (8.9) | 75 (11.7) | 133 (10.3) |
| Degree or similar | 249 (38.3) | 224 (34.8) | 473 (36.6) |
| Missing | 176 (27.1) | 168 (26.1) | 344 (26.6) |
*Four participants withdrew after randomisation.
†All melanomas diagnosed in white population.
Primary outcome: appropriateness of referral
The appropriateness of referrals did not differ significantly between the intervention and control groups (56.8% v 64.5%): absolute difference −8.1% (95% confidence interval −18.0% to 1.8%); P=0.12 (table 2). The results from the per protocol and intention to treat analysis were similar, and evidence was lacking of a difference in the effect of the intervention between the two age subgroups.
Table 2.
Appropriateness of referrals and clinician’s diagnostic performance in best practice (control) and MoleMate (intervention) groups. Values are percentages (number/number in group)
| Variables | Control group | Intervention group | % difference (95% CI) | P value |
|---|---|---|---|---|
| No of lesions assessed | 785 | 788 | ||
| % appropriate referral rate* | 64.5 (111/172) | 56.8 (130/229) | −8.1 (−18.0 to 1.8) | 0.11 |
| % appropriately managed in primary care† | 99.2 (588/593) | 99.6 (535/537) | 0.5 (−0·6 to 2·0) | 0.46 |
| % agreement with expert decision to biopsy or monitor (sensitivity)† | 95.7 (111/116) | 98.5 (130/132) | 2.8 (−1.8 to 7.4) | 0.26 |
| % agreement with expert assessment that lesion benign (specificity)† | 90.6 (588/649) | 84.4 (535/634) | −6.2 (−9·9 to −2·6) | <0.001 |
| Volume referred† | 22.4 (176/785) | 29.8 (235/788) | 7.4 (3·1 to 11·7) | 0.001 |
*Difference adjusted for clustering of lesions within patients; difference unadjusted for clustering is −7.8% (95% confidence interval −17.4% to 1.8%, P=0.12).
†Unadjusted for clustering of lesions within patients.
Of the 1573 lesions analysed in the control and intervention groups, 411 were referred (176 and 235, respectively) and 1162 were not referred (609 and 553). Table 3 shows the reference standard diagnoses of the referred lesions from both groups. Of the 241 lesions considered appropriately referred by the experts, 215 were biopsied and 22 were monitored. For the lesions that were neither biopsied nor monitored, and therefore were considered not appropriately referred, the reference standard was based on the expert clinical diagnosis.
Table 3.
Diagnoses of 411 referred lesions: 176 in best practice (control) group and 235 in MoleMate (intervention) group*
| Reference standard diagnosis | Appropriately referred | Not appropriately referred | |||||
|---|---|---|---|---|---|---|---|
| Control group | Intervention group | Total | Control group | Intervention group | Total | ||
| Total | 111 | 130 | 241 | 61 | 99 | 160 | |
| Method of diagnosis†: | |||||||
| Histology | 100 | 115 | 215 | — | — | — | |
| Monitored | 9 | 13 | 22 | — | — | — | |
| Expert | — | — | — | 61 | 99 | 160 | |
| Melanoma‡: | 17 | 18 | 35 | ||||
| Superficial spreading | 12 | 11 | 23 | — | — | — | |
| Nodular | 0 | 2 | 2 | — | — | — | |
| Lentigo maligna melanoma | 3 | 1 | 4 | — | — | — | |
| In situ§ or lentigo maligna | 2 | 4 | 6 | — | — | — | |
| Other skin cancers: | 1 | 5 | 6 | ||||
| Squamous cell carcinoma | 0 | 1 | 1 | — | — | — | |
| Basal cell carcinoma | 1 | 4 | 5 | — | — | — | |
| Other lesions: | 82 | 92 | 174 | 61 | 99 | 160 | |
| Dysplastic (atypical) naevus | 26 | 31 | 57 | 2 | 6 | 8 | |
| Seborrhoeic keratosis | 8 | 14 | 22 | 12 | 31 | 43 | |
| Dermatofibroma and haemangioma | 0 | 3 | 3 | 2 | 6 | 8 | |
| Lentigo | 5 | 4 | 9 | 0 | 1 | 1 | |
| Other benign¶ | 43 | 40 | 83 | 45 | 55 | 100 | |
*Participants totalling 10 referred lesions did not attend for dermatology assessment (four in control group, six in intervention group).
†Missing histology (two in control group, two in intervention group).
‡Invasive and pre-invasive.
§Clark level 1.
¶For example, benign naevus.
Thirty six histologically confirmed melanomas were diagnosed during the trial (tables 3 and 4). More than half were less than 1 mm thick with a good prognosis, and none were subsequently found to have metastases or lymph node involvement (control group: nodular melanoma 0, superficial spreading melanoma 13, lentigo maligna melanoma 3, melanoma in situ/lentigo maligna 2; intervention group: nodular melanoma 2, superficial spreading melanoma 11, lentigo maligna melanoma 1, melanoma in situ/lentigo maligna 4). Of the 18 melanomas diagnosed in the control group, 17 were considered suspicious by the lead clinicians and the patients appropriately referred: the non-referred lesion, identified by the two dermatology experts who reviewed all images in the trial, was a superficial spreading melanoma (Breslow thickness 1.2 mm). All of the 18 melanomas diagnosed in the intervention group were considered suspicious and the patients appropriately referred. In the referred participants six non-melanoma skin cancers were detected: one basal cell carcinoma in the control group and four basal cell carcinomas and one squamous cell carcinoma in the intervention group.
Table 4.
Diagnoses of lesions in 1162 non-referred patients: 609 in best practice (control) group and 553 in MoleMate (intervention) group*
| Diagnoses | Control group | Intervention group | Total |
|---|---|---|---|
| Total | 603 | 539 | 1142 |
| Benign | 588 | 535 | 1123 |
| Outcome unknown† | 10 | 2 | 12 |
| Dermatology expert review‡: | 5 | 2 | 7 |
| Monitor | 1 | 0 | — |
| Melanoma | 1 | 0 | — |
| Superficial spreading melanoma | 1 | — | |
| Other skin cancers | 3 | 2 | — |
| Basal cell carcinoma | 3 | 2 | — |
*19 non-referred lesions were excluded from follow-up due to violation of recruitment criteria or discontinued protocol (five in control group, 14 in intervention group), and one patient with one lesion died (control group).
†Did not attend lead clinician review at 3-6 months when uncertain reference diagnosis based on data from first lead clinician consultation.
‡Following lead clinician review at three months.
Among the referred but benign lesions the control and intervention groups did not differ significantly for diagnosis of seborrhoeic keratoses (histology: 7% (8/111) v 11% (14/130), P=0.34; expert opinion: 20% (12/61) v 31% (31/99), P=0.11) and dysplastic naevi (histology: 23% (26/111) v 24% (31/130), P=0.94; expert opinion: 3% (2/61) v 6% (6/99), P=0.71; table 3). Most of the non-referred lesions were considered at expert review to have been appropriately assessed (table 4). Reference diagnosis procedures for non-referred lesions identified six skin cancers. Both lesions in the intervention group were basal cell carcinomas, whereas the control group yielded one melanoma and three basal cell carcinomas.
Clinician outcomes
The median age of the lead clinicians (28 general practitioners, two nurse practitioners; 16 men) was 44 (range 35-56) years, and they had a median 15 (4-27) years’ experience. Seven had undergone some training in dermatology (three had a short dermatology training post, three were on clinical attachment to an out-patient clinic, and one was unspecified).
Diagnostic performance
Addition of the MoleMate system made no significant difference to the proportion of benign lesions appropriately managed in primary care or the percentage agreement with the expert decision to biopsy or monitor (table 2). The intervention reduced the percentage agreement with the expert assessment that the lesion was benign, and so resulted in a higher proportion of lesions referred.
Clinicians’ confidence
At completion of the trial the lead clinicians were confident and positive about using the MoleMate system. Compared with at the start of the trial at trial completion more clinicians thought that using MoleMate during a consultation would be simple (mean 4.8 (SD 1.56) v mean 5.9 (SD 0.65), P=0.001) and cost effective (mean 4.9 (SD 1.51) v mean 5.7 (SD 1.02), P=0.08). Furthermore, they thought that using MoleMate would be less likely to increase patients’ anxiety (mean 3.1 (SD 1.53) v mean 2.4 (SD 1.22), P=0.07) or to prolong consultations (mean 6.2 (SD 0·86) v mean 5.2 (SD 1.22), P<0.001). The clinicians agreed that MoleMate had been easy to use (median 6, interquartile range 5-6; Likert type scale 1=disagree strongly to 7=agree strongly) and fast enough to use (median 6, 5-6) in a consultation, and that the use of the seven point checklist (median 5, 4-6) or MoleMate (median 5, 4-6) had “often helped their decision about the referral.”
Clinician learning effects
There was no evidence of any learning effects and therefore contamination between the groups. The initial naive period and the remaining potentially contaminated period did not differ significantly for either the volume of referrals (naive period: control group 25.9% (85/328) v intervention group 29.9% (109/364), difference 4.0%; potentially contaminated period: control group 19.9% (91/457) v intervention group 29.7% (126/424), difference 9.8%; difference between periods 5.8% (95% confidence interval −3.0% to 14.5%), P=0.20) or the appropriateness of the referral (naive period: control group 66.7% (56/84) v intervention group 57.5% (61/106), difference −9.1%; potentially contaminated period: control group 62.5% (55/88) v intervention group 56.1% (69/123), difference −6.4; difference between periods 2.7% (−16.5% to 21.9%), P=0.78).
Patient outcomes
Patients’ satisfaction
The response rate to the patient questionnaire, completed within one week of the consultation with the lead clinician, was 74% (968/1293), and three months later was 80% (904/1135). The scores for patients’ satisfaction (five responses: poor, fair, good, very good, excellent) showed significant differences between the groups across all 12 items, although only a few responded to each item as poor or fair. Compared with the control group, more patients in the intervention group ranked their consultation as very good or excellent for thoroughness (control group 71.2% (n=475) v intervention group 83.1% (n=485), P<0.001), communication (control group 70.6% (n=479) v intervention group 82.1% (n=485), P<0.001), and reassuring care (control group 66.4% (n=474) v intervention group 77.2% (n=481), P<0.001).
Patients’ anxiety
General and skin cancer specific anxiety scores did not differ between groups immediately after the consultation or over time. Immediately after the consultation patients with non-referred lesions in the intervention group had lower mean general anxiety compared with people in the control group: 32.56 (SD 0.58) v 34.72 (SD 0.64), P=0.013; table 5.
Table 5.
General and skin cancer specific anxiety within one week of trial consultation and three month follow-up. Values are mean (SD) responses
| Variables | Within 1 week of trial consultation | 3 month follow-up | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Control group | Intervention group | Difference (95% CI) | P value | Control group | Intervention group | Difference (95% CI) | P value | ||
| Speilberger state trait anxiety inventory*: | |||||||||
| All | 34.91 (0.56) n=459 | 33.77 (0.50) n=472 | −1.14 (−2.62 to 0.34) | 0.13 | 34.59 (0.51) n=452 | 33.96 (0.52) n=450 | −0.63 (−2.07 to 0.83) | 0.39 | |
| Referred | 35.45 (1.15) n=118 | 36.18 (0.93) n=158 | 0.73 (−2.16 to 3.62) | 0.62 | 33.61 (0.98) n=118 | 33.85 (0.92) n=154 | 0.24 (−2.43 to 2.90) | 0.86 | |
| Non-referred | 34.72 (0.64) n=341 | 32.56 (0.58) n=314 | −2.16 (−3.87 to 0.46) | 0.013 | 34.94 (0.60) n=334 | 34.02 (0.63) n=296 | −0.92 (−2.64 to 0·80) | 0.29 | |
| Referred minus non-referred | 0.73 (1.28) | 3.62 (1.05) | 2.89 (−0.34 to 6.13) | 0.08 | −1.32 (1.17) | −0.17 (1.10) | 1.16 (−1.99 to 4.31) | 0.47 | |
| Skin cancer worry scale†: | |||||||||
| All | 10.06 (0.13) n=471 | 10.25 (0.15) n=478 | 0.19 (−0.20 to 0.58) | 0.34 | 9.45 (0.14) n=449 | 9.34 (0.14) n=458 | −0.12 (−0.51 to 0.27) | 0.56 | |
| Referred | 9.90 (0.25) n=119 | 10.30 (0.27) n=161 | 0.41 (−0.35 to 1.16) | 0.29 | 9.68 (0.34) n=115 | 9.53 (0.27) n=156 | −0.16 (−1.01 to 0·68) | 0.71 | |
| Non-referred | 10.12 (0.16) n=352 | 10.22 (0.17) n=317 | 0.11 (−0.35 to 0.56) | 0.56 | 9.37 (0.15) n=334 | 9.24 (0.16) n=302 | −0.13 (−0.56 to 0.30) | 0.55 | |
| Referred minus non-referred | −0.22 (0.30) | 0.08 (0.31) | 0.30 (−0.56 to 1.15) | 0.50 | 0.31 (0.33) | 0.28 (0.29) | −0.03 (−0.89 to 0.83) | 0.95 | |
*Six item response (1-4 Likert type responses); short form scaled up to 20 item range of 20-80.
†Six items; score 6-24.
Discussion
The MoleMate UK Trial assessed whether this novel computerised diagnostic tool, comprising a primary care scoring algorithm integrated with SIAscopy, could improve management of suspicious pigmented lesions in primary care; results in both study groups showed strong agreement with expert assessment of lesions. Adding the MoleMate system to best practice did not improve the appropriateness of referral; in fact, it resulted in lower agreement with expert assessment that the lesion was benign, resulting in a higher proportion of referrals overall. No melanomas were missed in the MoleMate group and only one was missed in the best practice group. Nevertheless, lead clinicians were confident that the MoleMate system enhanced their practice, and patients ranked satisfaction with consultations higher with the MoleMate system than with best practice alone, and were not anxious by the addition of this new diagnostic aid to best practice. By being perceived more positively, the novel technology provided false reassurance, as the systematic application of best practice guidelines ultimately proved more accurate.
Strengths and weaknesses of the study
An important strength of this rigorously conducted trial was its strong internal reliability. Unusually for skin cancer studies carried out in primary care, expert clinical diagnoses on all lesions were obtained, including lesions that were managed in primary care. We applied a rigorous randomisation method based on sealed envelopes produced by an independent statistician. Our sample size was met and interim analysis showed no significant clustering effects to reduce the power of the trial. We collected data between three and six months after the trial consultation to identify change over time and to confirm a benign diagnosis.20 Of 566 lesions reassessed, only two were found to be clinically significant: a dysplastic (atypical) naevus and a lentigo maligna. The choice of reference standard diagnosis was inevitably pragmatic as, for ethical reasons, we could not obtain histology for every lesion in the trial. From a total of 1573 lesions, only 2.7% did not have a reference standard diagnosis. Although there were eight more lesions in the best practice group managed in primary care for which we do not have a final diagnosis, we do not believe these would alter the key findings of this trial. Furthermore, all participants have been flagged with the Eastern Cancer Registry and Information Centre to identify any melanomas diagnosed over the next five years. In addition to gaining data on diagnostic performance we also obtained information on the acceptability of the intervention by patients and health professionals, which has not previously been examined in similar studies.
Several aspects of the trial design have inherent limitations. Our primary outcome measure was chosen as the one most relevant to an intervention designed to improve referral patterns of pigmented skin lesions. The definition of appropriateness of referral was based on the subsequent clinical management by an expert dermatologist. Thus any lesion considered by a dermatologist worthy of biopsying or monitoring would, from a primary care perspective, be an appropriate referral. As this was the first trial of the MoleMate system in primary care it would have been premature to power the trial to assess differences in detection rates of melanoma given the low prevalence. In fact, this trial is the largest done in primary care worldwide to date to test a new melanoma diagnostic aid.
In the absence of comparable data about people presenting with suspicious lesions in primary care, it is difficult to comment on the representativeness of the sample. The over-representation of people with higher education levels and under-representation of ethnic minority groups compared with the UK general practice population may slightly limit generalisability across the United Kingdom. As a new diagnostic test,26 the MoleMate system was implemented as part of a novel service model in which patients with a suspicious pigmented lesion were referred internally to a trained lead clinician. The trial population therefore represented patients referred to this new service model with a suspicious lesion. This internal referral pathway is consistent with similar service models and subspecialisation increasingly occurring within general practice.
We chose to compare the MoleMate system with standardised best practice rather than usual care for well considered reasons in the design of a trial in primary care27: firstly, a usual care arm would not have allowed us to obtain data on all lesions for reference standard diagnostic purposes because of a significant potential Hawthorne effect incurred through the required data collection procedures. We therefore would not have been able to assess appropriateness of management in primary care. Secondly, the risk of contamination between groups would have been much greater and therefore would have meant using a practice level, cluster randomised design, with consequent effects on the scale and feasibility of the trial. While this means that we do not have directly comparable data for usual care, the findings relating to best practice have important clinical implications.
Strengths and weaknesses in relation to other studies
Both the MoleMate and best practice groups showed impressively high diagnostic performances compared with other studies. Previous international estimates of sensitivity for the diagnosis of melanoma are in the order of 29-41% using history and naked eye assessment alone in primary care.14 28 29 Best practice performed much better, with a sensitivity of 95.7% for suspicious lesions, such that 17 out of 18 melanomas were correctly identified in this trial group. While there are no comparable data on appropriateness as defined in this trial, the ratio of melanomas to benign skin lesions was much higher in both trial groups than previously reported.4 Although the lead clinicians had no specialist training in dermatology before the trial, they were briefly trained in data collection procedures designed to systematically use best practice local guidelines, including the application of the seven point checklist. We did not detect any improvement in their performance during the trial and therefore do not believe that contamination between trial groups through learning effects is the explanation for the high performance in the best practice group. We propose that the formal implementation of the best practice guidelines contributed to the high performance in the comparison group.
SIAscopy has been shown to be an effective diagnostic aid in nurse led screening of pigmented lesions in secondary care17 but may not help expert dermatologists to distinguish melanomas from benign lesions.30 Dermoscopy is the most widely used non-invasive approach to aid diagnosis of melanoma; in the secondary care setting its use increases diagnostic accuracy compared with clinical visual inspection31 and SIAscopy.32 Dermoscopy has also been shown to improve accuracy of primary care doctors to triage lesions suggestive of skin cancer13; however, it is a difficult technique to learn, and takes time to become proficient.14 30 Computerised skin imaging devices based on dermoscopy have also been developed, such as MoleMax, SolarScan, and MelaFind.33 Few have yet been assessed in primary care and none using our rigorous approach of a randomised controlled trial. In a smaller non-randomised trial in Australian primary care, dermoscopy and short term sequential digital dermoscopy imaging showed a significant reduction in referrals or excisions of benign pigmented lesions while doubling the sensitivity for the diagnosis of melanoma.14
Implications for clinicians and policy makers
For melanoma, as for other cancers and any serious condition in primary care where prevalence is low, the inherent diagnostic problem remains of aiming for high sensitivity without resultant poor specificity. It has been suggested recently that tighter gatekeeping, as occurs in British general practice, may contribute to later cancer diagnosis because of a higher focus on specificity of referral or investigation.34 In this trial the lower specificity of the MoleMate system resulted in an increased volume of referrals, but the detection rates of melanoma in both arms were high.
By introducing an internal referral system within the practice, we tested two approaches to assess pigmented skin lesions in a more systematic way in primary care. The implementation of best practice guidelines performed better than the MoleMate system, and both performed better than reports of current practice. Improving the management of suspicious pigmented skin lesions could be rapidly effected by changes in general practice systems that ensure the routine application of the seven point checklist, perhaps further enhanced with internal referral and subsequent follow-up of all non-referred patients with lesions.
What is already known on this topic
Skin malignancy is an important cause of mortality in the United Kingdom and the incidence is rising every year; early detection and management will improve outcomes
Differentiating melanomas from other pigmented skin lesions is challenging, and diagnostic technology could improve primary care performance and appropriate referral of high risk pigmented skin lesions
What this study adds
Adding the MoleMate system (SIAscopy with primary care scoring algorithm) to the systematic application of best practice guidelines did not increase the proportion of patients appropriately referred with lesions; instead, the intervention led to a higher proportion of patients referred with lesions
Systematic application of best practice guidelines and the MoleMate system both performed much better than reports of current practice
Although the novel diagnostic technology was less accurate than best practice, clinicians and patients rated it higher than best practice for reassuring and thorough care, suggesting false reassurance
On current evidence the systematic application of best practice guidelines (including the seven point checklist) is the paradigm for management of suspicious skin lesions in primary care
MoleMate, Astron Clinica, SIAscopy, SIAscan, and SIAscope were initially trademarks of Astron Clinica, which kindly supplied the MoleMate systems for this trial. In August 2009, Astron Clinica was taken over by Biocompatibles International, by BTG (British Technology Group) in January 2011, and by MedX Health in June 2011, which now holds these trademarks. We thank the independent chair of our trial steering committee, Neil Campbell; lay member Marion Edwards; David Mant for earlier comments on the design; and Jonathan Mant for comments on the revised paper. This research would not have been possible without the help of the participating patients, general practitioners, nurses, managers, and administrative staff of the general practices involved. We thank the lead clinicians for their commitment to the study: Stuti Mukherjee and Sally Kaemer, Cherry Hinton Medical Centre, Cambridge; Jenny Wheatley and Alan Mills, Comberton Surgery; Ian Marshman and Sarah Burling, Cornford House Surgery, Cambridge; Yvonne Girgis-Hanna and Christopher Clayton-Payne, Gold Street Surgery, Saffron Walden; Dr Karen Newman & Dr Kumar Nagadev, Huntingdon Road Surgery, Cambridge; Antony Warren and Clare Goodhart, Lensfield Medical Practice, Cambridge; Jo Farnell and Miguel Arbide, Linton Health Centre; Paul Linehan and Fiona Cornish, Newnham Walk Surgery, Cambridge; Robert Dobler Selma Malik, Nuffield Road Medical Centre, Cambridge; Paul Saban and Emma Ramsay, Rookery Medical Centre, Newmarket; Andrew Douglas and Baz Sanghera, St Mary’s Surgery, Ely; Sharon Woods and James Morrow, Sawston Medical Practice; John Tweedale and Jeremy Blakeborough, Shelford Medical Practice; Caroline Lea-Cox and Angus Stewart, Trumpington Street Medical Practice, Cambridge; and Alistair Brown and Judith Lindeck, York Street Medical Practice, Cambridge.
Contributors: FMW led the study and wrote the first draft of the report, supported by JDE. FMW, JDE, ALK, PNH, and ATP designed the study. HCM and EH oversaw the running of the study and the data collection. PNH, NB, PN, and JW provided clinical expertise. ATP, LB, and ECFW undertook the analyses. ATP and FMW take responsibility for the integrity of the data and the accuracy of the data analyses. All authors participated in execution and oversight of the study, interpretation of the data, critical review of drafts, and approved the final submitted version. FMW will act as guarantor and made the final decision to submit for publication. The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report, or in the decision to submit for publication. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Funding: This study was funded by the National Institute for Health Research (NIHR) School for Primary Care Research. The views expressed in this paper are those of the authors and not necessarily those of the Department of Health. ALK is an NIHR senior investigator. Service support costs were obtained from the Department of Health with the support of NHS Cambridgeshire and the East of England Primary Care Research Network.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; JDE has received a research grant from Biocompatibles, PNH does consultancy for Lifescan & Health Screen UK: although PNH has longstanding intellectual involvement with the development of SIAscopy he has had no commercial involvement with Astron Clinica or Biocompatibles; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: This study was approved by Cambridgeshire 2 research ethics committee (reference 07/H0308/167).
Data sharing: The statistical code and dataset are available from the corresponding author at fmw22@medschl.cam.ac.uk.
Cite this as: BMJ 2012;344:e4110
References
- 1.Murchie P, Campbell NC. Pigmented lesions, cutaneous melanoma, and future challenges for primary care. Eur J Gen Pract 2007;13:151-4. [DOI] [PubMed] [Google Scholar]
- 2.Lens MB, Dawes M. Global perspectives of contemporary epidemiological trends of cutaneous malignant melanoma. Br J Dermatol 2004;150:179-85. [DOI] [PubMed] [Google Scholar]
- 3.Balch CM, Gershenwald JE, Soong SJ, Thompson JF, Atkins MB, Byrd DR, et al. Final version of 2009 AJCC melanoma staging and classification. J Clin Oncol 2009;27:6199-206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.All-Party Parliamentary Group on Skin. Report on the enquiry into the treatment, management and prevention of skin cancer. All-Party Parliamentary Group on Skin, 2003.
- 5.Pockney P, Primrose J, George S, Jayatilleke N, Leppard B, Smith H, et al. Recognition of skin malignancy by general practitioners: observational study using data from a population-based randomised controlled trial. Br J Cancer 2009;100:24-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Chen SC, Pennie ML, Kolm P, Warshaw EM, Weisberg EL, Brown KM, et al. Diagnosing and managing cutaneous pigmented lesions: primary care physicians versus dermatologists. J Gen Intern Med 2006;21:678-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jiwa M. Referral from primary to secondary care. BMJ 2010;341:1172-3. [DOI] [PubMed] [Google Scholar]
- 8.MacKie RM, Doherty VR. Seven-point checklist for melanoma. Clin Exp Dermatol 1991;16:151-3. [DOI] [PubMed] [Google Scholar]
- 9.Healsmith MF, Bourke JF, Osborne JE, Graham-Brown RA. An evaluation of the revised seven-point checklist for the early diagnosis of cutaneous malignant melanoma. Br J Dermatol 1994;130:48-50. [DOI] [PubMed] [Google Scholar]
- 10.National Institute for Health and Clinical Excellence. Referral guidelines for suspected cancer. NICE, 2005.
- 11.Goulart JM, Quigley EA, Dusza S, Jewell ST, Alexander G, Asgari MM, et al. Skin cancer education for primary care physicians: a systematic review of published evaluated interventions. J Gen Intern Med 2011;26:1027-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gerbert B, Bronstone A, Maurer T, Berger T, McPhee SJ, Caspers N. The effectiveness of an Internet-based tutorial in improving primary care physicians’ skin cancer triage skills. J Cancer Educ 2002;17:7-11. [DOI] [PubMed] [Google Scholar]
- 13.Argenziano G, Puig S, Zalaudek I, Sera F, Corona R, Alsina M, et al. Dermoscopy improves accuracy of primary care physicians to triage lesions suggestive of skin cancer. J Clin Oncol 2006;24:1877-82. [DOI] [PubMed] [Google Scholar]
- 14.Menzies SW, Emery J, Staples M, Davies S, McAvoy B, Fletcher J, et al. Impact of dermoscopy and short-term sequential digital dermoscopy imaging for the management of pigmented lesions in primary care: a sequential intervention trial. Br J Dermatol 2009;161:1270-7. [DOI] [PubMed] [Google Scholar]
- 15.Wood A, Morris H, Emery J, Hall PN, Cotton S, Prevost AT, et al. Evaluation of the MoleMate training program for assessment of suspicious pigmented lesions in primary care. Inform Prim Care 2008;16:41-50. [DOI] [PubMed] [Google Scholar]
- 16.Moncrieff M, Cotton S, Claridge E, Hall P. Spectrophotometric intracutaneous analysis: a new technique for imaging pigmented skin lesions. Br J Dermatol 2002;146:448-57. [DOI] [PubMed] [Google Scholar]
- 17.Govindan K, Smith J, Knowles L, Harvey A, Townsend P, Kenealy J. Assessment of nurse-led screening of pigmented lesions using SIAscope. J Plast Reconstr Aesthet Surg 2007;60:639-45. [DOI] [PubMed] [Google Scholar]
- 18.Emery JD, Hunter J, Hall PN, Watson AJ, Moncrieff M, Walter FM. Accuracy of SIAscopy for pigmented skin lesions encountered in primary care: development and validation of a new diagnostic algorithm. BMC Dermatol 2010;10:9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Walter FM, Morris HC, Humphrys E, Hall PN, Kinmonth AL, Prevost AT, et al. Protocol for the MoleMate UK Trial: a randomised controlled trial of the MoleMate system in the management of pigmented skin lesions in primary care [ISRCTN 79932379]. BMC Fam Pract 2010;11:36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Altamura D, Avramidis M, Menzies SW. Assessment of the optimal interval for and sensitivity of short-term sequential digital dermoscopy monitoring for the diagnosis of melanoma. Arch Dermatol 2008;144:502-6. [DOI] [PubMed] [Google Scholar]
- 21.Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, et al. The GRAIDS Trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer 2007;97:486-93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, et al. Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract 2000;50:882-7. [PMC free article] [PubMed] [Google Scholar]
- 23.Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). Br J Clin Psychol 1992;31:301-6. [DOI] [PubMed] [Google Scholar]
- 24.Lerman C, Schwartz MD, Lin TH, Hughes C, Narod S, Lynch HT. The influence of psychological distress on use of genetic testing for cancer risk. J Consult 1997;65:414-20. [DOI] [PubMed] [Google Scholar]
- 25.Newcombe RG. Interval estimation for the difference between independent proportions: comparison of eleven methods. Stat Med 1998;17:873-90. [DOI] [PubMed] [Google Scholar]
- 26.Ferrante di Ruffano T, Hyde CJ, McCaffery KJ, Bossuyt PM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ 2012;344:e686. [DOI] [PubMed] [Google Scholar]
- 27.Foster N, Little P. Methodological issues in pragmatic trials of complex interventions in primary care. Br J Gen Pract 2012;62:10-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Youl PH, Baade PD, Janda M, Del Mar CB, Whiteman DC, Aitken JF. Diagnosing skin cancer in primary care: how do mainstream general practitioners compare with primary care skin cancer clinic doctors? Med J Aust 2007;187:215-20. [DOI] [PubMed] [Google Scholar]
- 29.Chen SC, Bravata DM, Weil E, Olkin I. A comparison of dermatologists’ and primary care physicians’ accuracy in diagnosing melanoma: a systematic review. Arch Derm 2001;137:1627-34. [DOI] [PubMed] [Google Scholar]
- 30.Haniffa MA, Lloyd JJ, Lawrence CM. The use of a spectrophotometric intracutaneous analysis device in the real-time diagnosis of melanoma in the setting of a melanoma screening clinic. Br J Dermatol 2007;156:1350-2. [DOI] [PubMed] [Google Scholar]
- 31.Vestergaard ME, Macaskill P, Holt PE, Menzies SW. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting. Br J Dermatol 2008;159:669-76. [DOI] [PubMed] [Google Scholar]
- 32.Glud M, Gniadecki R, Drzewiecki KT. Spectrophotometric intracutaneous analysis versus dermoscopy for the diagnosis of pigmented skin lesions: prospective, double-blind study in a secondary reference centre. Melanoma Res 2009;19:176-9. [DOI] [PubMed] [Google Scholar]
- 33.Patel JK, Konda S, Perez OA, Amini S, Elgart G, Berman B. Newer technologies/techniques and tools in the diagnosis of melanoma. Eur J Dermatol 2008;18:617-31. [DOI] [PubMed] [Google Scholar]
- 34.Vedsted P, Olesen F. Are the serious problems in cancer survival partly rooted in gatekeeper principles? An ecologic study. Br J Gen Pract 2011;61:e508-12. [DOI] [PMC free article] [PubMed] [Google Scholar]