Key Points
Question
How is stratifying patients referred to the dermatology department for a new lesion into simple and interpretable risk categories associated with patient outcomes?
Findings
In this single-institution quality improvement cohort study of 37 478 patients referred to a dermatology department, a risk stratification pathway that was based on categorical referral reasons and risk factors provided by primary care physicians identified and prioritized patients at a higher risk of skin cancer (overall and melanoma specifically).
Meaning
The study findings suggest that stratifying patients based on this outlined pathway might expedite care for patients who require a biopsy and subsequent treatment, ensuring that individuals at higher risk for skin cancer receive timely treatment.
Abstract
Importance
Access to timely dermatologic care remains a challenge, especially for patients with new skin lesions. Assessing the efficiency of new triage pathways may assist in better resource allocation and shorter time to care.
Objective
To evaluate whether a rule-based triage system was associated with better skin cancer risk stratification of patients and reduced wait times.
Design, Setting, and Participants
This retrospective quality improvement cohort study of patients referred to Stanford University dermatology clinics was conducted between November 2017 and January 2023. A rules-based triage system based on a priori–determined high-risk lesion characteristics was implemented.
Exposures
Referral reasons and risk factors of patients provided by their primary care physicians.
Main Outcomes and Measures
Biopsy results of patients (diagnosis of any skin cancer and melanoma) at their visit or within 6 months after the visit. Regression models were used to assess the association between risk factors at referral and (1) biopsy outcomes and (2) time to first visit, adjusting for sociodemographic factors.
Results
Among 37 478 patients (mean [SD] age, 54 (18) years; 21 292 women [57%]), the rates of aggregate biopsy, malignant biopsy specimens, and melanoma were comparable across patients seen after (n = 12 302) and before (n = 25 176) the implementation of the new triage pathway. Patients seen through the lesion pathway had a higher risk of having malignant biopsy results (adjusted risk ratio [aRR], 1.6; 95% CI, 1.4-1.9) and melanoma (aRR, 2.0; 95% CI, 1.2-3.2) than those not seen through the pathway. Lesions that were concerning to referring clinicians for skin cancer were associated with an increased risk of skin cancer (all skin cancer: aRR, 2.8; 95% CI, 2.2-3.5; melanoma: aRR, 2.02; 95% CI, 1.1-3.7). Patients in the 3 high-risk lesion groups were seen faster in the new triage pathway (mean reduction, 26 days; 95% CI, 18-34 days).
Conclusions and Relevance
In this study, a new automated, rules-based referral pathway was implemented that expedited care for patients with high-risk skin cancer. This reform may have contributed to improving patient stratification, reducing the time from referral to first encounter, and maintaining accuracy in identifying malignant lesions. The findings highlight the potential to optimize clinical resource allocation by better risk stratification of referred patients.
This cohort study examines the association of a rule-based triage system with skin cancer risk stratification of patients and wait times.
Introduction
There is a well-described undersupply and maldistribution of dermatologists in the US1,2 that has been associated with excessive wait times averaging 30 to 100 days for appointments.2,3 For patients with new skin lesions, long wait times can be associated with advanced presentations at diagnosis and in turn more aggressive treatments, extensive resource use, and poor outcomes.4
The current referral process is frequently followed by delays and errors.5 Specialty referrals are often handwritten or free-text descriptors that frequently require clinician input to appropriately triage and schedule patients, in which is associated with increased physician administrative burden. In practices without formal triage workflows, patients are often defaulted to the next available appointment, unless specific “red-flag” complaints are identified by front-line staff. This lack of consistency leaves much room for improvement. Prior work has demonstrated that lesion-directed screening in the general population revealed skin cancer at rates similar to full-body skin examinations but was more than 5 times faster.6 This finding suggests a time-efficient and cost-effective approach for skin cancer detection through single-lesion risk stratification.
In 2021, we instituted a quality improvement (QI) intervention (Figure) to our referral and triage system reform to prioritize patients at higher risk for skin cancer. We outlined a strategy to evaluate the feasibility and efficiency of matching risk-stratified referrals with the timeliness of visits, allowing for greater practice efficiency by predicting the necessity of in-person clinic procedures and subsequent skin cancer treatment. We present this rules-based, single-lesion referral pathway as a specific model for this broad approach.
Figure. Dermatology Specialty Clinic Skin Cancer Triage Pathway Workflow.
In the new referral process, the patient self-refers or is referred by their primary care physician through the electronic health record system. Within the referral system, they were asked for the referral reason, which included (1) skin cancer screening; (2) single lesion of concern; and (3) rash/acne/rosacea. Within these reasons of referral, the referring clinician can then provide additional information via a checkbox system. Afterwards, the rule-based system assesses the risk of patients presenting single lesions of concern, and groups them as high risk (seen through expedited appointments) or average risk (seen at the next available in-clinic appointments). Patients referred for skin cancer screening, rash, acne or rosacea, were triaged with the current standard of care.
Methods
The study was conducted from November 2017 to January 2023. The QI intervention implemented a customized electronic medical record referral order that allows referring clinicians to select prespecified referral reasons. We then designed a rule-based system that automatically categorized the patients into high risk or average risk based on referral reasons (Figure). In cases of multiple reasons, we assigned patients to the highest-priority category.
Biopsy results, scheduling data, and demographic characteristics were extracted from the enterprise data warehouse using targeted queries and then analyzed using natural language processing tools in Python.7 To evaluate the accuracy of the inferred biopsied results, a board-certified dermatologist examined all cases of malignant biopsy specimens and 500 randomly chosen cases of nonmalignant biopsy specimens. The QI process was exempt from institutional review board approval, and we received institutional review board approval from Stanford University to publish the outcome of this study, who waived informed consent.
We evaluated 3 biopsy-related outcomes: (1) biopsy rates (proportion of referred patients who underwent biopsy); (2) rates of malignant biopsy specimens (proportion of patients with confirmed skin cancers in any biopsied lesions); and (3) biopsied melanoma specimen rates (proportion of patients with confirmed melanoma among any biopsied lesions). For each referral reason, we defined the control group as all pre–triage reform patients and post–triage reform patients not assigned with that reason. The adjusted relative risk (aRR) for (1) malignant biopsy specimens and (2) melanoma biopsy specimens were evaluated using Poisson regressions, adjusting for age, sex, race and ethnicity, and median household income. Differences in the referral to first encounter time for each reason were assessed using linear regressions. We used R software (version 4.1.1; R Foundation) for statistical analyses (with a robust variance estimator accounting for unequal variance across patients) and conducted a sensitivity analysis by excluding self-referred patients.
Results
Our study included 37 478 patients (mean [SD] age, 54 [18.6] years; 21 292 women [57%]; 1408 [4%] African American, 7740 [21%] Asian, 3844 [10%] Hispanic, and 21 148 [56%] White individuals) (eTable 1 in Supplement 1). Comparing patients before and after the new triage pathway, we saw broadly similar biopsy rates (9.7% and 10.1%, respectively), malignant biopsy specimen rates (28.5% and 25.3%, respectively) and melanoma biopsy specimen rates (2.5% and 3.7%, respectively) (Table 1). Baseline demographic characteristics associated with biopsy outcomes included age (15.0% of patients who were older than 80 years underwent biopsy, with 56.8% positive for cancer) and race (823 White patients underwent a biopsy, of which 34.6% yielded malignant results; eTable 2 in Supplement 1).
Table 1. Biopsy, Biopsied Cancer, and Biopsied Melanoma Rates and First Encounter From Referral in Aggregate and Stratified by Referral Reasons and Risk Factors Implemented in the New Skin Cancer Triage Pathway.
| Referral reasons and risk factors | No. | Rate, % (95% CI) | Mean first encounter from referral, d (95% CI)a | ||
|---|---|---|---|---|---|
| Biopsya | Biopsied specimena | ||||
| Cancer | Melanoma | ||||
| Patients seen before the implementation of the triage intervention | 25 176 | 9.7 (9.4-10.1) | 28.5 (26.7-30.2) | 2.5 (1.9-3.1) | 88 (87-90) |
| Patients seen after the implementation of the triage intervention | 12 302 | 10.1 (9.6-10.6) | 25.3 (22.9-27.7) | 3.7 (2.7-4.8) | 99 (98-101) |
| Specific lesions of concern | 3402 | 15.1 (13.9-16.3) | 28.2 (24.3-32.1) | 3.7 (2.1-5.3) | 87 (84-89) |
| High-risk subgroups | 825 | 25.3 (22.4-28.3) | 41.6 (34.9-48.3) | 4.8 (1.9-7.7) | 66 (62-71) |
| Appearance concerning to clinician for skin cancer | 519 | 27.9 (24.1-31.8) | 48.3 (24.1-31.8) | 4.8 (1.3-8.3) | 68 (63-74) |
| Bleeding or rapid evolution | 219 | 24.2 (18.5-29.9) | 32.1 (19.5-44.6) | 1.9 (0-5.5) | 61 (53-70) |
| Clinician concern for melanoma | 137 | 23.4 (16.3-30.4) | 25 (10-40) | 9.4 (0-19.5) | 60 (50-71) |
| Symptomatic (itching, pain) or bothersome to patient | 1269 | 11.5 (9.8-13.3) | 19.9 (13.4-26.3) | 2.7 (0-5.4) | 91 (87-95) |
| Prior skin biopsy performed for this lesion | 75 | 14.7 (6.7-22.7) | 45.5 (16.0-74.9) | 0 | 78 (61-95) |
| Skin cancer screening | 4040 | 12.1 (11.1-13.1) | 27.7 (23.7-31.7) | 4.5 (2.7-6.4) | 121 (119-124) |
| Personal history of skin cancer or actinic damage | 616 | 17.2 (14.2-20.2) | 48.1 (38.6-57.6) | 5.7 (1.3-10.1) | 107 (101-114) |
| Atypical nevi or >40 nevi | 482 | 17.4 (14.0-20.8) | 19.0 (10.7-27.4) | 4.8 (0.3-9.3) | 118 (110-125) |
| Family history of skin cancer | 1176 | 14.3 (12.3-16.3) | 38.1 (30.8-45.4) | 5.4 (1.9-8.8) | 118 (114-123) |
| Immunocompromised | 264 | 9 (5.6-12.6) | 33.3 (14.5-52.2) | 4.2 (0-12.2) | 120 (110-130) |
| Easily sunburns, has many freckles or blonde/red hair, or history of indoor tanning or blistering sunburns | 897 | 12.8 (10.6-15) | 20.9 (13.4-28.3) | 4.4 (0.1-8.1) | 122 (117-127) |
| Acne/rosacea | 482 | 3.1 (1.6-4.7) | 6.7 (0-19.3) | 0 | 106 (99-113) |
| Rash | 1424 | 7.6 (6.2-9.0) | 12.8 (6.6-19.1) | 0.07 (0-0.2) | 79 (75-83) |
Estimates are reported with 95% CIs using a normal approximation. Within a referral reason, the total number of patients across all the risk factor categories could exceed the total number of patients seen in the clinic, as referring physicians could cite multiple risk factors and reasons for referral.
We found that the single-lesion cohort had a higher biopsy rate (15.1% vs 8.3%; P < .001) despite a relatively similar malignant biopsy specimen rate (28.2% vs 22.8%; P = .04), suggesting that although this cohort underwent more biopsies, diagnostic accuracy was maintained. Within this cohort, specific risk factors associated with higher biopsy rates and malignant biopsy specimen rates included lesions that were bleeding and undergoing rapid evolution (24.2% and 32.1%, respectively), or lesions in which prior clinicians had concerns for melanoma (23.4% and 25.0%, respectively). Regarding new melanoma diagnoses, referrals with concern from a referring clinician for melanoma had the highest rate of malignant biopsy specimens at 9.4%.
Adjusting for demographic risk factors, patients with specific lesions of concern had a higher risk of having malignant biopsy results (aRR, 1.6; 95% CI, 1.4-1.9) and melanoma results (aRR, 2.0; 95% CI, 1.2-3.2) than those without (Table 2). Patients with lesions that were concerning to a clinician for (1) general skin cancer and (2) melanoma also had higher risks of malignant biopsy results (aRR, 2.8; 95% CI, 2.2-3.5; aRR, 2.02; 95% CI, 1.1-3.7) and higher risks of melanoma (aRR, 2.8; 95% CI, 1.3-6.0; aRR, 7.7; 95% CI, 2.5-23.8).
Table 2. Adjusted Associations Between Referral Groups in the Skin Cancer Triage Pathway and Malignant Biopsy Specimen Status, Melanoma Status, and Time to First Encounter From Referral.
| Groupa | aRR (95% CI) | Adjusted difference in first encounter from referral, d (95% CI)b | |
|---|---|---|---|
| Malignant biopsy specimensb | Melanomab | ||
| Triage intervention group | |||
| Specific lesions of concern | 1.63 (1.37 to 1.92) | 1.96 (1.20 to 3.19) | −5 (−8 to −3) |
| High-risk subgroup | 2.75 (2.24 to 3.39) | 3.19 (1.68 to 6.05) | −26 (−30 to −21) |
| Appearance concerning to clinician for skin cancer | 2.90 (2.33 to 3.54) | 3.09 (1.44 to 6.62) | −23 (−29 to −17) |
| Bleeding or rapid evolution | 2.35 (1.43 to 3.49) | 1.27 (0.18 to 9.21) | −30 (−39 to −21) |
| Clinician concern for melanoma | 2.85 (1.05 to 3.71) | 7.77 (2.49 to 24.3) | −30 (−41 to −20) |
| Symptomatic (itching, pain) or bothersome to patient | 0.90 (0.62 to 1.26) | 1.11 (0.41 to 2.97) | −0.2 (−4 to 4) |
| Prior skin biopsy performed for this lesion | 1.60 (0.60 to 3.20) | 0 | −13 (−30 to 4) |
| Skin cancer screening | 1.07 (0.90 to 1.28) | 1.68 (1.07 to 2.64) | 31 (29 to 34) |
| Personal history of skin cancer or actinic damage | 1.52 (1.15 to 1.96) | 2.10 (0.97 to 4.53) | 15 (9 to 21) |
| Atypical nevi or >40 nevi | 1.30 (0.77 to 2.01) | 2.57 (0.95 to 6.96) | 24 (17 to 32) |
| Family history of skin cancer | 1.42 (1.10 to 1.79) | 2.00 (1.05 to 3.82) | 25 (20 to 30) |
| Immunocompromised | 1.26 (0.59 to 2.27) | 1.32 (0.18 to 9.58) | 30 (20 to 40) |
| Easily sunburns, has many freckles or blonde/red hair, or history of indoor tanning or blistering sunburns | 0.84 (0.55 to 1.21) | 1.42 (0.58 to 3.45) | 28 (23 to 34) |
| Acne/rosacea | 0.24 (0.01 to 1.06) | 0 | 12 (5 to 20) |
| Rash | 0.50 (0.28 to 0.80) | 0.30 (0.04 to 2.12) | −12 (−16 to −8) |
Abbreviation: aRR, adjusted relative risk.
Reference group: patients not seen through a specific intervention group (historical controls before triage implementation and postreform patients not seen through that intervention group).
Poisson regressions were used to estimate aRRs for binary outcomes and linear regressions were used to estimate adjusted differences for continuous outcomes. The adjustment set included sex, age, race and ethnicity, and median household income of the reported zip code; we used robust variance estimators in regression models to account for unequal variance in patient outcomes. Reported aRR is 0 for cells with no observed melanoma cases.
In terms of wait times, we saw a slight increase over time (eFigure 2 in Supplement 1); patients in the a priori–determined high-risk categories, who were assigned an expedited lesion slot through a rules-based approach, were seen a mean of 66 days after referral, which was 26 days earlier (95% CI, 18-36 days; Table 2) than the rest of patients. We did not observe a difference in wait times between White patients and African American, Asian, and Hispanic patients (2 days; 95% CI, −8 to 11 days), although skin cancer was more common in former (eTable 2 in Supplement 1). The sensitivity analysis that excluded self-referral patients yielded similar results for both biopsy outcomes and wait times (eTable 3 in Supplement 1).
Discussion
This cohort study provides a granular model for how known skin cancer risk factors, when applied at the health system level, may better identify higher-risk patients requiring timely in-clinic care. The single lesion referral pathway was associated with similar biopsy frequency and accuracy, better risk stratification, and shorter wait times for patients with higher risks for skin cancer. Future QI efforts are focused on a matching capacity to further reduce waits for these and other identified higher-risk groups, including patients undergoing skin cancer screening. The UK’s National Institute for Health and Care Excellence recommended a specialist appointment within 28 days of referral for concerning lesions.8 In addition, multiple studies identified anxiety and stress that long wait times cause for patients with possible skin cancer diagnoses,9 underscoring the benefits of earlier diagnosis.
We found similar biopsy and cancer rates to those of previous studies. Sherban et al10 found that 27% of biopsied lesions via full-body skin examinations were malignant, similar to our findings of 27.7% for patients referred for skin cancer screening. We found that patients with a family or personal history of skin cancer were more likely to have a malignant biopsy specimen diagnosis, which is well established in prior work.11,12
Limitations
This study’s findings have limitations: given the observational nature, we cannot establish definitive causality, and the risk stratification and reduced wait times could be subject to uncontrolled confounding. Moreover, triage priorities can vary across practices. For example, this algorithm prioritized skin cancer risk over referrals for symptomatic lesions, although the latter can be associated with decreased quality of life. Such nonprioritized groups can experience relatively longer waits, highlighting the importance of understanding the postdeployment effect of triage decisions. Lastly, this cohort was from a single academic center, limiting the generalizability even after adjusting for patient demographic characteristics. In future work, we will explore a machine learning–augmented triage pathway integrating inputs, including lesion images.
Conclusions
In this cohort study, we instituted a new referral pathway to our academic center’s dermatology department. This was associated with improved patient stratification for skin cancer diagnoses and shortened referral to first encounter time by up to 30 days for higher-risk groups. Our findings might improve in-clinic resource allocation and reduce patient wait time by prioritizing those more likely to require biopsies.
eFigure 1. Study flow diagram illustrating inclusion and exclusion criteria to reach the study population
eFigure 2. Average time from referral to first encounter for patients overall and stratified by (i) high-risk skin cancer groups for which expeditated slots were given; and (ii) skin cancer screening group
eTable 1. Baseline demographics and clinical outcomes of interest for the study population, stratified before and after the implementation of the skin cancer triage pathway
eTable 2. Biopsy rates, biopsied malignancy rates, and biopsied melanoma rates in aggregate and stratified by referral reasons and risk factors implemented in the new skin cancer triage pathway
eTable 3. Adjusted associations between referral groups in the skin cancer triage pathway and (i) malignant biopsy status; (ii) melanoma; and (iii) first encounter from referral, for patients who are not self-referred
Data sharing statement
References
- 1.Kimball AB, Resneck JS Jr. The US dermatology workforce: a specialty remains in shortage. J Am Acad Dermatol. 2008;59(5):741-745. doi: 10.1016/j.jaad.2008.06.037 [DOI] [PubMed] [Google Scholar]
- 2.Resneck J Jr, Kimball AB. The dermatology workforce shortage. J Am Acad Dermatol. 2004;50(1):50-54. doi: 10.1016/j.jaad.2003.07.001 [DOI] [PubMed] [Google Scholar]
- 3.Stephens MR, Murthy AS, McMahon PJ. Wait times, health care touchpoints, and nonattendance in an academic pediatric dermatology clinic. Pediatr Dermatol. 2019;36(6):893-897. doi: 10.1111/pde.13943 [DOI] [PubMed] [Google Scholar]
- 4.Criscito MC, Martires KJ, Stein JA. A population-based cohort study on the association of dermatologist density and Merkel cell carcinoma survival. J Am Acad Dermatol. 2017;76(3):570-572. doi: 10.1016/j.jaad.2016.10.043 [DOI] [PubMed] [Google Scholar]
- 5.Pagani K, Lukac D, Olbricht SM, et al. Urgent referrals from primary care to dermatology for lesions suspicious for skin cancer: patterns, outcomes, and need for systems improvement. Arch Dermatol Res. 2023;315(5):1397-1400. doi: 10.1007/s00403-022-02456-7 [DOI] [PubMed] [Google Scholar]
- 6.Hoorens I, Vossaert K, Pil L, et al. Total-body examination vs lesion-directed skin cancer screening. JAMA Dermatol. 2016;152(1):27-34. doi: 10.1001/jamadermatol.2015.2680 [DOI] [PubMed] [Google Scholar]
- 7.Python Standard Library . Python documentation. Accessed September 25, 2023. https://docs.python.org/3/library/index.html
- 8.National Institute for Health and Care Excellence . Overview: suspected cancer: recognition and referral. Accessed November 6, 2023. https://www.nice.org.uk/guidance/ng12 [PubMed]
- 9.De Salins CA, Brenaut E, Misery L, Roguedas-Contios AM. Factors influencing patient satisfaction: assessment in outpatients in dermatology department. J Eur Acad Dermatol Venereol. 2016;30(10):1823-1828. doi: 10.1111/jdv.13652 [DOI] [PubMed] [Google Scholar]
- 10.Sherban A, Waseh S, Hugo A, Bui M, Daskalakis C, Jones E. Skin cancer biopsy and detection rates with total body skin examination: a cross-sectional retrospective analysis. J Am Acad Dermatol. 2022;86(4):929-931. doi: 10.1016/j.jaad.2021.03.061 [DOI] [PubMed] [Google Scholar]
- 11.Toussi A, Mans N, Welborn J, Kiuru M. Germline mutations predisposing to melanoma. J Cutan Pathol. 2020;47(7):606-616. doi: 10.1111/cup.13689 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Siegel JA, Korgavkar K, Weinstock MA. Current perspective on actinic keratosis: a review. Br J Dermatol. 2017;177(2):350-358. doi: 10.1111/bjd.14852 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure 1. Study flow diagram illustrating inclusion and exclusion criteria to reach the study population
eFigure 2. Average time from referral to first encounter for patients overall and stratified by (i) high-risk skin cancer groups for which expeditated slots were given; and (ii) skin cancer screening group
eTable 1. Baseline demographics and clinical outcomes of interest for the study population, stratified before and after the implementation of the skin cancer triage pathway
eTable 2. Biopsy rates, biopsied malignancy rates, and biopsied melanoma rates in aggregate and stratified by referral reasons and risk factors implemented in the new skin cancer triage pathway
eTable 3. Adjusted associations between referral groups in the skin cancer triage pathway and (i) malignant biopsy status; (ii) melanoma; and (iii) first encounter from referral, for patients who are not self-referred
Data sharing statement
