Accuracy of Skin Cancer Diagnosis by Physician Assistants Compared With Dermatologists in a Large Health Care System

Key Points

Question

Are physician assistants and dermatologists equally accurate in diagnosing skin cancer in patients undergoing screening?

Findings

In this medical record review of 33 647 skin cancer screening examinations in 20 270 unique patients, physician assistants needed to biopsy 39.4 pigmented lesions and dermatologists needed to biopsy 25.4 pigmented lesions to diagnose 1 case of melanoma. Patients screened by a physician assistant were significantly less likely than those screened by a dermatologist to be diagnosed with melanoma in situ.

Meaning

Compared with dermatologists, physician assistants have lower diagnostic accuracy for melanoma.

Abstract

Importance

Physician assistants (PAs) are increasingly used in dermatology practices to diagnose skin cancers, although, to date, their diagnostic accuracy compared with board-certified dermatologists has not been well studied.

Objective

To compare diagnostic accuracy for skin cancer of PAs with that of dermatologists.

Design, Setting, and Participants

Medical record review of 33 647 skin cancer screening examinations in 20 270 unique patients who underwent screening at University of Pittsburgh Medical Center–affiliated dermatology offices from January 1, 2011, to December 31, 2015. International Classification of Diseases, Ninth Revision code V76.43 and International Classification of Diseases and Related Health Problems, Tenth Revision code Z12.83 were used to identify pathology reports from skin cancer screening examinations by dermatologists and PAs.

Exposure

Examination performed by a PA or dermatologist.

Main Outcomes and Measures

Number needed to biopsy (NNB) to diagnose skin cancer (nonmelanoma, invasive melanoma, or in situ melanoma).

Results

Of 20 270 unique patients, 12 722 (62.8%) were female, mean (SD) age at the first visit was 52.7 (17.4) years, and 19 515 patients (96.3%) self-reported their race/ethnicity as non-Hispanic white. To diagnose 1 case of skin cancer, the NNB was 3.9 for PAs and 3.3 for dermatologists (P < .001). Per diagnosed melanoma, the NNB was 39.4 for PAs and 25.4 for dermatologists (P = .007). Patients screened by a PA were significantly less likely than those screened by a dermatologist to be diagnosed with melanoma in situ (0.2% vs 0.4% of visits, P = .04), but differences were not significant for invasive melanoma (0.2% vs 0.2% of visits, P > .99) or nonmelanoma skin cancer (6.1% vs 6.1% of visits, P = .98).

Conclusions and Relevance

Compared with dermatologists, PAs performed more skin biopsies per case of skin cancer diagnosed and diagnosed fewer melanomas in situ, suggesting that the diagnostic accuracy of PAs may be lower than that of dermatologists. Although the availability of PAs may help increase access to care and reduce waiting times for appointments, these findings have important implications for the training, appropriate scope of practice, and supervision of PAs and other nonphysician practitioners in dermatology.

This medical record review evaluates biopsy reports from skin cancer screening examinations to compare dermatologist vs physician assistant accuracy in diagnosing skin cancer in a large health care system.

Introduction

Advanced practice professionals (APPs), a term referring to nonphysician clinicians including nurse practitioners and physician assistants (PAs), provide an increasing proportion of dermatologic care, with one recent survey finding that 46% of dermatologists employed APPs in 2014.^1,2 Advanced practice professionals can help reduce wait times, improve access to dermatology care, and cost less to hire than dermatologists.³ However, ensuring quality of care provided by APPs is difficult because there is no formal training or certification program in dermatology for APPs.

The autonomy with which many APPs practice, including diagnosing and treating skin cancers without a board-certified dermatologist present, has drawn public criticism.⁴ One study found that APPs must perform more skin biopsies than dermatologists to diagnose 1 case of skin cancer.⁵ We compared skin cancer screening visits performed by PAs, the only APPs working in dermatology in our health care system, with those performed by dermatologists to determine whether rates of skin cancer detection and biopsy specificity (determined by number needed to biopsy [NNB] to find 1 case of skin cancer) differed by clinician type.

Methods

We identified and analyzed medical records of 33 647 skin screening examinations in 20 270 unique patients at University of Pittsburgh Medical Center–affiliated dermatology offices from January 1, 2011, to December 31, 2015, using International Classification of Diseases, Ninth Revision code V76.43 and International Classification of Diseases and Related Health Problems, Tenth Revision code Z12.83: “Encounter for screening for malignant neoplasm of the skin.” Skin cancer pathology reports from all visits in which a lesion was removed for pathologic examination for diagnostic purposes (on the day of or up to 1 month after the office visit), excluding reexcisions, were reviewed to categorize lesions as pigmented or nonpigmented and to obtain the final diagnosis. Number needed to biopsy to diagnose 1 case of skin cancer of any type (melanoma or nonmelanoma) and NNB to diagnose 1 case of melanoma were calculated by clinician type. For each visit, patient sex, age, personal history of melanoma, personal history of skin cancer, and clinician type (PA or board-certified dermatologist) were extracted. This project was reviewed and approved by the University of Pittsburgh Institutional Review Board, which granted a waiver of informed consent.

Years of clinical experience were calculated for each clinician as of the end date of the study period. Clinical experience was calculated from the start of employment in dermatology for PAs and from the year of dermatology residency completion for dermatologists. Physician assistant training in our system is not standardized and is at the discretion of the supervising physician but generally consists of shadowing with a physician and/or a more experienced PA. Financial support for continuing education is available for dermatologists and PAs. Physician assistants and dermatologists are encouraged but not required to attend monthly grand rounds, and have access to dermoscopic equipment, although standardized training in dermoscopy beyond that gained while shadowing or pursued independently is not provided to PAs. Physician assistants do not attend resident didactic sessions.

Statistical Analysis

Visit characteristics were compared between PAs and dermatologists using the unpaired, 2-tailed t test and χ² tests. Number needed to biopsy was calculated as the inverse of the absolute risk of skin cancer per biopsy in univariate regression models. To calculate NNB, only pigmented lesion biopsies were used for melanoma; all biopsies performed for diagnostic purposes were used as the denominator for any skin cancer. Number needed to biopsy for PAs and dermatologists was compared, and significance was determined using generalized linear regression, at α = .05. Years of experience for PAs and dermatologists were compared using a t test with unequal variance. All statistical analyses were performed in R, version 3.3.1 (The R Foundation).

Results

Patient and Clinician Demographic Characteristics

Of 20 270 unique patients, 12 722 (62.8%) were female, the mean (SD) age at the first visit was 52.7 (17.4) years, and 19 515 patients (96.3%) self-reported their race/ethnicity as non-Hispanic white. Most patients (13 151 [64.9%]) had a single screening visit during the study period. Patients with a history of melanoma were more likely to see a dermatologist, whereas those with a history of any type of skin cancer were more likely to see a PA (Table 1). Thirty clinicians (15 dermatologists and 15 PAs) provided skin cancer screening during the study period. Physicians had more years of experience (mean [SD], 13.5 [11.7] years) than PAs (mean [SD], 6.9 [3.6] years) (P = .03).

Table 1. Demographic Characteristics of 20 270 Unique Patients by Clinician Type.

Characteristic	No. (%)				P Valuea
	All Patients	Patients Seeing Both Clinician Types	PA Only	Dermatologist Only
No. of unique patients	20 270	2583	8037	9650	NA
No. of visits per patient, mean (SD)	1.66 (1.19)	3.08 (1.40)	1.60 (1.19)	1.33 (0.80)	<.001
Sex
Male	7548 (37.2)	1078 (41.7)	2837 (35.3)	3633 (37.6)	.001
Female	12 722 (62.8)	1505 (58.3)	5200 (64.7)	6017 (62.4)
Age at first visit, mean (SD), y	52.7 (17.4)	56.0 (16.4)	52.4 (17.5)	52.1 (17.5)	.18
Non-Hispanic whiteb	19 515 (96.3)	2526 (97.8)	7719 (96.0)	9270 (96.1)	.98
History of melanoma	1164 (5.7)	266 (10.3)	333 (4.1)	565 (5.9)	<.001
History of any skin cancer	4972 (24.5)	1026 (39.7)	2114 (26.3)	1832 (19.0)	<.001

Abbreviations: NA, not applicable; PA, physician assistant.

^aP value compares data for dermatologist only with data for PA only.

^bRace/ethnicity was self-reported.

Visit Analyses

Compared with those seeing a dermatologist, patients seeing a PA were slightly older (mean [SD] age, 52.4 [17.5] vs 52.1 [17.5] years), less likely to have a history of melanoma (333 patients [4.1%] vs 565 [5.9%]), and more likely to have a history of any skin cancer (2214 patients [26.3%] vs 1832 [19.0%]) (all P < .001). Physician assistants performed more biopsies overall (3928 visits [22.9%] vs 3427 [20.8%], P < .001) and of pigmented lesions (2213 visits [12.9%] vs 1826 [11.1%], P < .001) than dermatologists. Screenings performed by dermatologists were more likely to result in a diagnosis of melanoma in situ (58 visits [0.4%] vs 40 [0.2%], P = .04), but not invasive melanomas (25 visits [0.2%] vs 26 [0.2%], P > .99) or nonmelanoma skin cancer (993 visits [6.1%] vs 1031 [6.1%], P = .98). Breslow depth for invasive melanomas did not differ significantly by clinician type (Table 2). To diagnose 1 case of skin cancer, the mean NNB was 3.9 (95% CI, 3.7-4.1) for PAs and 3.3 (95% CI, 3.2-3.5) for dermatologists (P < .001). To diagnose 1 case of melanoma, the mean NNB was 39.4 (95% CI, 31.1-49.8) for PAs and 25.4 (95% CI, 20.6-31.3) for dermatologists (P = .007) (Table 3).

Table 2. Characteristics of Skin Cancer Screening Visits by Clinician Type Based on 33 647 Unique Visits.

Visit-Level Patient Characteristic	No. (%) of Visits			P Valuea
	All Clinicians	PA Only	Dermatologist Only
No. of visits	33 647	17 162 (51.0)	16 485 (49.0)	NA
Sex				.12
Male	13 657 (40.6)	6895 (40.2)	6762 (41.0)
Female	19 990 (59.4)	10 267 (59.8)	9723 (59.0)
Age, mean (SD), y	55.5 (17.1)	56.2 (17.0)	54.8 (17.2)	<.001
Personal history of melanoma	3572 (10.6)	1644 (9.6)	1928 (11.7)	<.001
History of any skin cancer	13 120 (39.0)	7689 (44.8)	5431 (32.9)	<.001
Outcome
Total biopsies performed	7355 (21.9)	3928 (22.9)	3427 (20.8)	<.001
Biopsy of pigmented lesions	4039 (12.0)	2213 (12.9)	1826 (11.1)	<.001
NMSC diagnosed	2024 (6.1)	1031 (6.1)	993 (6.1)	.98
Melanoma diagnosed
In situ (% visits)	98 (0.3)	40 (0.2)	58 (0.4)	.04
Invasive (% visits)	51 (0.2)	26 (0.2)	25 (0.2)	>.99
Breslow depth, median (IQR), mm	0.42 (0.3-0.63)	0.41 (0.29-0.59)	0.44 (0.35-0.76)	.36
Breslow depth, mean (SD), mm	0.72 (1.25)	0.86 (1.72)	0.58 (0.32)	.44

Abbreviations: IQR, interquartile range; NA, not applicable; NMSC, nonmelanoma skin cancer; PA, physician assistant.

^aP value compares data for dermatologist only with data for PA only.

Table 3. Number Needed to Biopsy by Cancer Type and Health Care Provider Type.

Cancer Type	NNB, Mean (95% CI)			P Valuea
	All Clinicians	PA Only	Dermatologist Only
Any Skin Cancer
All patients	3.6 (3.5-3.7)	3.9 (3.7-4.1)	3.3 (3.2-3.5)	<.001
Personal history of melanoma
Yes	3.6 (3.3-4.0)	4.5 (3.7-5.4)	3.2 (2.8-3.7)	.004
No	3.6 (3.5-3.7)	3.8 (3.6-4.1)	3.4 (3.2-3.6)	<.001
Melanoma
All patients	31.6 (27.0-36.9)	39.4 (31.1-49.8)	25.4 (20.6-31.3)	.007
Personal history of melanoma
Yes	19.8 (14.3-27.3)	21.3 (12.9-35.3)	18.6 (12.2-28.5)	.69
No	35.2 (29.4-42.0)	44.4 (34.0-57.9)	27.6 (21.7-35.1)	.01

Abbreviations: NNB, number needed to biopsy; PA, physician assistant.

^aP value compares data for dermatologist only with data for PA only.

Discussion

Physician assistants performed more skin biopsies to detect melanoma and nonmelanoma skin cancer than did dermatologists. In addition, PAs were less likely than dermatologists to diagnose melanoma in situ during a skin cancer screening visit.

Our findings are consistent with those of Nault et al,⁵ who found a significantly higher NNB among APPs, primarily nurse practitioners, compared with dermatologists submitting diagnostic specimens for dermatopathologic evaluation. However, our study included more patients and data from nearly 7 times as many biopsies. By focusing on visits coded as skin cancer screenings, we could determine the NNB and the proportion of visits resulting in skin cancer diagnosis. Although few data are available on the NNB for PAs, a large German skin cancer screening initiative, in which dermatologists made the decisions to biopsy or not, reported an NNB of 28 to diagnose 1 case of melanoma,⁶ similar to our mean NNB of 25.4 for dermatologists. However, both are higher than the NNB of 17.4 for dermatologists reported by Nault et al.⁵

The lower detection rate among PAs of melanomas in situ, which are often more challenging to diagnose than invasive melanomas, likely reflects lower clinician sensitivity. Physician assistants and dermatologists had similar detection rates for invasive melanomas and nonmelanoma skin cancers, which tend to be more clinically obvious, suggesting that this difference is not the result of a difference in the risk pool of patients seen by each clinician type. This distinction is further supported by the finding that, in comparison with patients seen by a dermatologist, patients seen by a PA were somewhat less likely to have a history of melanoma but more likely to have a history of skin cancer overall.

In terms of mortality, the consequences of diagnosing melanoma in situ are unclear. However, assuming that some fraction of these lesions will progress to ultimately fatal invasive melanoma, early detection and treatment of melanoma at the in situ stage should be beneficial to the patient in terms of prognosis and decreased morbidity for the requisite surgical intervention. In addition to its likely favorable consequences for patient outcomes, treatment of melanoma in situ is significantly less expensive than treatment of invasive melanoma.⁷

Dermatology is one of the highest employers of APPs in medicine, and this trend is likely to continue, particularly as more dermatology practices are acquired by private equity firms with an obligation to shareholders to maximize profits.⁸ Most procedures performed independently by APPs are diagnostic skin biopsies, suggesting that a large portion of skin cancer diagnosis in the United States is being performed by these practitioners.¹ Measuring the quality of care delivered by practitioners is challenging. The American Academy of Dermatology recommends that APPs should provide care only after a dermatologist has evaluated the patient, made a diagnosis, and developed a treatment plan.⁹ Determining proficiency to practice when no objective measure, such as specialty certification for PAs, exists means that dermatologists who use PAs must take seriously the responsibility to train and supervise them appropriately and monitor the quality of care delivered. We propose that measuring NNB for each clinician may help to objectively assess performance. This undertaking may not be practical for all settings, but if a practice sends all its pathology specimens to a single laboratory, NNB could be estimated by dividing the sum of the number of melanomas, nevi, and seborrheic keratoses biopsied by the number of melanomas biopsied per clinician. This quotient would give some feedback about each clinician’s performance, allowing dermatologists to identify outlier APPs (and dermatologists).

Strengths and Limitations

Strengths of our study include the large number of visits analyzed and the ability to measure NNB and rates of skin cancer detection from screening visits. However, there are some limitations to our approach. We may have missed skin cancer screenings occurring during other visit types. However, this approach was previously validated with sensitivity analyses¹⁰ and missed less than 20% of skin examinations occurring in other visit types. Although we attempted to control for differences in patient risk factors for skin cancer retrospectively by stratifying results by patient history of melanoma, we were not able to capture all risk factors (eg, history of sun exposure, nevus count, and family history of melanoma), and patients who believed they were at high risk for melanoma may have chosen to see a dermatologist rather than a PA. Physician assistants in our system have dermatologists available for consultation; therefore, it is likely that occasionally the decision to perform a biopsy was made by the consulted dermatologist, and this would not be captured because the PA would be listed as the visit clinician. Our findings also may not be representative of all practice settings. Physician assistants at University of Pittsburgh Medical Center–affiliated dermatology offices are likely better trained than most, as they have the opportunity to attend some educational sessions and grand rounds and are provided time and funding to attend educational meetings; thus, we would expect that the differences we observe would only be magnified in other practice settings, such as those in which PAs receive limited training and practice with no supervising dermatologist on-site.

Conclusions

In the age of cost-conscious medicine, it is important to consider more than just clinician salary in determining cost of care. Visits in which skin cancers are missed and/or biopsies are performed on benign lesions owing to lower diagnostic accuracy are low-value visits and increase the potential harm to patients. This information should be factored into policy decisions about scope of practice, hiring decisions, supervision of APPs, and patient decisions about who provides their dermatologic care.