The validity of diagnostic and treatment codes for actinic keratosis in electronic health records

Dear Editor,

Actinic keratoses (AK) are among the most common dermatologic diagnoses,^{1, 2} reported to cost nearly $1 billion yearly in the United States (US).³ Recent AK research has relied on diagnosis and procedure codes to identify AKs in claims and electronic health record (EHR) data.^{3, 4} However, there has been no investigation of whether these codes accurately identify AKs. Validation of these codes is essential to ensure that using them for research is appropriate and to allow researchers to use single codes (if reliable), rather than more conservative algorithms, simply because a validation has not yet been done.

We conducted a retrospective chart review at the University of Pennsylvania. We identified all office visit encounters with a diagnosis code for AK (International Classification of Diseases[ICD]-9 702.0 or ICD-10 L57.0), and also those with a procedure code for destruction of premalignant lesion (Current Procedural Terminology[CPT] 17000, 17003, or 17004) between July 2008 and June 2018. For diagnosis codes, we determined from clinical notes whether a patient had an AK physically present. If not, we recorded the possible reason that an AK diagnosis was coded (e.g. follow up for prior AK). For procedure codes, we determined whether the procedure was performed on an AK. If not, we recorded the alternate diagnosis. Each encounter was reviewed by a trained medical record abstractor (OGC), with a second review by a dermatologist (MRW) when notes were unclear or when initial review determined that no AK was present or treated.

We calculated a positive predictive value (PPV) separately for diagnosis codes and procedure codes. PPV was defined as the number of encounters with verified AK diagnoses or verified destructions of AKs, respectively, divided by the total number of encounters in each category. A priori, we calculated a sample size of 196 to detect a PPV of 85% with a 95% confidence interval of 80%−90%, chosen as an appropriately large PPV to justify use of single code alone. We reviewed 200 encounters each for diagnosis codes and for procedure codes selected by random number generator. Data were analyzed using Stata 15. This study was approved by University of Pennsylvania Institutional Review Board.

We identified 121,705 encounters with an AK diagnosis code and 87,813 encounters with a premalignant destruction procedure code. The majority of encounters were with dermatology (93.2% of diagnosis code encounters, 95.7% of procedure code encounters) and with patients who were male (57.6% of diagnosis code encounters, 57.8% of procedure code encounters). The mean ages were 68.5 (SD 11.3) and 67.8 (SD 12.8) in diagnosis and procedure code encounters, respectively.

We reviewed 200 randomly selected charts each for diagnosis and procedure codes. Table 1 displays reasons and alternate diagnoses for diagnosis code and procedure code encounters that did not have AKs present or treated. The PPV of having an AK physically present at the time of AK diagnosis code was 90.5% (95% CI 85.6%−94.2%). We investigated whether an algorithm requiring prior AK diagnosis codes would improve the PPV, and we found that it did not. The majority of AK diagnosis codes without an AK physically present (seen in dermatology and non-dermatology encounters) were related to previous AKs. When we expanded our criteria and calculated the PPV of the encounter relating to an AK diagnosis whether or not an AK was physically present (e.g. including follow up visit for prior AK, past medical history of AK), it was 96.5% (95% CI 92.9%−98.6%). The PPV of having an AK treated at the time of a premalignant destruction procedure code was 94.0% (95% CI 89.8%−96.9%). Alternate diagnoses (seen in dermatology and non-dermatology encounters) included warts, seborrheic keratoses, and skin tags.

Table 1.

Reasons and alternate diagnoses for diagnosis code and procedure code encounters that did not have AKs present or treated

AK diagnosis code encounters without AK physically present (n=19)			Premalignant destruction procedure code encounters without treatment of an AK (n=12)
	n (%)			n (%)
Follow up visit for previous AK	7 (36.8)		Warts (e.g. verruca vulgaris, condyloma)	5 (41.7)
Past medical history of AK	5 (26.3)		Seborrheic keratoses	2 (16.7)
Actinic damage described without discrete AKs	1 (5.3)		Skin tags	2 (16.7)
AK not mentioned in note	5 (26.3)		Other	1 (8.3)
No clinical note present	1 (5.3)		Procedure not mentioned in note	2 (16.7)

In this retrospective chart review, we assessed the validity of the diagnosis codes and premalignant destruction procedure codes for AK in EHR data. PPV was chosen as the outcome because this is a critical measure for claims and EHR research.^{5, 6} This study is limited in that it uses EHR data from a single institution in the US and may not be generalizable to other EHR or claims data. However, this is consistent with similar studies, as many large claims and EHR datasets do not provide chart access and so validation cannot be done directly.

Based on the PPVs of greater than 90%, our results indicate that both the ICD codes for AK diagnosis and the premalignant destruction CPT codes for AK treatment are valid for use in AK and skin cancer research using claims and EHR data, which is crucial for prior and future studies utilizing these data sources.