To the Editor: Cutaneous immune-related adverse events (cirAEs) are the most common toxicities of immune checkpoint inhibitor (ICI) therapy and have been found to correlate with significant survival benefits.1,2 However, cirAEs remain largely understudied due to resource demand from manual phenotyping, limiting population-based investigations. We aim to design a rule-based approach using the International Classification of Diseases (ICD), ninth/Tenth revisions, and medications for automatic phenotyping of cirAEs.
We identified a retrospective cohort of 4409 ICI recipients between 2011 and 2020 at Mass General Hospital (MGH) (2534) and Dana-Farber Cancer Institute and Brigham and Women’s Hospital (DFCI/BWH) (1875), among which 422 (16.7%) and 624 (33.3%) patients, correspondingly, visited a dermatologist within 2 years from ICI initiation. Respectively, the median follow-up after ICI initiation was 14 (IQR: 4-35) and 17 (IQR: 6-34) months; the median age at ICI initiation was 66 (IQR: 57-74) and 64 (IQR: 56-72) years old. Two independent reviewers conducted manual chart review to ascertain the presence of cirAEs, according to rash timing, morphology, absence of competing risk factors, histologic confirmation, and response to therapy. There were 373 (16%) and 450 (20%) cirAE cases within 2 years of ICI with median onset time of 58 (IQR: 20-151) and 70 (IQR: 22-176) days from ICI initiation in the 2 cohorts. We designed our rule-based algorithm using the MGH cohort and validated it on the DFCI/BWH cohort.
We extracted data from the Research Patient Data Registry3 and then applied the following approach to identify cirAEs: (1) Include ICD codes of cutaneous events associated with ICIs in literature and expert consensus4 and topical anti-inflammatory medications (Table I) within 2 years after ICI initiation. To reduce the false positive signals, we only included codes given by dermatologists; (2) Create a sliding time window of 3 months with a step size of 15 days. If a patient had more than 1 cutaneous event and more than 1 prescription of medications in any time window, the patient is considered as having a cirAE.
Table I.
Diagnoses and topical anti-inflammatory medications used to identify cirAEs
| Cutaneous event | Total cases§ |
ICD code | |
|---|---|---|---|
| MGH | DFCI/BWH | ||
| Diagnosis | |||
| Acne | 8 | 22 | ICD10: L70; ICD9: 706.1 |
| Alopecia | 0 | 0 | ICD10: L63.0; L63.1; L63.2; L63.8; ICD9: 704.01 |
| Eczema | 45 | 98 | ICD10: L20.9; L20.89; L30.8; L30.9; ICD9: 691.8; 692.9 |
| Erythema multiforme | 5 | 1 | ICD10: L51.0; L51.8; L51.9; ICD9: 695.10; 695.11; 695.12; 695.19 |
| Lichen planus | 6 | 20 | ICD10: L43; L66.1; L44.3; ICD9: 697.0 |
| Mucositis | 6 | 8 | ICD10: K12; ICD9: 528.00 |
| Sweets disease | 0 | 5 | ICD10: L98.2; L88; ICD9: 686.01 |
| Bullous dermatitis | 7 | 15 | ICD10: L12; L10; Q82.8; L13.0; L01.03; L13.8; ICD9: 694.0; 694.2; 694.5; 694.6; 694.8 |
| Pruritus | 28 | 67 | ICD10: L29.8; L29.9; ICD9: 698.8 |
| Psoriasis | 14 | 40 | ICD10: L40; L41; ICD9: 696.0; 696.1; 696.2; 696.8 |
| Rash NOS∗ | 86 | 280 | ICD10: R21; ICD9: 782.1 |
| SJS/TEN† | 2 | 1 | ICD10: L51.1; L51.2; L51.3; ICD9: 695.13; 695.14; 695.15 |
| Urticaria | 1 | 5 | ICD10: L50.0; L50.1; L50.8; L50.9; ICD9: 708.8 |
| Vitiligo | 15 | 32 | ICD10: L80; ICD9:709.01 |
| Maculopapular | 1 | 14 | ICD10: L27.1; ICD9: 693.0 |
| Drug hypersensitivity NOS | 30 | 99 | ICD10: L27.0; T88.7; ICD9: 693.0 |
| Panniculitis | 0 | 1 | ICD10: M79.3; M54.0; L93.2; ICD9: 729.30; 723.6 |
| DRESS‡ | 0 | 0 | ICD10: D72.12 |
| Folliculitis | 6 | 31 | ICD10: L72.9; L73.9; L11.0; L87.0 |
| Exfoliative dermatitis | 0 | 0 | ICD10: L26 |
| Dermatomyositis | 0 | 2 | ICD10: D49.9; M33.0; M33.1; M33.20; M33.90; ICD9: 710.3; 239.9 |
| Hyperpigmentation | 23 | 8 | ICD10: L81.9; ICD9: 709.00 |
| Grover's disease | 0 | 11 | ICD10: L11.1; ICD9: 702.8 |
| Photosensitivity | 1 | 13 | ICD10: L56.8; ICD9: 692.79 |
| Vasculitis | 0 | 1 | ICD10: L95.8; L95.9; D69.0; ICD9: 709.1; 287.0 |
| Sarcoidosis | 0 | 1 | ICD10: D86.3; D86.89; L92.9; ICD9: 135 |
| Scleroderma | 0 | 1 | ICD10: M34.1; M34.2; M34.9; ICD9: 710.1 |
| Rosacea | 11 | 31 | ICD10: L71.8; L71.9; ICD9: 695.3 |
| Lupus erythematosus | 0 | 2 | ICD10: L93.0; L93.1; L93.2; ICD9: 695.4 |
| Seborrheic dermatitis | 19 | 37 | ICD10: L21.9; ICD9: 690.1 |
| Erythema nodosum | 0 | 1 | ICD10: L52; ICD9: 695.2 |
| Acral erythema | 1 | 1 | ICD10: L53.8; ICD9: 695.89 |
| Erythematous condition | 2 | 1 | ICD10: L53.9; ICD9: 695.9 |
| Pityriasis rubra pilaris | 0 | 0 | ICD10: L44.0; ICD9: 696.4 |
| Granuloma annulare | 0 | 2 | ICD10: L92.0 |
| Hyperhidrosis | 2 | 0 | ICD10: R61; L74.519; L74.52; ICD9: 780.8; 705.21; 705.22 |
| Hyperkeratosis | 2 | 3 | ICD10: L85.9; ICD9: 701.9 |
| Keratoacanthoma | 1 | 8 | ICD10: L85.8; ICD9: 701.8 |
| Onycholysis | 1 | 1 | ICD10: L60.1; ICD9: 703.8 |
| Pityriasis rosea | 0 | 0 | ICD10: L42; ICD9: 696.3 |
| Actinic keratosis | 98 | 123 | ICD10: L57.0; ICD9: 702.0 |
| Other | 35 | 86 | ICD10: L85.3; K11.7; ICD9: 706.8; 527.7 |
| Medication type | Medication | ||
|---|---|---|---|
| Topical medication | |||
| Corticosteroids | 672 | 994 | Hydrocortisone, fluticasone, fluocinolone; fluocinonide, mometasone, alclometasone, desoximetasone, halobetasol, betamethasone, clobetasol, triamcinolone, desonide, amcinonide, diflorasone |
| Calcineurin inhibitors | 0 | 0 | Pimecrolimus, tacrolimus |
| Vitamin D analogs | 9 | 14 | Calcitriol, calcipotriene |
cirAEs, Cutaneous immune-related adverse events; ICD, International Classification of Diseases.
NOS: not otherwise specified.
SJS/TEN: Stevens-Johnson syndrome/toxic epidermal necrolysis.
DRESS: Drug reaction with eosinophilia and systemic symptoms.
Total cases collected from dermatology diagnosis.
Our rule-based approach achieved an accuracy of 0.87, positive predictive value (PPV) of 0.83, and negative predictive value of 0.87 using a 3-month sliding window (Table II). We further examined the performance using windows of different sizes and validated on the DFCI/BWH cohort. The cross-institutional validation showed consistent performance with only a mild decrease in the average PPV from 0.81 to 0.69. Furthermore, the results displayed robustness with different window sizes. When also including diagnoses from oncology departments, the PPV decreased by 12% and 5% in the 2 cohorts (Table II).
Table II.
Cross-institutional validation performance of the rule-based cirAE phenotyping
| Department∗ | Sliding window size | Institution | Accuracy† | PPV‡ | NPV§ |
|---|---|---|---|---|---|
| Dermatology only | 1 mo | MGH | 0.86 | 0.79 | 0.86 |
| DFCI/BWH | 0.83 | 0.73 | 0.83 | ||
| 2 mo | MGH | 0.87 | 0.80 | 0.87 | |
| DFCI/BWH | 0.83 | 0.67 | 0.84 | ||
| 3 mo | MGH | 0.87 | 0.83 | 0.87 | |
| DFCI/BWH | 0.83 | 0.68 | 0.84 | ||
| 6 mo | MGH | 0.87 | 0.83 | 0.87 | |
| DFCI/BWH | 0.83 | 0.66 | 0.85 | ||
| Average | MGH | 0.87 | 0.81 | 0.87 | |
| DFCI/BWH | 0.83 | 0.69 | 0.84 | ||
| Dermatology or oncology | 1 mo | MGH | 0.87 | 0.70 | 0.88 |
| DFCI/BWH | 0.83 | 0.68 | 0.84 | ||
| 2 mo | MGH | 0.87 | 0.70 | 0.88 | |
| DFCI/BWH | 0.83 | 0.63 | 0.85 | ||
| 3 mo | MGH | 0.87 | 0.70 | 0.88 | |
| DFCI/BWH | 0.83 | 0.63 | 0.85 | ||
| 6 mo | MGH | 0.87 | 0.66 | 0.88 | |
| DFCI/BWH | 0.83 | 0.61 | 0.86 | ||
| Average | MGH | 0.87 | 0.69 | 0.88 | |
| DFCI/BWH | 0.83 | 0.64 | 0.85 |
The bold values highlight the best performance we achieved using different sliding window size.
cirAE, Cutaneous immune-related adverse event; DFCI/BWH, Dana-Farber Cancer Institute and Brigham and Women’s Hospital.
Department: The diagnosis department that our rule included.
Accuracy: the number of patients with cirAE status identified correctly divided by the number of patients in the cohort.
PPV: positive predictive value.
NPV: negative predictive value.
In summary, we present the first rule-based algorithm for cirAEs' phenotyping. It achieved a reliable performance and can be scaled to enable further cirAE research, using claim databases with diverse populations that can improve the generalizability of research outcomes. A major restriction of this study is that it is subjected to dermatology diagnosis, which limits its generalizability. Moreover, not all cirAEs are defined by a specific ICD code, putting constraints on accuracy. However, our method can provide guidance for natural language processing pipelines that incorporate medical notes to identify cirAEs.
Conflicts of interest
Y.R.S. is an advisory board member/consultant and has received honoraria from Incyte Corporation, Castle Biosciences, Galderma, and Sanofi outside of the submitted work.
Footnotes
Drs Chen and Wan are cofirst authors.
Drs Kwatra and Semenov are co-senior authors.
Funding sources: S.G.K. is supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases, United States of the National Institutes of Health, United States under award number K23AR077073. Y.R.S. is supported in part by the Department of Defense, United States under Award number W81XWH2110819 and by the Dermatology Foundation, United States under the Medical Dermatology Career Development Award. The other authors received no funding for this research.
IRB approval status: Reviewed and approved by Mass General Brigham Institutional Review Board (Protocol # 2020P002307).
Key words: cutaneous immune-related adverse events; ICD codes; immune checkpoint inhibitor; immunotherapy; medication; oncology.
References
- 1.Kalinich M., Murphy W., Wongvibulsin S., et al. Prediction of severe immune-related adverse events requiring hospital admission in patients on immune checkpoint inhibitors: study of a population level insurance claims database from the USA. J Immunother Cancer. 2021;9(3) doi: 10.1136/jitc-2020-001935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Tang K., Seo J., Tiu B.C., et al. Association of cutaneous immune-related adverse events with increased survival in patients treated with anti–programmed cell death 1 and anti–programmed cell death Ligand 1 therapy. JAMA Dermatol. 2022;158(2):189–193. doi: 10.1001/jamadermatol.2021.5476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Nalichowski R., Keogh D., Chueh H.C., Murphy S.N. Calculating the benefits of a research patient data repository. AMIA Annu Symp Proc. 2006;2006:1044. [PMC free article] [PubMed] [Google Scholar]
- 4.Wongvibulsin S., Pahalyants V., Kalinich M., et al. Epidemiology and risk factors for the development of cutaneous toxicities in patients treated with immune-checkpoint inhibitors: a United States population-level analysis. J Am Acad Dermatol. 2021;86(3):563–572. doi: 10.1016/j.jaad.2021.03.094. [DOI] [PMC free article] [PubMed] [Google Scholar]