Use of Patient-Reported Outcomes in Risk Prediction Model Development to Support Cancer Care Delivery: A Scoping Review

Abstract

PURPOSE

The integration of patient-reported outcomes (PROs) into electronic health records (EHRs) has enabled systematic collection of symptom data to manage post-treatment symptoms. The use and integration of PRO data into routine care are associated with overall treatment success, adherence, and satisfaction. Clinical trials have demonstrated the prognostic value of PROs including physical function and global health status in predicting survival. It is unknown to what extent routinely collected PRO data are used in the development of risk prediction models (RPMs) in oncology care. The objective of the scoping review is to assess how PROs are used to train risk RPMs to predict patient outcomes in oncology care.

METHODS

Using the scoping review methodology outlined in the Joanna Briggs Institute Manual for Evidence Synthesis, we searched four databases (MEDLINE, CINAHL, Embase, and Web of Science) to locate peer-reviewed oncology articles that used PROs as predictors to train models. Study characteristics including settings, clinical outcomes, and model training, testing, validation, and performance data were extracted for analyses.

RESULTS

Of the 1,254 studies identified, 18 met inclusion criteria. Most studies performed retrospective analyses of prospectively collected PRO data to build prediction models. Post-treatment survival was the most common outcome predicted. Discriminative performance of models trained using PROs was better than models trained without PROs. Most studies did not report model calibration.

CONCLUSION

Systematic collection of PROs in routine practice provides an opportunity to use patient-reported data to develop RPMs. Model performance improves when PROs are used in combination with other comprehensive data sources.

BACKGROUND

Patient-reported outcomes (PROs) are data submited directly by patients without clinical interpretation.¹ Previous studies have demonstrated prognostic value of PROs.^2-5 The use of PROs to develop risk prediction models (RPMs), exclusively or in combination with other data elements, is a relatively new and promising idea.^6,7 Traditionally, the use of PROs has been limited to symptom surveillance in clinical trials,^8,9 to guide symptom management at or between clinic visits,¹⁰ and to assess quality of care.¹¹ These use cases have been well-documented; however, the use case for PROs to develop RPMs is lacking. RPMs are increasingly being developed to support clinical decision making in oncologic care.^12,13 Most contemporary RPMs in oncology are developed using data from clinical trials, disease registry, or the electronic health record (EHR).^13-17 Examples of RPM using clinical data include a model that assesses the risk of potentially preventable acute care visits within 6 months of initiating antineoplastic therapy¹⁰ and various prostate cancer-specific calculators and nomograms.^18,19 Several RPMs have been developed but few are integrated into routine practice.^7,20

Systematic collection of PROs in routine clinical practice has been lagging.²¹ The lack of integration with EHRs is one of the major barriers.²² To overcome the integration barrier, EHR vendors have begun to offer capabilities to integrate PRO data collection,^23,24 enabling health systems to collect PRO data in a more structured and systematic manner.²¹ EHR integration of PROs in routine practice opens an opportunity to extend the use of PROs as it allows an additional stream of potentially prognostic data to develop RPMs. Additionally, from a patient-centered care perspective, PROs enable the incorporation of patients' data in the model development process, potentially making RPMs more responsive to patient's symptom burden and may offer improved predictions of outcomes.

Despite the potential for PROs to improve model performance, it is unknown to what extent oncologic RPMs have used PRO data to predict an outcome of interest. The objective of this scoping review was to assess the use of PROs to train RPMs in cancer care. This review summarizes the current landscape of using PROs in the model development process, including the use of PROs as predictors in the model training process and the deployment of prediction models in routine cancer care delivery.

METHODS

We conducted a scoping review using the methodology put forward by JBI, formerly known as the Joanna Briggs Institute^25,26 to assess these three research questions:

1. What are the characteristics of RPMs developed using PROs as predictors?

2. How are PROs used in the development of RPMs?

3. How are RPMs evaluated and compared?

Data Sources

We searched four databases including MEDLINE (Ovid), Embase (Elsevier), CINAHL Complete (EBSCO), and Web of Science Core Collection (Clarivate) in March-May 2023.

Search Strategy

We sought studies that incorporated PROs to develop RPMs. The search strategy was developed by a trained and experienced medical librarian (C.W.) in consultation with the two reviewers (R.P. and S.D.). The detailed search strategy is included in Appendix Table A1. The search was developed using keywords and controlled vocabulary for cancer and oncology care in combination with machine learning, quality of life, and patient-reported outcomes.

Eligibility Criteria

We adopted the US Food and Drug Administration's definition of PROs as symptom reports submitted directly by patients (or their proxies) without interpretation from anyone on the clinical team.¹ We defined RPMs as mathematical equations or machine learning algorithms commonly used in oncology to predict the probability of or risk of an outcome of interest.²⁷ The following three inclusion criteria were used to identify eligible studies: (1) identified study used PROs as predictors to train RPMs; (2) RPMs described in the study were developed to support symptom assessment or management in cancer care or used to predict patient outcomes related to cancer care including survival, acute care use, quality of life etc; (3) and the identified study defined predictors, outcomes of interest, specified modeling algorithms, and reported model performance. We included studies that followed some commonly recommended strategies including split sample development and validation or testing of model performance on an external data set.²⁷ The following exclusion criteria were used: (1) studies developed prediction models without PRO data as predictors in the model training process or (2) developed or deployed models using PROs in nononcologic settings or (3) did not perform model training, testing, or validation. We also excluded editorials, perspectives, meta-analysis, scoping or systematic reviews, conference proceedings, or dissertations (Appendix Table A2).

Selection of Sources of Evidence

The identified studies were uploaded into Rayyan²⁸ and Covidence.²⁹ The selection process involved two stages: (1) a title and abstract screening in Rayyan and (2) full-text review and data extraction in Covidence. All titles and abstracts were screened independently by two reviewers (R.P. and S.D.) and checked for duplicates. Abstracts were considered for a second round of review in Covidence if they met the inclusion criteria. Reasons for exclusion were recorded. Full-text review of studies meeting inclusion criteria were then conducted by the reviewers. The Preferred Reporting Items for Systematic reviews and Meta-Analyses³⁰ flow diagram (Fig 1) illustrates the search process with the numbers of studies included/excluded along each step of the search process.

FIG 1.

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; PROs, patient-reported outcomes; RPM, risk prediction model.

Data Extraction Process

Data were extracted by reviewers and discussed jointly during analysis. The following data were extracted: details on the citation including authors, year of publication, country of origin, study setting, methodology including study design, and outcomes. Data were also extracted to identify the aspects of PROs and how they were used in the model training process. Furthermore, to assess how the RPMs were developed, the following data were also extracted: learning algorithms, model performance, sample sizes, model comparison, model validation or testing, and the clinical applicability of the models. Extracted data were entered into Covidence independently using a template developed for the review. Both reviewers assessed the extracted data, and any disagreements were resolved through discussion.

RESULTS

The search retrieved 1,773 records, which were deduplicated in EndNote using the method outlined by Bramer et al.³¹ An additional study was identified via hand searching. The team screened 1,254 records, of which 1,063 records were initially excluded. The most common reasons for initial exclusion were (1) studies were unrelated to cancer or were not conducted in humans; (2) PROs were not used as predictors in the model training process; (3) did not perform model training, testing or validation; and (4) were methodological, comparative effectiveness or epidemiological analysis without predicting an outcome of interest. One hundred ninety one reports were sought for retrieval, of which 168 were further excluded. Most common reasons for exclusion at this stage were criteria for RPM not met and studies were either descriptive, psychometric, protocol papers, abstracts only, or duplicate studies. Finally, 23 were assessed in full text for eligibility. Eighteen studies met inclusion criteria and underwent the final extraction process (Fig 1).

The description of studies and the prediction models trained are shown in Tables 1 and 2. Briefly, 12 studies^{32-36,39-41,44,45,47,49} were retrospective studies based on data from cancer registry, EHR, or population databases while six studies^{37,38,42-44,46} performed retrospective analysis of prospectively collected cohort data. Most studies were published between 2018 and 2023 except for one study³⁸ published in 2010. Most studies were conducted in the United States,^{32,34,40,41,44,46,48,49} followed by Canada,^38,39,45,47 the Netherlands,^36,43 Denmark,^33,35 the United Kingdom,³³ Singapore,³⁷ and Spain.⁴² Twelve studies^{32,33,35,36,38-40,42,43,45,47,48} were multicenter, provincial, national, or multinational. Six studies^{34,37,41,44,46,49} were single-center studies. Seven studies^{32,33,36,39,43,46,48} included patients with a single cancer type while 11 studies included patients with multiple cancer types.^{34,35,37,38,40-42,44,45,47,49} Sixteen studies specified treatment settings, including five studies in supportive or palliative setting,^{35,37,38,46,49} five in post-treatment or survivorship,^{32,33,36,43,48} and six in active treatment.^{34,40-42,44,47}

TABLE 1.

Characteristics of Selected Studies and PROs Used in Model Training

Study	Country	Outcomes	Context	Setting	PROs Used for Model Training	Aspects of PROs Used	PRO Scores Use
Alam et al32	The United States	Mortality among patients with renal cell carcinoma	Retrospective analysis of a prospective registry	Patients with renal cell carcinoma	SF-36 and VR-12, MCS, PCS	Mental and physical health QOL	QOL scores categorized as discrete variables
Battersby et al33	The United Kingdom, Denmark	Risk of postoperative bowel dysfunction (LARS score) at least 1 year after surgery	Retrospective cohort study involving multiple sites across two countries	Patients with rectal cancer who underwent anterior resection and completed the LARS score	EORTC QLQ, LARS score, Wexner incontinence scores	LARS score	Preoperative LARS scores classified patients into 3 severity groups
Giri et al34	The United States	Survival (1-year)	Retrospective analysis of a prospective registry	Adult patients with cancer in a registry	PROMIS (1 question)	Self-rated global health	Item dichotomized as poor (poor and fair) and good (good, very good, excellent)
Hansen et al35	Denmark	1-month or 1-week mortality	Retrospective analysis of a palliative care registry	Adult patients with cancer on palliative care	EORTC QLQ-C15-PAL	Symptoms and function/QOL	Scores converted into 0-100 scale
Hasannejadasl et al36	The Netherlands	Frequency of erections and erectile dysfunction at 1-2 years postdiagnosis	Retrospective analysis of a prospective study and cancer registry data	Patients with localized or locally advanced prostate cancer	EPIC-26	Frequency of erectile dysfunction	Frequency of erections categorized into binary never had an erection v the rest at year 1 and year 2
Hum et al37	Singapore	30- and 90-day mortality	Prospective cohort study from single institution	Patients with incurable cancer receiving palliative care	ESAS (patient), PPS V2 (observer)	Functionality, symptom burden, comorbidity burden	ESAS summative scores
Martin et al38	Canada	Survival after clinical assessment	Prospective study Provincial databases and registries	Patients with advanced cancer on palliative care	ESAS, PG-SGA	Nutritional and Karnofsky performance status information	Change in weight
Noel et al39	Canada	14-day ED use or hospitalization after symptom assessment	Retrospective study of provincial databases and registries	Patients with head and neck cancer	ESAS	9 cancer-associated symptoms	Symptom scores treated as a separate variable
Parikh et al40	The United States	180-day mortality within index encounter	Retrospective study of routine care data from multiple institutions	Adult oncology patients	PRO-CTCAE and PROMIS	10 symptoms from PRO-CTCAE; 1 QOL question from PROMIS	Standard symptom scores (Likert scale)
Peterson et al41	The United States	180-day ED use or hospitalization after treatment	Retrospective study of routine care data from single institution	Patients receiving chemotherapy	PROMIS	12-item PROMIS and global mental and physical health scores	PRO scores used in subanalysis
Quintana et al42	Spain	Having a reoperation or readmission during the first year of follow-up after the main surgical intervention	Prospective cohort study using routine care data from multiple institutions	Patients with surgically resectable colon or rectal cancer	HADS, EuroQOL-5D, EORTC QLQ, BI, DUFSSQ	Generic QOL, anxiety, depression screening tool, social support network	Change score
Revesz et al43	The Netherlands	12-month postoperative QOL scores	Prospective cohort studies from multiple institutions	Patients with stage I-III colorectal cancer enrolled in two longitudinal cohort studies	EORTC QLQ- C30	Seven domains (subscales): global QOL, cognitive, emotional, physical, fatigue, role, and social functioning	Summative score (0-100 points): Higher scores meant better HRQOL except for fatigue where higher scores meant worse fatigue
Rossi et al44	The United States	30-day postsurgical outcomes (complications and unplanned readmissions)	Prospective pilot studies using routine care data from single institution	Surgery patients with lung and GI cancer	MDASI and PGHD (VivoFit)	13 cancer symptoms	Change score (a movement of 1.2 points considered clinically meaningful)
Seow et al45	Canada	Mortality from date of diagnosis (up to year 4)	Retrospective study of provincial databases and registries	Adult patients diagnosed with all cancer types	ESAS, ECOG functional score (patient-reported)	Symptoms and performance status	Individual symptom score
Sidey-Gibbons et al46	The United States	180-day mortality among women with ovarian cancer	Prospective study using routine care data from single institution	Patients with ovarian cancer	MDASI-OC, EQ-5D, CESD, GAD-7, FACT-O	Symptom specific, QOL, depression, and anxiety	Change score
Sutradhar et al47	Canada	ED visits within 7 days of assessment	Retrospective study of provincial databases and registries	Patients with cancer	ESAS	9 cancer-associated symptoms	Scores categorized into none, mild, moderate, severe
Van Dyk et al48	The United States	Clinically meaningful neurocognitive decline in patients with breast cancer from presystemic therapy to 24 months	Retrospective study of multisite prospective cohort	Breast cancer survivors 60+ years with nonmetastatic disease	FACT-Cog (PCI-18), TIADL, CES-D, STAI, FACT-F	Cognitive functioning and neurocognitive data, depression, anxiety, fatigue	Change scores (baseline to 12 months) for 59 neurocognitive measures and 3 behavioral measures
Xu et al49	The United States	180-day mortality after clinic visit	Retrospective study of routine care data from single institution	Patients with advanced cancer seen in an outpatient setting	ESAS	PSS, GDS	One-time composite scores

Abbreviations: BI, Barthel Index; CES-D, center for epidemiological studies depression; DUFSSQ, Duke-UNC Functional Social Support Questionnaire; ECOG, Eastern Cooperative Oncology Group score; ED, emergency department; EORTC QLQ, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (Core); EORTC QLQ-C15-PAL, Shortened version of EORTC QLQ-C30 adapted for palliative care; EPIC-26, Expanded Prostate Cancer Index Composite-26; ESAS, Edmonton Symptom Assessment System; EuroQOL-5D/EQ-5D, Euro QOL 5-Dimensional; FACT-Cog, Functional Assessment of Cancer Therapy-Cognition; FACT-F, Functional Assessment of Cancer Therapy—Fatigue; FACT-O, Functional Assessment of Cancer Therapy—Ovarian; GAD-7, General Anxiety Disorder-7; GDS, Global Distress Score; HADS, Hospital Anxiety and Depression Scale; HRQOL, health-related quality of life; LARS, low anterior resection syndrome; MCS, mental component summary; MDASI, MD Anderson Symptom Inventory; MDASI-OC, MD Anderson Symptom Inventory for Ovarian Cancer; PCS, physical component summary; PGHD, patient-generated health data; PG-SGA, Patient-Generated Subjective Global Assessment; PHS, Physical Symptom Score; PPS V2, Palliative Performance Scale; PRO-CTCAE, Patient-Reported Outcomes-Common Terminology Criteria for Adverse Events; PROMIS, Patient-Reported Outcomes Measurement Information System; PSS, Psychosocial Distress Score; QOL, quality of life; SF-36, Short Form Health Survey-36; STAI, Spielberger State-Trait Anxiety Inventory; TIADL, timed instrumental activities of daily living; VR-12, Veterans RAND 12-Item Health Survey.

TABLE 2.

Characteristics of Prediction Models Used in Selected Studies

Study	Learning Method	Model Performance	Sample	Model Comparison	Discrimination Performancea	Validation	Clinical Applicability
Alam et al32	CPH	C-Statistic, AIC	Training: 1,494 Testing: 479	Comparison between models trained with and without PRO data	C-Statistic 0.72 (with PRO) 0.70 (without PRO)	External testing on a different registry	Addition of QOL metrics into prognostic models could improve risk stratification
Battersby et al34	Linear regression with LASSO	LASSO estimations, Mallow's CP, Somer's D test Harrel's C-statistic, calibration plots	Development: 463 (the United Kingdom) Validation: 938 (Denmark)	Comparison between development and validation cohorts (mean LARS scores)	Somer's D 0.26 (the United Kingdom) 0.28 (Denmark) C-Statistic 0.615 (the United Kingdom) 0.625 (Denmark)	External validation on a different cohort	Nomogram and online preoperative LARS score to predict bowel dysfunction. Recommendation for patient management
Giri et al34	CPH	AUC, sensitivity, specificity, PPV, NPV, hazard ratio, odds ratio	708 older adults but 603 underwent GA and reported on SRH at initial consult visit	Model performance compared between poor and good self-reported health (SRH)	AUC (frailty) 0.82 (derivation) 0.71 (validation)	External validation on a different registry and intervention trials	Single-item SRH may identify vulnerable older adults who require further evaluation with a GA
Hansen et al35	LR, CPH, Other	AUC	Development: 30,969 Validation: 5,508	Model performance compared with and without symptoms or problems	AUC 0.77 (clinical data + PRO) 0.63 (clinical data only) 0.76 (clinical + physical function) 0.74 (physical function) (1-month palliative care)	Validation on a separate data set from a different year	Physical function was a stronger predictor of survival than all clinical variables
Hasannejadasl et al36	LR with recursive feature elimination	AUC, sensitivity, specificity, overall accuracy, recalibration in the large (updating intercept); calibration plots	1-year cohort: 848 2-year cohort: 670	Model performance compared using year 1 v year 2 data	AUC 0.84 (year 1) 0.81 (year 2)	External validation on data set gathered at a different location	Nomograms created for 1-year and 2-year models
Hum et al37	CPH	AUC, sensitivity, specificity, PPV, NPV	Training: 560 Validation: 280	Model comparison with and without PPS data	AUC 0.69-0.75 (30-day survival) 0.64-0.68 (90-day survival)	Six-fold cross-validation	Prediction tool may help identify patients earlier for supportive, palliative care
Martin et al38	CPH	C-Statistic	Training: 1,164 Validation: 603	Model comparison performed between base and full model	C-Statistic 0.88 (base model) 0.87 (full model)	External validation on data set gathered at a different location	Model may be useful in planning for care for patients referred to palliative care
Noel et al39	LR with LASSO, RF, DT, ANN, GBM, kNN	AUC, sensitivity, specificity, calibration plots	Training: 9,409 Testing: 2,352	Model comparison between PRO only model and full model	AUC (GBM models) 0.80 (full model) 0.72 (PRO only)	5-fold CV and 4:1 random split between training and testing sets	Protype prediction model may be used in EHR to risk-stratify patients with head and neck cancer
Parikh et al40	LR with LASSO	AUC, AUPRC, TPR, FPR, decision curves (net benefit)	Academic: 8,555 Community: 3,795	Model comparison among PRO only, PRO + EHR, and EHR-only data sets in two cohorts	AUC (academic practice) 0.86 (EHR + PRO) 0.82 (EHR) 0.74 (PRO) AUC (community practice) 0.89 (EHR + PRO) 0.86 (EHR) 0.77 (PRO)	70/30 random split between training and testing	Integrating PROs and EHR data into routine prognostic algorithm improves risk prediction
Peterson et al41	LR with LASSO, Ridge, Elastic Net, RF, NN, GBM, SVM, kNN etc	AUC, calibration plot	8,439	Model comparison using EHR data and separately with PRO features	AUC (main analysis) 0.806 (Ensemble) 0.78 (LASSO) AUC (subanalysis) 0.735 (EHR) 0.736 (EHR + PRO)	Grid search with 10-fold CV. 80/20 random split between training and test sets	Models may help with risk stratification
Quintana et al42	LR	AUC, Hosmer-Lemeshow test for calibration	Patients with rectal cancer = 708, patients with colon cancer = 1,817	Model comparison between patients with colon and rectal cancers	AUC (readmission) 0.642 (colon) 0.694 (rectal) AUC (reoperation) 0.686 (colon) 0.646 (rectal)	50/50 split between derivation and validation, bootstrap resampling	Models may help develop interventions to reduce readmissions and reoperations
Revesz et al43	LR	AUC, sensitivity, specificity, R2, Brier score, Calibration in the large, H-S test, calibration plots	EnCoRe study = 276 COLON study = 1,320	Comparison between original and updated models	AUC 0.77-0.85 (original models) 0.78-0.85 (updated models) (AUCs for 7 domains reported)	Original models externally validated and updated in the study	Models may help identify colorectal cancer survivors of low HRQOL early
Rossi et al44	LR with LASSO or Ridge Regularization	AUC, AUPRC, other	47 completed ePROs at least once; 40 wore wearable devices (step counter)	Comparison among several models with and without PROs	AUC 0.62 (baseline) 0.70 (MDASI) 0.72 (steps) 0.74 (MDASI + Steps)	Leave-one-out cross-validation	PROs and PGHD may complement clinical data to predict postoperative outcomes
Seow et al45	CPH	C-Index, grouping patients into deciles and calibration plots	255,494	No direct comparison but models refit annually	C-Index: 0.90 (year 0) 0.91 (year 1 to year 4)	60/40 random split. Validated on test set	Potential to be an online tool to predict survival over time
Sidey-Gibbons et al46	LR, DT, NN, XGBM/GBM, SVM, other	AUC, sensitivity, specificity, other	243 women	Comparison among several models	AUC 0.73 (GLM) 0.71 (GAM) 0.72 (regression trees) 0.61 (boosted trees) 0.74 (MARS) 0.71 (SVM) 0.76 (NN)	10-fold CV to tune models. 2:1 random split	Models may inform end of life clinical decision making and palliative care utilization
Sutradhar et al47	Logistic regression with GEE for clustering	Sensitivity, specificity, accuracy, and discrimination (AUC), calibration plots	Training: 170,092 (80%) Testing: 42,523 (20%)	Comparison between models trained with and without PRO data	AUC 0.74 (EHR + PRO) 0.70 (EHR)	80/20 random split. Models validated on test set	Feasible to use administrative data to predict ED use and prompt timely interventions
Van Dyk et al48	LR (LASSO, ElasticNet), RF, GBM	AUC, sensitivity, specificity	Intervention arm = 435 Control arm = 441	Comparison among several models	AUC 0.74 (LASSO) 0.95 (GBM) 0.798 (Elastic net) 0.56 (RF)	5-fold cross-validation	Models may support screening for cognitive decline and referrals for further evaluation
Xu et al49	LR, NN, XGBoost, SVM, kNN, MARS	AUC, sensitivity, specificity, PPV, NPV, calibration plots	Training set = 504, testing set = 126	Top 4 models evaluated against each other	AUC 0.69 (XGBoost) 0.67 (NN) 0.66 (GLM)	3 repeated 10-fold CV. Selected 4 algorithms with best CV performance. 4:1 random split	Models trained on PRO data may promote goal-concordant end-of-life care

Abbreviations: AIC, Akaike Information Criteria; ANN, artificial neural network; AUPRC, area under the precision recall curve; C-Index/Statistic, concordance index; CPH, Cox proportional hazards; CV, cross-validation; DT, decision trees; ED, emergency department; EHR, electronic health record; FPR, false-positive rate; GA, geriatric assessment; GBM, gradient boosting machines; GEE, generalized estimating equation; H-S, Hosmer-Lemeshow Test; KM, Kaplan-Meier Curves; kNN, k-Nearest Neighbors; LARS, Low Anterior Resection Syndrome Score; LASSO, least absolute shrinkage and selection operator; LR, logistic regression; NN, neural nets; NPV, negative predictive value; PHS, Physical Symptom Score; PPS, Psychosocial Distress Score; PPV, positive predictive value; PROs, patient-reported outcomes; QOL, quality of life; RF, random forests; SRH, self-reported health; SVM, support vector machine; TPR, true-positive rate; XGBM, extreme gradient boosting machines.

^aDiscriminative performance in the testing set (as applicable).

PROs as Predictors

Our review found that several instruments were used to collected patient-reported outcomes data. Most studies used cancer-specific instruments, including the Edmonton System Assessment System (ESAS), the European Organization for Research and Treatment of Cancer Quality of Life of Cancer Patients (EORTC-QLQ), and the MD Anderson Symptom Inventory (MDASI). A few studies used the Patient-Reported Outcomes Measurement Information System (PROMIS) and other generic quality-of life instruments. Of the 18 studies included in this review, six studies^{37-39,45,47,49} used the ESAS instrument, four studies^33,35,42,43 used EORTC-QLQ, three studies^34,40,41 used PROMIS items, and two studies^44,46 used the MDASI questionnaire while the remaining three studies used instruments including the Expanded Prostate Cancer Index-Composite-26,³⁶ Functional Assessment of Cancer Therapy-Cognitive Function,⁴⁸ or SF-36.³² Seven studies^{32,36,39-41,45,47} used individual symptom scores as discrete variables, five studies^{38,42,44,46,48} used scores that compared status at two or more time points (ie, change scores) and four studies^35,37,43,49 used one-time composite or summative scores. Two studies^33,34 created risk or severity groups based on a cutoff or threshold. One study⁴⁴ used patient-generated health data (ie, step counter) as an additional predictor, and another study⁴⁸ used various cognitive functioning items in their models. Nine studies^{32,35,37-41,44,47} compared the performance against models trained using EHR data alone or PRO and EHR data combined. Similarly, three studies^33,36,46 used cancer-specific functional patient-reported outcome measures (PROMs), and the rest used generic PROMs to collect patient-reported data. Cancer diagnosis, cancer-related symptoms, quality of life, mental function, physical function, and performance-related questions were widely used predictors.

Outcomes and Clinical Implications

Most studies predicted mortality or assessed survival after surgery or systemic therapy initiation. Seven studies^{32,35,37,40,45,46,49} predicted mortality while two studies^34,38 assessed survival. Among studies assessing postoperative complications, two assessed reoperation or readmissions,^42,44 and one study each assessed postoperative quality-of-life scores,⁴³ erectile dysfunction,³⁶ or bowel dysfunction.³³ Three studies^39,41,47 assessed post-treatment emergency department use or hospitalization while one study⁴⁸ each assessed neurocognitive decline. One study³⁵ found that physical function was a stronger predictor of survival than all clinical variables, and another study³⁴ found that single-item self-reported health may identify vulnerable older adults who might require further evaluation. Among the studies that discussed clinical implications of their prediction models, discussions focused on developing risk stratification strategies, developing novel interventions, or using PROs to improve overall predictive performance of RPMs to support patient care were prominent.

Using prediction models trained on PRO data to risk-stratify patients for early intervention including identification of patients for supportive or palliative care,^37,38 identification of cancer survivors of low health-related quality of life for early intervention⁴³ and screening for cognitive decline and referrals for further evaluation⁴⁸ were key clinical implications highlighted in the studies. Development of interventions to support patient care was also mentioned including interventions to reduce readmissions and reoperations,⁴² informing end-of-life clinical decision-making and palliative care use,⁴⁶ and promoting goal-concordant end-of-life care.⁴⁹ Other studies discussed using PROs to improve predictive performance, including the integration of PROs and other patient-generated health data into EHR data into routine prognostic algorithm to improve risk prediction.^32,35,40,44

Modeling Algorithms

Most studies fit logistic regressions^35,42,43,47 or logistic regression with L1 (LASSO) or L2 (Ridge), ElasticNet regularization, or recursive feature elimination.^{35,36,39-41,44,47-49} Five studies^{32,34,35,37,38} fit Cox proportional hazards models to predict time to event outcomes. For nonlinear models, four studies trained variations of Neural Nets^39,41,46,49 and five studies used various boosting algorithms including Gradient Boosting Machines or eXtreme Gradient Boosting.^{39,41,46,48,49} Similarly, three studies each used Random Forest,^39,41,48 k-Nearest Neighbors,^39,41,49 or Support Vector Machines,^41,46,49 and two studies used decision-trees.^39,46

Model Performance

Discriminative performance of models was assessed using AUC^{37,39-44,47-49} or the Concordance Index (C-Index).^32,33,38,45 Similarly, studies also assessed sensitivity, specificity, positive predictive values, negative predictive values^{34,36,37,39,40,46-49} and/or R-Squared, Brier Score⁴³ among other test statistics. Model calibrations were discussed in eight studies that visually assessed calibration using calibration plots^{33,36,39,41-43,45,47,49} or the Hosmer-Lemeshow test or used calibration in the large to assess mean calibration.^42,43

Model Performance Comparison

The modeling approaches varied across studies to provide a valid head-to-head comparison of model performance. Eight studies compared performance between models developed with PROs versus without PROs or EHR data only versus PRO and EHR data combined.^{32,35,37,39-41,44,47} The discriminative performance of PRO only models were around 0.70, 0.72, and 0.74,^40,41,49 while the discriminative performance of models with PRO and EHR data combined ranged from 0.72 to 0.87.^{32,35,38-41,47} The addition of PRO data to EHR data improved model performance from 0.70 to 0.74,⁴⁹ 0.74 to 0.86,⁴¹ and from 0.63 to 0.77.³⁵

Similarly, two studies compared performance between the original versus updated models with a range of AUCs reported (Table 2).^38,43 One study³³ tested the performance of a model developed in another country, yet another study⁴² compared models between patients with colon and rectal cancer with similar and modest AUCs. Performances of different modeling approaches were compared in four studies^41,46,48,49 while three studies^36,37,45 performed no direct comparison between models but assessed model performance temporally. One study⁴⁴ compared performance between patient-reported symptoms versus patient-generated data (step counts) against the baseline model with AUCs ranging from 0.62 for baseline model to 0.74 for symptoms and steps count model.

Model Validation and Testing

Seven studies^{39-42,45,46,49} randomly split data between training, validation, or testing sets with most studies using k-fold cross-validation for model tuning or testing purposes.^{37,39,41,44,46,48,49} Seven studies^32-36,38,43 used external validation with previously unseen data collected at a different location or at a different time. Most studies briefly discussed clinical applicability of models to support care delivery, if and when deployed in the clinical settings. Two studies^33,36 discussed developing nomograms or online tools to predict sexual or bowel function at a future time point.

DISCUSSION

This scoping review synthesized 18 studies to assess the landscape of prediction model development using patient-reported outcomes in oncology. The findings provide an overview on the types of studies, modeling approaches, assessment of model performance, and clinical implications of RPMs. First, we found that most studies conducted retrospective analyses of prospectively collected cohort, cancer registry, or other population databases. Second, we found PROs can be predictive, but the predictive performance depends on the PRO instrument being used and on the outcome being predicted. In most cases, PROs improved model performance when incorporated with demographic and or clinical data.

Third, we observed wide variations in how PRO scores are used in the modeling process. For example, some studies used individual symptom scores, some created risk or severity groups, and others used one-time composite or summative scores or compared scores at two or more time points. We further observed a wide range of model validation and calibration techniques used in the model development process. Most studies used cross-validation or bootstrapping for internal validation, while only a few conducted external validations on unseen data collected at a different time or place. Furthermore, we found that most studies performed head-to-head comparison between modeling approaches, but very few performed head-to-head comparison of models and tested on external samples. Studies that performed head-to-head comparison between modeling approaches and tested models on external data sets are likely to have high internal and external validity.^27,50 Furthermore, we found that most studies assessed discriminative performance (AUC or C-index) but did not provide information on how model calibration was assessed, if at all. This is a severe limitation of prediction models in oncology as model calibration relates to the accuracy of risk predicted on an individual patient. Discrimination is an important aspect of model performance; however, the accuracy of the prediction is also important in RPMs.^51,52

Symptom surveillance using PROs are increasingly being used to support active symptom management in routine practice. Despite the evidence to suggest PROs are associated with improved survival, symptom control, and overall treatment success,^8,53-55 key clinical questions have yet to be fully answered. First, are patient-reported symptoms good predictors of patient outcomes? We found most models trained using patient-reported symptom data performed marginally better than models trained on clinical data only. By contrast, systematic reviews have documented the prognostic value of overall physical function status and global health status, in models predicting overall survival.^4,56 Future studies should assess whether physical function and global health status are more predictive of patient outcomes compared with common symptoms.

Second, should prediction models trained on PRO data be integrated within the EHRs to support patient care? This question has yet to be answered as most prediction or prognostic models (trained with or without PROs) are rarely developed with future clinical use as an explicit consideration⁷ or once developed, have rarely been implemented in routine clinical practice.⁵⁷ Furthermore, whether PROs alone are adequately predictive and whether RPMs trained on PROs can substantially improve outcomes in clinical care remain to be seen. Future studies should investigate how prediction models using PROs can help improve patient outcomes. Future studies should also assess the best outcomes to consider for RPMs and which PRO instruments are more predictive of key outcomes of interest.

There are several limitations to this scoping review. First, although our search strategy was comprehensive, there is a possibility that we may not have captured all relevant literature on this topic, and therefore, we may have missed studies that included PROs in the model training process. Second, there are differences in terms used in traditional statistics and machine learning fields; thus, potentially eligible studies using different terminologies may not have been identified. Third, our search was limited to peer-reviewed articles in English and thus studies of PROs in RPMs in other languages were excluded. Fourth, we did not perform the risk of bias in our review; hence, we are not able to assess the extent of bias associated with patient populations included in or excluded from the modeling studies, biases associated with statistical or modeling approaches, and algorithmic or other biases associated with study design. Despite these limitations, this review focused on studies that split data into training, validation, or testing sets as previous reviews included studies with or without internal or external validation of their results, limiting the robustness of the models' discriminative performance in unseen data. Furthermore, we included studies that used diverse data sources including clinical trials, longitudinal or cross-sectional studies, and registry or EHR data, making the review relevant to those interested in using diverse data sources for model development.

In conclusion, PROs enable the incorporation of patient data in the model training process, making prediction models more patient-centered. Assessing the prognostic value of PROs is important to extending the use of PROs beyond the traditional use cases. Our findings indicate that predictive performance improves when PROs are combined with other comprehensive data sources.

APPENDIX

TABLE A1.

Search Strategy

Database (Platform)	Search	Results
Medline (Ovid)	1. exp neoplasms/OR exp medical oncology/OR exp oncology nursing/OR exp Chemoradiotherapy/OR exp “chemotherapy, adjuvant”/OR Drug Therapy/OR exp radiotherapy/OR exp Cancer patients/OR (Cancer* OR carcinoma* OR Neoplas* OR Tumo?r OR tumor* OR tumour* OR Malignan* OR Oncolog* OR Leuk?emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy).ti,ab,kf,kw 2. exp patient reported outcome measures/OR ((patient* adj3 (data OR outcome OR symptom* OR self) adj3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”).ti,ab,kf,kw 3. (exp algorithms/AND exp “models, statistical”/) OR (algorithm* OR ((prognostic* OR predict* OR survival*) adj3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*” OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”).ti,ab,kf,kw 4. exp “quality of life”/OR risk factors/OR exp patient care/OR ((symptom* adj3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) adj3 (predict* OR risk)) OR “quality of life” OR (Patient? adj1 care) OR (clinic* adj3 need*)).ti,ab,kf,kw 5. and/1-4	267
Embase (Elsevier)	1. ('malignant neoplasm'/exp OR 'childhood cancer survivor'/exp OR 'cancer survivor'/exp OR 'cancer therapy'/exp OR 'drug therapy'/exp OR 'radiotherapy'/exp OR 'oncology'/exp OR 'oncology ward'/exp OR 'oncological procedure'/exp OR 'cancer patient'/exp) OR (Cancer* OR carcinoma* OR Neoplas* OR Tumo?r OR tumor* OR tumour* OR Malignan* OR Oncolog* OR Leuk?emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy):ti,ab 2. ('patient-reported outcome'/exp OR 'self report'/exp) OR ((patient* NEAR/3 (data OR outcome OR symptom* OR self) NEAR/3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR 'electronic self-report*'):ti,ab 3. 'machine learning'/exp OR ('algorithm'/exp AND 'statistical model'/exp) OR (algorithm* OR ((prognostic* OR predict* OR survival*) NEAR/3 (model* OR system*)) OR 'risk model*' OR 'statistical model*' OR 'machine learning*' OR 'clinical decision rule*' OR 'clinical prediction rule*' OR 'neural network*' OR 'artificial Intelligence'):ti,ab 4. 'Patient care'/exp OR 'risk factor'/exp OR ((symptom* NEAR/3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) NEAR/3 (predict* OR risk)) OR 'quality of life' OR (Patient? NEAR/1 care) OR (clinic* NEAR/3 need*)):ti,ab 5. (#1 AND #2 AND #3 AND #4) AND [embase]/lim	976
CINAHL Complete (Ebsco)	1. MH (“Cancer Care Facilities” OR “Cancer Fatigue” OR “Cancer Survivors” OR “Cancer Screening” OR “Cancer Patients” OR “Early Detection of Cancer” OR “Neoplasms+” OR “Oncology+” OR “Oncology Care Units” OR “Oncologic Nursing+” OR “Oncologic Care+” OR “Oncology Surgery+” OR “Chemotherapy, Cancer+” OR “Radiotherapy+”) OR TI (Cancer* OR carcinoma* OR Neoplas* OR tumo#r* OR Malignan* OR Oncolog* OR Leuk#emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy) OR AB (Cancer* OR carcinoma* OR Neoplas* OR Tumo#r OR Malignan* OR Oncolog* OR Leuk#emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy) 2. MH (“Patient-Reported Outcomes+” OR “Self Report+”) OR TI ((patient* N3 (data OR outcome OR symptom* OR self) N3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”) OR AB ((patient* N3 (data OR outcome OR symptom* OR self) N3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”) 3. MH (“Machine Learning+” OR “Algorithms”) OR TI (algorithm* OR ((prognostic* OR predict* OR survival*) N3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*” OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”) OR AB (algorithm* OR ((prognostic* OR predict* OR survival*) N3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*” OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”) 4. MH(“Disease Management+” OR “Patient Care+” OR “Quality of Life+” OR “Risk Factors”) OR TI ((symptom* N3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) N3 (predict* OR risk)) OR “quality of life” OR (Patient? N1 care) OR (clinic* N3 need*)) OR AB ((symptom* N3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) N3 (predict* OR risk)) OR “quality of life” OR (Patient# N1 care) OR (clinic* N3 need*)) 5. S1 AND S2 AND S3 AND S4	213
Web of science Core (Clarivate) Editions searched: Sci-expanded, ssci, ahci, cpci-s, cpci-ssh, bkci-s, bkci-ssh, esci, Ccr-expanded, ic	1. ((TI=(Cancer* OR carcinoma* OR Neoplas* OR tumo$r* OR Malignan* OR Oncolog* OR Leuk$emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy)) OR AB=(Cancer* OR carcinoma* OR Neoplas* OR tumo$r* OR Malignan* OR Oncolog* OR Leuk$emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy)) OR AK=(Cancer* OR carcinoma* OR Neoplas* OR tumo$r* OR Malignan* OR Oncolog* OR Leuk$emia* OR Metasta* OR Adenoma* OR Adenocarcinoma* OR Adeno-carcinoma* OR Lymphoma* OR Melanoma* OR Sarcoma* OR Myeloma* OR Blastoma* OR Mesothelioma* OR Thymoma* OR Hepatoma* OR Hepatoblastoma* OR Hepato-blastoma* OR glioma* OR Ganglioglioma* OR Glioblastoma* OR Neuroblastoma* OR Retinoblastoma* OR Meningioma* OR Seminoma* OR Carcinosarcoma* OR Angiosarcoma* OR Chondrosarcoma* OR Cholangiocarcinoma* OR Medulloblastoma* OR Astrocytoma* OR Ependymoma* OR Germinoma* OR Craniopharyngioma* OR chemo* OR radiation OR radiotherapy) 2. ((TI=((patient* NEAR/3 (data OR outcome OR symptom* OR self) NEAR/3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”)) OR AB=((patient* NEAR/3 (data OR outcome OR symptom* OR self) NEAR/3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”)) OR AK=((patient* NEAR/3 (data OR outcome OR symptom* OR self) NEAR/3 (report* OR generate*)) OR PRO or ePRO OR telePRO OR “electronic self-report*”) 3. ((TI=(algorithm* OR ((prognostic* OR predict* OR survival*) NEAR/3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*” OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”)) OR AB=(algorithm* OR ((prognostic* OR predict* OR survival*) NEAR/3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*„ OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”)) OR AK=(algorithm* OR ((prognostic* OR predict* OR survival*) NEAR/3 (model* OR system*)) OR “risk model*” OR “statistical model*” OR “machine learning*” OR “clinical decision rule*” OR “clinical prediction rule*” OR “neural network*” OR “artificial Intelligence”) 4. ((TI=((symptom* NEAR/3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) NEAR/3 (predict* OR risk)) OR “quality of life” OR (Patient$ NEAR/1 care) OR (clinic* NEAR/3 need*))) OR AB=((symptom* NEAR/3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) NEAR/3 (predict* OR risk)) OR “quality of life” OR (Patient$ NEAR/1 care) OR (clinic* NEAR/3 need*))) OR AK=((symptom* NEAR/3 (manage* OR alert* OR surveil* OR prognos* OR predict* OR forecast*)) OR ((mortality OR factor*) NEAR/3 (predict* OR risk)) OR “quality of life” OR (Patient$ NEAR/1 care) OR (clinic* NEAR/3 need*)) 5. #1 AND #2 AND #3 AND #4	317
Total		1,773

Abbreviation: PRO, patient-reported outcome.

TABLE A2.

Inclusion and Exclusion Criteria

Criterion	Inclusion	Exclusion
Population	Adult patients diagnosed with solid tumor cancers	Patients diagnosed with other diseases
Concept	Patient-reported outcomes or patient-generated health data	Nonsymptom data, such as satisfaction with care or other survey instruments
Context	Risk prediction or prognostic calculation of an outcome of interest	Epidemiological or health services research that did not involve predicting an outcome of interest for cancer patients
Study type	Prospective or retrospective analysis, cohort studies, clinical trials, observational data analysis	Opinions, perspectives, conference proceedings, reviews, meta-analysis, abstracts, editorials

AUTHOR CONTRIBUTIONS

Conception and design: Roshan Paudel, Michael J. Hassett

Administrative support: Christine Cronin

Collection and assembly of data: Roshan Paudel, Samira Dias, Carrie G. Wade

Data analysis and interpretation: Roshan Paudel, Samira Dias, Christine Cronin, Michael J. Hassett

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

No potential conflicts of interest were reported.