Review of Electronic Patient-Reported Outcomes Systems Used in Cancer Clinical Care

The number of electronic patient-reported outcome (PRO) systems has increased; systems can be programmed to have numerous features that facilitate PRO assessment integration and routine monitoring in clinical care.

Abstract

Purpose:

The use of electronic patient-reported outcomes (PRO) systems is increasing in cancer clinical care settings. This review comprehensively identifies existing PRO systems and explores how systems differ in the administration of PRO assessments, the integration of information into the clinic workflow and electronic health record (EHR) systems, and the reporting of PRO information.

Methods:

Electronic PRO (e-PRO) systems were identified through a semistructured review of published studies, gray literature, and expert identification. System developers were contacted to provide detailed e-PRO system characteristics and clinical implementation information using a structured review form.

Results:

A total of 33 unique systems implemented in cancer clinical practice were identified. Of these, 81% provided detailed information about system characteristics. Two system classifications were established: treatment-centered systems designed for patient monitoring during active cancer treatment (n = 8) and patient-centered systems following patients across treatment and survivorship periods (n = 19). There was little consensus on administration, integration, or result reporting between these system types. Patient-centered systems were more likely to provide user-friendly features such as at-home assessments, integration into larger electronic system networks (eg, EHRs), and more robust score reporting options. Well-established systems were more likely to have features that increased assessment flexibility (eg, location, automated reminders) and better clinical integration.

Conclusion:

The number of e-PRO systems has increased. Systems can be programmed to have numerous features that facilitate integration of PRO assessment and routine monitoring into clinical care. Important barriers to system usability and widespread adoption include assessment flexibility, clinical integration, and high-quality data collection and reporting.

Introduction

Health information technology (HIT) is increasingly integrated into US medical care, with 57% of office-based physicians now using electronic health records.¹ Simultaneously, Internet use spans ages, races/ethnicities, and incomes, partly as a result of increased access via laptops and handheld devices.² HIT promotes high-quality, patient-centered cancer care³ by collecting and incorporating standardized patient-reported information into clinical settings.

Routine cancer care assessment of patient-reported outcomes (PROs), including symptoms, function, and quality of life, has been shown to improve symptom management,^4,5 identification of psychosocial problems,^6,7 and patient-provider communication.^8,9 Providers and patients express satisfaction with using PRO information in clinical care,^8,10,11 and research shows that PRO assessments can be performed without adversely affecting clinic workflow or visit length.⁹ PRO use is largely facilitated by available real-time electronic platforms that collect, store, and report PRO data to inform clinical care. Electronic PRO (e-PRO) assessment systems allow efficient standardized assessments, decreased response burden, increased satisfaction, improved ease of use, and fewer missing data compared with paper-based PRO measures.^12–14 Furthermore, Web-based systems provide patients new ways to report symptoms, function, and quality of life outside their clinical visits.

Although e-PRO system development and implementation have occurred in a wide range of “early adopter” cancer clinical care settings, to our knowledge, no systematic review to date has identified these systems and their features. This study therefore identified systems implemented over the past 12 years and evaluated their administration, data collection, and reporting features. This semistructured review of published and gray literature focused on (1) system design and software features and (2) integration of e-PRO collection and reporting into cancer clinical care. Other critical elements, such as PRO instruments used and analytic approaches, are not discussed here.

Methods

Eligible systems were defined as those used in clinical oncology practice settings or related treatment areas (eg, radiation, surgery), that assess PROs electronically and summarize patients' responses to the provider. Because PRO system hardware is adapted and implemented in different settings, we limited our review to a system's current hardware iteration, thus ensuring that the same system was not double-counted. When multiple systems were identified from the same institution or corporation, each was reviewed for design similarity. Systems that exhibited more than 80% similarity across our review features were considered the same system. No two systems had the same set of features; all systems listed in this review had unique characteristics. If developers indicated that systems at the same organization were developed for separate purposes (ie, clinic based versus Web based), they were considered separate. Applications of the same e-PRO system at multiple sites were not counted separately.

Systems were identified through publications, conference abstracts, and reporting by researchers and clinicians. PubMed and MEDLINE searches used the following terms: patient-reported outcomes (outcome assessment, quality of life, health status indicators, patient-reported), clinical care (patient care, clinical care, delivery of health care), and oncology. This resulted in 190 articles, from which 18 eligible e-PRO systems were identified. Abstracts from the 2009 to 2011 meetings of ASCO, International Society for Quality of Life Research, International Society for Pharmacoeconomics and Outcomes Research, and American Medical Informatics Association were also reviewed, leading to identification of three additional eligible systems. Twelve additional systems were identified through sources such as gray literature (eg, Web sites and press releases) and field experts. Systems are listed in Appendix Table A1 (online only).

System characteristics and clinical implementation were identified and abstracted using a structured review form created by the authors. A form prepopulated with available information was sent to system developers for verification and to obtain missing items. For a system to be included in this review, follow-up and verification from the developer via a minimum of two e-mails and one telephone call was required. System characteristics were summarized and categorized as either treatment or patient centered. Treatment-centered systems tracked patients during anticancer treatment, whereas patient-centered systems captured patient data across the cancer care continuum, including treatment and survivorship.

Results

The review identified 33 e-PRO systems. Twenty-seven developers responded to our information request, yielding an 81% response rate. A majority of systems are in the United States (70%) and are used solely by one entity (56%). One third of systems were implemented in a single academic institution. Most systems were used in medical oncology clinics; 15% were used only in surgical or radiation oncology consultations, and 7% were used in palliative care settings. One system was developed for nonclinical treatment use. Most were developed in house with foundation or government-funded grants and have been used by 500 or more patients (Table 1).

Table 1.

System Administration

Feature	No. (N = 27)	%	% of Patient Centered (n = 19)	% of Treatment Centered (n = 8)
System launched
≥ 5 years ago	17	63	63	63
1-4 years ago	10	37	37	37
How developed*
In house	24	89	84	100
External developers	3	11	16	0
Commercially available	8	30	32	25
No. of users
> 500	12	44	42	50
< 500	7	26	32	13
Not currently in use	8	30	26	38
Institutional setting
Domestic	19	70	68	75
Academic	6	22	21	25
Community hospital or clinic	4	15	5	37
Multisite	9	33	42	12
International	8	30	32	25
Academic	4	15	21	0
Community hospital or clinic	1	4	0	12
Multisite	3	11	11	12
Primary location system used
Medical oncology clinic	21	78	78	75
Radiation/surgery	4	15	11	25
Palliative care/hospice	2	7	11	0
Access location
Clinic	9	33	26	50
Home	8	30	37	13
Both	10	37	37	37
System accessibility*
Computer	17	63	68	50
Tablet	14	52	53	50
Kiosk	6	22	26	13
Cell phone	4	15	16	13
Training available*
Clinicians and staff	23	85	90	75
Patients	16	59	63	50
Available, but unspecified	1	4	0	13
None	3	10	10	13
Security features*
Secure log-in	21	78	79	75
Encryption	14	52	63	25
PHI protection*	5	17	26	0
Not Web-based	3	10	11	13
IRB approval and patient consent
Required	16	59	58	63
Conditional	4	15	16	13
Exempt	7	26	26	25
Cancer population
All (no restrictions)	11	41	47	25
≤ 5 types of cancer	9	33	32	38
> 5 types of cancer	7	26	21	38
Most common cancer sites*
Breast	12	44	37	63
Prostate	10	37	32	50
Gastrointestinal	9	33	21	63
Gynecological	9	33	21	63
Lung	7	26	11	63
Phase of care*
Active treatment	17	63	53	88
Survivorship	11	41	58	0
End of life	3	11	16	0
Prior to treatment	1	4	0	13
Not specified	6	22	32	0
System focus*
Two or more clinical areas	20	74	79	38
Treatment specific	9	33	21	63
Chemotherapy	5	19	5	50
Radiation	2	7	0	25
Surgery	2	7	5	13
Palliative care	2	7	10	0

Abbreviation: PHI, personal health information.

^*Denotes multiple response items; total ≠ 100%.

Most systems (70%) provided patients with in-clinic system access, and 37% allowed patients to fill out PRO assessments either at home or in the clinic. Web-based assessments were the primary method of data collection, both inside and outside the clinic. Interactive voice response by telephone was offered for 26% of systems. In addition to computers, system access options included tablets (41% touch-screen access, 26% stylus only), cell phones (15%), and clinic-based kiosks (22%). All Web-based systems used security (such as secure log-in or data encryption; Table 1).

Most systems (63%) were intended for use during treatment, and many (40%) were also used in follow-up care. One third of systems restricted assessments to monitoring specific cancer treatments, most commonly chemotherapy. Although many systems were open to all cancers, a quarter of systems were limited to specific cancers (Table 1).

Format varied with regard to question administration and data capture (Table 2). A majority of the systems (56%) assessed one question at a time, and required a mouse click to advance to the next question (63%). Systems handled missing patient data differently. Most allowed for multiple patient log-ins (to answer the same PRO questionnaire) and missing data (a questionnaire could be completed even if items were skipped). Two systems allowed patients to skip items and provided a prepopulated “neutral” response (ie, “Neither Agree or Disagree”) to PRO questions. If patients did not change the response option, that default response would be recorded. Treatment-focused systems were less likely than patient-centered systems to allow patients to select a “Not Applicable” option (25% v 42%). More than half of the systems (63%) sent assessment reminders. The most common reminder method was e-mail. Only one third of systems offered reminders by phone, text, or letter.

Table 2.

Data Collection and Assessment Information

Feature	No. (N = 27)	%	% of Patient Centered (n = 19)	% of Treatment Centered (n = 8)
Question format
One question per page	15	56	58	50
Multiple questions per page	10	37	32	50
Both options available	2	7	11	0
Page features
Progress bar	11	41	42	38
Visual graphics	12	44	47	38
Question advancement
Mouse click only	17	63	63	63
Automatic only	7	26	21	37
Both options available	3	11	15	0
Data capture*
Multiple logins per assessment	18	67	68	63
Allows N/A response	10	37	42	25
Default response preselected	2	7	5	13
PRO assessment selection*
Automatic (by system)	18	67	58	88
Provider	11	41	47	25
Patient	4	15	16	13
Notification types for completing questionnaires*
Patient reminders	17	63	63	63
Automatic	14	52	58	38
Manual	4	14	5	38
PRO score result alerts	23	85	95	63
No alerts	4	15	5	37
Patient reminder format*
E-mail	14	52	58	38
Telephone call	3	11	11	13
Text message/SMS	4	15	16	13
Verbal	2	7	11	0
Paper	2	7	5	13
N/A	10	37	37	38
PRO score results alert format*
E-mail	17	63	68	50
Text message/SMS	5	19	21	13
Verbal	2	7	11	0
N/A	4	15	5	38
Alert recipient*
Provider	18	67	79	38
Staff	16	59	63	50
Patients	8	29	32	25
Caregiver	3	11	15	0
N/A	4	15	5	37
Flexible system features*
Both home and clinic log-in access	10	37	37	37
Multiple assessment scheduling options	24	93	95	75
Two or more sources for PRO selection (patient, provider, system)	4	15	16	13
Self-identification of important issues by patient	11	41	53	13

Abbreviations: N/A, not applicable; PRO, patient-reported outcomes; SMS, short message service.

^*Denotes multiple response items; total ≠ 100%.

After patients completed their questionnaires, 85% of systems sent real-time alerts tied to patient responses, primarily directed to providers and staff. Half of the systems also sent alerts to patients. Alerts were typically sent by e-mail or text message, with 22% of systems sending alerts in multiple formats. Alerts were most commonly set up to prompt follow-up by clinician (74%), but some systems generated automatic referrals to pharmacists, social workers, and other service staff when necessary. Forty-four percent of e-PRO systems were directly integrated into the clinic's electronic health record (EHR) system. Some systems also provided patient education (eg, fatigue management tips), and more than half administered educational modules within the PRO system itself. (Table 3). System-wide automation (patient reminders, score alerts, and follow-up referral) was reported by one treatment-centered and five patient-centered systems. System feature flexibility varied. Some allowed for flexibility in location of administration (clinic or home) and PRO questionnaires administered (Table 3).

Table 3.

System Integration and Reporting

Feature	No. (N = 27)	%	% of Patient Centered (n = 19)	% of Treatment Centered (n = 8)
Report access*
Provider	26	96	100	88
Patient	17	63	58	75
Staff	11	41	37	50
Caregiver	3	10	16	0
PRO reports accessible*
Immediately	19	70	79	50
At visit	12	44	42	50
Report content*
Current scores	25	93	95	88
Longitudinal change	25	93	100	75
Interpretation included	21	77	79	75
Cut scores (eg, low, medium, high)	19	70	79	50
Population norms or reference values	14	52	63	25
Identification of meaningful change	14	52	53	50
Reports modifiable	13	48	58	25
General guidelines	9	33	32	38
Five or more content features available	19	70	84	38
Visual presentation of PRO scores*
Graphs	22	81	89	63
Tables	16	59	63	50
Numbers	6	22	16	38
Two or more options	15	56	58	63
Clinical response*
Prescribed response	21	78	79	75
Clinician/staff follow-up	20	74	79	63
Patient education	7	26	26	25
Automatic referral	7	26	26	25
Patient notified	1	4	0	13
N/A	6	22	21	25
Linkage to other systems*
EHR	12	44	53	25
Appointment/scheduling	11	41	47	25
Patient portal	5	19	21	25
Billing	4	15	21	0
Two or more features	11	14	53	13
No linkage	12	44	37	63
Accessibility*
Cancer specific	16	59	68	38
Other care	12	44	53	25
Inaccessible elsewhere	10	37	32	62
Patient education*
Administered in system	15	56	63	38
Linked to PRO scores	13	48	58	25
Documents actions	6	22	26	13
No education through system	12	44	37	63
Other features*
Allows both patients and clinicians to access PRO scores	23	85	89	75
Backup reporting option for missed assessments	18	67	74	50
Quality of care evaluation possible	17	63	68	50
Decision aids	16	59	58	63
Patient satisfaction collected	10	37	43	25
Accreditation reporting	3	11	16	0
Two or more features	23	85	89	75

Abbreviations: EHR, electronic health record; N/A, not applicable; PRO, patient-reported outcomes.

^*Denotes multiple response items; total ≠ 100%.

Type and format of reporting varied considerably (Table 2). Almost all systems (96%) provided summaries of patient-reported data to prespecified providers, but only one allowed multiple providers to access this information. Most systems (93%) provided summaries of patient-reported data for individual assessments and over repeated assessments (93%). Three quarters of systems had the functionality to give reports to patients, but only 63% actually provided them (eg, through a patient portal or immediately generated results). More than half (70%) allowed access to scores immediately after patients completed assessments, whereas other systems restricted access to results until the clinical encounter. Data interpretation guidelines and indicators of meaningful change in patient scores were the least common features. Ten systems reported availability of at least seven of the 10 report features identified in Table 3.

Discussion

This review illustrates the features and functionality of e-PRO systems used in clinical oncology settings. The systems were generally developed to improve symptom management, identify psychosocial problems, and facilitate patient-provider communication by (1) treatment-centered monitoring during intensive therapeutic periods and (2) patient-centered monitoring across care transitions. Systems designed to support therapeutic monitoring provided features most useful during specific care periods, such as identification and monitoring of treatment-related adverse effects. In contrast, patient-oriented systems were designed for monitoring patients across the cancer care trajectory, from active treatment to survivorship. Although both system types are appropriate in oncology, there is a need for long-term monitoring across the cancer care continuum.¹⁵ Our study identified increased development of patient-centered systems over the past 5 years, which may be due in part to this report, as well as the increasing ability to electronically capture patient data in care settings. Ideally, it would be useful for e-PRO systems to monitor patients throughout the course of care while capturing detailed information about patients' responses to specific treatments in order to balance these important objectives.

System Design

Flexibility allows most systems to bridge treatment-specific and long-term PRO assessment. By providing a wide range of software features, some systems can adapt to specific patient or provider needs without additional programming. For example, flexibility in location allows both in-clinic and at-home monitoring during follow-up. Flexible assessment selection gives providers and/or patients the ability to choose what data to track. Flexible assessment frequency permits specified recurrences or an open-ended schedule (eg, whenever a patient comes into the clinic or chooses to complete an assessment online). This flexibility also allows systems to grow beyond oncology-specific treatment settings, facilitating patient-centered care by documenting a wide number of PROs, from daily treatment symptoms to long-term health-related quality-of-life concerns.

Successful systems should integrate both treatment- and patient-centered perspectives into one health informatics model. Currently, all identified systems require manual PRO content selection to ensure relevancy. However, future e-PRO systems will provide opportunities for automatic integration of PRO content tailored to individual patient needs. Features facilitating both treatment-focused and continuum-focused monitoring can be combined with the right PRO instrument to accomplish specific needs. For example, reports to clinicians could provide graphical overlays for toxicity data during treatment and provide less detailed longitudinal monitoring during survivorship. To achieve these goals, it is critical to combine software features with informatics capabilities that support reporting and data visualization solutions.

Another key design difference was intended audience. Some systems relied on patients to complete PRO assessments when they wanted to identify issues, whereas others relied on provider selection of assessment frequency and topics. One system alerted patients about their scores, accompanied by a suggestion to speak to the physician, without reporting scores directly to the physician. Some patient-centered systems presented educational materials, generated mental health referrals, or alerted social workers if warranted by a patient's PRO scores, thus helping patients address symptoms outside of clinical interactions. Finally, some systems allowed patients to view their PRO scores, whereas others gave that information to their health care provider. Whereas most systems focused on providing information needed by clinicians, systems designed for patients provide an important method to identify their needs.

A patient-centered focus depends on patients actively communicating their concerns. This approach recognizes that patients' and providers' perspectives regarding needs and symptoms can be meaningfully different.^16–18 Few systems currently identify and communicate patients' self-identified PRO concerns, and although some current systems do allow for write-in text, future research should determine whether provider- and patient-focused systems generate different PROs, as well as how patient free-text information can inform these scores. As shared clinical decision making becomes increasingly important, newer systems may provide PRO administration tailored to patient preferences.

Data Collection Features

System usability and clinical care integration were important system characteristics. Generally, systems should be user friendly for patients, staff, clinicians, and researchers. Systems must provide an efficient, easy experience that patients are willing to repeat by providing features such as an option to save data when sessions are interrupted, easily understood page layouts, and the ability to move quickly through questionnaires. This study showed that these features are not uniformly included, but that patients' experiences could be substantially improved with minimal programming effort.

A successful e-PRO system must impose minimal burden on the staff. Although e-PRO systems may reduce staff burden by streamlining PRO collection and scoring,¹⁹ and by automating symptom identification outside of patient care visits (ie, patients reporting from home), few systems currently offer this level of automation. Many systems require staff time to contact patients with assessment reminders or regarding missing information before appointments. While automation using reminders, alerts, and follow-up (eg, automatic referrals based on patient responses) is possible, few systems identified currently provide this measure of automation. Until systems consistently offer full automation with a high degree of validity and reliability (eg, consistently identifying clinically appropriate referrals), staff burden could be a considerable barrier to sustainable use.

We also identified disparate technical support for these systems. Although many large cancer centers have resources necessary to develop, run, and maintain an e-PRO system, limited capacity in smaller rural and community-based practice settings, where real-time monitoring is especially useful, can pose an issue.^15,20 As systems move further away from clinical settings, managing score-based alerts becomes challenging. System alerts occurring outside the clinic can be in the form of an electronic communication between patient and provider. Therefore, commonly cited physician barriers to electronic communication, such as workload and time demands, security, and payment/reimbursement,²¹ should be thoroughly evaluated after system implementation.

Another issue identified was balancing system practicality with capturing high-quality data. System developers must carefully adapt paper PRO assessments into electronic formats in order to ensure measurement equivalence when possible. For instance, e-PRO systems can be programmed to require item response or to allow preselected default responses to certain questions. Each of these design features may change how patients answer questionnaires. Providing default responses makes it easier for patients to answer quickly, but patients may be less likely to select a different response. Conversely, removing the ability to skip an item allows for more complete data but may provide inaccurate information if patients are unwilling to answer or believe that none of the possible answers are applicable. Formal paper-electronic equivalence testing should be done to ensure that e-PRO features do not limit data validity. An International Society for Pharmacoeconomics and Outcomes Research task force has provided general guidelines and recommendations.²² Although geared toward a research-oriented audience, these are important to consider from a clinical perspective to ensure the validity and reliability of PRO data.

Assessment Reporting and Workflow Integration

For clinicians, previous e-PRO implementation studies have suggested that ease of use and focus on clinically relevant issues are necessary for sustainability.²³ The systems reviewed here exhibited considerable variation in both areas. Overall, EHR integration was limited among these systems but has enormous potential. Integrating PRO data into EHRs allows clinicians to incorporate PROs into care without accessing a different system. Only the systems that provided EHR integration also linked to other features such as the patient portal and coding and billing information and were likely to be accessible across other noncancer care settings and to be linked to scheduling. Oncology care typically requires complex coordination of HIT to effectively link to other clinical care,²⁴ and integrating PRO information into other HIT applications is necessary to sustain oncology e-PRO systems. This allows PRO data to be used to improve quality of care by, for example, monitoring provider performance²⁵ and facilitating comparative effectiveness research.²⁶

PRO information must be well integrated into the clinical experience to be accessed regularly. Previous research has highlighted the importance of providing clinicians with PRO information that is actionable.^27,28 This requires systems to report scores to clinicians in formats that allow for quick, accurate interpretation. However, the least common reporting features were inclusion of general interpretation guidelines; identification of meaningful changes; and ability to report e-PRO scores in a numerical text-based format, with or without graphical representation of information. Numerical text-based reporting is particularly important because it allows for PRO scores and lab values to be displayed on the same report or EHR system, and graphical displays that allow pattern recognition are particularly important for efficient clinical care.

This study has important limitations. First, because results are based on information reported by system developers, rather than through direct observation by the authors, some information could have been reported incorrectly. Second, the amount of detail requested on the study survey may have deterred respondents from providing complete answers, and some companies may have been unwilling to provide proprietary information. Finally, although the review was as exhaustive as possible in identifying e-PRO systems used in clinical cancer settings, little published information about these systems exists. It is possible that the review did not capture existing systems, such as e-PRO features integrated into larger EHR systems. However, these findings build on previous e-PRO system reviews limited to published, randomized control trials²⁹ and include an in-depth review of system features and integration into clinical settings. In addition, given our high response rate and the lack of a previous review of this kind, it seems likely that the trends we observed are representative of systems that may have been omitted. Despite these limitations, this study adds important information about the current state and focus of e-PRO systems used in oncology.

Conclusion

PRO data collection continues to become increasingly significant to health care. The National Institutes of Health, the Agency for Healthcare Research and Quality, and the new Patient-Centered Outcomes Research Institute have all enthusiastically supported the development of PRO methods for collection and use in clinical care. The Centers for Medicare and Medicaid Services and other payers, as well as the Food and Drug Administration, have begun to use PROs to evaluate interventions. Examples of PRO applications for care reporting are seen in both the Centers for Medicare and Medicaid Services EHR Meaningful Use requirements and efforts such as the ASCO's CancerLinQ program, both of which illustrate how nationwide PRO data collection efforts can be used to monitor and improve care and patient outcomes. This study showed that current e-PRO systems vary dramatically in their focus and features. As electronic PRO collection increases, understanding the range of available features and characteristics available will help to ensure that e-PRO systems remain accessible and useful within the care setting.

Appendix

Table A1.

Identified e-PRO Systems

System	Institution	In Use	Focus
Advanced Symptom Management System in Palliative Care (ASyMSp)	University of Stirling, Stirling, Scotland	No	Patient
BrightOutcome (clinic-based system)	BrightOutcome, Buffalo Grove, IL	Yes	Patient
BrightOutcome (Web-based system)	BrightOutcome	Yes	Patient
Comprehensive Health Enhancement Support System (CHESS)	University of Wisconsin	Yes	Patient
Computer-based Health Evaluation System (CHES)	Medical University of Innsbruck (Evaluation Software Development), Innsbruck, Austria	Yes	Patient
Dynamic Clinical Systems	Dynamic Clinical Systems, Hanover, NH	Yes	Patient
Electronic Patient-reported Outcomes from Cancer Survivors (ePOCS)	Clinical Centre in Leeds, St James's University Hospital, University of Leeds, Leeds, United Kingdom	Yes	Patient
Electronic Patient Self-Assessment and Management (SAM)	Memorial Sloan-Kettering Cancer Center and University of California, San Francisco	Yes	Patient
Electronic Self Report Assessment-Cancer (ESRA-C)	University of Washington and Dana-Farber Cancer Institute	Yes	Patient
EPIC-AC	Northwestern University	Yes	Patient
Navigating Cancer (Web-based system)	Navigating Cancer	Yes	Treatment
Ontario Symptom Management Collaborative (OSMC)	Cancer Care Ontario, Toronto, Ontario, Canada	Yes	Patient
Patient Assessment, Care and Education (PACE)	West Clinic, Memphis, TN	Yes	Patient
Patient Viewpoint	Johns Hopkins University	Yes	Patient
Quality of Life In Childhood Oncology (QLIC-ON)	Academic Medical Centre/Emma Children's Hospital, Amsterdam, the Netherlands	No	Patient
Symptom Monitoring and Management SyMON-1	Northwestern University	Yes	Treatment
Symptom Tracking and Reporting (STAR)	Memorial Sloan-Kettering Cancer Center	Yes	Treatment
Support Screen	City of Hope National Medical Center	Yes	Treatment
Tell Us	Johns Hopkins University; the Medical Decision Logic	No	Patient
The Personal Well-Being Checklist (PWBC)	Tom Baker Cancer Centre, Calgary, Alberta, Canada	No	Treatment
Un-named	Mayo Clinic	No	Treatment
Un-named	Oregon Health & Science University	No	Treatment
Un-named Indiana QOL	School of Nursing, Indiana University	No	Patient
VisionTree	VisionTree Software	Yes	Patient
WebChoice	Rikshospitalet Medical Center, Oslo, Norway	Yes	Patient
WebCore	Memorial Sloan-Kettering Cancer Center	Yes	Patient
Wireless Health Outcomes Monitoring System (WHOMS)	Istituto Nazionale Tumori, Milan, Italy	No	Treatment

Systems Identified Only (not abstracted)
Advanced Symptom Management System (ASyMS)	University of Dundee, Dundee, Scotland	Yes
Advanced Symptom Management System in Young Adults (ASyMS-YG)	UCL Institute of Child Health/Great Ormond Street Hospital for Children, National Health Service, London, United Kingdom	No
Cancer Care Coordination/Home-Telehealth (CCHT) Program	Office of Telehealth Services, Department of Veterans Affairs	Unknown
Computerized Outcome Assessment Tool (COAT)-HNC	Chang Gung University, Gueishan, Taiwan	Unknown
PAINReportIt-Plus	University of Illinois at Chicago	Unknown
Quality of Life Informatics Platform (QoLIP)/onQoL	Fernando Pessoa University, Ponte de Lima, Portugal	Unknown

Authors' Disclosures of Potential Conflicts of Interest

Although all authors completed the disclosure declaration, the following author(s) and/or an author's immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: Amy P. Abernethy, American Association of Hospice and Palliative Medicine (U), Advoset (education company, including contracts from Novartis) (C), Orange Leaf Associates LLC (C) Consultant or Advisory Role: Amy P. Abernethy, Pfizer (C), Novartis (C) Stock Ownership: None Honoraria: None Research Funding: Amy P. Abernethy, Biovex, Dara, Helsinn, Mico, Pfizer, Alexion, Amgen, Dendreon, Celgene Expert Testimony: None Patents, Licenses or Royalties: None Other Remuneration: None

Author Contributions

Conception and design: Roxanne E. Jensen, Claire F. Snyder, Amy P. Abernethy, Ethan Basch, Deena R. Loeffler, Bryce B. Reeve

Financial support: Arnold L. Potosky

Administrative support: Roxanne E. Jensen, Aaron C. Roberts

Collection and assembly of data: Roxanne E. Jensen, Aaron C. Roberts, Deena R. Loeffler

Data analysis and interpretation: Roxanne E. Jensen, Claire F. Snyder, Amy P. Abernethy, Arnold L. Potosky, Aaron C. Roberts, Deena R. Loeffler, Bryce B. Reeve

Manuscript writing: All authors

Final approval of manuscript: All authors