INTRODUCTION
Over the past several decades, there has been a shift within healthcare to be more person-centered and regulatory agencies have identified patient-reported outcomes (PROs) as a critical component of medical research and practice.1 While PRO measurements may increase overall patient survival2, the methods by which PRO data are collected and analyzed have been inconsistent.3,4
Electronic data capture systems (EDCs), applications or “apps”, and single-purpose devices, have provided more sophisticated methods of PRO measurement but research on the implementation in a practice setting is limited. Additionally, the majority of research including the implementation of ePRO measures in practice does not include family caregivers.
As the National Quality Forum asserts, the term “patient” in “patient-reported outcomes” represents “all persons, including patients, families, caregivers, and consumers.”5 While, caregivers of cancer patients have been shown to suffer from significant emotional stress and experience physical maladies, screening with PRO assessments for caregivers has yet to become standard.6
Therefore, the purpose of this study was to assess the feasibility of an electronic-PRO (ePRO) collection system in caregivers of cancer patients.
METHODS
Design
This study utilized a prospective, repeated measures design. Data were collected at three time-points: study enrollment (±14 days of the start of the patient’s treatment) and three and six months after enrollment (±14 days). Subjects were assigned login information for the online survey. Cancer patients provided a release of medical record for treatment information.
Study Participants
Eligible subjects were at least 18 years old, literate in English or Spanish, intended to serve as an active caregiver throughout the study, and had internet access (other than on a smartphone). Up to three family caregivers per cancer patient were permitted to participate. Caregivers of all cancer patients registered at the National Institutes of Health Clinical Center who were initiating a new cancer treatment, across the inpatient and outpatient setting, were considered eligible.
ePRO survey
Measures of outcomes considered clinically relevant (e.g. screening) for cancer caregivers were included as Computerized Adaptive Tests (CATs) and fixed-length forms. The survey included the Patient-Reported Outcomes Measurement Information System (PROMIS®) measures of global mental and physical health, anxiety, depression, and fatigue, the Neuro-QoL instrument, Applied Cognition-General Concerns, and the NIH Toolbox measures of meaning and purpose, loneliness, self-efficacy, and perceived stress.7
Feasibility
Feasibility of the ePRO system was assessed through study specific usability questions evaluating subject’s satisfaction with the system, as well as through enrollment metrics. Usability questions included level of agreement with the following statements: 1) I am satisfied with the ease of completing the survey; 2) I am satisfied with the amount of time it took to complete the survey; and 3) I am satisfied with the support information (online-line help, messages, documentation) when completing the survey. Statements were rated on a Likert Scale with response options ranging from Strongly Disagree (0) to Strongly Agree (7), higher scores representing greater satisfaction. At the end of study, subjects were given the opportunity to respond to an open-ended question regarding their experience. Eligibility and exclusion criteria were recorded in an enrollment log to track factors that influenced ePRO system access.
Statistical Analysis
Descriptive statistics were performed on demographic variables, as well as the usability questions. Enrollment data were used to identify barriers to subjects enrolling on the study. Qualitative analysis was performed using thematic analysis at the latent level on responses to the open-ended question. Two raters (SZB, MC) independently read and coded text to identify themes, which were finalized by team consensus.
RESULTS
Demographics and Outcomes
Demographic information is described in Table 1.
Table 1.
Baseline demographics (n=137a)
| Characteristic | n (%) |
|---|---|
| Age, M (SD) | 48.5 (11.7) |
| Sex, Female | 94 (68.6) |
| Race/Ethnicityb | |
| Hispanic/Latino(a) | 19 (14.0) |
| Non-Hispanic - White | 96 (70.6) |
| Non-Hispanic - Other | 21 (15.4) |
| Marital Status, married/partnered | 114 (83.2) |
| Relationship to patient | |
| Spouse | 67 (48.9) |
| Parent | 48 (35.0) |
| Other (sibling, aunt, significant other) | 22 (16.1) |
| Caregiver Role | |
| Sole | 65 (47.4) |
| Part of a team | 72 (52.6) |
| Patient Type | |
| Adult | 102 (74.5) |
| Pediatric | 35 (25.5) |
| Patient treatment | |
| Single Therapy (chemotherapy, radiation, surgery) | 26 (19.0) |
| Combination Therapy (allogeneic HSCT) | 21 (15.3) |
| Biotherapy/Immunotherapy | 90 (65.7) |
n=137, 2 withdrew after enrollment, no initial survey.
n=136, 1 declined response.
n=129, 8 failed to complete baseline survey.
Abbreviations: M, mean; SD, standard deviation; HSCT, hematopoietic stem cell transplantation.
Feasibility
The baseline survey was completed by 129 subjects, 53% of the 244 eligible. Of the 309 caregivers screened, 55 caregivers were deemed not eligible: enrolled on a competing protocol (n=4), not speaking English or Spanish (n=3), patient treatment plans changing (n=38), no internet access (other than smartphone) (n=9), and caring for a patient outside of the NIH (n=1). There were 115 eligible subjects who declined participation: caregiver did not follow-up with study team following invitation to participate (n=98), not interested (n=7), dislike of online surveys (n=3), and perceived increase burden (n=7). Of the 139 subjects enrolled, 10 withdrew due to lack of interest (n=8) or changes in patient treatment plans (n=2).
Mean usability scores at baseline were 6.5 (±0.8) for ease of completion, 6.7 (±0.9) for time to complete, and 6.3 (±1.2) for supportive information. Forty-two subjects (30.2%) provided specific comments regarding their experience, revealing both positive and negative themes. The two most common positive themes were favorable of the technology and comprehensiveness of survey content. More specifically, subjects reported, “there were no technical issues” and “the survey itself seemed thorough and extensive.” The most common negative theme was difficulties with the survey, with subjects recognizing that “the survey was a bit too long.’ Additional themes included wanting to obtain the results of the survey and miscellaneous expressions, such as feeling “unfortunate timing in [the caregiver and patient’s] lives.”
DISCUSSION
Results of this study suggest that an ePRO system can provide providers with a useful tool to assess relevant outcomes in caregivers of cancer patients. Subjects were highly satisfied with the overall process of providing PRO data via an ePRO system. However, this study has several limitations. Usability data was not obtained from subjects who declined participation, including those who may have been too burdened. Thus, the feasibility for all cancer caregivers may be lower than reported. Additionally, the feasibility of the ePRO system may be specific to the system used in this study and an objective measure of response burden (e.g. time to completion) was not obtained.
Still, previous research has found that PRO use in routine clinical care and symptom monitoring improves patient-provider communication, patient satisfaction, and understanding of medical treatment effects8,9 but some have argued that clinical outcomes do not benefit from measurement alone and that the adoption of a PRO system has not been proven as an efficient use of resources.10 Future research on the implementation of ePROs in practice should include the documentation in EMRs, pattern and impact of referral, and impact on provider workflow. Three primary categories of complexities of integrating an ePRO collection system were observed in this study, including instrument selection, technical capabilities, and study coordination issues. Considerations for successful implementation of an ePRO collection system are presented in Table 2.
Table 2.
Considerations for implementing an ePRO measurement system
| Category | Consideration |
|---|---|
| Instrument Selection | • Does the system support Computerized Adaptive Test? • Can the system link with instrument databases? • How are custom instruments (i.e. demographics) collected? • Are selected instruments validated for online use? • Are multiple languages available? |
| Technical Capabilities | • Can the system support different study designs? • Are there device, browser, or location limitations? • Where are data stored and is it compliant with regulations? • Can branching or skip-logic be used? • Will non-English languages be supported? • Can instruments be scored within the system? • Are there access or timing restrictions (i.e. multiple entries, time limits)? • Are there alerts and notifications? • How can data be exported? |
| Study Coordination | • What skills are needed within the study team (i.e. clinical, data management, informational technology)? • How are subjects enrolled and tracked? • What troubleshooting support is available? • How will communication occur with subjects? |
KEY POINTS.
Patient-reported outcomes (PROs) are a key component of psycho-oncology research and practice but inconsistent collection systems have restricted their widespread adoption.
The term “patient” in PROs encompasses patients and caregivers, a population often suffering from physical and psychological problems that are not routinely or systematically assessed.
The objective of this study was to evaluate the feasibility of an electronic PRO collection (ePRO) system in caregivers of cancer patients.
Caregivers who completed the study were highly satisfied with the collection system. Considerations for future feasibility research include the documentation in electronic medical record (EMR) systems, the pattern and the impact of referral, and impact on provider workflow.
While ePRO collection systems can provide efficient and reliable outcome measures, instrument selection, technical capabilities, and study coordination considerations are essential for integrating web-based PRO measurement in healthcare settings.
ACKNOWLEDGEMENTS
The authors would like to thank Bagus Adiyanto, Gladys Wang, Yang Fann, Jim Witter, Sohee Shim, and Marina Noronha Ferraz de Arruda Colli, as well as the Northwestern University’s Department of Medical Social Sciences Informatics group for their support. We also thank the participants, their families, and the research teams of the NIH Clinical Center. This study was funded by the Intramural Research Program of the NIH ().
Footnotes
CONFLICT OF INTEREST
All authors have declared no conflict of interest.
REFERENCES
- 1.Vodicka E, Kim K, Devine EB, Gnanasakthy A, Scoggins JF, Patrick DL. Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007–2013). Contemp. Clin. Trials 2015;43:1–9. [DOI] [PubMed] [Google Scholar]
- 2.Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bottomley A, Pe M, Sloan J, et al. Analysing data from patient-reported outcome and quality of life endpoints for cancer clinical trials: a start in setting international standards. Lancet Oncol. 2016;17(11):e510–e514. [DOI] [PubMed] [Google Scholar]
- 4.Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual. Life Res. 2012;21(8):1305–1314. [DOI] [PubMed] [Google Scholar]
- 5.NQF. Patient Reported Outcomes (PROs) in Performance Measurement. Washington, DC: National Quality Forum;2013. [Google Scholar]
- 6.NAC. Cancer Caregiving in the U.S. An Intense, Episodic, and Challenging Care Experience. Bethesda, MD: National Alliance for Caregiving;2016. [Google Scholar]
- 7.HealthMeasures. PROMIS. 2017; http://www.healthmeasures.net/explore-measurement-systems/promis. Accessed 2/17/2017.
- 8.Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J. Clin. Oncol. 2016;34(6):557–565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kotronoulas G, Kearney N, Maguire R, et al. What Is the Value of the Routine Use of Patient-Reported Outcome Measures Toward Improvement of Patient Outcomes, Processes of Care, and Health Service Outcomes in Cancer Care? A Systematic Review of Controlled Trials. J. Clin. Oncol 2014;32(14):1480–1501. [DOI] [PubMed] [Google Scholar]
- 10.Kroenke K, Cheville AL. Symptom Improvement Requires More Than Screening and Feedback. J. Clin. Oncol 2016;34(27):3351–3352. [DOI] [PubMed] [Google Scholar]