Physicians’ Use of Patients’ Daily Reports of Quality of Life to Evaluate Treatment Response in Phase I Cancer Trials

Abstract

For cancer patients on Phase I trials, one of the most important physician decisions is whether or not patients are deriving benefit from therapy. With an increasing number of cytostatic treatment agents, the criteria to determine patient response to Phase I treatment has become harder to define. Physicians are increasingly looking to patient-reported outcomes (PROs) such as quality of life (QOL) to help evaluate treatment response. Electronic daily diary (EDD) devices can be used by patients to report their QOL over extended periods of time, thereby providing a more accurate picture of how patients are affected by treatment on a daily basis. However, questions remain about how to integrate this patient-reported information into decisions about Phase I treatment. This study investigated how physicians use patients’ daily QOL reports to evaluate patient response to Phase I treatment. Data were collected over a 4-month period from Phase I patients (N = 30) and physicians (N = 3) in an NCI-designated comprehensive cancer center. Patients completed daily QOL reports using EDD devices and physicians were provided with a summary of patients’ QOL before each visit. After the visit, doctors recorded their treatment decision and also rated the importance of four biomedical factors (Toxicity, Imaging, Labs, and Performance Status) and QOL in their treatment decision for that visit. Although physicians rated QOL as being very important in evaluating treatment response, in practice, when predictors of their decisions were analyzed, results showed they relied exclusively on biomedical data (Toxicity, Imaging) to make Phase I treatment decisions. Questions remain about the utility and effective integration of QOL and biomedical data in clinical decision-making processes in Phase I clinical trials.

1. Introduction

Although patient symptoms (e.g. Adverse Events) serve as the basis for labeling claims on drugs approved by the US Food and Drug Administration (FDA), symptom reports are largely based on physician, rather than patient, observations [1]. Both the FDA and the National Cancer Institute have proposed that, for measuring response to treatment, patient-reported outcomes (PROs) rather than a physician’s interpretation of patient functioning are the “gold standard” of assessment [2–5].

PROs have particular relevance in Phase I cancer clinical trials. Traditionally, these trials have focused on the efficacy of cytotoxic treatments, which are designed to kill cancer cells and rely on objective biomedical endpoints (e.g., tumor response, progression, toxicity) [2]. However, an increasing number of cytostatic agents (e.g., biologic targeted therapies, angiogenic inhibitors) are being evaluated as cancer treatments. Rather than killing cells, these agents are designed to suppress cell growth, and as a result, biomedical endpoints are more difficult to evaluate. Although guidelines exist for evaluating these agents (i.e. Response Evaluation Criteria in Solid Tumors (RECIST) guidelines [6]), the guidelines are better suited for evaluating outcomes of cytotoxic rather than cytostatic agents. Thus, PROs can provide an additional, and valuable, source of data for physicians to evaluate treatment response.

Obtaining PROs such as quality of life (QOL) can be of particular importance when different treatments show only minimal differences in tumor response and/or survival outcomes. Understanding how a patient’s QOL is impacted by treatment can provide critical information that may help in determining the best treatment and the best treatment course for that patient [2]. For example, biomedical endpoints do not capture many of the debilitating side effects of treatment, such as pain, fatigue, and depression; in contrast, patients’ reports of functioning can provide this valuable information [2]. As a result, physicians are increasingly using PROs to develop a richer understanding of the ways in which patients are impacted by treatment. Further, studies show that when used systematically PRO data improves clinical outcomes in cancer patients (e.g., patient-physician communication, patient satisfaction with care) [7].

Although traditionally PROs have been collected using paper-and-pencil methods, this methodology can be problematic. Handwritten reports can require additional time for scoring, delay the relay of information to medical staff, and be difficult to integrate with electronic records [8]. Many assessment measures also rely on one-time and/or retrospective reports of patient symptoms. This process of “looking back” can lead to inaccurate, incomplete, or misleading reports of patient well-being. Asking patients to summarize their QOL since the previous visit (which could be 3 days or 3 weeks ago depending on treatment cycles) may lead patients to incorrectly conclude that their symptoms have not changed much and/or that they have experienced little day-to-day variability in functioning. People may also assign meaning to events/experiences after the fact to make them more consistent with events/experiences that follow [9]. For example, a patient may not remember exactly how they felt (“I can’t remember if I was tired or not”) but make assumptions based on other information (“I had just finished treatment, so I must have been pretty tired after that”). Biases associated with retrospective recall of symptoms can be particularly problematic in patients undergoing intensive treatment (i.e. Phase I cancer patients) who frequently experience multiple symptoms with varying patterns of intensity across time.

One solution for concerns about the validity of PRO data is electronic daily diary (EDD) devices, which offer the ability to gather patient reports of multiple symptoms, such as QOL, that change daily [10–12]. Using a portable handheld device, patients can enter reports of their physical symptoms, mood, or QOL every day while at home engaging in their usual routines and activities. Thus, these devices offer a minimally intrusive way to gather patient reports that are free of biases associated with retrospective recall or even being in the doctor’s office [7]. Patients are also more likely to report symptoms when using electronic diaries rather than paper diaries [10].

Previous research has successfully used EDD devices to study health behaviors (e.g. smoking, alcohol use, exercise) in healthy populations [13–17] and symptoms in disease populations (e.g. asthma, heart disease, inflammatory bowel disease, rheumatoid arthritis) [18–21]. Comparatively, there are few reports of EDD device use in cancer patients, and no studies of cancer patients on clinical trials [7,22]. However, given the relatively low patient burden, EDD devices can be an ideal method for collecting information about patients’ QOL as reported by patients, thereby providing oncologists with an accurate and timely picture of the impact of treatment on patients to consider when evaluating treatment response [7].

Although collecting PRO data has been strongly advocated, it is not clear how well (or even if) physicians are making use of patient-reported QOL in evaluating treatment response to Phase I therapies. The data can be variable, integrating patient-reported data (i.e. QOL) with “objective” metrics (e.g. results of blood tests) can be confusing, and physicians may lack access or understanding of how to interpret this data [23].

The current study seeks to understand whether physicians, if provided with patient-reported QOL data prior to clinic visits, will find this information clinically meaningful in evaluating patients’ response to Phase I clinical cancer treatments. Specifically, we sought to understand: 1) Which biomedical and patient-reported decision factors are physicians using to evaluate Phase I treatment response, and 2) are these biomedical and patient-reported decision factors the same physicians subjectively rate as important in making treatment decisions in Phase I cancer patients?

2. Methods

2.1. Participants and Procedures

Participants were patients and physicians at an NCI-designated comprehensive cancer center in a large Mid-western city in the United States. Patient eligibility criteria were age ≥ 18; physician-confirmed eligibility for Phase I clinical cancer trial; and ability to read, write, and speak English. Physician eligibility criteria were 1) practicing physician at KCI, and 2) ability to enroll patients in Phase I trials. The study received approval from the hospital and university Institutional Review Boards. Participants signed informed consent and HIPAA documents in accordance with IRB regulations.

Patients were asked to complete daily QOL reports for approximately 4 months (which for most patients was at least 2 complete cycles of treatment). Prior to each clinic visit, patients’ QOL data was downloaded from the EDD device to a secure Web-based database. Using a scoring algorithm embedded in the database, a research assistant generated a summary report that displayed the patient’s highest, lowest, and average patient-reported QOL during the past 7 days. This summary report was provided to the physician prior to the start of the patient’s visit. After each visit, physicians completed the Physician Treatment Decision-Making questionnaire (described below) to record their perceived importance of biomedical and patient-reported decision factors in evaluating treatment response for that visit. Each patient’s medical record was reviewed after the visit to confirm the treatment decision recorded by the physician.

2.2. Measures

Patient Quality of Life (Patient-Reported QOL)

Patients were asked to report their daily quality of life using the 15-item QLQ-C15-PAL (Palliative Care), a version of the 30-item EORTC QLQ-C30 [24]. The original 30-item measure is one of the most widely used instruments for assessing physical and psychosocial symptoms patients with cancer in palliative settings [25]. The QLQ-C15-PAL yields scores that are directly comparable with the 30-item version and is considered a reliable and valid alternative to the 30-item scale [24]. Questions 1–14 are rated using a 4-point scale (ranging from 0 = “not at all” to 3 = “very much”). A total patient-reported QOL score was calculated by summing daily item responses for each patient for these 14 questions; higher scores reflected more negative patient-reported QOL outcomes.

Physician Treatment Decision-Making Questionnaire

After each patient visit, physicians were asked to report on three aspects of the patient visit:

• First, physicians recorded the Results of the following biomedical data: Toxicity, Lab Work, Imaging, and Performance Status [24]. Toxicity, Lab Work, and Imaging results were determined by routine lab and imaging tests performed on the patient prior to each visit. Performance Status was rated by the physician during the visit. Each of these results was recorded by the physician as either supporting more treatment (1 = “continue”) or stopping treatment (0 = “stop”). The exception was Imaging results, which the physician coded according to the patient’s response to treatment using a 3-point scale (0 = “Progression”, 1 = “Stable”, and 3 = “Response”). These responses were later collapsed into “continue” (Stable, Response) or “stop” (Progression) treatment categories.

• Second, physicians were asked to record their Treatment Decision for the visit (1 = “continue” or 0 = “stop”).

• Third, physicians rated their perceived Importance of the following factors in their treatment decision for that visit: Toxicity, Lab Work, Imaging, Performance Status, and Patient-Reported QOL. The rating scale ranged from 0 = “not important at all” to 5 = “extremely important”.

3. Results

All Phase I physicians (N = 3) agreed to participate in the study. Patient recruitment rate was 81% (30 of 37 patients approached agreed to participate). The sample, although largely Caucasian (only 6.7% minority), was almost evenly split by sex (47% female, 53% male) and had a wide age range (31 – 80 years old; M = 56.7; SD = 12.4). Table 1 shows demographic and disease characteristics for the patient sample split by sex.

Table 1.

Patient demographics and clinical factors.

	Total (n = 30)	Men (n = 16)	Women (n = 16)
Age (mean)	56.65 (SD = 12.41)	34.36 (SD = 18.15)	28.82 (SD = 15.39)
Education	43% ≤ HS diploma	38% ≤ HS diploma	50% ≤ HS diploma
Ethnicity	93% White	88% White	100% White
Marital status	77% married	75% married	79% married
Relationship length	31.59 years (16.66)	34.36 years (18.51)	28.8 years (15.39)
Work status	33% disabled 37% retired	38% disabled 38% retired	29% disabled 36% retired
Income	50% < $40K	13% < $40K	57% < $40K
Primary cancer (top 2 sites)	43% colorectal 10% lung, breast (tied)	38% colorectal 13% lung	50% colorectal 21% breast
Stage at diagnosis1	81% Stage 4	80% Stage 4	81% Stage 4

¹All patients had progressed to Stage 4 at time of entry into the study.

Average length of time in the study was 52 days (SD = 31.5, range = 4 – 142 days) and the average number of clinic visits per patient was 4.65. During the 4-month collection period, patients completed a total of 1390 daily reports; average number of daily reports per patient was 46.3 reports (SD = 30).

There was an 88% completion rate for daily reports with five patients completing 100% of their daily reports (range = 56% – 100%). Completion rates were not related to patient income, education, gender, or age, all ps > 0.05. Number of days in the study was unrelated to completion rates, patient income, education, gender, and age, all ps > 0.05.

Predictors of Treatment Decisions

Five decision factors were tested as predictors of physician treatment decisions: Toxicity, Lab Work, Imaging, Performance Status, and Patient-Reported QOL. The model was tested using GENMOD (SAS 9.1.3; SAS Institute, 2003) 9.1.3 and General Estimating Equations (GEE). GEE models control for non-independence in the data (i.e. patients having multiple reports) and allow for a binominal outcome variables [26,27]. The model used Physician Treatment Decision (continue or stop treatment) as an outcome variable and controlled for patients’ demographics (age, education, income, gender) and patient oncologist¹. As Physician Treatment Decision is a binary outcome, results were interpreted as odds ratio. The criterion for statistical significance for all analyses was set at p ≤ 0.05.

The GEE model used patients’ biomedical Results (Toxicity, Lab Work, Imaging, Performance Status) and Patient-Rated QOL (averaged across the 7 days prior to the visit) as predictors. Results showed the only significant predictors of Physician Treatment Decision were Toxicity (B = 5.02, SE = 1.23, odds ratio = 151.4, p < 0.001) and Imaging (B = 5.58, SE = 1.5, odds ratio = 265.1, p < 0.01). If Toxicity results were rated as within range, then physicians were 151 times more likely to continue treatment. Similarly, if the results of Imaging were rated as within range (i.e. Stable or Response), physicians were 265 times more likely to continue treatment.

Next, we compared the results of the GEE model, which showed the decision factors actually used by physicians, to factors physicians subjectively rated as being most important in evaluating treatment response. As shown in Table 2, physicians rated Toxicity (M = 4.63) as being the most important evaluation criteria, which was consistent with their treatment decisions. By comparison, Imaging was rated as only “somewhat important” (M = 2.06) and yet was a significant predictor of treatment decisions. In contrast, Patient-Reported QOL (M = 3.89), Labs (M = 4.18), and Performance Status (M = 4.0), which were subjectively rated by physicians as “very important”, did not significantly predict Physician Treatment Decision (p > 0.05).

Table 2.

Physician-rated importance of decision factors.

	Mean	Standard deviation	Range	Confidence interval
Toxicity	4.63	0.78	2 – 5	4.50 – 4.75
Labs	4.18	0.93	1 – 5	4.03 – 4.32
Imaging	2.06	1.83	0 – 5	1.77 – 2.35
Performance status	4.0	0.96	1 – 5	3.85 – 4.15
PT QOL	3.89	0.78	2 – 5	3.76 – 4.01

Note: All scales had a possible range of 0 = not at all important to 5 = extremely important.

4. Discussion

Although incorporating PRO data such as patient QOL into physician decision-making has been advocated for some time, it is not clear how well physicians are making use of patient reports in evaluating treatment response to Phase I therapies. At issue has been the use of reporting methodologies that rely on retrospective recall, whether physicians perceive QOL information to be useful, and if so, how this information is being used in treatment decision-making. This study sought to collect information about patients’ QOL over time as reported by patients using EDD devices, and further, to understand whether physicians, if provided with this patient-reported QOL information prior to visits, would find the information to be clinically meaningful in evaluating treatment response in a Phase I context.

Although patient-reported QOL data is increasingly being used in clinical cancer trials decision-making [28], few studies have collected daily QOL data from patients, which has the benefits of minimizing recall bias and increasing the validity of patients’ reports. By gathering reports over multiple days, as was done in the current study, QOL scores are less likely to contain random error variance, and as a result, increase the methodological strength of the study [29]. Our study further demonstrated the feasibility of collecting daily QOL reports from patients with advanced cancer who were participating in taxing Phase I clinical trials. We had both low refusal rates for participation and high completion rates for daily reports, reinforcing that adding daily diary methodology to clinical trials will not add significant patient burden. Further, although not explicitly studied, physician participants in our study spontaneously reported the daily diary information to be more useful than both their usual assessments and single-occasion patient reports of QOL elicited during clinic visits.

With respect to physicians’ treatment decisions, an interesting pattern of findings emerged when comparing decision factors rated as important by physicians in their treatment decisions to factors physicians actually used in making those decisions. With the exception of Imaging, physicians rated the other three biomedical factors—Toxicity, Labs, and Performance Status—as very important. The patient-reported QOL physicians received prior to each visit was also rated as very important in their decisions. Interestingly, the comparison of factors physicians rated as important compared to the factors they actually used in their decisions showed an interesting contradiction. That is, while patient-reported QOL was perceived by physicians as very important in evaluating a patient’s response to treatment, patient-reported QOL data was not actually factored into decisions about whether or not to continue with treatment. In reality, treatment decisions were based only on the results of Toxicity and Imaging tests.

Research on decision-making would suggest that treatment decisions based on toxicity and imaging alone are consistent with “heuristic processing” models in which decisions are more likely to be made based on a few readily available and applicable factors as the number of decision factors increases [30,31]. A second potential explanation is that these findings are unique to a Phase I context in which patients often have few other treatment options. All things being equal, physicians may be more likely to continue a Phase I patient on treatment, despite reports of poor QOL, than a patient who has less advanced disease and/or other available treatment options (e.g. second or third line therapies). In a Phase I context, the default may likely be to continue a patient on treatment until otherwise disallowed by criteria specified by the trial protocol (i.e. toxicity, disease progression). In fact, our data, which showed 78% of visits resulted in treatment despite very poor patient-reported QOL, provide support for this explanation.

Decisions about treatment, however, may be very different in a Phase II or Phase III treatment situation. The possibility of alternate treatment options combined with less advanced disease and/or a more positive prognosis likely provides physicians more “degrees of freedom” in their decisions about whether to keep a patient with relatively poor QOL on treatment. As a result, greater weight may be placed on patient-reported QOL, and in particular poor patient-reported QOL, given that other treatment agents with less adverse effects for patients are available. In short, patient-reported QOL may be more influential in decisions about Phase II or III treatment than in Phase I treatment decisions precisely because there are more treatment options from which to choose. Rather than more decision options leading to reliance on heuristic processing (e.g. using what is available) or “choice overload” [32], a larger set of available treatment options may in reality provide the physician with more latitude and flexibility to respond to patients’ subjective response to treatment, particularly if that response is negative.

Future studies would benefit from examining the use and utility of patient-reported QOL in treatment decision-making in other contexts, including both Phase II/III and non-trial settings. The parameters by which treatment response is evaluated in these contexts likely deviate from those in a Phase I trial. It is also possible that other clinical factors, such as level of side effects (mild, moderate, severe) and the extent to which these can be controlled, might be influencing treatment decisions. That is, if a patient is experiencing treatment side effects, but those effects can be controlled, then treatment may be more likely to continue than when the side effects are uncontrollable.

4.1. Limitations

While this study design had methodological strengths (i.e. daily diary methodology, PRO data), our findings suggest considering the possibility of alternate explanations. It is possible that physicians’ subjective importance ratings of patient QOL reports may in part be influenced by a desire to provide socially desirable answers to the question. That is, physicians may believe that they should find patient-reported QOL reports important even if they are unlikely to use them to make treatment decisions. Equally plausible is that patient QOL was not accurately measured. Although given the methodological strength and rigor of our data collection methods (e.g. daily diary, EORTC QOL measure), we feel confident that our findings are not the result of measurement error. Yet another possibility is that physicians did not receive or received insufficient information about patient-reported QOL or were unsure how to interpret results. A research assistant was present in clinic to personally provide each physician with patient-reported QOL summary reports prior to every visit, all but ensuring the same access to and understanding of patient-reported QOL information prior to making a treatment decision. Further, treatment decision questionnaires were presented and explained to each physician prior to the visit; however, it is possible that further training or explanation was required or that a different presentation of QOL data—per-haps a visual data summary—might have been more effective. In fact, we are planning to use visual summaries in a physician-making intervention we are currently piloting and hope to shed some light on this issue of method of presentation. We also acknowledge that our sample size was relatively small, and future studies would benefit from testing these research questions in larger, more diverse samples.

4.2. Summary

Toxicity and Imaging were the most important factors in physicians’ decisions in Phase I treatment situations. In contrast, patient-reported QOL did not factor into physicians’ treatment decisions, despite physicians rating this patient information as subjectively important in evaluating treatment response. These findings suggest a need to further explore how PRO such as QOL can be better utilized and integrated with existing biomedical data in Phase I trials to be more clinical meaningful even with the often stringent criteria of trial protocols.

Abbreviations

PRO

Patient-Reported Outcomes

EDD

Electronic Daily Diary

QOL

Quality of Life

FDA

US Food and Drug Administration

RECIST

Response Evaluation Criteria in Solid Tumors