Abstract
Purpose
A cross-sectional study was conducted to investigate the feasibility of implementing routine collection of the Euro-Qol 5 dimensions (EQ-5D) questionnaire, to inform drug and health technology reimbursement decision making.
Methods
Women with breast cancer were recruited during scheduled clinic visits to an academic cancer centre. EQ-5D-5L was self-administered using electronic tablets. Diagnostic and treatment data were abstracted from patient charts. Feasibility was assessed primarily by the proportion of patients who fully completed EQ-5D-5L and by their willingness to complete the instrument at each clinic visit.
Results
588 women were approached for study participation, 341 were enrolled. Fully completed EQ-5D-5L questionnaires were obtained in 323 participants (95% of participants, 95% CI 92–97%). Median time for EQ-5D-5L completion was 1.5 minutes (range:0.35 to 14.7). Mean age of participants was 58 years old. Most women who completed EQ-5D were White, born outside Canada and presented a high education level; one-quarter had metastatic disease. Most participants reported “No problems” in all EQ-5D-5L dimensions. Mean EQ-5D-5L index and mean EQ-5D-5L VAS values for all participants were respectively 0.83 (SD 0.13) and 75.7 (SD 17.45), with patients with metastatic disease scoring the lowest values. Seventy-eight percent of participants were willing to complete EQ-5D-5L at each clinic visit; lower Charlson comorbidity index and higher education level were predictors of willingness to continue to answer EQ-5D-5L.
Conclusions
Tablet-based collection of EQ-5D-5L in the context of routine clinical practice proved to be feasible. However, many patients declined study participation or reported being in full health, raising concerns about whether this method of collecting EQ-5D adequately represents the health status of all breast cancer patients.
Introduction
Breast cancer is the most frequent cancer in the world among women [1]. In developed countries, survival has improved significantly in the last decades, as a result of screening programs and more effective treatments [2–5]. Traditional outcome measures, such as overall survival, are central to cancer decision-making. However, as the number of breast cancer survivors increases, up-to-date real-world health-related quality of life (HRQOL) data can provide additional information regarding the overall burden of cancer and the effectiveness of medical interventions [6–8].
Preference-based HRQOL measures, in the form of utilities, are used by many countries in economic evaluations that inform drug and health technology reimbursement decision making [9–12]. Utilities are used to calculate survival, adjusted for quality of life, as Quality-Adjusted Life Years (QALYs) [13–16]. Cost per QALY estimates are often sensitive to utility values. However, utility studies often have small sample sizes and use a range of methods [17]. A sustainable and methodologically rigorous approach to collecting preference-based HRQOL measures at a population-level, across the full spectrum of a disease, could have broad benefits for epidemiological and economic analyses.
In Ontario, the Canadian province with the greatest population (13.5 million), regional cancer centres have successfully implemented routine standardized electronic symptom and functional status screening as part of routine care for ambulatory patients, demonstrating that patient-reported outcome (PRO) collection at population-level, with the primary purpose of improving individual patient care, is feasible [18].
In this study, we investigated the feasibility of administering the EuroQol 5-dimensions (EQ-5D) questionnaire, a generic measure of health status, in the context of routine clinical practice, in a cancer centre in Ontario. In this context, the purpose of EQ-5D collection was to obtain data to inform drug and health technology reimbursement decision making. EQ-5D scores, although available to clinicians, were not intended to be used for improving individual patient care. Other studies have investigated EQ-5D collection, but have significant differences in terms of objectives, methodology, population included and setting, when compared to our study. They either included the general population or patients with other diseases, used trained research assistants to help administer the questionnaires, discussed EQ-5D results with patients, assessed both patients´ and doctors’ perspectives or the equivalence of paper and electronic versions of EQ-5D, among other differences [19–21].
Thus, we were interested in assessing feasibility in the context when data collection was not anticipated to have direct individual benefit for respondents. Feasibility was judged by examining several outcomes, including completeness of EQ-5D responses and the willingness of patients to continue to answer the questionnaire at subsequent clinic visits.
Methods
Research design, setting and participants
We conducted a cross-sectional study to assess the feasibility of routinely collecting EQ-5D-5L questionnaires among a convenience sample of women with breast cancer, recruited during clinic visits at the Louise Temerty Breast Cancer Centre (Sunnybrook Health Sciences Centre, Toronto). Participants were eligible for the study if they were: 1) females over 17 years of age; 2) diagnosed with stages I to IV breast cancer; 3) undergoing treatment or active surveillance for their cancer; and 4) English literate. Women were excluded if they could not give written informed consent or complete the study questionnaires on their own. Each woman was asked to participate only once.
The study was designed to interfere as little as possible with the normal clinical workflow. Patients were approached during their scheduled clinic visit, preferably after completing Symptom Screening and Functional Status questionnaires, which are routinely administered in waiting room kiosks. All cancer centres in Ontario have such kiosks (Fig 1).
Fig 1. Standard of care and study procedures.
Patients were approached for study participation after completing the standard of care questionnaires.
The Symptom Screening Questionnaire (ESAS, Edmonton Symptom Assessment System) is a self-reported instrument that measures nine common symptoms of cancer: pain, tiredness, drowsiness, nausea, lack of appetite, shortness of breath, depression, anxiety, and well-being [22,23]. ESAS responses are scored on a scale from 0 (no symptoms) to 10 (worst possible symptom). The Patient-Reported Functional Status tool consists of a version of the Eastern Cooperative Oncology Group performance status tool [18]. It was designed to be completed by the patient and asks the patients to rate their activity in the past month from 0 (“normal with no limitation”) to 4 (“pretty much bedridden, rarely out of bed”). Both questionnaires take less than 3 minutes to be competed [24,25]. Answers to these questionnaires are reviewed by the clinical care team during the appointment.
Written informed consent was obtained prior to any study procedures. While the answers to the standard of care questionnaires were meant to be discussed with the clinical care team, study participants were informed in the written informed consent and orally during recruitment, that EQ-5D-5L answers would not be discussed during the visit or in the future if the questionnaire was also implemented in routine clinical practice. The main objective of EQ-5D collection was to obtain data for drug and health technology reimbursement and decision making.
The study questionnaires were self-administered using electronic tablets (iPad®). Participants could both interrupt answering the study questionnaires at any time and return to answering the questionnaires later during their visit. The recruiters recorded reasons that patients chose not to join the study if they volunteered a reason.
The wireless hospital internet network was used to access the questionnaires on the tablets through a specific web link. We abstracted diagnostic and treatment data from patient charts. All information was stored in the main research database.
EQ-5D-5L health questionnaire
The EQ-5D-5L questionnaire consists of the EQ-5D-5L descriptive system and the EQ visual analog scale (EQ VAS). EQ-5D-5L has been formally translated and validated into Canadian English and French [26]. The descriptive system includes 5 dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has 5 levels: no problems (level 1), slight problems (level 2), moderate problems (level 3), severe problems (level 4), extreme problems (level 5). A unique health state is defined by one level from each of the five dimensions (for example, 11111, if no problems in all 5 dimensions). Each health state is converted into a utility, a cardinal value anchored at 0 (representing death) and 1 (representing full health), by applying a formula that attaches weights to each level in each dimension of the descriptive system [27]. Values below 0, which represent states worse than death, are also possible [28]. We used weights from the Canadian EQ-5D-5L value set estimated by Xie et al [27]. The EQ VAS records the respondents’ self-rated health on a vertical, visual analogue scale numbered 0 to 100, where the endpoints are labelled “The best health you can imagine” (100) and “The worst health you can imagine” (0).
The EQ-5D-5L was followed by a question assessing participants’ ongoing willingness to answer the questionnaire at subsequent clinic visits. Participants were also asked to provide additional demographic and clinical information.
Classification into breast cancer states
We classified study participants into five mutually exclusive breast cancer states, based on their disease status at the time of recruitment. Breast cancer states were defined based on a review of the literature, with the intention of being relevant both to clinical practice and economic modeling [17,29]:
“First year after primary breast cancer” (State 1): Patients diagnosed with invasive breast cancer (stage I to III) in the 12 months prior to study enrolment and were being treated with curative intent at the time of recruitment or before.
“First year after recurrence or new primary breast cancer” (State 2): Patients diagnosed with at least a second metachronous invasive breast cancer or a local-regional recurrence treated with curative intent, in the 12 months prior to study enrolment and without metastatic disease.
“Second to fifth year after a primary breast cancer or recurrence treated with curative intent” (State 3): Patients diagnosed with the latest primary invasive breast cancer or a loco-regional recurrence treated with curative intent more than one year, but fewer than five years, prior to study enrolment and without metastatic disease.
“Sixth and following years after a primary breast cancer or recurrence treated with curative intent” (State 4): Patients diagnosed with the latest primary invasive breast cancer or a loco-regional recurrence more than five years prior to study enrolment and without metastatic disease.
“Metastatic Breast Cancer” (State 5): Patients with distant or inoperable disease.
Outcomes
The main outcome was the proportion of participants who fully completed EQ-5D-5L. We also assessed the proportion of eligible patients recruited, the proportion of patients who were ineligible, and the proportion of patients who declined study participation. Other secondary outcomes included: 1) The proportion of participants willing to complete the EQ-5D-5L at each clinic visit; 2) Median completion time for the EQ-5D-5L; 3) Clinical and socio-demographic characteristics associated with willingness to continue to complete the EQ-5D routinely at future clinic visits.
Sample size
The provincial standard for performance in symptom screening is that each Regional Cancer Centre in Ontario screen 70% of their ambulatory oncology patients at least once per month using the Symptom Screening questionnaire [18]. Based on this standard, we a priori defined feasible routine administration of EQ-5D-5L as at least 70% of recruited patients fully completing the questionnaire and defined a fully completed questionnaire as one in which all the items had been answered. We aimed for a sample size of 341 patients, which produced a two-sided 95% confidence interval with a width equal to 10%, when the proportion of patients who fully complete the EQ-5D-5L was 70%.
Analysis
Descriptive statistics were calculated for all variables and study outcomes. We used the chi-square test or Fisher’s exact test for categorical variables to compare characteristics of patients willing to continue to answer EQ-5D-5L routinely and those who were not willing. Multivariable logistic regression was used to identify predictors of willingness to continue to answer EQ-5D-5L routinely. We hypothesised that younger age (<45 years old vs. older), a lower Charlson Comorbidity Index (0 versus higher), higher education level (above 8th grade vs. 8th grade or lower), native English speaking, and absence of metastatic disease (Health States 1–4 versus 5) would be associated with willingness to continue to answer the EQ-5D-5L routinely. In an exploratory analysis, we tested the hypothesis that “no problems” in each of EQ-5D-5L dimensions would also be associated with willingness to continue to answer EQ-5D-5L routinely. We assessed open-ended questions, such as the reasons offered for non-consent, qualitatively.
Statistical analysis was performed using SAS version 9.4 (SAS Institute, Cary, NC). A two-tailed p-value less than 0.05 was considered statistically significant.
This study was approved by the Sunnybrook’s research ethics board (protocol number 2551) and by the University of Toronto Research Ethics Board (protocol number 38199) and was performed in line with the principles of the Declaration of Helsinki.
Results
Patient recruitment, eligibility, and EQ-5D-5L completion
Between November 2016 and June 2017, we approached 588 women, of whom 341 were eligible and enrolled (recruitment rate: 58%; 95% CI 0.54–0.62), 169 (29%) declined participation and 78 (13%) were ineligible (Fig 2). The most common reason for ineligibility was not being English literate (N = 53; 68%). Reasons for declining study participation were volunteered by 87 women (51%) and are presented in Table 1. The most common reasons were not feeling well enough to participate (N = 30; 34%), lack of time (N = 19; 22%), participation in other studies at the cancer centre (N = 13; 15%), and concerns over privacy or not wanting to provide access to personal health information (N = 12;14%). Only 6 women (7%) cited the use of tablets as a reason for declining participation.
Fig 2. Patient disposition.
Flow chart summarizing patient recruitment, eligibility, and primary outcome.
Table 1. Reasons given by patients for declining study participation.
| Reasons for declining study participation | Number of Patients (%) |
|---|---|
| Not feeling well enough to participate (too symptomatic or overwhelmed) | 30 (34%) |
| Lack of time to answer the survey | 19 (22%) |
| Participation (currently or in the past) in other studies involving surveys | 13 (15%) |
| Declined participation due to concerns over privacy (feared inappropriate access to private health information or declined chart review) | 12 (14%) |
| Does not like / know how to use tablets | 6 (7%) |
| Does not feel surveys are helpful to herself or other patients and/or preferred to speak with oncologist directly | 6 (7%) |
| Already answered the symptom screening questionnaire | 1 (1%) |
| Total | 87 |
Our primary study outcome was the proportion of fully completed EQ-5D-5L questionnaires by patients recruited; our estimate was 95% (323 of 341 patients recruited; 95% CI 92–97%; Fig 2). This number represented 63% (323/509; 95% CI 59–67%) of all eligible patients approached (Fig 2). Median time for EQ-5D-5L completion was 1.5 minutes (range: 0.35 to 14.7 minutes). Only 2% (N = 5) of the recruited patients were enrolled in a clinical trial at the time of recruitment. The Symptom Screening and Functional Status questionnaires were completed in the waiting room kiosks by 198 (61%) of the patients who completed EQ-5D-5L, as part of their routine care.
Answers to the EQ-5D-5L were missing for 18 (5%) of the patients enrolled (Fig 2). Of these, 4 (1%) participants were called for their appointments while they were starting to answer the study questionnaires and returned the tablets without answering any questions. Answers of the other 14 (4%) patients were lost during data transfer to the temporary database (either because the tablets were inactive for more than one hour and the system logged out for privacy reasons or because of interruptions in the wireless network). All data were successfully transferred from the temporary database to the main research database.
Patient characteristics
Participants who fully answered the EQ-5D-5L had a mean age of 58 years, and most were White (N = 172; 53%), born outside Canada (N = 182; 57%), and had a high level of education (N = 182; 56% attended or graduated college or university; N = 73; 23% had postgraduate or professional degrees) (Table 2). Although all participants were English literate, 35% (N = 115) primarily spoke other languages at home. Most participants (N = 228; 71%) had been diagnosed with breast cancer in the 7 years prior to enrollment and were receiving systemic treatment (N = 230; 71%). After we classified participants into our five mutually exclusive breast cancer states, only 5 were included in Health State 2 (“First Year after recurrence or new primary breast cancer”) (Table 2).
Table 2. Characteristics of the patients who completed the EQ-5D-5L.
| Characteristics | N = 323 (%) |
|---|---|
| Age (Mean; SD; min-max) | 58 (SD:12;28–90) |
|
Age < 45 years 45–64 years ≥65 years |
38 (12%) 192 (59%) 93 (29%) |
|
Menopausal Status Pre-menopausal Post-menopausal |
51 (16%) 227 (70%) |
|
Marital Status Married or common-law Separated, divorced, or widowed Single or never married |
225 (70%) 69 (21%) 27 (8%) |
|
Education ≤ Grade 8 Attended or graduated high school Attended or graduated college or university Postgraduate or professional degree |
11 (3%) 56 (17%) 182 (56%) 73 (23%) |
|
Employment Status Retired Unemployed Working full-time or part-time Other (e.g, on leave, disability) |
103 (32%) 28 (9%) 139 (43%) 51 (16%) |
|
Annual Family Income <$30,000 $30,000 to $59,999 $60,000 to $89,999 $90,000 to $119,999 $120,000 to $149,999 >$150,000 |
37 (11%) 58 (18%) 51 (16%) 26 (8%) 27 (8%) 47 (15%) |
|
Born in Canada Yes No |
129 (40%) 183 (57%) |
|
Primary Language spoken at home English Other |
205 (63%) 115 (36%) |
|
Racial or ethnic group Asian—East (eg, Chinese, Japanese, Korean) Asian—South (eg, Indian, Pakistani, Sri Lankan) Asian—South East (eg, Malaysian, Filipino, Vietnamese) Black—Caribbean (eg, Barbadian, Jamaican) Indian—Caribbean (eg, Guyanese with origins in India) Latin American (eg, Argentinean, Chilean, Salvadoran) Middle Eastern (eg, Egyptian, Iranian, Lebanese) Mixed heritage (eg, Black—African and White—North American) White—European (eg, English, Italian, Portuguese, Russian) White—North American (eg, Canadian, American) |
45 (14%) 22 (7%) 22 (7%) 8 (2%) 8 (2%) 9 (3%) 26 (8%) 3 (1%) 65 (20%) 107 (33%) |
|
Charlson Comorbidity Index 0 1–2 ≥3 |
221 (68%) 85 (26%) 17 (5%) |
|
Breast Cancer stage at diagnosis Stage I Stage II Stage III Stage IV |
100 (31%) 142 (44%) 61 (19%) 16 (5%) |
|
Breast Cancer Subtype Hormone-Receptor Positive HER2 Positive Triple negative |
223 (69%) 55 (17%) 40 (12%) |
|
Breast cancer Surgery Mastectomy Lumpectomy SLNB ALND |
132 (41%) 159 (49%) 169 (52%) 118 (37%) |
|
Current treatment Chemotherapy (+/- targeted therapy) Hormonal treatment (+/- targeted therapy) Targeted therapy (only) Radiotherapy |
48 (15%) 170 (53%) 12 (4%) 2 (1%) |
|
Breast Cancer State First year after primary breast cancer First year after recurrence or new primary breast cancer Second to fifth year after primary breast cancer or recurrence Sixth and following years after primary breast cancer or recurrence Metastatic Breast Cancer |
78 (24%) 5 (2%) 104 (32%) 55 (17%) 81 (25%) |
N, number; SD, standard deviation; Min, minimum; Max, maximum; SLNB, sentinel lymph node biopsy; ALND, axillary lymph node dissection.
EQ-5D-5L results
Across all health states, most participants answered the EQ-5D-5L questions reporting “No problems” (level 1) for “Mobility” and “Self-Care” (Table 3). Most patients in State 4, who were further away from their breast cancer diagnosis and had been treated with curative intent, reported “No Problems” (level 1) for “Usual Activity”, “Pain/Discomfort” and “Anxiety/ Depression” dimensions, while participants in the other States reported “Slight” to “Moderate” problems (levels 2 and 3, respectively). Very few participants reported “Severe” to “Extreme” problems (levels 4 and 5, respectively) in any dimension. As expected, patients with metastatic breast cancer (State 5) reported the greatest number of problems.
Table 3. EQ-5D-5L health profile by breast cancer state.
| EQ-5D Dimensions | Breast Cancer States (N; percent of column total) | Total (N = 323) |
|||||
|---|---|---|---|---|---|---|---|
| State 1 (N = 78) |
State 2 (N = 5) |
State 3 (N = 104) |
State 4 (N = 55) |
State 5 (N = 81) |
|||
| Mobility | No problems Slight problems Moderate problems Severe problems Extreme problems |
61; 78% 11; 14% 6; 8% - - |
3; 60% 2; 40% - - - |
72; 69% 22; 21% 9; 9% 1;1% - |
42; 76% 8; 15% 2; 4% 2; 4% 1; 2% |
42; 52% 25; 31% 10; 12% 3; 4% 1; 1% |
220; 68% 68; 21% 27; 8% 6; 2% 2; 1% |
| Self-Care | No problems Slight problems Moderate problems Severe problems Extreme problems |
66; 85% 10; 13% 1; 1% - 1; 1% |
3; 60% 2; 40% - - - |
92; 88% 10; 10% 2; 2% - - |
51; 93% 3; 5% 1; 2% - - |
63; 78% 13; 16% 4; 5% - 1; 1% |
275; 85% 38; 12% 8; 2% - 2; 1% |
| Usual Activity | No problems Slight problems Moderate problems Severe problems Extreme problems |
31; 40% 34; 44% 11; 14% 1; 1% 1; 1% |
1; 20% 2; 40% 2; 40% - - |
59; 57% 33; 32% 11; 11% 1; 1% - |
44; 80% 5; 9% 6; 11% - - |
28; 35% 35; 43% 15; 19% 1; 1% 2; 2% |
163; 50% 109; 34% 45; 14% 3; 1% 3; 1% |
| Pain/ Discomfort | No problems Slight problems Moderate problems Severe problems Extreme problems |
38; 49% 28; 36% 10; 13% 1; 1% 1; 1% |
1; 20% 2; 40% 2; 40% - - |
31; 30% 51; 49% 16; 15% 6; 6% - |
33; 60% 12; 22% 8; 15% 1; 2% 1; 2% |
20; 25% 33; 41% 23; 28% 4; 5% 1; 1% |
123; 38% 126; 39% 59; 18% 12; 4% 3; 1% |
| Anxiety/ Depression | No problems Slight problems Moderate problems Severe problems Extreme problems |
35; 45% 30; 38% 11; 14% 1; 1% 1; 1% |
3; 60% 1; 20% 1; 20% - - |
45; 43% 44; 42% 14; 13% 1; 1% - |
28; 51% 18; 33% 8; 15% 1; 2% - |
24; 30% 37; 46% 19; 23% 1; 1% - |
135; 42% 130; 40% 53; 16% 4; 1% 1; 0.3% |
State 1, “First year after primary breast cancer”; State 2, “First year after recurrence or new primary breast cancer”; State 3, “Second to fifth year after a primary breast cancer or recurrence treated with curative intent”; State 4, “Sixth and following years after a primary breast cancer or recurrence treated with curative intent”; State 5, “Metastatic Breast Cancer”.
The mean EQ-5D-5L utility value for all breast cancer patients was 0.83 (SD 0.13), with the highest values (0.86; SD 0.14) in State 4 and the lowest values (0.78; SD 0.14) in patients with State 5 (Table 4). The mean EQ-5D-5L VAS value (Table 4) for all breast cancer patients was 75.7 (SD 17.45), with the highest value -also for State 4 (81.7; SD14.0) and the lowest for State 5 (68.1; SD 19.4).
Table 4. EQ-5D-5L utility and VAS values by breast cancer state.
| EQ-5D-5L | Breast Cancer State | N | % | Mean | SD | Median | IQR | Min | Max |
|---|---|---|---|---|---|---|---|---|---|
| Index Values | State 1 | 78 | 24 | 0.84 | 0.14 | 0.87 | 0.82–0.93 | 0.23 | 0.95 |
| State 2 | 5 | 2 | 0.82 | 0.09 | 0.78 | 0.76–0.87 | 0.73 | 0.95 | |
| State 3 | 104 | 32 | 0.84 | 0.12 | 0.86 | 0.81–0.91 | 0.21 | 0.95 | |
| State 4 | 55 | 17 | 0.86 | 0.14 | 0.91 | 0.85–0.95 | 0.36 | 0.95 | |
| State 5 | 81 | 25 | 0.78 | 0.14 | 0.81 | 0.70–0.89 | 0.29 | 0.95 | |
| Total | 323 | 100 | 0.83 | 0.13 | 0.87 | 0.79–0.91 | 0.21 | 0.95 | |
| VAS values | State 1 | 78 | 24 | 76.0 | 18.7 | 80.0 | 61–90 | 10 | 100 |
| State 2 | 5 | 2 | 82.4 | 12.3 | 80.0 | 73–89 | 70 | 100 | |
| State 3 | 104 | 32 | 77.8 | 14.7 | 80;5 | 69–90 | 36 | 100 | |
| State 4 | 55 | 17 | 81.7 | 14.0 | 82.0 | 74–94 | 49 | 100 | |
| State 5 | 81 | 25 | 68.1 | 19.4 | 70.0 | 52–84 | 21 | 100 | |
| Total | 323 | 100 | 75.7 | 17.5 | 80.0 | 62–90 | 10 | 100 |
State 1, “First year after primary breast cancer”; State 2, “First year after recurrence or new primary breast cancer”; State 3, “Second to fifth year after a primary breast cancer or recurrence treated with curative intent”; State 4, “Sixth and following years after a primary breast cancer or recurrence treated with curative intent”; State 5, “Metastatic Breast Cancer”; N, number; SD, standard deviation; IQR, intra-quartile range; Min, minimum; Max, maximum.
Willingness to complete EQ-5D-5L routinely
Of participants who completed EQ-5D-5L, 135 (42%) said they would definitely continue to answer EQ-5D-5L routinely at subsequent clinic visits, 117 (36%) were very likely to do so, 42 (13%) were unsure, 18 (6%) were very unlikely, and 10 (3%) would definitely not (Table 5). Reasons for choosing one of the latter three options were provided by 32 participants (46%) (S1 Table) and frequently included: the time consuming/repetitive nature of the questions; taking time away from their appointment; not seeing the benefit of answering the questionnaire; indicating that the questions were not appropriate for their situation; stating that the EQ-5D questionnaire was too similar to the symptom screening questionnaire; not being symptomatic in most visits or being healthy; and preferring to discuss symptoms with as oncologist.
Table 5. Willingness to continue to answer EQ-5D-5L routinely at subsequent clinic visits.
| Willingness to answer EQ-5D-5L | Breast Cancer States (N; percentage of column total) | Total (N = 323) |
||||
|---|---|---|---|---|---|---|
| State 1 (N = 78) |
State 2 (N = 5) |
State 3 (N = 104) |
State 4 (N = 55) |
State 5 (N = 81) |
||
| Definitely | 34; 44% | - | 49; 47% | 23; 42% | 29;36% | 135; 42% |
| Very likely | 27; 35% | 3; 60% | 36; 35% | 22; 40% | 29; 36% | 117; 36% |
| Total | 61; 78% | 3; 60% | 85; 82% | 45; 82% | 58; 72% | 252; 78% |
| Unsure | 11; 14% | - | 13; 13% | 7; 13% | 11; 14% | 42; 13% |
| Very unlikely | 5; 6% | 1; 20% | 3; 3% | 1; 2% | 8; 10% | 18; 6% |
| Definitely not | - | 1; 20% | 3; 3% | 2; 4% | 4; 5% | 10; 3% |
| Total | 5; 6% | 2; 40% | 6; 6% | 3; 6% | 12; 15% | 28; 9% |
Participants were asked if they were willing to continue to answer EQ-5D-5L routinely, at subsequent clinic visits, if asked to do so. Possible answers were “Definitely”, “Very likely”, “Unsure”, “Very unlikely”, “Definitely not”. Patients who chose on of the latter 3 options, were asked to volunteer a reason for their answer. State 1, “First year after primary breast cancer”; State 2, “First year after recurrence or new primary breast cancer”; State 3, “Second to fifth year after a primary breast cancer or recurrence treated with curative intent”; State 4, “Sixth and following years after a primary breast cancer or recurrence treated with curative intent”; State 5, “Metastatic Breast Cancer”; N, number.
In the unadjusted analysis (S2 Table), the Charlson Comorbidity Index and the educational level were associated with willingness to answer EQ-5D routinely, while any level of reported “problems” in the EQ-5D-5L mobility dimension was associated with unwillingness to answer EQ-5D-5L routinely. In multivariable analysis (Table 6), patients with a Charlson comorbidity index of 1 to 2 (OR 0.77, 95% CI 0.40–1.49) and ≥ 3 (OR 0.25, 95% CI 0.08–0.73) were less willing to answer EQ-5D-5L routinely (joint p = 0.039) compared with patients with an index value of 0, and patients with an education level of high school (OR 11.98, 95% CI 2.60–55.29) or higher (College / University OR 10.95, 95% CI 2.61–45.90 and Postgraduated/ Professional OR 16.55, 95% CI 3.59–76.37) more willing to answer EQ-5D-5L routinely than patients with an eighth grade education or less (joint p = 0.004). In the exploratory analysis that added problems/ no problems in each of the EQ-5D-5L dimensions to the multivariable model, only the level of education continued to be significantly associated with willingness to continue to answer EQ-5D-5L at subsequent visits (S3 Table).
Table 6. Characteristics associated with willingness to continue to answer EQ-5D at each clinic visit.
| Characteristics | Willing N = 252; 78% |
Unsure / Unwilling N = 70; 22% |
Adjusted OR (95% CI) | p-value |
|---|---|---|---|---|
|
Age < 45 years 45–64 years ≥ 65 years |
29; 76% 152; 80% 71; 76% |
9; 24% 39; 20% 22; 24% |
reference 1.59 (0.66–3.84) 1.66 (0.61–4.52) |
0.555 |
|
Charlson Comorbidity Index 0 1–2 ≥ 3 |
179; 81% 64; 75% 9; 53% |
41; 19% 21; 25% 8; 47% |
reference 0.77 (0.40–1.49) 0.25 (0.08–0.73) |
0.039 |
|
Education Grade 8 education or less Some of completed High school Some of completed college or university Some of completed postgraduate / professional |
3; 27% 45;80% 142; 78% 62; 85% |
8; 73% 11; 20% 40; 22% 11; 15% |
reference 11.98 (2.60–55.29) 10.95 (2.61–45.90) 16.55 (3.59–76.37) |
0.004 |
|
Primary Language English Other |
163; 80% 87; 76% |
42; 20% 28; 24% |
reference 0.77 (0.43–1.38) |
0.388 |
|
Breast CancerState State 1 State 2 State 3 State 4 State 5 |
61; 79% 3; 60% 85; 82% 45; 82% 58; 72% |
16; 21% 2; 40% 19; 18% 10; 18% 23; 28% |
1.76 (0.79–3.90) 0.55 (0.08–3.61) 1.73 (0.84–3.57) 1.50 (0.62–3.64) reference |
0.429 |
We hypothesised that younger patients (younger than 45 years old), with a lower Charlson Comorbidity Index (0 versus higher), higher education level (versus ≤ 8 grade), English native speakers and who did not present metastatic disease (States 1–4 versus 5) would be more willing to continue to answer the EQ-5D-5L routinely. OR, odds ratio.
Discussion
We found that routine administration of EQ-5D-5L for breast cancer patients is feasible in a clinical care setting where patients routinely answer PROs using a web-based system, with 95% of patients fully completing EQ-5D-5L. The distribution of breast cancer staging and subtype in our study was typical of that described in the literature, but our pre-specified breast cancer states had limitations since we only recruited 5 patients to breast cancer state 2 [30].
The median time for EQ-5D-5L completion was acceptable and most participants were willing to complete EQ-5D-5L at subsequent clinic visits. Reasons given by participants for unwillingness to answer the questionnaire at subsequent visits included the overlap between some questions of EQ-5D-5L and the symptom screening questionnaire, not seeing the benefit of answering the questionnaire and indicating that the questions were not appropriate for their situation. Addressing these reasons is important for increasing patient participation and retention. The use of tablets was well-accepted and did not seem to interfere with study participation, although these findings might not be generalizable to all clinical settings, since patients in Ontario are used to answer PROs in electronic format.
However, the number of patients who fully completed EQ-5D-5L represents only 58% of presumed eligible patients approached for study participation. Although we were not able to determine if there were significant differences between study participants and non-participants, we found that participants were less willing to routinely answer the EQ-5D-5L if they had comorbidities or a lower education level. These results are exploratory and need confirmation, because the model was prone to overfitting because it had many predictors.
The mean EQ-5D-5L utility and VAS values obtained for all breast cancer patients were high (0.83 and 75.7, respectively). These results were lower than the normative values for EQ-5D-5L found for Canadian women, which were respectively 0.869 and 82.8, and close to the values reported for Canadian women with chronic health conditions (utility index 0.841 and VAS value 79.3) [31]. The highest values in our study, which are similar to “healthy” Canadian women, were observed for patients who had breast cancer treated with curative intent more than six years prior to enrolment, and the lowest for patients with metastatic disease Surprisingly, our patients reported “no problems” or a low level of “problems” in most EQ-5D-5L dimensions, even if we consider patients with metastatic disease, which raises the question of whether EQ-5D adequately captures the health status of these patients or whether our study population was representative of all breast cancer patients. The dimensions in which a higher level of problems was reported were “Usual activities” and “Anxiety / Depression”.
Our findings are comparable to those of other studies that have incorporated the collection of PRO questionnaires into the routine care of cancer patients. According to the Ontario Cancer System Quality Index, the screening rate for the Symptom Screening and Functional Status questionnaires for breast cancer patients, which are collected without the need of a consent form and are used in direct clinical care, was 53% in 2018. Population-based initiatives without a clinical care component that collect PRO from cancer survivors using clinical staff to obtain informed consent [32], have consent rates ranging from 61% (in the electronic Patient-reported Outcomes from Cancer Survivors (ePOCS) system) [26,33,34] to 95% (in the Cancer of the Prostate Strategic Urologic Research Endeavour (CaPSURE) database) [35–37] Within the ePOCS system, which also uses a web-based system for electronic capture of different PROs, including the EQ-5D, a decrease in questionnaire completion rates was observed over time, with 85% of recruited patients completing the questionnaires at baseline and 66% at 15 months, although 86% of recruited patients said they would be willing to use the system long-term [26]. We hypothesize that the cross-sectional nature of our study, as well as the explanation from study staff about why data was being collected, might explain the high completion rate for EQ-5D-5L. Participant fatigue might decrease participation over time in clinics that implement routine collection of the EQ-5D-5L. Alternatively, collection without informed consent may increase participation over what we have observed, although our study was very pragmatic apart from this requirement. The net effect of these factors is a topic for future study.
Our study has limitations. We recruited patients from an outpatient clinic of a single tertiary cancer centre, and although our study population is representative of the patients followed at our centre, these patients might not be representative of breast cancer patients in other settings. Patients recruited were mostly young, White, and well-educated. We did not include patients with more severe disease (patients admitted to hospital or to palliative care, for example) or long-term survivors who were no longer followed at the cancer centre. We excluded patients who were not English literate. Our study may also have been prone to non-response bias, as we included patients who were feeling relatively well, as patients reported feeling ill as the main reason for not participating, including being too symptomatic and too overwhelmed (for example, because they were waiting for a diagnosis or exam results).
Our findings can inform future initiatives regarding routine collection of PROs. Asking patients to complete EQ-5D-5L, the Symptom Screening and the Functional Status questionnaires together, at the time of their scheduled appointments, is an opportunity for patients to get acquainted with the web-based system and to collect data at important times in each patients’ cancer trajectory (e.g., diagnosis, start of a new treatment, progression of disease). The EQ-5D-5L is appealing to add as a routinely collected PRO because it is a short questionnaire that can be easily answered outside the clinic, in a computer, or on a smart phone, a few days before or after the clinic appointment, and even after patients are discharged from the cancer clinic, improving patient convenience, and potentially, participation and retention. The possibility of answering EQ-5D outside cancer clinics is especially relevant to capture data on quality-of-life over time and to include long-term survivors or patients at the end of life. In the future, a system of automatic electronic reminders could be set up at specified time points, to investigate if patient participation would increase.
Although the EQ-5D can be used in individual patient consultations, for example for supporting clinical decisions or for shared decision making, our primary goal with this study was not patient care but to use the collected data to inform drug and health technology reimbursement and decision making [38]. Providing patients with practical examples of how the data is used, and even explore possible applications of the data collected in clinical practice, might possibly increase their willingness to answer EQ-5D routinely [32]. Routinely collected EQ-5D data is already used to inform patient-provider decision making in other areas, such as knee or hip replacement surgery [39,40]. The clinical usefulness of integrating patient-reported outcome measures (including the Edmonton Symptom Assessment System- revised: Renal- ESAS-r and the EQ-5D-5L) in the clinical management of hemodialysis patients is being assessed in a randomized clinical trial in Alberta and Ontario (NCT03535922). Similar research is lacking in cancer patients.
Conclusions
In conclusion, our study demonstrated that tablet-based collection of EQ-5D-5L in the context of routine clinical practice was feasible for breast cancer patients. EQ-5D-5L can be easily administered, in addition to the Symptom Screening and Functional Status questionnaires. However, many patients declined study participation or reported being in full health, raising concerns about whether this method of collecting EQ-5D adequately represents the health status of all breast cancer patients.
Supporting information
(PDF)
(PDF)
(PDF)
Data Availability
Data cannot be shared publicly because in accordance with the study’s consent form, participant information collected will be handled in a confidential manner and will be stored on site in a secure, privacy protected data warehouse, developed under the support of The Canada Foundation for Innovation at Sunnybrook Research Institute. As such, study data should not be transferred to a public repository. Data are available for researchers who meet the criteria for access to confidential data. For data access please contact the Sunnybrook Research Ethics Board at 416-480- 6100 x 688144 or Reb@sunnybrook.ca.
Funding Statement
The authors received no specific funding for this work.
References
- 1.Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, et al. Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012. International Journal of Cancer. 2015;136(5):E359–E86. doi: 10.1002/ijc.29210 [DOI] [PubMed] [Google Scholar]
- 2.Shields M, Wilkins K. An update on mammography use in Canada. Health reports. 2009;20(3):7–19. Epub 2009/10/10. . [PubMed] [Google Scholar]
- 3.Holford TR, Cronin KA, Mariotto AB, Feuer EJ. Changing patterns in breast cancer incidence trends. Journal of the National Cancer Institute Monographs. 2006;(36):19–25. Epub 2006/10/13. doi: 10.1093/jncimonographs/lgj016 . [DOI] [PubMed] [Google Scholar]
- 4.Edwards BK, Brown ML, Wingo PA, Howe HL, Ward E, Ries LA, et al. Annual report to the nation on the status of cancer, 1975–2002, featuring population-based trends in cancer treatment. Journal of the National Cancer Institute. 2005;97(19):1407–27. Epub 2005/10/06. doi: 10.1093/jnci/dji289 . [DOI] [PubMed] [Google Scholar]
- 5.Canadian Cancer Statistics 2017. Toronto, Ontario: Canadian Cancer Society, 2017. [Google Scholar]
- 6.Basch E, Abernethy AP, Mullins CD, Reeve BB, Smith ML, Coons SJ, et al. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J Clin Oncol. 2012;30(34):4249–55. Epub 2012/10/17. doi: 10.1200/JCO.2012.42.5967 . [DOI] [PubMed] [Google Scholar]
- 7.Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22(8):1889–905. Epub 2013/01/05. doi: 10.1007/s11136-012-0344-y . [DOI] [PubMed] [Google Scholar]
- 8.Lipscomb J, Gotay CC, Snyder CF. Patient-reported outcomes in cancer: a review of recent research and policy initiatives. CA: a cancer journal for clinicians. 2007;57(5):278–300. Epub 2007/09/15. doi: 10.3322/CA.57.5.278 . [DOI] [PubMed] [Google Scholar]
- 9.Report from the Canadian Coordinating Office for Health Technology Assessment (CCOHTA). Guidelines for economic evaluation of pharmaceuticals: Canada. International journal of technology assessment in health care. 1995;11(4):796–7. Epub 1995/01/01. doi: 10.1017/s0266462300009247 . [DOI] [PubMed] [Google Scholar]
- 10.Detsky AS. Guidelines for economic analysis of pharmaceutical products. PharmacoEconomics. 1993;3(5):354–61. [DOI] [PubMed] [Google Scholar]
- 11.Brady B, Amin M, Mensinkai S. PHP26 CCOHTA GUIDELINES FOR THE ECONOMIC EVALUATION OF HEALTH TECHNOLOGIES: CANADA 2004. Value in Health. 2004;7(3):374–5. [Google Scholar]
- 12.Excellence NIfHaC. Guide to the Methods of Technology Appraisal. NICE. [PubMed]
- 13.Conner-Spady BL, Cumming C, Nabholtz JM, Jacobs P, Stewart D. A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation. Bone Marrow Transplantation. 2005;36(3):251–9. doi: 10.1038/sj.bmt.1705032 [DOI] [PubMed] [Google Scholar]
- 14.Hamashima C. Long-term quality of life of postoperative rectal cancer patients. Journal of gastroenterology and hepatology. 2002;17(5):571–6. Epub 2002/06/27. doi: 10.1046/j.1440-1746.2002.02712.x . [DOI] [PubMed] [Google Scholar]
- 15.Korfage IJ, Essink-Bot ML, Borsboom GJ, Madalinska JB, Kirkels WJ, Habbema JD, et al. Five-year follow-up of health-related quality of life after primary treatment of localized prostate cancer. Int J Cancer. 2005;116(2):291–6. Epub 2005/04/01. doi: 10.1002/ijc.21043 . [DOI] [PubMed] [Google Scholar]
- 16.Pickard AS, Wilke CT, Lin HW, Lloyd A. Health utilities using the EQ-5D in studies of cancer. PharmacoEconomics. 2007;25(5):365–84. Epub 2007/05/10. doi: 10.2165/00019053-200725050-00002 . [DOI] [PubMed] [Google Scholar]
- 17.Peasgood T, Ward SE, Brazier J. Health state utility values in breast cancer: A review and metaanalysis. Value in Health. 2011;14(7):A462. [DOI] [PubMed] [Google Scholar]
- 18.Pereira J, Green E, Molloy S, Dudgeon D, Howell D, Krzyzanowska MK, et al. Population-based standardized symptom screening: Cancer Care Ontario’s Edmonton Symptom Assessment System and performance status initiatives. Journal of oncology practice / American Society of Clinical Oncology. 2014;10(3):212–4. Epub 2014/04/24. doi: 10.1200/JOP.2014.001390 . [DOI] [PubMed] [Google Scholar]
- 19.Ng APP, Liu KSN, Cheng WHG, Wong CKH, Cheng JKY, Lam JSM, et al. Feasibility and acceptability of electronic EQ-5D-5L for routine measurement of HRQOL in patients with chronic musculoskeletal problems in Hong Kong primary care. Health and quality of life outcomes. 2022;20(1):137. doi: 10.1186/s12955-022-02047-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wallwiener M, Heindl F, Brucker SY, Taran FA, Hartkopf A, Overkamp F, et al. Implementation and Feasibility of Electronic Patient-Reported Outcome (ePRO) Data Entry in the PRAEGNANT Real-Time Advanced and Metastatic Breast Cancer Registry. Geburtshilfe Frauenheilkd. 2017;77(8):870–8. Epub 20170824. doi: 10.1055/s-0043-116223 ; PubMed Central PMCID: PMC5570586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bagattini ÂM, Camey SA, Miguel SR, Andrade MV, de Souza Noronha KVM, de C. Teixeira MA, et al. Electronic Version of the EQ-5D Quality-of-Life Questionnaire: Adaptation to a Brazilian Population Sample. Value in Health Regional Issues. 2018;17:88–93. 10.1016/j.vhri.2017.11.002. [DOI] [PubMed] [Google Scholar]
- 22.Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. Journal of palliative care. 1991;7(2):6–9. Epub 1991/01/01. . [PubMed] [Google Scholar]
- 23.Nekolaichuk C, Watanabe S, Beaumont C. The Edmonton Symptom Assessment System: a 15-year retrospective review of validation studies (1991–2006). Palliative medicine. 2008;22(2):111–22. Epub 2008/03/29. doi: 10.1177/0269216307087659 . [DOI] [PubMed] [Google Scholar]
- 24.Ontario CC. Available from: https://www.cancercareontario.ca/en/guidelines-advice/symptom-side-effect-management/symptom-assessment-tool.
- 25.Wong A, Tayjasanant S, Rodriguez-Nunez A, Park M, Liu D, Zapata KP, et al. Edmonton Symptom Assessment Scale Time Duration of Self-Completion Versus Assisted Completion in Patients with Advanced Cancer: A Randomized Comparison. Oncologist. 2021;26(2):165–71. Epub 20201221. doi: 10.1002/onco.13619 ; PubMed Central PMCID: PMC7873322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ashley L, Jones H, Thomas J, Newsham A, Downing A, Morris E, et al. Integrating patient reported outcomes with clinical cancer registry data: a feasibility study of the electronic Patient-Reported Outcomes From Cancer Survivors (ePOCS) system. Journal of medical Internet research. 2013;15(10):e230. Epub 2013/10/29. doi: 10.2196/jmir.2764 ; PubMed Central PMCID: PMC3841364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Xie F, Pullenayegum E, Gaebel K, Bansback N, Bryan S, Ohinmaa A, et al. A Time Trade-off-derived Value Set of the EQ-5D-5L for Canada. Medical care. 2016;54(1):98–105. Epub 2015/10/23. doi: 10.1097/MLR.0000000000000447 ; PubMed Central PMCID: PMC4674140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Froberg DG, Kane RL. Methodology for measuring health-state preferences—I: Measurement strategies. Journal of clinical epidemiology. 1989;42(4):345–54. Epub 1989/01/01. doi: 10.1016/0895-4356(89)90039-5 . [DOI] [PubMed] [Google Scholar]
- 29.Lidgren M, Wilking N, Jonsson B, Rehnberg C. Health related quality of life in different states of breast cancer. Quality of life research: an international journal of quality of life aspects of treatment, care and rehabilitation. 2007;16(6):1073–81. doi: 10.1007/s11136-007-9202-8 [DOI] [PubMed] [Google Scholar]
- 30.Voduc KD, Cheang MCU, Tyldesley S, Gelmon K, Nielsen TO, Kennecke H. Breast Cancer Subtypes and the Risk of Local and Regional Relapse. Journal of Clinical Oncology. 2010;28(10):1684–91. doi: 10.1200/JCO.2009.24.9284 . [DOI] [PubMed] [Google Scholar]
- 31.Yan J, Xie S, Johnson JA, Pullenayegum E, Ohinmaa A, Bryan S, et al. Canada population norms for the EQ-5D-5L. The European Journal of Health Economics. 2023. doi: 10.1007/s10198-023-01570-1 [DOI] [PubMed] [Google Scholar]
- 32.Corsini N, Fish J, Ramsey I, Sharplin G, Flight I, Damarell R, et al. Cancer survivorship monitoring systems for the collection of patient-reported outcomes: a systematic narrative review of international approaches. Journal of cancer survivorship: research and practice. 2017;11(4):486–97. Epub 2017/04/05. doi: 10.1007/s11764-017-0607-2 . [DOI] [PubMed] [Google Scholar]
- 33.Ashley L, Jones H, Forman D, Newsham A, Brown J, Downing A, et al. Feasibility test of a UK-scalable electronic system for regular collection of patient-reported outcome measures and linkage with clinical cancer registry data: the electronic Patient-reported Outcomes from Cancer Survivors (ePOCS) system. BMC medical informatics and decision making. 2011;11:66. Epub 2011/10/28. doi: 10.1186/1472-6947-11-66 ; PubMed Central PMCID: PMC3212976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Ashley L, Jones H, Thomas J, Forman D, Newsham A, Morris E, et al. Integrating cancer survivors’ experiences into UK cancer registries: design and development of the ePOCS system (electronic Patient-reported Outcomes from Cancer Survivors). British Journal of Cancer. 2011;105(1):S74–S81. doi: 10.1038/bjc.2011.424 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Lubeck DP, Litwin MS, Henning JM, Carroll PR. Measurement of Health-Related Quality of Life in Men with Prostate Cancer: The CaPSURE™ Database. Quality of Life Research. 1997;6(5):385–92. doi: 10.1023/a:1018439528024 [DOI] [PubMed] [Google Scholar]
- 36.Lubeck DP, Litwin MS, Henning JM, Stier DM, Mazonson P, Fisk R, et al. The CaPSURE database: a methodology for clinical practice and research in prostate cancer. CaPSURE Research Panel. Cancer of the Prostate Strategic Urologic Research Endeavor. Urology. 1996;48(5):773–7. Epub 1996/11/01. doi: 10.1016/s0090-4295(96)00226-9 . [DOI] [PubMed] [Google Scholar]
- 37.Porten SP, Cooperberg MR, Konety BR, Carroll PR. The example of CaPSURE: lessons learned from a national disease registry. World J Urol. 2011;29(3):265–71. Epub 02/24. doi: 10.1007/s00345-011-0658-3 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ernstsson O, Janssen MF, Heintz E. Collection and use of EQ-5D for follow-up, decision-making, and quality improvement in health care—the case of the Swedish National Quality Registries. Journal of Patient-Reported Outcomes. 2020;4(1):78. doi: 10.1186/s41687-020-00231-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Gutacker N, Street A. Use of large-scale HRQoL datasets to generate individualised predictions and inform patients about the likely benefit of surgery. Quality of Life Research. 2017;26(9):2497–505. doi: 10.1007/s11136-017-1599-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Bansback N, Trenaman L, MacDonald KV, Hawker G, Johnson JA, Stacey D, et al. An individualized patient-reported outcome measure (PROM) based patient decision aid and surgeon report for patients considering total knee arthroplasty: protocol for a pragmatic randomized controlled trial. BMC Musculoskelet Disord. 2019;20(1):89–. doi: 10.1186/s12891-019-2434-2 . [DOI] [PMC free article] [PubMed] [Google Scholar]