Skip to main content
Cancers logoLink to Cancers
. 2020 Dec 22;13(1):1. doi: 10.3390/cancers13010001

Incorporating the Patient Voice in Sarcoma Research: How Can We Assess Health-Related Quality of Life in This Heterogeneous Group of Patients? A Study Protocol

Dide den Hollander 1,2, Marco Fiore 3, Javier Martin-Broto 4, Bernd Kasper 5, Antonio Casado Herraez 6, Dagmara Kulis 7, Ioanna Nixon 8, Samantha C Sodergren 9, Martin Eichler 10, Winan J van Houdt 11, Ingrid M E Desar 2, Isabelle Ray-Coquard 12, Claire Piccinin 7, Hanna Kosela-Paterczyk 13, Aisha Miah 14, Leopold Hentschel 10, Susanne Singer 15,16, Roger Wilson 17, Winette T A van der Graaf 1,18,, Olga Husson 1,19,*,
PMCID: PMC7792589  PMID: 33561018

Abstract

Simple Summary

Sarcomas are a rare group of heterogeneous tumors. Because treatment side-effects detract from the survival benefit of treatment, it is important to assess treatment effectiveness in terms of patient-reported health-related quality of life (HRQoL). However, a sarcoma specific measurement instrument or strategy does currently not exist. This study aims to (1) develop a list of all HRQoL issues relevant to sarcoma patients; (2) determine a strategy for sarcoma-specific HRQoL assessment. An international, multicenter study will be conducted, searching existing literature and conducting interviews with adult sarcoma patients and healthcare professionals (HCPs) to create a list of HRQoL issues. Subsequently, another group of sarcoma patients and HCPs will be asked to rate and prioritize the issues. This information will help create a strategy to measure HRQoL in sarcoma patients, taking into account the heterogeneity of sarcomas, which will improve the collection of personalized patient-reported outcome data in future research and clinical care.

Abstract

Sarcomas comprise 1% of adult tumors and are very heterogeneous. Long-lasting and cumulative treatment side-effects detract from the (progression-free) survival benefit of treatment. Therefore, it is important to assess treatment effectiveness in terms of patient-reported outcomes (PROs), including health-related quality of life (HRQoL) as well. However, questionnaires capturing the unique issues of sarcoma patients are currently lacking. Given the heterogeneity of the disease, the development of such an instrument may be challenging. The study aims to (1) develop an exhaustive list of all HRQoL issues relevant to sarcoma patients and determine content validity; (2) determine a strategy for HRQoL measurement in sarcoma patients. We will conduct an international, multicenter, mixed-methods study (registered at clinicaltrials.gov: NCT04071704) among bone or soft tissue sarcoma patients ≥18 years, using EORTC Quality of Life Group questionnaire development guidelines. First, an exhaustive list of HRQoL issues will be generated, derived from literature and patient (n = 154) and healthcare professional (HCP) interviews (n = 30). Subsequently, another group of sarcoma patients (n = 475) and HCPs (n = 30) will be asked to rate and prioritize the issues. Responses will be analyzed by priority, prevalence and range of responses for each item. The outcome will be a framework for tailored HRQoL measurement in sarcoma patients, taking into account sociodemographic and clinical variables.

Keywords: sarcomas, soft tissue sarcoma, bone sarcoma, health-related quality of life, patient-reported outcomes

1. Introduction

Sarcomas are a group of rare malignant solid neoplasms of mesenchymal origin, comprising more than 100 histological and more than 250 molecular subtypes [1]. Significant heterogeneity exists among cases with widely different patterns of the stage at diagnosis, tumor location at almost any anatomical site, prognosis, treatments, and age at diagnosis [2]. Sarcomas can be classified into soft tissue sarcomas (STS), including gastrointestinal stromal tumors (GIST) (84% of all sarcomas) and bone sarcomas (BS; 14%), and other sarcomas [3]. The overall incidence of sarcoma is about 7 in 100,000 persons, with 30,000 new cases a year in Europe [3].

Approximately 10% of patients present with de novo metastatic disease. Furthermore, many sarcomas have aggressive biological behavior, and around half of patients with high-grade tumors will eventually develop incurable metastatic disease [4,5]. The five-year relative survival in Europe (period 2000–2007) is 60% for STS and 50% for BS [6]; however, survival in the metastatic setting is poor; for example, in STS, median overall survival is 12–20 months [6,7] but may be many years in exceptional cases.

Treatment for localized STS usually consists of conservative surgery with pre- or postoperative radiotherapy, however in retroperitoneal STS extended multivisceral resection is often needed in order to obtain complete resection and reduce the risk of recurrence [8]. The evidence for (neo-)adjuvant chemotherapy is controversial; however, it may be considered in high-risk tumors. In patients with advanced STS, not amenable to curative surgical resection, systemic treatment with chemotherapy is the mainstay of therapy with the aim to relieve symptoms, slow tumor progression and prolong survival. In some sarcoma subtypes, other systemic options are available, including tyrosine kinase inhibitors [9,10,11], but access may be limited in some countries. Immunotherapy has shown some promise in trials of some advanced STS. However, no treatment has yet been approved [12]. Surgical resection, stereotactic ablative radiotherapy or local treatments such as radiofrequency ablation of oligo-metastases can be considered in selected cases [13]. Generally, conventional radiotherapy is applied in the palliative setting, mainly for pain reduction.

The treatment of BS also differs per subtype. In patients with localized chondrosarcoma, surgery alone is the standard of treatment. Ewing sarcoma and osteosarcoma patients will have surgery in addition to (neo)adjuvant chemotherapy. For Ewing sarcoma patients, radiotherapy can be delivered either instead of surgery, in cases where surgery would be morbid or significantly impact functional outcomes, but more often in the (neo) adjuvant setting. For patients with advanced BS, palliative chemotherapy and/or radiotherapy are potential therapeutic options.

Despite the effectiveness of treatments for sarcoma, the varying degrees of long-lasting and cumulative treatment side-effects and morbidity in a substantial number of patients contribute negatively to their overall outcome [8,14,15]. Information on survival only is insufficient to determine the net clinical (or true) benefit of a treatment for patients, and subjective patient-reported outcomes (PROs), including health-related quality of life (HRQoL), should be considered as well. HRQoL is a multidimensional concept that includes the patient’s perception of the impact of their disease and treatment on physical, psychological, and social functioning in daily life [16]. Measurement of HRQoL facilitates the calculation of quality-adjusted life years (QALY), defined as “the measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life” [17]. PROs are increasingly used as important endpoints in cancer clinical trials, and implementation has become more standardized [18,19]. Furthermore, studies have shown that using PROs in clinical practice leads to better patient-provider communication, more symptom control and patients are more willing to self-report their symptoms from home [20,21,22,23].

The availability of high-quality HRQoL data of sarcoma patients is limited but shows they have more physical and psychological problems than the general population [24,25]. Patients experience disabling disease- and treatment-related symptoms such as fatigue and pain, interference in family, social and vocational life, and limitations in leisure activities, which can, in turn, impact their mental health and confidence [14,26,27]. Existing HRQoL data in sarcoma patients is mostly based on the use of generic (SF-36, EQ-5D) or cancer-generic (EORTC-QLQ-C30, FACT-G) questionnaires; however, these measures do not cover all sarcoma- or sarcoma treatment-specific problems. For example, in the PALETTE trial (EORTC 62072), a double-blind, randomized, phase 3 trial of pazopanib versus placebo as second-line or later treatment for patients with advanced STS, HRQoL was an exploratory endpoint, using the EORTC QLQ-C30 [28]. Although HRQoL did not improve with pazopanib, the observed improvement in progression-free survival without impairment of HRQoL was considered a relevant result. The toxicity profile of pazopanib was reflected in the patients’ self-reported symptoms (e.g., diarrhea) but did not translate into significantly worse overall global health status during treatment. However, several other side-effects common to angiogenesis inhibitors (e.g., hand-foot syndrome or skin reactions) were not assessed as PROs in this study. Another study reported that bone sarcoma survivors struggle with their altered appearance and display of functional impairments (e.g., walking with a limp or needing a crutch), which impacts their self-esteem and social life [29]. This is not covered by existing (cancer-)generic questionnaires. Thus, to fully understand and assess in detail the impact of this sarcoma treatment on all domains of HRQoL, in both clinical research and care, the use of HRQoL measure(s) that includes sarcoma-specific treatment symptoms or issues is needed.

Usually, the development of a tumor-specific module resolves the problem of insufficient content coverage of a generic measure. A comprehensive sarcoma-specific HRQoL measure for use in clinical trials is currently lacking, as existing HRQoL measures for sarcoma patients are focused on physical functioning only (e.g., Toronto extremity salvage score (TESS) as a measure of physical disability [30]) or patient experiences (e.g., sarcoma assessment measure (SAM) [31]). However, it was found during the development process of the SAM that tumor location and type of treatment received are important factors that determine which HRQoL issues subgroups of sarcoma patients are dealing with, in addition to established factors including sex and age [32,33]. For example, five issues (not specified) were identified to be only relevant in patients who had undergone amputation, and nine issues were relevant to head and neck sarcoma patients [31]. This heterogeneity of the disease was also reflected in HRQoL outcomes in German sarcoma patients and survivors [34]. These data show that the development of a single instrument is challenging, and it may not be the best option to meet the needs of academia and industry to assess the impact of new treatments [15,35].

The European Organization for Research and Treatment of Cancer Quality of Life Group (EORTC QLG) is currently developing a new measurement strategy, shifting to a more flexible approach that allows for the use of customized measures, in addition to the “static” standardized questionnaires, where relevant [36]. Traditionally, questionnaires are developed during a 4-phase process which takes several years, i.e., (1) generation of HRQoL issues, (2) construction of an item list, (3) pretesting the provisional questions, (4) field-testing to determine reliability, validity and responsiveness [37]. The new approach entails the use of a combination of standardized HRQoL questionnaires (i.e., the EORTC Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) and existing disease-, site, or population-specific modules to enable population comparison, and validated items from the so-called Item Library (an online platform comprised of more than 900 individual items from over 60 EORTC questionnaires) to cover the full range of issues relevant for the study population (depending on the research question) [38]. Item banks also enable computer adaptive testing (CAT) techniques, tailoring the selection of items based on responses to previous items [39,40]. This flexible approach offers the possibility to encompass the full range of age-, location- and treatment-specific issues relevant to sarcoma patients and to improve the relevance for patients in answering questions that are meaningful to them. Furthermore, it also allows for shorter questionnaires, which makes compliance to a repeat routine more likely and opens the way to longitudinal measurement, which patients find improves understanding [41].

This study aims to develop an exhaustive list of all HRQoL issues relevant to sarcoma patients (irrespective of their sarcoma stage, subtype, treatment, age) and to determine content validity. Subsequently, a strategy for HRQoL measurement in sarcoma patients will be determined that will allow for tailored, longitudinal assessment based on sociodemographic and clinical variables in future clinical trials.

2. Methods and Analysis

2.1. Study Design and Setting

We will conduct a cross-sectional, international, multicenter, mixed-method phase 1 questionnaire development study among bone and soft tissue sarcoma patients at more than thirty centers world-wide, following EORTC QLG questionnaire development guidelines [37]. First, a list of HRQOL issues will be generated, derived from the available literature and patient- and healthcare professional (HCP) interviews. Subsequently, another group of sarcoma patients and HCPs will be asked to rate the issues on relevance and to prioritize the most important issues. Clinical data will be extracted from patient records. The study started already with recruitment at the first center in May 2019, and a systematic review of the literature has recently been published [42].

2.2. Patient and Public Involvement

Implementation of HRQoL and other PRO assessments in sarcoma research is one of the research priorities of the Sarcoma Patients EuroNet (SPAEN), an international network of sarcoma, GIST, and Desmoid patient advocacy groups. Furthermore, public involvement is an integral aspect of patient-reported outcome measure (PROM) development [43]. Patient representatives were involved in the design of the study (feedback on the protocol and patient information form) and will be involved in reporting the results. In several countries, local patient advocacy groups will disseminate information about this study among their members to increase awareness about this project and to encourage participation. Study results will also be shared with patients through these organizations.

2.3. Eligibility Criteria

Inclusion criteria are: (1) confirmed diagnosis of sarcoma (according to the ICD-10 codes C40 and C41 for bone sarcoma and C49 for soft-tissue sarcoma). Patients with all stages of the disease, and both during treatment or posttreatment follow-up, will be included. The primary types of treatment to be included are surgery, radiotherapy, systemic treatment, or a combination; (2) age ≥ 18 years. Although sarcoma is a disease affecting all ages and sarcoma patients aged < 18 years are seen within adult health services in some countries, only those aged 18 years and older will be included because the EORTC QLQ-C30 is only validated among cancer patients aged 18 years and older. The EORTC Quality of Life Group is currently conducting a study to identify unique HRQoL issues specific to adolescents and young adults (AYAs; aged 14–39 years) with cancer and to validate the EORTC QLQ-C30 in young people with cancer [44]. Depending on the results of this study, a measurement strategy for AYA sarcoma patients can be determined at a later stage (3) having the mental capacity to provide informed consent and participate in the study (as determined by their treating physician). Patients must be able to rate or complete questionnaires themselves, as a prerequisite for patient-reported outcome measures; (4) medical professionals, e.g., surgical oncologist, medical oncologist, radiation oncologist or clinical oncologist, nurse, social worker, psychologist, a physiotherapist with experience of working with sarcoma patients.

Exclusion criteria are: (1) being too ill (on the judgment of the treating physician); the following diagnoses: (2) GIST; (3) Kaposi sarcoma; (4) carcinosarcoma; (5) benign and locally aggressive mesenchymal tumors.

2.4. Data Collection

2.4.1. Literature Review

First, a search of the literature through four medical search engines has been performed to identify all relevant HRQoL issues and existing HRQoL questionnaires currently used among sarcoma patients. The search strategy was created with the help of a librarian and combined medical subject headings (MeSH) terms and keywords for sarcoma and quality of life [42]. The reference lists of all identified publications will be reviewed to retrieve other relevant publications. There will be no restrictions with regard to the year of publication. This literature search will result in a list of HRQoL issues relevant to sarcoma patients, which will be used to develop the issue list.

2.4.2. Part A: Interviews

First, the respondent will be asked an open question about his/her sarcoma history and experience and then be prompted according to the interview schedules (one for patients and one for HCPs) that will be provided for all collaborators to ensure homogeneity in data collection (Supplementary Material, text S1). Subsequently, the respondent will be asked to review the EORTC QLQ-C30 and a site-specific module, if available, for the topic and item relevance (to be scored on a 4-point Likert scale of relevance for the survivorship period). Those with a sarcoma in the head and neck region will be asked to complete the EORTC QLQ-H&N43 and those with a uterine sarcoma the EORTC QLQ-EN24 to determine the relevance of these existing site-specific items of these questionnaires in sarcoma patients. All patients will receive a copy in their own language. The respondent will be encouraged to “think aloud”, providing feedback on the reasons for his/her ratings. The respondents will also be asked to identify HRQoL issues that they believe to be important that are not included in the current core questionnaire and, where relevant, site-specific module.

This list of HRQoL issues generated by the literature search and semi-structured patient and HCP interviews will be consolidated into a comprehensive list of issues for all languages of collaborating countries.

2.4.3. Part B: Review Issue-List

The new list of HRQoL issues will be administered to another group of sarcoma patients and HCPs. Patients and HCPs will be asked to rate the HRQoL issues on relevance (4-point Likert scale: (1) not relevant–(4) very relevant), to prioritize the 25 most important issues, and to indicate relevant issues missing from this list. Patients will also be asked to complete the EORTC QLQ-C30. In addition, those with a sarcoma in the head and neck region are asked to complete the EORTC QLQ-HN43 and those with a uterine sarcoma the EORTC QLQ-EN24.

2.5. Recruitment

Patients will be recruited from Northern Europe (e.g., Germany, The Netherlands, Norway), Central/Eastern Europe (e.g., Poland), Southern Europe (e.g., Cyprus, France, Italy, Spain), the United Kingdom and also in countries outside Europe, including Australia, Canada, Hong Kong, India, Israel, Jordan, and the United States.

The study coordinator will use stratification to ensure that subjects represent the target population and will communicate to the local collaborators which patients can be invited. Purposive sampling will aim to achieve good representation according to sex, age, sarcoma location and treatment. When selecting the patients for interviews, care will be taken to include both younger and older patients, males and females, all types of treatment, as well as newly diagnosed patients and patients in posttreatment follow-up. We will stratify our sample according to the 6 locations of sarcoma (upper and lower extremities, axial skeleton, retroperitoneal/intra-abdominal, head and neck including the scalp, thorax including breast, and pelvic organs including urogenital), disease stage (localized, metastatic disease/local relapse) and subtype (bone sarcoma, soft tissue sarcoma) to ensure the heterogeneity of sarcomas in terms of subtypes, treatments, age, sex, and physical impairments is adequately covered (Table 1). These characteristics are based on the clinical experience of health care professionals and researchers in the sarcoma field and input from patient representatives.

Table 1.

Stratification matrix phase 1a and 1b.

Stratification by Disease Stage, Subtype and Treatment Extremities Axial Skeleton (Including Pelvic Bones) Head and Neck (Including Scalp) Thoracic (Including Breast) Retroperitoneal and Intra-Abdominal Pelvic Organs Including Urogenital Total
Phase 1a
Localized
Bone
SU 4 2 2 - a - -
SU + RT 4 4 - 74
SU(+RT) + ST 4 4 -
Soft tissue
SU 2 - 2 6 6 10
SU+RT 8 - 2 4 4 -
Metastatic/local relapse 80
Bone
Local treatment (+ST) 4 2 2 - a - -
ST 6 8 8 - -
Soft tissue
Local treatment (+ST) 2 - - - - -
ST 6 - 8 8 8
BSC 2 - 2 2 2 2 10
Total
Per location 154
Per treatment
SU only: 34
SU + RT: 26
SU + RT + ST: 14
Surgery total: 74
RT total: 42 20 32 20 20 20 40
ST only: 60
ST total: 74
Local: 10
Per subtype
Bone: 54
Soft tissue: 100
Phase 1b
Localized
Bone
SU 20 4 5 - a - -
SU + RT 20 8 -
SU(+RT) + ST 20 8 - 235
Soft tissue
SU 10 - 4 12 24 20
SU + RT 40 - 4 8 16 -
SU(+RT) + ST - - 12 - - -
Metastatic/local relapse
Bone
Local treatment (+ST) 20 4 4 - a - -
ST 30 16 16 - - 240
Soft tissue
Local treatment (+ST) 10 - - - - -
ST 30 - 16 16 32 16
BSC 10 - 4 4 8 4 30
Total
Per location 210 40 65 40 80 40 475
Per treatment
SU only: 99
SU + RT: 96
SU + RT + ST: 40
Surgery total: 235
RT total: 136
ST only: 172
ST total: 212
Local: 38
Per subtype
Bone: 175
Soft tissue: 300

a Covered by axial group; SU = surgery; ST = systemic therapy (e.g., chemotherapy, tyrosine kinase inhibitors, immunotherapy); RT = radiotherapy; BSC = best supportive care; local treatment = SU and/or RT in metastatic setting.

During part A, eligible patients will be identified from the Medical Oncology, Radiation Oncology or Surgery departments of the participating hospitals and through patient organizations. The health care professional will introduce the study to eligible patients. If a patient expresses interest, he or she will be approached by the researcher for further details about the study, the interview, and the agreement to participate.

In part B, patients will be selected by the clinical team in the participating hospitals and invited to participate in the study using an invitation letter signed by the treating physician or in person during a routine follow-up appointment. Within one week, they will be contacted by phone to explain the study and to ask if they are willing to participate. If they are willing to participate, they will receive by mail an informed consent form, the issue list and the EORTC QLQ-C30 questionnaire (and EORTC QLQ-H&N43 or -EN24 module if applicable).

During both part A and part B, HCPs with experience of working with sarcoma patients will be invited by the study coordinator or the local PI.

2.6. Case Report Forms

Sociodemographic data will be collected at study entry, including age, gender, level of education and employment status, living arrangements, travel distance to hospital, and access to centralized care. These data may be extracted from the dossier or obtained from the patient directly. Clinical data will also be collected, including primary diagnosis, stage of the disease, type of therapy (if any), and comorbidity using the Charlson comorbidity index and international prognostic index (IPI) [45]. These clinical data will be extracted from the medical file and noted on the demographic and clinical data form. The information will be recorded and stored in accordance with the Dutch General Data Protection Regulation. Data linkage will be done at the Netherlands Cancer Institute.

2.7. Questionnaires

Three existing EORTC quality of life questionnaires will be reviewed by patients and HCPs, for the topic and item relevance. The EORTC QLQ-C30 is provided for all patients, the EORTC QLQ-H&N43 for patients with sarcoma in the head and neck region and the EORTC QLQ-EN24 for patients with uterine sarcoma. Before the start of phase 1b, based on the issues found in phase 1a, it will be evaluated if more location-specific modules (e.g., prostate, breast or colorectal) can be included. All scales and single-item measures range from 0 to 100 after linear transformation. A higher score on functional scales and global QoL means better functioning and HRQoL, whereas a higher score on the symptom scales means more complaints.

2.7.1. EORTC-Quality of Life Questionnaire (EORTC QLQ-C30)

The EORTC QLQ-C30 version 3.0 was developed to measure HRQoL in patients with cancer [46]. It was translated into and linguistically validated into over 110 languages. This 30-item questionnaire consists of five functional scales (physical, role, cognitive, emotional and social), a global health status/quality of life scale, three symptom scales (fatigue, pain, nausea and vomiting) and six single items assessing common symptoms (dyspnea, loss of appetite, insomnia, constipation and diarrhea) and perceived financial impact of the disease.

2.7.2. EORTC-QLQ-Head and Neck Module (EORTC QLQ-H&N43)

The EORTC QLQ-Head and Neck Module (EORTC QLQ-H&N43) measures HRQoL in patients with head and neck cancer [47]. It has been translated into and linguistically validated in 29 languages. It consists of 43 items, divided into 12 symptom scales (pain in the mouth, swallowing, problems with teeth, dry mouth and sticky saliva, problems with senses, speech, body image, social eating, sexuality, problems with shoulder, skin problems and fear of progression) and 7 single items (problems opening the mouth, coughing, social contact, swelling in the neck, weight loss, problems with wound healing and neurological problems).

2.7.3. EORTC-QLQ-Endometrial Cancer Module (EORTC QLQ-EN24)

The EORTC QLQ-Endometrial Cancer Module (EORTC QLQ-EN24) was developed for HRQoL measurement in patients with endometrial cancer and consists of 24 items [48]. It has been translated into and linguistically validated in 35 languages. Five symptom scales (lymphedema, urological symptoms, gastrointestinal symptoms, poor body image and sexual/vaginal problems), five single symptoms items (pain in back and pelvis, tingling/numbness, muscular pain, hair loss and taste change) and three single functional items (sexual interest, sexual activity and sexual enjoyment) are measured.

2.8. Objectives

The primary objective is to develop an HRQoL measurement strategy for sarcoma patients, indicating which issues can be included for all sarcoma patients (sarcoma-generic) and which issues should be evaluated in certain subgroups (e.g., based on treatment or sarcoma location) only. This is the first step towards the implementation of sarcoma specific PROs in future clinical trials. Secondary endpoints are: (1)to develop an exhaustive list of all HRQoL issues relevant to sarcoma patients (irrespective of their sarcoma subtype, treatment, time since diagnosis and age, and including generic/specific issues), (2) to determine the proportion of patients in each subgroup (according to tumor localization, type of sarcoma, disease stage and treatment) rating each item and the items from the EORTC QLQ C30 as relevant (i.e., rating 2–4 on Likert scale indicates relevance), (3) to determine the proportion of respondents with head neck or uterine localization rating each site-specific item of the H&N43 or EN24 questionnaire as relevant (i.e., rating 2–4 on Likert scale indicates relevance), (4) to determine the proportion of subgroups rating each item as relevant (i.e., half subgroups rating an item as relevant indicates a sarcoma-generic issue), and (5) to determine subgroup-specific (according to localization, type of treatment, subtype and disease stage) issues using a comparative approach (issues cited at least 1.5 times more indicates subgroup-specificity).

2.9. Sample Size Calculation

Part A: No formal sample size calculation can be made for the interviews. Data collection continues until saturation is reached, which is usually after 12 patients in phenomenological research [49]. Saturation occurs when adding more participants to the study (or, in this case, subgroup) does not result in additional perspectives or information. A stratification matrix (Table 1) will be used that ensures that at least 12 patients per location and 10 per treatment will be interviewed, considering the incidence of sarcoma location and treatment. In total, 154 patients will be recruited. Although some locations will be overrepresented in this sample compared to the population-based incidence numbers, this proportion was chosen to make sure that enough patients are enrolled per group to be able to conduct subgroup analyses.

Part B: Although the EORTC guidelines for development state that no more than 10 patients should be included in this part, the heterogeneity of the sarcoma population requires greater numbers for adequate subanalyses. Again, at least 10 patients per location and 10 per treatment will be recruited (Table 1), leading to a total of 475 patients.

Healthcare professionals: According to the EORTC module development guidelines, at least 5 HCPs should be included; therefore, 5 HCPs will be interviewed for each location (i.e., 6 in total), which means 30 HCPs in total. This applies to both parts A and B.

2.10. Statistical Analysis

We will perform thematic analysis of the qualitative data. Transcripts of the patient and HCP interviews will be examined by two coders in detail in order to identify basic patterns and recurrent themes using line-by-line coding to examine, compare and begin to develop conceptual categories. Categories will be developed inductively using the constant comparison method. Comparing each item with the rest of the data to create analytical categories, and then grouping categories together will make it possible to identify key themes [50]. The consolidated criteria for reporting qualitative research (COREQ) will be followed to ensure the accuracy of this qualitative study [51].

Data from patient and HCP interviews and questionnaires will be analyzed according to the EORTC module development guidelines [36], using basic quantitative analyses, including descriptive statistics (e.g., missing data, means and standard deviations, floor and ceiling effects), prevalence (number of patients who experienced each complaint, i.e., who scored 2, 3 or 4, divided by the total number of patients who completed that item, multiplied by 100), priority ratings (number of patients and professionals who have given priority ratings to each item) and range of responses for each item.

In order to facilitate transparent decision-making concerning the best approach to take in developing a sarcoma measurement strategy, the existing decision criteria will be followed: (1) an inclusive approach will be taken with regard to relevant items; (2) inclusion of an issue will be considered when more than 30% of patients (in total or per subgroup) mention the issue as missing; (3) issues that have a low (e.g., mean <1.5) mean score for relevance or importance will be considered for exclusion.

To distinguish “sarcoma generic” from “subgroup-specific issues”, issues that are more or less frequent in specific subgroups will be evaluated. An issue will be coded as endorsed by a respondent if it was scored 2 (“a little relevant”) or higher on the 4-point response scale. An issue needs to be endorsed by at least 30% of the patients in a subgroup to be included as an issue in that subgroup. If an issue is endorsed by patients from 3 out of 6 tumor locations and 2 out of 4 treatment groups, it will be considered to be a generic sarcoma issue; otherwise, it can be considered a sarcoma subgroup-specific issue.

Additional analyses will be performed per subgroup for location, treatment(s) received (surgery; chemotherapy/targeted therapy; radiotherapy; no treatment), type of sarcoma (soft tissue vs. bone sarcoma), and disease stage (localized vs. metastatic). An issue needs to be endorsed by at least 30% of the patients in a subgroup, and the mean score needs to be 1.5 or higher to be included as an issue in a subgroup. Then the prevalence of each issue between the aforementioned subgroups (according to location, type of treatment, subtype and disease stage and combinations of these factors) will be compared to decide whether an issue is subgroup-specific. Issues that are cited by at least 1.5 times as many respondents in group A compared to group B (a ratio of 3:2) may be considered as subgroup-specific.

For head and neck, and uterine sarcoma patients, the scores on the H&N43 and EN24 will be analyzed. We will assess the prevalence (percentage of the subjects who score higher than 2 is >30% and a mean score of 1.5 or higher) of each site-specific item. When at least 80% of the items are prevalent, using the site-specific module for those sarcoma locations will be recommended.

All the collected information will help to advise researchers as to which factors need to be taken into account when creating an item list for a specific study focusing on subgroups of sarcoma patients.

2.11. Missing Data

We will describe the number of missing items (i.e., number of items or questions that were not rated for relevance by patients or HCPs) and describe these results in our findings.

3. Conclusions

Sarcoma diagnosis and treatment have a substantial impact on a patient’s HRQoL, which should be taken into account in research and organization of care for this patient group. This study will be the first to investigate the optimal strategy for the assessment of HRQoL in sarcoma patients. Measurement of HRQoL in the sarcoma population is challenging because of heterogeneity in terms of locations, subtypes, treatments, age, and prognosis, and a holistic measurement strategy or instrument does currently not exist. The multicenter, international design of the study enables adequate sample sizes for specific subgroups of this rare cancer and ensures a multi-ethnic background of participants. Furthermore, the use of purposive sampling allows for coverage of the previously described heterogeneity of sarcomas. Our results will help to ensure the collection of high-quality, personalized patient-reported outcome data in future clinical trials and observational studies, eventually improving tailored supportive care for sarcoma patients.

Acknowledgments

This study will be conducted on behalf of the EORTC Quality of Life Group and EORTC Soft Tissue and Bone Sarcoma Group and is registered at clinicaltrials.gov (NCT04071704).

Supplementary Materials

The following are available online at https://www.mdpi.com/2072-6694/13/1/1/s1, Text S1: Interview scripts for phase 1a (patients and HCPs).

Author Contributions

Conceptualization, O.H. and W.T.A.v.d.G.; methodology, all authors; D.K. and C.P. contributed regarding the use of the EORTC Item Library; R.W. contributed as a patient representative; formal analysis, D.d.H. and O.H.; investigation, D.d.H., M.F., J.M.-B., B.K., A.C.H., I.N., S.C.S., M.E., W.J.v.H., I.M.E.D., I.R.-C., H.K.-P., A.M., L.H., S.S., and W.T.A.v.d.G.; resources, D.d.H. and O.H.; data curation, D.d.H., writing—original draft, D.d.H., writing—review and editing, all authors; supervision, O.H. and W.T.A.v.d.G.; project administration, D.d.H.; funding acquisition, O.H. and W.T.A.v.d.G. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by a research grant of the EORTC Quality of Life Group (QLG; grant no. 003–2018).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.WHO Classification of Tumours Editorial Board . Soft Tissue and Bone Tumours, WHO Classification of Tumours. 5th ed. WHO Press; Geneva, Switzerland: 2020. [Google Scholar]
  • 2.Andritsch E., Beishon M., Bielack S., Bonvalot S., Casali P., Crul M., Delgado Bolton R., Donati D.M., Douis H., Haas R., et al. ECCO Essential Requirements for Quality Cancer Care: Soft Tissue Sarcoma in Adults and Bone Sarcoma. A critical review. Crit. Rev. Oncol. Hematol. 2017;110:94–105. doi: 10.1016/j.critrevonc.2016.12.002. [DOI] [PubMed] [Google Scholar]
  • 3.RARECAREnet Information Network on Rare Cancer. [(accessed on 30 March 2020)]; Available online: http://app.rarecarenet.eu/analysis.php.
  • 4.Coindre J.M., Terrier P., Guillou L., Le Doussal V., Collin F., Ranchère D., Sastre X., Vilain M.O., Bonichon F., N’Guyen Bui B. Predictive value of grade for metastasis development in the main histologic types of adult soft tissue sarcomas: A study of 1240 patients from the French Federation of Cancer Centers Sarcoma Group. Cancer. 2001;91:1914–1926. doi: 10.1002/1097-0142(20010515)91:10&#x0003c;1914::AID-CNCR1214&#x0003e;3.0.CO;2-3. [DOI] [PubMed] [Google Scholar]
  • 5.Dangoor A., Seddon B., Gerrand C., Grimer R., Whelan J.S., Judson I. UK guidelines for the management of soft tissue sarcomas. Clin. Sarcoma Res. 2016;6:20. doi: 10.1186/s13569-016-0060-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Stiller C., Botta L., Brewster D.H., Ho V., Frezza A.M., Whelan J., Casali P.G., Trama A., Gatta G. Survival of adults with cancers of bone or soft tissue in Europe—Report from the EUROCARE-5 study. Cancer Epidemiol. 2018;56:146–153. doi: 10.1016/j.canep.2018.08.010. [DOI] [PubMed] [Google Scholar]
  • 7.Savina M., Le Cesne A., Blay J.-Y., Ray-Coquard I., Mir O., Toulmonde M., Cousin S., Terrier P., Ranchere-Vince D., Meeus P., et al. Patterns of care and outcomes of patients with METAstatic soft tissue SARComa in a real-life setting: The METASARC observational study. BMC Med. 2017;15:78. doi: 10.1186/s12916-017-0831-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Gamboa A.C., Gronchi A., Cardona K. Soft-tissue sarcoma in adults: An update on the current state of histiotype-specific management in an era of personalized medicine. CA A Cancer J. Clin. 2020;70:200–229. doi: 10.3322/caac.21605. [DOI] [PubMed] [Google Scholar]
  • 9.Judson I., Morden J.P., Kilburn L., Leahy M., Benson C., Bhadri V., Campbell-Hewson Q., Cubedo R., Dangoor A., Fox L., et al. Cediranib in patients with alveolar soft-part sarcoma (CASPS): A double-blind, placebo-controlled, randomised, phase 2 trial. Lancet Oncol. 2019;20:1023–1034. doi: 10.1016/S1470-2045(19)30215-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Martin-Broto J., Stacchiotti S., Lopez-Pousa A., Redondo A., Bernabeu D., De Alava E., Casali P.G., Italiano A., Gutierrez A., Moura D.S., et al. Pazopanib for treatment of advanced malignant and dedifferentiated solitary fibrous tumour: A multicentre, single-arm, phase 2 trial. Lancet Oncol. 2019;20:134–144. doi: 10.1016/S1470-2045(18)30676-4. [DOI] [PubMed] [Google Scholar]
  • 11.Yamaguchi M., Erdenebaatar C., Saito F., Motohara T., Miyahara Y., Tashiro H., Katabuchi H. Long-Term Outcome of Aromatase Inhibitor Therapy with Letrozole in Patients With Advanced Low-Grade Endometrial Stromal Sarcoma. Int. J. Gynecol. Cancer. 2015;25:1645–1651. doi: 10.1097/IGC.0000000000000557. [DOI] [PubMed] [Google Scholar]
  • 12.Dufresne A., Brahmi M. Immunotherapy in sarcoma: Combinations or single agents? In whom? Curr. Opin. Oncol. 2020;32:339–343. doi: 10.1097/CCO.0000000000000651. [DOI] [PubMed] [Google Scholar]
  • 13.Sutton C., Zhang Y., Kim D., Yarmohammadi H., Ziv E., Boas F.E., Sofocleous C.T., Tap W.D., D’Angelo S.P., Erinjeri J.P. Analysis of the Chemotherapy-Free Interval following Image-Guided Ablation in Sarcoma Patients. Sarcoma. 2020;2020:3852420. doi: 10.1155/2020/3852420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Chan A., Lim E., Ng T., Shih V., Quek R., Cheung Y.T. Symptom burden and medication use in adult sarcoma patients. Support. Care Cancer. 2015;23:1709–1717. doi: 10.1007/s00520-014-2533-4. [DOI] [PubMed] [Google Scholar]
  • 15.Soomers V.L., Husson O., Desar I.M.E., Van De Sande M.A., De Haan J.J., Verhoef C., Vriens I.J.H., Van Houdt W.J., Van De Poll-Franse L., A Van Der Graaf W.T. Patient and diagnostic intervals of survivors of sarcoma: Results from the SURVSARC study. Cancer. 2020;126:5283–5292. doi: 10.1002/cncr.33181. [DOI] [PubMed] [Google Scholar]
  • 16.Kluetz P.G., Slagle A., Papadopoulos E.J., Johnson L.L., Donoghue M., Kwitkowski V.E., Chen W.-H., Sridhara R., Farrell A.T., Keegan P., et al. Focusing on Core Patient-Reported Outcomes in Cancer Clinical Trials: Symptomatic Adverse Events, Physical Function, and Disease-Related Symptoms. Clin. Cancer Res. 2016;22:1553–1558. doi: 10.1158/1078-0432.CCR-15-2035. [DOI] [PubMed] [Google Scholar]
  • 17.National Institute for Health and Care Excellence Glossary. [(accessed on 16 November 2020)]; Available online: https://www.nice.org.uk/glossary.
  • 18.Vodicka E., Kim K.G., Devine E.B., Gnanasakthy A., Scoggins J., Patrick D.L. Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007–2013) Contemp. Clin. Trials. 2015;43:1–9. doi: 10.1016/j.cct.2015.04.004. [DOI] [PubMed] [Google Scholar]
  • 19.Coens C., Pe M., Dueck A.C., Sloan J., Basch E., Calvert M., Campbell A., Cleeland C., Cocks K., Collette L., et al. International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: Recommendations of the SISAQOL Consortium. Lancet Oncol. 2020;21:e83–e96. doi: 10.1016/S1470-2045(19)30790-9. [DOI] [PubMed] [Google Scholar]
  • 20.Basch E.M., Deal A.M., Kris M.G., Scher H.I., Hudis C.A., Sabbatini P., Rogak L., Bennett A.V., Dueck A.C., Atkinson T.M., et al. Symptom Monitoring with Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J. Clin. Oncol. 2016;34:557–565. doi: 10.1200/JCO.2015.63.0830. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kotronoulas G., Kearney N., Maguire R., Harrow A., Di Domenico D., Croy S., MacGillivray S. What Is the Value of the Routine Use of Patient-Reported Outcome Measures Toward Improvement of Patient Outcomes, Processes of Care, and Health Service Outcomes in Cancer Care? A Systematic Review of Controlled Trials. J. Clin. Oncol. 2014;32:1480–1501. doi: 10.1200/JCO.2013.53.5948. [DOI] [PubMed] [Google Scholar]
  • 22.Warrington L., Absolom K., Conner M., Kellar I., Clayton B., Ayres M., Velikova G. Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review. J. Med. Internet Res. 2019;21:e10875. doi: 10.2196/10875. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kennedy F., Absolom K., Clayton B., Rogers Z., Gordon K., Francischetto E.O., Blazeby J.M., Brown J., Velikova G. Electronic Patient Reporting of Adverse Events and Quality of Life: A Prospective Feasibility Study in General Oncology. JCO Oncol. Pr. 2020:Op2000118. doi: 10.1200/OP.20.00118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.McDonough J., Eliott J., Neuhaus S.J., Reid J., Butow P.N. Health-related quality of life, psychosocial functioning, and unmet health needs in patients with sarcoma: A systematic review. Psycho-Oncol. 2019;28:653–664. doi: 10.1002/pon.5007. [DOI] [PubMed] [Google Scholar]
  • 25.Storey L., A Fern L., Martins A., Wells M., Bennister L., Gerrand C., Onasanya M., Whelan J.S., Windsor R., Woodford J., et al. A Critical Review of the Impact of Sarcoma on Psychosocial Wellbeing. Sarcoma. 2019;2019:9730867. doi: 10.1155/2019/9730867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Fauske L., Bruland Ø.S., Grov E.K., Bondevik H. Cured of Primary Bone Cancer, But at What Cost: A Qualitative Study of Functional Impairment and Lost Opportunities. Sarcoma. 2015;2015:484196. doi: 10.1155/2015/484196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Parsons J.A., Eakin J.M., Bell R.S., Franche R.L., Davis A.M. “So, are you back to work yet?” Re-conceptualizing ‘work’ and ‘return to work’ in the context of primary bone cancer. Soc. Sci. Med. 2008;67:1826–1836. doi: 10.1016/j.socscimed.2008.09.011. [DOI] [PubMed] [Google Scholar]
  • 28.Van Der Graaf W.T.A., Blay J.-Y., Chawla S.P., Kim D.-W., Bui-Nguyen B., Casali P.G., Schöffski P., Aglietta M., Staddon A., Beppu Y., et al. Pazopanib for metastatic soft-tissue sarcoma (PALETTE): A randomised, double-blind, placebo-controlled phase 3 trial. Lancet. 2012;379:1879–1886. doi: 10.1016/S0140-6736(12)60651-5. [DOI] [PubMed] [Google Scholar]
  • 29.Fauske L., Lorem G.F., Grov E.K., Bondevik H. Changes in the body image of bone sarcoma survivors following surgical treatment—A qualitative study. J. Surg. Oncol. 2015;113:229–234. doi: 10.1002/jso.24138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Davis A.M., Wright J.G., Williams J.I., Bombardier C., Griffin A., Bell R.S. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual. Life Res. 1996;5:508–516. doi: 10.1007/BF00540024. [DOI] [PubMed] [Google Scholar]
  • 31.Martins A., Bennister L., A Fern L., Gerrand C., Onasanya M., Storey L., Wells M., Whelan J., Windsor R., Woodford J., et al. Development of a patient-reported experience questionnaire for patients with sarcoma: The Sarcoma Assessment Measure (SAM) Qual. Life Res. 2020;29:2287–2297. doi: 10.1007/s11136-020-02481-x. [DOI] [PubMed] [Google Scholar]
  • 32.Davidson D., Barr R.D., Riad S., Griffin A.M., Chung P.W., Catton C., O’Sullivan B., Ferguson P.C., Davis A.M., Wunder J.S. Health-related quality of life following treatment for extremity soft tissue sarcoma. J. Surg. Oncol. 2016;114:821–827. doi: 10.1002/jso.24424. [DOI] [PubMed] [Google Scholar]
  • 33.Paredes T., Canavarro M.C., Simões M.R. Anxiety and depression in sarcoma patients: Emotional adjustment and its determinants in the different phases of disease. Eur. J. Oncol. Nurs. 2011;15:73–79. doi: 10.1016/j.ejon.2010.06.004. [DOI] [PubMed] [Google Scholar]
  • 34.Eichler M., Hentschel L., Richter S., Hohenberger P., Kasper B., Andreou D., Pink D., Jakob J., Singer S., Grützmann R., et al. The Health-Related Quality of Life of Sarcoma Patients and Survivors in Germany—Cross-Sectional Results of a Nationwide Observational Study (PROSa) Cancers. 2020;12:3590. doi: 10.3390/cancers12123590. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Gundle K.R., Cizik A.M., Jones R.L., Davidson D.J. Quality of life measures in soft tissue sarcoma. Expert Rev. Anticancer Ther. 2014;15:95–100. doi: 10.1586/14737140.2015.972947. [DOI] [PubMed] [Google Scholar]
  • 36.Bottomley A., Reijneveld J.C., Koller M., Flechtner H., Tomaszewski K.A., Greimel E., Ganz P.A., Ringash J., O’Connor D., Kluetz P.G., et al. Current state of quality of life and patient-reported outcomes research. Eur. J. Cancer. 2019;121:55–63. doi: 10.1016/j.ejca.2019.08.016. [DOI] [PubMed] [Google Scholar]
  • 37.Johnson C.D., Aaronson N.K., Blazeby J. Guidelines for Developing Questionnaire Modules. 4th ed. EORTC; Brussels, Belgium: 2011. [Google Scholar]
  • 38.EORTC Item Library. [(accessed on 19 November 2020)]; Available online: https://www.eortc.be/itemlibrary/
  • 39.Groenvold M., Aaronson N.K., Darlington A.-S.E., Fitzsimmons D., Greimel E., Holzner B., Reijneveld J.C., Tomaszewski K.A., Leeuw I.V.-D., Van De Poll-Franse L.V. Focusing on Core Patient-Reported Outcomes in Cancer Clinical Trials—Letter. Clin. Cancer Res. 2016;22:5617. doi: 10.1158/1078-0432.CCR-16-1529. [DOI] [PubMed] [Google Scholar]
  • 40.Kulis D., Piccinin C., Bottomley A., Darlington A., Grønvold M. PCN45 What to Choose? Creating a Patient-Reported Outcomes (Pro) Assessment Strategy with Eortc Measures and Item Library that Can Meet the Needs of Regulators. Value Health. 2019;22:S64. doi: 10.1016/j.jval.2019.04.169. [DOI] [Google Scholar]
  • 41.Wilson R. Patient led PROMs must take centre stage in cancer research. Res. Involv. Engagem. 2018;4:7. doi: 10.1186/s40900-018-0092-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Hollander D.D., Van Der Graaf W.T., Fiore M., Kasper B., Singer S., Desar I.M., Husson O. Unravelling the heterogeneity of soft tissue and bone sarcoma patients’ health-related quality of life: A systematic literature review with focus on tumour location. ESMO Open. 2020;5:e000914. doi: 10.1136/esmoopen-2020-000914. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Grundy A., Keetharuth A.D., Barber R., Carlton J., Connell J., Buck E.T., Barkham M., Ricketts T., Robotham D., Rose D., et al. Public involvement in health outcomes research: Lessons learnt from the development of the recovering quality of life (ReQoL) measures. Heal. Qual. Life Outcomes. 2019;17:60. doi: 10.1186/s12955-019-1123-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Sodergren S.C., Husson O., Rohde G.E., Tomaszewska I.M., Griffiths H., Pessing A., Yarom N., Hooker L., Din A., Darlington A.S., et al. Does age matter? A comparison of health-related quality of life issues of adolescents and young adults with cancer. Eur. J. Cancer Care. 2018;27:e12980. doi: 10.1111/ecc.12980. [DOI] [PubMed] [Google Scholar]
  • 45.Charlson M.E., Pompei P., Ales K.L., MacKenzie C. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J. Chronic Dis. 1987;40:373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
  • 46.Aaronson N.K., Ahmedzai S., Bergman B., Bullinger M., Cull A., Duez N.J., Filiberti A., Flechtner H., Fleishman S.B., De Haes J.C.J.M., et al. The European Organization for Research and Treatment of Cancer QLQ-C30: A Quality-of-Life Instrument for Use in International Clinical Trials in Oncology. J. Natl. Cancer Inst. 1993;85:365–376. doi: 10.1093/jnci/85.5.365. [DOI] [PubMed] [Google Scholar]
  • 47.Singer S., Amdal C.D., Hammerlid E., Tomaszewska I.M., Silva J.C., Mehanna H., Santos M., Inhestern J., Brannan C., Yarom N., et al. International validation of the revised European Organisation for Research and Treatment of Cancer Head and Neck Cancer Module, the EORTC QLQ-HN43: Phase IV. Head Neck. 2019;41:1725–1737. doi: 10.1002/hed.25609. [DOI] [PubMed] [Google Scholar]
  • 48.Greimel E., Nordin A., Lanceley A., Creutzberg C.L., Van De Poll-Franse L.V., Bjelic-Radisic V., Galalae R., Schmalz C., Barlow E., Jensen P.T., et al. Psychometric validation of the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Endometrial Cancer Module (EORTC QLQ-EN24) Eur. J. Cancer. 2011;47:183–190. doi: 10.1016/j.ejca.2010.08.014. [DOI] [PubMed] [Google Scholar]
  • 49.Guest G., Bunce A., Johnson L.R. How Many Interviews Are Enough? Field Methods. 2006;18:59–82. doi: 10.1177/1525822X05279903. [DOI] [Google Scholar]
  • 50.Morse J.M., Barrett M., Mayan M., Olson K., Spiers J. Verification Strategies for Establishing Reliability and Validity in Qualitative Research. Int. J. Qual. Methods. 2002;1:13–22. doi: 10.1177/160940690200100202. [DOI] [Google Scholar]
  • 51.Tong A., Sainsbury P., Craig J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int. J. Qual. Health Care. 2007;19:349–357. doi: 10.1093/intqhc/mzm042. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials


Articles from Cancers are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES