What Questionnaires To Use When Measuring Quality Of Life In Sacral Tumor Patients? The Updated Sacral Tumor Survey

Abstract

Background context

Patient reported outcomes are becoming increasingly important when investigating results of patient and disease management. In sacral tumor patients symptoms can vary substantially, therefore no single questionnaire can adequately account for the full spectrum of symptoms and disability.

Purpose

The purpose of this study is to analyze redundancy within the current sacral tumor survey and make a recommendation for an updated version based on the results and patient and expert opinions.

Study design/setting

A survey study from a tertiary care orthopaedic oncology referral center was used for this study.

Patient sample

The patient sample included 70 patients with sacral tumors (78% chordoma).

Outcome measures

The following ten questionnaires included in the current sacral tumor survey were evaluated: the Patient Reported Outcomes Measurement Information System (PROMIS) Global Item short form, PROMIS Pain Intensity short form, PROMIS Pain Interference short form, PROMIS Neuro-QOL v1.0 Lower Extremity Function short form, PROMIS v1.0 Anxiety short form, the PROMIS v1.0 Depression short form, the International Continence Society (ICS) Male short form, the Modified Obstruction-Defecation Syndrome (MODS) questionnaire, the PROMIS Sexual Function Profile v1.0, and The Stoma Quality Of Life tool.

Methods

We performed an exploratory factor analysis to calculate possible underlying latent traits. Spearman rank correlation coefficients were used to measure to what extent the questionnaires converged. We hypothesized the existence of six domains based on current literature: mental health, physical health, pain, gastrointestinal symptoms, sexual function, and urinary incontinence. To assess content validity, we surveyed 32 patients, nine orthopaedic oncologists, one medical oncologist, one radiation oncologist, and an orthopaedic oncology nurse practitioner with experience in treating sacral tumor patients on the relevance of the domains.

Results

Reliability as measured by Cronbach alpha ranged from 0.65 to 0.96. Coverage measured by floor and ceiling effects ranged from 0 to 52% and from 0 to 30%, respectively. Explanatory factor analysis identified three traits to which the questionnaires that were expected to measure a similar construct correlated the most; mental health, physical function, and pain. Content validity index demonstrated low disagreement among patients (range: 0.10 to 0.18) and high agreement among physicians (range: 0.91 to 1.0) on the relevance of the proposed domains. Social health was identified by 50% of the commenting patients as an important yet missing domain.

Conclusions

The current sacral tumor survey is incomplete, time consuming, and not all surveys are appropriate for the sacral tumor population. Our recommended survey contains less than half the questions and includes the newly recognized domain social health.

Introduction

Primary malignant bone tumors of the sacrum are often treated by partial sacrectomy with or without radiation [14, 33]. The impact of these major surgeries on neurological, physical, psychological, social, and emotional functioning is substantial and can have a major impact on a patients’ quality of life [7, 27, 31, 36, 44]. It is important to accurately measure these outcomes in order to: (1) understand the impact of treatment on patients, (2) educate future patients, and (3) compare treatments. Although providers seem to agree about the importance of measuring these outcomes, there is little consensus about what tools to use to establish these outcomes in this patient population. Because symptoms can vary substantially, no single questionnaire can cover the full spectrum of symptoms and disability. In 2013, the Sacral Tumor Study Group –an international collaboration of orthopaedic oncologists, medical oncologists, and radiation-oncologists from multiple institutions– compiled a list of questionnaires specifically for sacral tumor patients during an official meeting. The development of survey has not been published as a whole but parts of the survey have been used in previous studies looking at quality of life after sacral resection [31, 43].

This study aims to analyze the coverage and reliability of the current survey developed by the Sacral Tumor Study Group. Secondarily, we assessed redundancy of questionnaires in an attempt to shorten the survey without losing valuable information. Based on these analyses, and supported by a survey among patients and expert clinicians about what aspects of disease they consider important, we provide recommendations for a new shorter yet more revealing survey to evaluate outcomes in patients with sacral tumors.

Methods

Study Design

Our Institutional Review Board approved this cross-sectional survey study. All patients with a sacral tumor who visit our clinic are asked to complete the sacral tumor survey for quality improvement purposes. Patients younger than 18 years of age or non-native English speakers are excluded. Between February 2013 and August 2014, a total of 119 sacral tumor patients were seen at our Orthopaedic Oncology unit and were eligible to complete the sacral tumor survey. Eighty-eight (74%) patients completed the survey. We excluded 18 (20%) patients who completed less than half of the questionnaires that comprise the current sacral tumor survey. Patients were included irrespective of their tumor type and treatment in order to obtain input from a heterogeneous group of patients. Seventy patients remained for analysis; when multiple surveys per patients were completed, only the first one was included to avoid a learning curve on the survey completion and avoid violation of the statistical rule of independence. Patients completed the survey using a tablet computer and data was collected through REDCap (Vanderbilt University, Nashville, Tennessee, United States of America). REDCap is an online data collection tool allowing for creation of study-specific surveys to capture participant data securely online.

Outcome measures

Ten different questionnaires are included in the sacral tumor survey in the following order: 1) the Patient Reported Outcomes Measurement Information System (PROMIS) Global Item short form; 2) PROMIS Pain Intensity short form 3a; 3) PROMIS Pain Interference short form 6b; 4) PROMIS Neuro-QOL v1.0 Lower Extremity Function short form; 5) PROMIS v1.0 Anxiety short form (6a); 6) the PROMIS v1.0 Depression short form (6a) 7) the International Continence Society (ICS) Male short form; 8) the Modified Obstruction-Defecation Syndrome (MODS) questionnaire; 9) the PROMIS Sexual Function Profile v1.0: and 10) The Stoma Quality Of Life tool. The PROMIS questionnaires are designed to measure specific domains in the general population, and where not specifically designed for sacral tumor patients.

For the PROMIS questionnaires a T-score can be calculated. A higher score indicates more of the construct being measured. The T-score rescales the raw score into a standardized score with a mean of 50 and a standard deviation (SD) of ten, where a score of 50 represents the mean score of the general US population. This allows comparison of the patients’ score with the score of the general US population. Other countries may also develop such reference values. Comparison of outcomes among groups is still possible. Translations of PROMIS questionnaires are constantly being validated making the tool more internationally useful [1].

The PROMIS Global Health short form (ten items) [32] allows for the assessment of mental health (four items) and physical health (four items), plus two items (scored separately) about global health and satisfaction with social roles. The PROMIS Pain Intensity short form 3a (3 items) [18] assesses how much a person has been hurting over the last seven days. The PROMIS Pain Interference short form 6b (six items) measures the self-reported consequences of pain on relevant aspects of one’s life, over the last seven days. The PROMIS Neuro-QOL v1.0 Lower Extremity Function short form [23] (eight items) measures one’s ability to carry out various activities involving the trunk region and increasing degrees of bodily movement, ambulation, balance or endurance. The PROMIS v1.0 Anxiety short form 6a (six items) [15] assesses self-reported fear, anxious misery, hyperarousal, and somatic symptoms related to arousal, over the last seven days. The PROMIS v1.0 Depression short form 6a (six items)[18] assesses self-reported negative mood, views of self, and social cognition, as well as decreased positive affect and engagement, over the last seven days. The ICS Male short form (14 items) [9] is divided into five domains; quality of life (one item), nocturia (one item), frequency (one item), voiding (five items), and incontinence (six items). The MODS questionnaire (eight items) [25] measures to what extent a patient has difficulties to evacuate. A higher score indicates more difficulties to evacuate. The PROMIS Sexual Function Profile v1.0 – Male (eight items) or Female (ten items) [2] contains seven different subdomains of sexual function pertaining to the past 30 days; interest in sexual activity, vaginal discomfort (women only), lubrication (women only), erectile function (men only), orgasm, and global satisfaction with sex life. The Stoma Quality Of Life tool (19 items) [4] contains five domains: work/social function (six items), sexuality/body image (five items), stoma function (six items), financial concerns (one item), and skin irritation (one item). A higher score indicates more negative influence of the stoma on the domains subject. This tool is only applicable for patients with a stoma. Only five patients completed this questionnaire, since the placement of a stoma is not standard of care at our institution. We therefore excluded the Stoma QoL tool from this analysis. Realizing standard of care varies between institutions, we set out to include a new questionnaire appropriate for patient with and without a stoma.

Baseline characteristics were collected from the electronic medical records and consisted of age at time of survey completion, sex, previous surgical treatment, systemic treatment, radiation treatment, tumor volume, and tumor diagnosis.

Statistical analysis

Categorical variables are calculated as frequencies with percentages, and continuous variables as median with interquartile range (IQR). We used median imputation to estimate a missing item (i.e. question) within a questionnaire if only one item was not completed; the value was rounded down. There were 33 of a total of 4213 items missing (0.78%).

Assessment of coverage and reliability was performed for each of the questionnaires separately. Coverage is described as floor and ceiling effects; the situation when patients score at respectively the lowest (floor) or the highest possible score (ceiling) of a questionnaire, leaving it impossible to discriminate between patients at either end of the scale. This can cause a skew of the data and influence comparative analysis [8]. There are no widely accepted thresholds to determine acceptable floor or ceiling effects. However, we consider less than 5% of the responses at one end of the scale to be acceptable, between 5% and 20% as moderate and more than 20% as substantial [20, 22, 26]. To assess reliability we calculated a Cronbach alpha value. This is a measure of internal consistency –how closely related a set of items are as a group– ranging from 0 to 1, where a higher score indicates a higher internal consistency. A score of 0.9 or higher might reflect redundancy of items within the questionnaire, a Cronbach alpha between 0.7 and 0.9 is recommended [30, 37].

Subsequently, we performed an exploratory factor analysis to calculate possible underlying latent traits within the sacral tumor survey and subsequently correlate (i.e. factor loading) the questionnaires with the mathematically derived traits. A factor loading of 1 indicates perfect association of the questionnaire with the underlying trait, 0 indicates no association, and −1 indicates perfect inverse association of the questionnaire with the underlying trait. Spearman rank correlation coefficients were subsequently used to measure to what extent the questionnaires converged. High correlation coefficients indicate a strong relationship between the questionnaires. We considered correlation coefficients higher than 0.4 to be indicative of redundancy for inclusion within the sacral tumor survey [29, 30]. For both analyses we only included questionnaires that were thought to possibly measure a similar trait; i.e. global-mental health, anxiety, depression were expected to measure mental health; global-physical health and lower extremity function were expected to measure physical health; pain intensity and pain interference were expected to measure pain. Sixty-three of the seventy patients in the study group completed all questionnaires and were therefore available for exploratory factor analysis.

To determine what issues should be addressed in a survey for patients with sacral tumors, we hypothesized the existence of six domains based on the current literature: mental health, physical health, pain, gastrointestinal symptoms, sexual function, and urinary incontinence [6, 7, 27, 36, 39, 41, 45]. To assess content validity, we contacted 80 patients (of whom 32 responded, i.e. 40%), nine orthopaedic oncologists, one medical oncologist, one radiation oncologist, and an orthopaedic oncology nurse practitioner with experience in treating sacral tumor patients. The orthopaedic oncologists were all members of the Sacral Tumor Study group and represented five different institutions from three countries. We asked each of these individuals to rate the importance of the six domains on a 4-point Likert scale (i.e. 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = highly relevant). From this data a content validity index [38] can be established. We decided that the patients’ opinion on the relevance on a domain does need to meet a threshold and all domains with at least one score of quite or highly relevant were included. We did decide to use a threshold to exclude a domain, because not all patients have yet or will ever endure the complete spectrum of possible complaints and might therefore find a specific domain not relevant were others would. Therefore, we were interested in the proportion of patients scoring a domain as “not relevant” and set the exclusion threshold at 0.90, meaning that a domain would be potentially excluded if at least 90% of the patients agreed that a domain was not relevant. For the experts we calculated the proportion of experts scoring a domain “quite relevant” or “highly relevant” and set the inclusion threshold at 0.8 (i.e. 80% of the experts agreed that the domain was relevant). If a domain did not meet the expert threshold for inclusion and exceeded the patient threshold for exclusion and, the domain was excluded. Patients and experts were given the opportunity to suggest additional subjects that in their opinion were missing from the survey.

Baseline characteristics

The median age of the 70 sacral tumor patients was 61 years (IQR, 50 to 67 years) of whom 38 (54%) were male and 32 (46%) were female. At the time of survey completion 39 patients (56%) had already undergone sacral resection and 43 (61%) patients had received, or were in the process of receiving, radiotherapy. Most of the patients (51 patients, 73%) were diagnosed with chordoma, five (7%) with giant cell tumor, five (7%) with a metastatic lesion, two (3%) with solitary fibrous tumor, two (3%) with schwannoma, one (1%) with chondrosarcoma, one (1%) with haemanginoma, one (1%) with notochordal cell tumor, one (1%) with osteosarcoma, and one (1%) with plasmacytoma. Median tumor volume was 136 cm³ (IQR, 40 to 420). Tumor volume data was not available for 13 patients. The most cephalad involved sacral vertebrae was S1 in 14 (20%) of the cases, S2 in 22 (31%) of the cases, S3 in 16 (23%) of the cases, S4 in 5 (7%) of the cases, S5 in 5 (7%) of the cases, and the coccyx in 8 (11%) of the cases. Of the 39 cases that underwent surgery prior to completing the survey, 24 (62%) were operated using a single posterior approach and 15 (38%) were operated using a combined anterior and posterior approach.

Results

Assessing coverage and reliability of the sacral tumor survey

The global health questionnaire showed adequate internal consistency, and moderate coverage for the mental health domain (Cronbach alpha: 0.78; no floor effect and 11% ceiling effect) and adequate coverage in the physical health domain (Cronbach alpha: 0.70; no floor effect and a 2% ceiling effect) (Table 1). The pain intensity questionnaire showed high internal consistency (Cronbach alpha of 0.90) and moderate coverage due to 20% floor effect. The pain interference questionnaire showed similar internal consistency (Cronbach alpha: 0.96) and coverage (16% floor effect). The lower extremity questionnaire showed a high internal consistency (Cronbach alpha: 0.95) but poor coverage due to a substantial ceiling effect (29%). The PROMIS anxiety questionnaire showed high internal consistency (Cronbach alpha: 0.94) and poor coverage due to a 24% floor effect. The PROMIS depression questionnaire showed high internal consistency (Cronbach alpha: 0.94) and poor coverage due to a 37% floor effect. For the two domains of the ICS questionnaire consisting of more than one question, the internal consistency was high (incontinence, Cronbach alpha of 0.81 and voiding, Cronbach alpha of 0.91). Coverage was poor due to floor effects ranging from 18% in the voiding domain to 52% in the quality of life domain. Coverage was moderate for the MODS questionnaire due to a 16% floor effect and internal consistency was found to be adequate (Cronbach alpha of 0.78). In the sexual function questionnaire a moderate to high consistency was found in all domains (Cronbach alpha ranged between 0.65 and 0.96) except for the vaginal discomfort domain (Cronbach alpha: 0.49). Coverage was poor due to floor and ceiling effects that ranged from zero to 30%.

Table 1.

Internal consistency, floor and ceiling effect and score distribution per questionnaire of the sacral tumor survey

Questionnaires	Items per scale	Responses	Median (IQR)	Range	Possible range	Cronbach alpha†	Floor effect (%)	Ceiling effect (%)	High score indicates
PROMIS v1.0 Anxiety SF 6a	6 items	70	54 (46–61)	39 – 74	39 – 83	0.94	17 (24)	0	More anxiety
PROMIS v1.0 Depression SF 6a	6 items	70	48 (38–55)	38 – 80	38 – 80	0.94	26 (37)	1 (1)	More depressive symptoms
PROMIS Global Item SF
Mental Health	4 items	63	45 (40–51)	24 –68	21 – 68	0.78	0	7 (11)	Better mental health
Physical Health	4 items	63	48 (44–56)	25 – 68	21 – 68	0.70	0	1 (2)	Better physical health
Neuro_QOL v1.0 Lower Extremity Function SF 1	8 items	70	45 (38–59)	23 – 59	17 – 59	0.95	0	20 (29)	Better lower extremity function
PROMIS Pain Interference SF 6b 2	6 items	70	59 (51–65)	41 – 78	41 – 78	0.96	11 (16)	1 (1)	More pain interference
PROMIS Pain Intensity 3a 3	3 items	70	48 (44–55)	31 – 72	31 – 72	0.90	14 (20)	2 (3)	Higher pain intensity
PROMIS Sexual Function
Interest	2 items	61	44 (40–51)	33 – 70	33 – 70	0.92	14 (23)	1 (2)	More interest
Satisfaction	2 items	26	50 (48–55)	31 – 66	31 – 66	0.93	1 (4)	5 (19)	More satisfaction
Lubrication	2 items	10	47 (37–65)	37 – 65	37 – 65	0.96	3 (30)	3 (30)	Better lubrication
Vaginal discomfort	3 items	11	51 (48–55)	45 – 70	34 – 70	0.49	0	1 (9)	More discomfort
Erectile function	3 items	16	59 (48–57)	47 – 61	37 – 67	0.65	0	0	Better function
Orgasm	1 item	31	3 (2–4)	1 – 5	1 – 5	*	7 (23)	2 (6)	Better ability
International Continence Society Male Q
Qol	1 item	58	0 (0–1)	0 – 3	0 – 3	*	30 (52)	4 (7)	More QoL
Nocturia	1 item	58	1 (1–2)	0 – 4	0 – 4	*	7 (12)	1 (2)	More nocturia
Frequency	1 item	55	1 (1–2)	0 – 3	0 – 3	*	11 (20)	2 (4)	More frequent
Voiding	5 items	57	4 (1–10)	0 – 20	0 – 20	0.91	10 (18)	1 (2)	More voiding
Incontinence	6 items	65	3 (1–5)	0 – 17	0 – 24	0.81	16 (25)	0	More incontinence
Modified Obstruction-Defecation Score 4	8 items	70	5 (2–10)	0 – 19	0 – 24	0.78	11 (16)	0	More obstruction

^†Cronbach alpha represents internal consistency

^*Single item domains do not allow for the calculation of a Cronbach alpha

¹Median imputation was used to estimate the score for one missing item in four patients (6%)

²Median imputation was used to estimate the score for one missing item in 11 patients (16%)

³Median imputation was used to estimate the score for one missing item in five patients (7%)

⁴Median imputation was used to estimate the score for one missing item in 13 patients (19%)

Assessing redundancy of questionnaires

Exploratory factor analysis of questionnaires that were expected to measure similar constructs, derived four latent traits (ie, factors). Each questionnaire correlated with each trait in varying degrees, which is demonstrated by the different correlation coefficients (ie, factor loadings) (Table 2). The negative correlation coefficients are explained by the different directions of the questionnaires relative to the latent trait. We identified three traits to which the questionnaires correlated the most; mental health, physical function, and pain. The fourth trait contained very low correlation coefficients and was therefore disregarded.

Table 2.

Exploratory factor analysis (n = 63)

Expected trait	Questionnaire	Factor 1	Factor 2	Factor 3	Factor 4
Mental health	Global - Mental	− 0.73	− 0.14	0.43	0.21
	Depression	0.81	0.25	− 0.10	0.10
	Anxiety	0.85	0.22	− 0.08	0.01
Physical health	Global - Physical	− 0.29	− 0.44	0.71	0.01
	Lower Extremity	− 0.01	− 0.49	0.50	0.25
Pain	Pain Intensity	0.15	0.85	− 0.22	0.01
	Pain Interference	0.37	0.81	− 0.24	0.03

-Bold indicates the highest factor loading per questionnaire

-Seven of the total 70 patients did not complete all the questionnaires and were therefore excluded from the exploratory factor analysis

Spearman rank correlation analysis revealed substantial correlations (rho > 0.40) between questionnaire scores that were expected to measure similar constructs: the global-mental score and the depression score (rho: −0.64), the global-mental score and the anxiety score (rho: −0.70), the global-physical score and the lower extremity score (rho: 0.62), the pain intensity score and the pain interference score (rho: 0.81) (Table 3). High correlations were also identified between the global-physical health score and both the pain intensity score (rho: −0.58) and the pain interference score (rho: −0.63).

Table 3.

Spearman correlation coefficients (n = 63)

	Global-Mental	Depression	Anxiety	Global-Physical	Lower Extremity	Pain Intensity
Depression	− 0.64	.	.	.	.	.
Anxiety	− 0.70	0.80	.	.	.	.
Global - Physical	0.63	− 0.40	− 0.38	.	.	.
Lower Extremity	0.20	− 0.18	− 0.12	0.62	.	.
Pain Intensity	− 0.31	0.36	0.38	− 0.58	− 0.40	.
Pain Interference	− 0.49	0.58	0.53	− 0.63	− 0.48	0.81

-Bold indicates higher than 0.4

-Seven of the total 70 patients did not complete all the questionnaires and were therefore excluded from Spearman correlation analysis

Content validity

Based on the current literature, we developed a framework for a survey containing six potential domains, i.e. mental health, physical health, pain, gastrointestinal symptoms, sexual function, and urinary incontinence. To what extent the hypothesized domains were relevant was assessed by calculation of the content validity index (Table 4). None of the domains reached our predetermined thresholds for domain exclusion in either the expert or patient groups. There was high agreement (index value: 0.91 to 1.0) among the 12 experts on the relevance of the domains. Among the 32 patients there was little agreement that domains were not relevant (index value: 0.10 to 0.18). Of the ten patients (31%) that commented on the proposed survey domains, five (50%) identified the lack of a domain addressing social heath (e.g. “ability to return to work”, “ability to enjoy movie/dinners”, “ability to go on vacation”). The remaining five patients (50%) addressed issues concerning mental and physical health (e.g. “worrying about the future”, “weakness in legs”, “always uncomfortable”).

Table 4.

Content validity index

Domains	Patients†	Specialists*
Mental Health	0.18	0.91
Physical Health	0.12	1.00
Pain	0.18	1.00
Gastrointestinal Symptoms	0.10	0.91
Sexual Function	0.10	0.91
Urinary Incontinence	0.12	0.91

^†An index value of 0.9 or higher indicates exclusion

^*An index value of 0.8 or lower indicates exclusion

Discussion

Sacral tumors are rare, but their treatment can significantly impact many aspects of a patients’ life. It is therefore important to uniformly collect patient reported outcomes to understand the impact of different treatment options (e.g. surgery using different surgical approaches and definitive or (neo-)adjuvant radiotherapy), educate future patients, and allow for comparison of outcomes. However, there is no consensus on what questionnaires should be used, and the current sacral tumor survey contains over 100 questions, which is a burden to patients, physicians, and researchers. We assessed coverage, reliability, and redundancy within the sacral tumor survey and collected patient and expert opinions to identify the most relevant aspects of quality of life for patients undergoing treatment for sacral tumors. We found moderate to poor coverage for eight questionnaires and redundancy of four questionnaires, indicating the potential for shortening the survey. Patients addressed the lack of questions pertaining to social health, but agreed with experts on the importance of six other predefined domains. We therefore recommend measuring outcomes for sacral tumor patients using seven questionnaires which each address one of the following seven domains: mental health, physical health, pain, gastrointestinal symptoms, urinary incontinence, sexual function, and social health.

This study has several limitations. First, we analyzed a sample of 70 patients while 119 potentially eligible patients were encountered in our outpatient clinic. This might have introduced a selection bias, and our sample might therefore not be representative of the entire sacral tumor population. However, a comparison between the patients who did and who did not complete the survey did not demonstrate any differences (p > 0.05 for gender, age, tumor volume, and type of tumor). Also, we do not believe this had a substantial impact on the internal consistency, correlation, or exploratory factor analysis, as our sample was heterogeneous in terms of sex, treatment modality, timing of survey completion, and outcomes. Studies assessing validity, coverage, and reliability of a survey benefit by having a heterogeneous patient sample as the survey needs to be able to differentiate between patients in all disease stages. We also did not find a difference between the 63 patients who were included in the explanatory factor analysis and the seven patients who were excluded (p > 0.05 for gender, age, tumor volume, and type of tumor). Second, not all questionnaires were filled out completely and statistical analyses did not allow missing values. We therefore used median imputation to resolve this issue. Because of the relative small number of missing items (0.78%) we anticipate minor, if any, impact on the validity measurements.

Third, we used traditional methods of analyzing reliability, validity, and coverage; while more advanced techniques, such as item response theory (IRT), are available. Item response theory can further measure the survey’s performance; however, IRT requires a large sample size. Future studies with larger numbers and pre- and post-treatment measurements should thoroughly validate the newly recommended sacral tumor survey in a multi-institutional cohort of sacral tumor patients, to establish external validity, reliability, responsiveness and coverage. Fourth, we had limited expert opinions available to calculate the content validity index. However, the high level of agreement on domain relevance and the absence of suggestions for additional domains are reassuring for the consensus among experts.

The depression and anxiety questionnaires measured the same construct as the mental health domain of the global health questionnaire, but demonstrated poor coverage. This is in line with previous studies that found comparable substantial floor effects for the depression and anxiety questionnaires, suggesting that these two questionnaires are not capable of distinguishing between patients with relatively little mental health issues [3, 10, 24]. The physical health domain of the global health questionnaire measured the same construct as the lower extremity function questionnaire, but the latter demonstrated worse coverage. These findings are in line with previous studies that demonstrated a ceiling effect in the lower extremity function questionnaire [23] and a high correlation (rho, 0.55) between the lower extremity score and the physical function score [28]. We therefore decided to include the global health questionnaire –including the mental health domain and the physical function domain– in the new sacral tumor survey (Table 5). The pain intensity and the pain interference questionnaire measured a similar construct and, in line with previous studies, demonstrated high interquestionnaire correlation [3]. Other studies also reported good coverage of the pain scores [3], in contrast with our findings of moderate coverage. We believe the pain interference does not quantify the patient’s pain, but rather assesses how pain interferes with daily functioning. Also considering the brevity of the pain intensity questionnaire (three items for the pain intensity questionnaire versus six items for the pain interference questionnaire), we recommend using the pain intensity score in the new sacral tumor survey. The high correlation between the global physical function score and the pain sores suggests that pain might influence (i.e. interferes with) physical function [3]. Despite this, we believe that they are two separate aspects of quality of life and need to be measured independently.

Table 5.

Survey recommendation

Mental Health
PROMIS Global - Mental	4 items
Physical Health
PROMIS Global - Physical	4 items
Pain
PROMIS Pain Intensity	3 items
Gastrointestinal Symptoms †
PROMIS Gastrointestinal Symptoms Scale
Constipation	9 items
Bowel Incontinence/Soilage	4 items
Sexual Function
PROMIS Sexual Function*	8 – 10 items
Urinary Incontinence †
Urogenital Distress Inventory	6 items
Social Health**†
PROMIS Ability to participate in social-roles and activities	4 – 12 items

^†Newly added questionnaire

^*Contains gender-specific questions

^**CAT = computer adaptive test

The sexual function questionnaire allows patients to opt out if they had no sexual activity in the last 30 days, except for the questions about interest in sex. This explains why the response rate for the sexual interest domain is high but the scores themselves were low, as almost a quarter of the patients replied “not at all interested” in the sexual interest domain. For all other domains most patients opted out, making assessment of coverage less reliable. Even though, reliability of the domains was comparable with those reported in the literature [11, 12], there is a lack of reported floor and ceiling effects. Taking into account that both patient and experts agree that the sexual function domain is an important aspect of quality of life, all patients should have an opportunity to report on their sexual function. Maybe the PROMIS Sexual Function questionnaire is more appropriate for measuring sexual function over time within the same patient. Even though, we believe the PROMIS Sexual Function questionnaire could have proper coverage for sacral tumor patients who are sexually active and therefore recommend including it in the new survey.

The International Continence Society Male questionnaire (14 questions) is designed to assess lower urinary tract symptoms in men with prostatic disease [9, 19], and demonstrated poor coverage in our analysis. For the urinary symptoms domain, we therefore recommend using the Urogenital Distress Inventory (UDI-6) [40]. This questionnaire has been validated for both males and females in several countries [5, 13, 42], demonstrated good coverage [41, 42], and good to moderate inter-consistencies [5, 13, 41, 42], consists of only six items, and addresses difficulty with emptying the bladder, as well as incontinence instead of focusing only on incontinence.

The MODS questionnaire was originally designed as part of a decision algorithm focusing on whether to surgically treat patients with obstructed defecation syndrome [34]. As far as we know, no thorough validation of this questionnaire has been performed. The MODS is very rarely used in the literature, which makes comparison of results with other patient cohorts or other institutions difficult. In our analysis we found adequate reliability but only moderate coverage. We recommend measuring gastrointestinal symptoms using two domains of the novel PROMIS gastrointestinal symptoms scale [35], i.e. constipation (nine items) and bowel incontinence/soilage (four items). The PROMIS gastrointestinal symptoms scale is not designed for a disease specific population, but rather for anyone experiencing gastrointestinal symptoms. Although the PROMIS gastrointestinal symptoms scale was designed using a patient population in which none had a colostomy [35], we still recommend that patients with a colostomy also fill out these questions, as there are comparable questions validated in a colostomy population [45]. We do not recommend collecting additional colostomy-specific outcomes, since this would increase completion time and only add information for colostomy patients.

The qualitative methodology used in this study to better understand what the patients feel is relevant to measure their quality of life, allowed us to identify a new domain, namely, social health. Interestingly, this was not recognized by any of the experts. Another study, which focused on the qualitative assessment of patient experiences following sacrectomy, also found social health to be a common theme [7]. The PROMIS initiative provides validated questionnaires on social health. We believe that the “PROMIS Ability to participate in social roles and activities” [17] questionnaire would be a valuable addition to the new survey. This questionnaire assesses a patient’s perceived ability to perform one’s usual social roles and activities. Considering the burden of a lengthy survey on patients and the superiority in terms of coverage, we recommend using the computer adaptive test (CAT) version of the questionnaire. The computer adaptive testing item-bank offers a dynamic selection of questions wherein each question is based on the answer of the previous question, resulting in a patient-specific series of questions [21]. This method minimizes floor and ceiling effects and reduces the numbers of questions and therefore the completion time.

In conclusion, in an effort to shorten the sacral tumor survey without losing valuable information based on current literature, patient and expert opinion, we made a recommendation for the questionnaires that should be included in the revised sacral tumor survey (Table 5). This resulted in a shorter survey (minimum 42 and maximum 54 questions, that can be completed in approximately 15 minutes) that covers all relevant aspects of quality of life for sacral tumor patients. All PROMIS questionnaires, including the CAT versions, are readily available on Assessment Center [16]; a free, online data collection tool that enables researchers to create study-specific websites for capturing participant data securely online. Assessment Center also allows for creation of custom questions. The Urogenital Distress Inventory [40] is also readily online available.