The development and validation of a needs assessment tool for use with YOUng adult survivors of a CentrAl Nervous system tumor (YOU-CAN)

Abstract

Background

Adolescent and young adult (AYA) survivors of a central nervous system (CNS) tumor represent a vulnerable group who can experience: social isolation, low rates of employment, and achieving independence can be compromised, leading to poorer quality of life compared with survivors of other cancer types. The aim of this study is to develop and evaluate the validity of a needs assessment tool (NAT) for AYA survivors of a CNS tumor.

Methods

Items generated using data from 29 qualitative studies and cognitive interviews (n = 8) produced NAT V1.1 (49 items). 128 of 316 eligible participants attending neuro-oncology clinics at 4 NHS sites between June 2022 and March 2023 completed the NAT V1.1 to allow for item reduction and refinement and to evaluate reliability and validity. A pilot study (n = 6) using YOU-CAN in routine follow-up concluded the study.

Results

Hierarchical analysis and Rasch analysis identified 18- and 15-items for removal, respectively. YOU-CAN, comprised of the remaining 16 items, demonstrates excellent test-retest reliability (intra-class correlation coefficient, 0.901, n = 40) and sufficient correlation with the European Quality of Life questionnaire and Supportive Care Needs Survey (Pearson r = 0.433 and 0.590, respectively). Pilot testing showed YOU-CAN triggered discussions of unmet needs in consultations and highlighted the importance of multidisciplinary support.

Conclusions

YOU-CAN is a valid and reliable instrument containing items related to concerns about physical and emotional health; family and relationships; self-acceptance; and independence. Future efforts should examine YOU-CAN’s feasibility, and develop guidance for managing unmet needs. Routine use of YOU-CAN may improve the identification of otherwise undiscussed unmet needs and opportunities to deliver personalized support.

Central nervous system (CNS) tumors account for 25% of malignancies in children and 15% in adolescents and young adults (AYA). The survival rates of CNS tumors exceed 70% in children and AYAs¹ and there are therefore a growing number of long-term survivors. Eighty-two percent of survivors of a CNS tumor experience one or more ongoing late effects of treatment² and life-altering symptoms continue to be prevalent up to 30 years after the primary diagnosis.³ Symptoms often include persistent anxiety and depression, suicidal thoughts, and feelings of inadequacy.^3,4 Physical, neurologic, and neurocognitive impairments affect individuals’ ability to gain employment,^5–7 and achieve independence.^8,9 Ongoing symptoms contribute to poor health-related quality of life (HRQoL), which is lower than in survivors of other cancer types.¹⁰ In addition, survivors report feeling socially isolated, while often relying on family for social support.^4,11,12

A large proportion of young CNS tumor patients attend long-term follow-up clinics, which aim to support survivors to return to “normality” as quickly as possible while detecting and managing ongoing toxicities from treatment and disease.¹³ There is an increasing call to provide individualized follow-up care that supports individuals to self-manage symptoms, but despite this, many patients report a lack of knowledge of possible late effects of treatment and feel less supported as time after their treatment goes on.^3,14,15 This suggests that ongoing needs experienced by this population may remain unmet.

Patient-reported outcomes (PROs), including Needs Assessment Tools (NATs), have the potential to improve communication between health care professionals and patients, improve symptom management and HRQoL, and enable personalized care to be delivered.¹⁶ PROs validated for use in AYA survivors of a CNS are scarce, and currently, none measure unmet need.¹⁷ PROs and NATs validated in adult and pediatric populations of CNS survivors require further validation before they can be recommended for routine use in the AYA population.^18,19 Indeed, studies assessing quality of life or unmet needs using adult or child-focused instruments with AYAs may produce misleading and unreliable results if they do not reflect age-appropriate language and relevant domains.^20,21

This study aimed to develop and validate a new NAT for use with adolescent and YOUng adult (aged 16–39 years) survivors of a CentrAl Nervous system tumor (YOU-CAN) diagnosed during childhood or young adulthood.

Methods

The development of YOU-CAN was guided by the Consensus-based standards for the selection of health measurement instruments checklist for PRO measurement instruments,²² and the Food and Drug Administration²³ PRO development guidelines. Figure 1 shows that our sequential mixed methods study design consisted of the following phases: item generation to generate the NAT V1.0, followed by cognitive interviews to produce V1.1 (Phase 1); cross-sectional study and application of hierarchical and Rasch analysis of NAT V1.1 to select items for the final, unidimensional YOU-CAN version (Phase 2); preliminary validation and reliability testing of YOU-CAN (Phase 3); and a pilot study using YOU-CAN (Phase 4). Ethical approval was received from the National Research Ethics Service (North West; 21/NW/0344) for all phases.

Figure 1.

Study design.

Patient Experience Group and Clinical Expert Involvement

A patient experience group was established and their involvement was incorporated into all phases of the study (see Supplementary Material 1). Participants for this were invited via an advert placed on an NHS closed Facebook page. Group participants contributed to Phases 1 and 2 during 2 online focus groups and at other times via email. They were compensated for their time with a £15 shopping voucher for each focus group they attended. A clinical expert group also contributed to Phases 1 and 2 and consisted of 1 consultant endocrinologist, 3 consultant oncologists working in late effects clinics, an AYA Lead Nurse, an AYA Psychologist, a Youth Worker, and an AYA physiotherapist.

Study Recruitment

Participants for all phases of the study were recruited from 4 neuro-oncology outpatient follow-up clinics at 3 NHS sites in North West England between June 2022 and March 2023. Participants were eligible for all study phases if they were diagnosed between birth and 25 years of age with a CNS tumor, had completed treatment at least 6 months previously, were aged 16–39 years, and could provide written informed consent to take part. Participants were excluded if they had an expected prognosis of less than 1 year, were unable to independently answer items or understand English, or if the clinical team deemed them unsuitable to approach for other reasons. A member of the clinical team provided eligible participants with study information when they attended a face-to-face appointment or sent information via the post if their appointment was virtual. For Phase 1, participants were purposively sampled (n = 8) to include diversity in ethnicity; age, gender, and CNS tumor diagnosis. Patients recruited for the pilot study (n = 6) were required to be study naïve.

Data Collection

Phase 1: Initial Item Generation and Cognitive Interviews.—

We previously conducted a systematic review and meta-ethnography examining the needs of AYA survivors of a CNS tumor aged 15–39 years and synthesized 29 qualitative studies to develop a conceptual framework.²⁴ The conceptual framework highlighted ongoing needs throughout the continuum from adolescence to adulthood. Despite this age range being a period of rapid growth, and social and developmental change, the framework represented the needs of this population together. We assumed the conceptual framework represented a reflexive model where all items were a manifestation of 1 underlying trait, namely unmet need. We generated items using published qualitative data included in our meta-ethnography systematic review, and then reduced and refined the initial item list (NAT Version 1.0) through discussion within the research team and with our patient experience and clinical expert groups. At this stage, they could add any items they thought relevant based on their lived experience of the disease.

Participants for the cognitive interviews were provided a copy of the NAT Version 1.0 prior to the interviews to familiarize themselves with the content. Cognitive interviews use a combination of the “think aloud” and “probing” techniques of questioning.²⁵ Topics included comprehensibility, question style, response options, respondent burden, and comprehensiveness, aimed at ensuring that the items reflected unmet needs experienced in survivorship. All cognitive interviews were audio recorded and transcribed by the researcher (Kate Law). We conducted them via telephone to maintain social distance precautions due to the COVID-19 pandemic. Findings from the interviews informed further item reduction and refinement, which produced NAT Version 1.1.

The NAT (V 1.1) included 49 items representing issues relating to emotional and physical issues; achieving independence; friendships and relationships; family functioning; information and support needs. Items represented themes within the conceptual framework. Response options were given on a 5-point Likert scale with an additional option of “not applicable.”

Phase 2: Cross-sectional Study for Item Reduction and Refinement.—

In line with established guidance,²⁶ we estimated 150 participants to be an adequate sample size for Rasch analysis to evaluate validity. Eligible participants were provided with a study pack containing: a cover letter describing the study and its requirements, the NAT (V 1.1); and the Supportive Care Needs Survey (SCNS)²⁷ and EQ-5D²⁸ (see Phase 3). Questionnaires could be completed on paper or electronically by scanning a QR code. Implied consent was assumed on the return of a completed questionnaire. Demographic (age, gender, diagnosis, ethnicity, employment, and marriage status) and diagnosis details were provided by participants. To assess the representativeness of our study sample, we extracted data on age, gender, and cancer diagnosis from the health care records of potential participants who were invited to take part but declined or did not respond; no reminders were sent. Findings from the quantitative analysis were discussed with a patient experience group and clinical experts to review the items suggested for inclusion in YOU-CAN, as well as those excluded.

Phase 3: Test–Retest Reliability and Construct Validity Evaluation.—

On receipt of a completed NAT (V.1.1) by the study team, participants were invited to complete a second NAT (V1.1) 1 week later via a pre-paid, self-addressed envelope until 40 completed NATs were returned. Since all participants were at least 6 months beyond the end of treatment, and most were beyond 5 years from the end of treatment, participants were assumed to be clinically stable during the week interval between completion of the first and second questionnaires.

The construct validity of YOU-CAN was assessed using the SCNS and EQ-5D as comparator instruments. The SCNS (34 items) is a widely used instrument, validated in 18–85-year-olds diagnosed with cancer, that measures patients’ perception of their need for help in 5 domains: psychological; health system; physical and daily living; patient care and support; sexuality, providing a score between 0 and 100 validations in adult survivors of cancer aged.²⁷

EQ-5D (5 items) is a generic measure of HRQoL, validated in adults aged over 16 years, across 5 domains: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression, and includes a visual analog scale. It is used in multiple areas of clinical care and in research worldwide.²⁸

Phase 4: Pilot Study.—

The aim of this phase was to provide preliminary data regarding the clinical utility of YOU-CAN in routine follow-up to direct future research and development. A member of the clinical team invited eligible participants to complete the YOU-CAN on paper prior to seeing the medical consultant for their annual follow-up. YOU-CAN data and the time taken to complete the questionnaire were recorded. Following consultations, 1 researcher (Kate Law) conducted semi-structured individual interviews with consented clinicians and patients who used the YOU-CAN during their clinic appointment. The interview guide for patients and clinicians aimed to gather opinions on the clinical utility of YOU-CAN including the content of YOU-CAN; ease of completion/use; impact on discussions during consultation; impact on approach to offering support; and emotional response, when completing or discussing the results of YOU-CAN. Written interview notes were taken.

Data Analysis

Phase 1: Item Generation and Cognitive Interviews.—

Potential items were generated by listing all quotes from qualitative literature identified from the meta-ethnography²⁴ in an item tracking matrix. One researcher (Kate Law) grouped and merged quotes according to similarity and interpreted them into a statement of need in discussion with the other members. The original wording of each item was maintained where possible to maximize readability and understanding and to reduce cognitive demand and complexity of items. Alterations were documented in the item tracking matrix to provide evidence of decision-making. Items were altered based on patients’ understanding following thematic analysis of transcripts.²⁹

Phase 2: Cross-sectional Study for Hierarchical and Rasch Analysis.—

Demographic and item responses for each questionnaire were analyzed using descriptive statistics. Statistical significance was at P < .05. Statistical analyses were conducted using SPSS (version 28.0.1.0 [142] Inc; Chicago, IL).

Hierarchical analysis was applied to the NAT V1.1 data to identify potential items for deletion, including items with: statistically significant age bias (Pearson’s correlation) or sex bias (independent t test); floor and ceiling effects (<10% and >90% endorsed extreme categories, respectively); substantial inter-item correlations (r > 0.7); and missing data per item (>20%).^30,31 Following this process, the remaining items were included in the Rasch analysis.

Rasch Measurement Theory is an established method used to validate questionnaires based on 2 assumptions: the instrument is unidimensional, and the probability of answering an item is unrelated to the probability of answering any other item for people with the same amount of trait.³² The underlying trait of unmet need is measured using items reflecting a range of item difficulty.³³ “Item difficulty” in this case relates to the level of unmet need expressed by the item and is calculated according to observed and expected responses in RUMM software 2030 (www.rummlab.com). Respondents were grouped according to their level of need into class intervals (of approximately equal sizes) using their observed scores in RUMM. Data from class intervals were compared using Chi-squared statistics. Summary statistics presented in Rasch analysis include the Person Separation Index. Analogous to Cronbach alpha it gives an estimate of internal consistency and was considered good when >0.90.³² Rasch analysis involves identifying which items do not fit the Rasch model so they can be removed individually and iteratively until a unidimensional fit is achieved. Unidimensionality was indicated by a nonsignificant difference between the person estimates of items showing the greatest difference when ordered by the first component in Principal Component Analysis.³⁴ Residuals (summation of individual person and item deviations) between ±2.5 logits indicate adequate fit to the Rasch model, the scale of unmet need is measured in logits in RUMM software.³⁴ Items outside this range were flagged for removal individually and iteratively. Additional tests of fit, as shown in Table 1, are displayed in RUMM software 2030 and were also used to identify items for potential removal. Rasch analysis resulted in the NAT version named YOU-CAN.

Table 1.

Tests of Fit in Rasch Analysis

Tests of fit	Explanation
Item characteristic curves (ICC)	ICC were examined after each item was removed to identify where remaining items over/under discriminate and therefore which items show misfit the Rasch model and could be removed. Under-discrimination is seen when more people are observed to have a low need for this item than expected and fewer people have a higher need than expected.
Targeting	Targeting displays provided a visual representation of how well items target the participants by showing the relative distribution of participants and items on the same logit scale. This display was examined after each item was removed and the aim was to achieve a final set of well targeted items.
Category probability curves	It is assumed that there is a logical order in the use of responses across the underlying trait of unmet need. A threshold refers to the point between adjacent response categories for each item where it is equally likely to obtain either score of two adjacent categories. Category Probability Curves are a visual representation of the order of these thresholds and were examined to identify where items or the responses did not fit the Rasch model. Thresholds should be ordered and can be merged where disordered thresholds occurs.35
Local dependency	A source of misfit could be due to presence of local dependency between items (r > 0.2 above the average of residual correlations), that is where the response to one item has a bearing on their response to a different item, illustrating which items can be removed.

Scoring.—

Items were scored 1 (strongly agree; indicating agreement to having a “need”) to 5 (strongly disagree), and an item scored zero for any missing data. All items were post hoc reversed scored so that the transformed score was on a scale from 1 (lowest possible level of need) to 80 (highest possible level of need).

Missing Data.—

Questionnaires for individual responders were removed prior to analysis if there was more than 30% missing data. Missing data in the remaining questionnaires was left blank for Rasch analysis, where there was less than 5% missing it was not expected to affect results.³² Since responses needed to be on a Likert scale of increasing/decreasing sequence for Rasch analysis, responses to “not applicable” were marked as missing. RUMM2030 automatically assumes missing data is missing at random.³⁶

Phase 3: Test–Retest Reliability and Construct Validity Evaluation.—

Test-retest reliability for the total raw score of YOU-CAN was evaluated using intra-class correlation coefficients. Sufficient reliability is demonstrated when ICC ≥ 0.70.²²

The SCNS and EQ-5D were analyzed as per the developers’ instructions.^27,28 We hypothesized that the items in YOU-CAN would be sufficiently correlated (Pearson’s r ≥ 0.5) with the SCNS as they both measure unmet needs, albeit for different populations. We expected that YOU-CAN would correlate with the EQ-5D to a lesser extent (Pearson’s r = 0.3–0.5) as this is designed to measure HRQoL, a related but different concept to “unmet need.”²²

Phase 4: Pilot Study.—

Thematic analysis²⁹ of interview data from patient and clinician participants was conducted. Descriptive statistics were used to analyze YOU-CAN data to show the level of need of participants in this phase.

Results

Study Participant Characteristics

In total, 316 out of 326 young people met eligibility criteria, of whom 128 (41%) returned completed NAT V1.1 questionnaires as part of Phase 2; 104 were completed on paper and 24 completed online. Time since treatment was not recorded, though the majority (n = 92) were recruited from a late effects service where patients were at least 5 years beyond the completion of treatment. Table 2 shows that their characteristics represented a range of diagnoses and ages across the spectrum 16–39 years, with an equal number of men and women and 78% identifying as White British. There were no significant differences between the age (P = .220), gender (P = .157), or diagnosis (P = .078) of responders and nonresponders.

Table 2.

Characteristics of Study Participants

Characteristic	Phase 1 n (%) (n = 8)	Phase 2 n (%) (n = 128)	Phase 3 n (%) (n = 40)	Phase 4 n (%) (n = 6)
Women	4 (50)	65 (51)	21 (52)	3 (50)
Men	4(50)	63 (49)	19 (48)	3(50)
Age group: 16–20 y		28 (22)	9 (23)
21–25 y	4 (50)	33 (27)	10 (25)	2 (33)
26–30 y	2 (25)	33 (25)	10 (25)	2 (33)
31–35 y	1 (12)	23 (17)	6 (14)	2 (33)
36–39 y	1 (12)	11 (9)	5 (13)
Diagnosis
Craniopharyngioma	1 (12)	20 (16)	5 (13)	1(16)
IDH wildtype diffuse astrocytoma		2 (2)		1 (16)
Neurocytoma		2 (2)
Germ cell tumor and Germinoma	1 (12)	19 (15)	8 (20)
Low grade glioma	1 (12)	37 (29)	11 (8)
Medulloblastoma	2 (25)	28 (22)	8 (20)	1 (16)
Oligodendrocytoma		2 (2)
Rhabdoid tumor		1 (1)	1 (2)
Xanthroastrocytoma		1 (1)
Meningioma		2 (2)	1(2)	1 (16)
Ependymoma		7 (5)	4 (10)
Pineal tumour	1 (12)	4 (4)	1 (2)	1 (16)
Choroid plexus tumour	2 (25)	1 (1)	1 (2)	1 (16)
Gliosarcoma		1 (1)
Glioblastoma		1 (1)
Ethnicitya:				Not recorded
African		1 (1)
White British	7 (88)	98 (78)	35 (88)
Asian	1 (12)	13 (10)	5 (12)
Chinese		1 (1)
Missing data		12 (10)
Employment statusa: employed	3 (37)	79 (63)	28 (70)	Not recorded
Marriage statusa: married	2 (25)	16 (13)	6 (15)	Not recorded

Note:

^aInformation on ethnicity, employment, and marriage status was recorded with informed consent, this information was not recorded where participants declined to participate. Demographic information on nonresponders is in Supplementary Material 4.

Phase 1: Item Generation and Cognitive Interviews

Initially, 227 quotes from 29 qualitative studies (n = 246 individual participants) resulted in a list of 55 potential items; Supplementary Material 2 shows examples of decision-making at this stage. Items were generated from participants who were between 6 months and 27 years post-completion of treatment thereby covering a broad range of time points within this period. After discussion with members of the patient experience group and clinical expert group, we produced a 41-item NAT V 1.0.

Eight cognitive interviews lasting between 35 and 60 min were conducted. Table 3 shows examples of modifications to item wording and 8 additional items based on interviewees’ suggestions; Supplementary Material 3 contains the complete overview of modifications. This phase resulted in the NAT V1.1 which comprised 49 items.

Table 3.

Examples of Quotes from Cognitive Interviews which Supported Item Amendment

Potential draft item: Currently, I feel like I need help with …	Quote from cognitive interviews to support change	Amended draft item: Currently, I feel like I need help with …
finding ways to overcome side effects of my cancer or treatment	So basically, finding ways to, like for example my muscle cramps … I read it as physical … physical and emotional well-being are two different things … (Interview 6)	finding ways to overcome physical issues (eg, Poor balance, fatigue, memory loss, loss of hearing, and loss of sight)
		Additional item: finding ways to overcome emotional issues
Dealing with symptoms more common in older people (eg, menopause)	So for this one, kind of symptoms that wouldn’t usually happen in someone in my age group … the phrase more common in older people, I don’t know, I wouldn’t want anyone to get offended by that … if they are struggling with it, oh, that kind of sucks (Interview 5)	dealing with symptoms that wouldn’t usually happen to someone in my age group
worry that I won’t be able to have children	I’d say “starting a family” and worry about infertility (I4) I did worry a lot, what if my kid also has the same tumour or what if I pass down my cancer or something … (I6) Starting a family is different (to infertility) because there are so many things you can do like surrogacy, IVF, adoption, like that (Interview 6)	worry about starting a family
		Additional item: worry about infertility
relying on my parents for support for my mental health	I did have a question about this … this question the way it is phrased is like I need help not doing this thing, if that makes sense … people don’t want to be relying on their parents so I think it could be phrased in a more positive way … (Interview 5)	accessing support for my mental health outside of my family

Phase 2: Cross-sectional Study for Hierarchical and Rasch Analysis

Three questionnaires were excluded due to >30% missing data from the draft NAT and 125 were included in the final analysis. There was minimal missing data in the final data set: out of 125, 11 participants had less than 5% missing data and 5 participants had 5%–14% missing.

Hierarchical Item Reduction.—

Hierarchical analysis identified 18 items for removal; the reasons are shown in Supplementary Material 5. No items showed ceiling effect or age bias.

Rasch Analysis.—

The remaining 31 items were included in the Rasch analysis. Summary statistics demonstrated an acceptable level of fit to the Rasch unidimensional model after an iterative process of removing 15 items (final item set: χ² = 43.58, P = .08; 9 significant tests out of 118, 7.63%, lower bound confidence interval [CI] = 0.037). The fit statistics for the remaining 16 items, which make up the final YOU-CAN version, are shown in Table 4. YOU-CAN items formed a unidimensional fit to the Rasch model with a Person Separation Index of 0.92. The final list of items making up YOU-CAN can be found in Supplementary Material 6.

Table 4.

Fit Statistics for Remaining Items that Form YOU-CAN

Item: “currently, I feel like I need help with …”	Item location	Item fit residual	Local dependency (r)	ICC fit (Chi-square probability)
1. Finding ways to overcome physical issues (eg, Poor balance, fatigue, memory loss, loss of hearing, loss of sight).a	−0.869	1.102	Item 2 0.212	0.602
2. Finding ways to overcome emotional issues	−0.483	1.566		0.682
3. Mourning the loss of my old life	0.073	−0.214		0.575
4. Belonging to a friendship group	0.215	1.648		0.371
5. Not worrying my family with my problems	0.019	0.530	Item 2 0.197	0.805
6. Wanting to help my family manage their worries	0.018	−0.493	Item 5 0.383	0.287
7. Managing my worries about starting a family	−0.119	6.595		0.00b
8. Not feeling defined by my brain tumor	0.275	−1.979		0.083
9. Accepting who I have become	−0.041	−0.350	Item 8 0.243	0.399
10. Feeling like my previous self	0.124	−0.376	Item 3 0.233, item 8 0.273, item 9 0.157	0.541
11. Having a positive attitude	0.063	0.748	Item 9 0.145, item 10 0.153	0.811
12. Finding ways to live independently	−0.057	−0.287		0.827
13. Going out safely alone	0.194	−0.406	Item 12 0.275	0.241
14. Learning daily life skills (eg, using public transport, buying things)c	0.348	−0.495	Item 12 0.137, item 13 0.457	0.323
15. Finding ways to take part in hobbies	0.318	−0.638	Item 13 0.231, item 14 0.423	0.625
16. Accessing support for my mental health outside of my family	−0.079	0.559		0.552

Note:

^aLowest item location represents the item reflecting lowest level of unmet need.

^bClassic under-discrimination.

^cHighest item location rsepresenting the item expressing greatest level of unmet need.

The average score for YOU-CAN of 125 participants was 43.49 (n = 125; SD = 16.43) on a scale ranging from 1 (lowest possible level of need) to 80 (highest possible level of need). The mean person location for the instrument was −0.37 (SD = 2.0; item fit = 0), demonstrating reasonable fit.³⁴ Examining the tests of fit of the final 16 items did not result in any other items being removed, these items were well targeted and no differential item functioning was seen in these items using Bonferroni adjusted statistics. There was no significant local dependency between items. Further information on tests of fit can be seen in Supplementary Material 7.

Phase 3: Test–Retest Reliability and Construct Validity Evaluation

Forty participants completed the NAT at 2-time points a week apart. The intraclass correlation coefficient (ICC) between test and retest was 0.901 (CI: 0.814–0.948). Correlations between YOU-CAN and the EQ-5D and SCNS were r = 0.433 (P < .001) and r = 0.590 (P < .001), respectively.

Phase 4: Pilot Study

Six patient participants completed YOU-CAN prior to seeing the clinician. The mean need score was 51.33 (SD = 14.58). A score of 50 can indicate a high level of need for 10 issues or a lower level of need for a higher number of issues. The mean time taken to complete the NAT was 2.12 min (SD = 0.85). Six patient participants and 4 clinician interviews were conducted, each lasting 10–25 min. Two themes were constructed: (1) Impact on consultations: length, focus of discussion, addressing needs; (2) Practical issues.

Themes.—

Consultation: Length

Clinicians reported mixed views regarding the impact of the NAT on the length of consultation. One clinician felt it made consultations shorter as it focused on discussion, and 3 clinicians felt it made them longer but acknowledged that “if we get used to it, it might speed up clinics.”

Consultation: Focus of Discussion

One clinician reported that the NAT confirmed the issues they anticipated to be present, while 3 others reported that YOU CAN raise issues not previously known to be a problem to the patient. Similarly, 4 patients reported that using the NAT allowed them to raise issues they would not have otherwise wanted to “bother the doctor with.” These patient participants said it changed their discussion and provided them with answers to ongoing issues. All patients and health care professionals agreed there was nothing missing from the list of items.

Consultation: Addressing Needs

All clinicians mentioned the need for guidance on how to respond when a patient identifies they need support. One clinician reported the current lack of resources to provide support made them feel awkward as “we want to give solutions to problems; it’s uncomfortable when we don’t have resources to help the well-being of our patients.” Three clinicians commented on the lack of multidisciplinary support to address the needs identified by YOU-CAN. YOU-CAN highlighted that addressing the diverse nature of unmet needs required a multidisciplinary approach: “ a lot of it is social work, we could do with a social worker, a lot isn’t medical,” “we need more help, a late effects nurse, psychologist...” One patient reinforced the desire for a multi-disciplinary team approach: “it would be helpful if there was someone else to talk to, not a doctor, a doctor isn’t the right person to help with everything. We need someone in clinic to signpost us.” All patient participants felt comfortable answering the questions.

Practical Issues

One patient who completed the NAT while sitting next to their mother said their presence influenced their answers as their mother could see the responses. Another patient said they wished for the NAT to be given on paper during the clinic as “emails and links to online questionnaires would just get lost.” All patients commented on the ease of completion. Clinicians noted that they would prefer to see the results prior to the consultation to give some time for preparation, and that results would be most useful if they were accessible on the electronic patient record.

Discussion

YOU-CAN is a 16-item questionnaire, designed to measure unmet needs in AYA survivors of a CNS tumor. To our knowledge, this is the first NAT designed and validated with AYA survivors of a CNS tumor (aged 16–39 years). The demographics of the participants in this study were similar to those who were invited to take part but chose not to respond. Our population was also representative of the ethnic diversity of those diagnosed with a CNS tumor in the United Kingdom.³⁷ Despite guidance to provide personalized support in follow-up, it is widely accepted that this population can experience ongoing issues and an objective assessment of issues is often lacking.^14,15 PROs designed to measure HRQoL exist for this population, however, when assessing which issues are a priority to the patient, NATs are more suitable as they identify which issues the individual wants support with.³⁸ Age-appropriate, but not disease-specific tools to improve assessments of need are available to guide conversations between health care professionals and patients, such as the Integrated Assessment Map³⁹ though this lacks psychometric validation, and is not intended to quantify levels of unmet need using a total score. In addition, validated NATs exist that have been developed in other cancer populations, for example the Patient Concerns Inventory, developed in adult populations of CNS survivors.⁴⁰ However, it is known that measurement instruments designed for adult or pediatric populations may overlook age-specific issues resulting in misleading results.^20,21

We have demonstrated the content validity of YOU-CAN by ensuring the comprehensiveness of items generated from a systematic literature review in addition to patient and health care involvement during its development and cognitive interviews. Consideration of the complexity of items was necessary given this population is susceptible to cognitive impairment.⁴¹ The choice of recall period was influenced by the patient experience group and qualitative data from cognitive interviews. Using “currently” as a recall period, as opposed to “in the last week/ month” which is common in other NATs,^42,43 minimizes complex processing for people who have poor memory and may find being asked to recall how they felt over a longer period a challenge. This may also reflect disease-specific preferences, as a time frame is not specified in the NAT developed for adult CNS patients.⁴⁰

YOU-CAN demonstrated excellent test-retest reliability for the overall score of unmet need and good construct validity with the EQ-5D and SCNS.^27,28 However more testing at item test-retest level is needed since it is necessary for the scores for each item to remain constant in order to demonstrate responsiveness.⁴⁴

The concept of unmet need can be considered a multi-dimensional construct.⁴⁵ However, Rasch analysis identified 16 items that contribute reliably to measuring 1 underlying trait (unmet need) while still including a range of important aspects of survivorship relevant to the underlying conceptual framework: concerns about physical and emotional health; family and relationships; self-acceptance; and independence. These are known to be areas of ongoing concern for the AYA CNS tumor survivor population.^3,8,9 Qualitative data from the pilot study and patient experience group suggested endorsement of the content and comprehensiveness of the final instrument. However, the clinical expert group noted a lack of items regarding employment and education, which are known issues for this population.^5,9 Items relating to education demonstrated a floor effect and indicated redundancy so were removed during hierarchical analysis. During Rasch analysis, items related to employment demonstrated misfits and were removed to create a unidimensional instrument where data fits the Rasch model. Further research is necessary to examine the clinical utility of YOU-CAN and whether an item about employment is necessary.

Limitations

The majority of qualitative studies included to generate items for NAT V1.0 consisted of mainly White European and American populations, therefore, the items may under-represent the needs of people from other cultures. In addition, due to the small number of participants from ethnic minority groups, we were unable to examine whether ethnicity affects item functioning. Information on the ethnicity of those eligible but who did not respond was also missing, so we could not assess whether our study population was representative of the overall sample in terms of ethnicity. Further research is necessary to establish the cross-cultural validity of YOU-CAN.

The response option “Neither agree/ disagree” was underused and therefore demonstrated some misfit to the Rasch model and this option could not be merged post hoc as recommended³³ since it is impossible to determine whether it should be combined with “agree” or “disagree” which are the options either side of this response. The alternative is to code this as missing, however, such a large amount of missing data would decrease the reliability of the analysis, therefore the response was kept despite it being under-used.³² If respondents did not have this as a response option and were forced to answer agree, disagree, or leave blank, it is possible that data would fit differently to the Rasch model and items forming the final unidimensional instrument may have been different.

Implications for Practice

We have demonstrated here that the YOU-CAN is quick and simple for patients to use and, in a very small pilot study, that it may have clinical utility. YOU-CAN allows patients to report what matters to them, thereby contributing to overcoming clinician bias and enabling a consistent person-centered approach to identifying the needs that patients deem important.

Many NATs have been developed, validated, and successfully implemented into routine practice in other populations to provide a short, reliable, and systematic method of identifying needs while also being used to collect routine data for research purposes and service development.^46,47 The purpose of YOU-CAN is not to identify all possible unmet needs experienced by an individual. Instead, it can be used to provide a starting point for conversations between HCP and patients, while collecting valid and reliable data on the unmet needs of AYA survivors of a CNS tumor attending long-term follow-up. This means that, informed by the patient involvement group and the cognitive interviews, we kept some items purposefully broad while complementing them with more specific prompts (see Supplementary Material 6). This helped us to keep the instrument short while optimizing its clinical utility. Furthermore, identifying unmet needs with YOU-CAN should only be considered the first step toward resolving the issue. Patients’ responses to YOU-CAN can help clinical teams to characterize ongoing needs, direct support to those who need it most, or initiate implementation of appropriate interventions into the service where support does not already exist, for example, referral to a fertility specialist, psychological support services and social services to support independent living. Pilot data suggests the use of YOU-CAN gave clinicians and patients’ permission to discuss different issues related to gaining independence, which are not routinely raised but are ongoing concerns for patients. It should be possible for issues around employment to be raised if there are issues with living independently or learning life skills. However, the lack of employment-specific items highlights the need for further research to examine the clinical utility and content of YOU-CAN.

Further research to examine the responsiveness and clinical utility on a larger scale is important and necessary before it can be incorporated into routine follow-up clinics. If YOU-CAN demonstrates responsiveness proven to detect change in unmet needs over time with good reliability at the item level and it is able to detect when a real change has occurred, then there is an opportunity to use this to improve the collection of routine data to better characterize the unmet needs specific to AYA survivors of CNS cancer. Reliable data can then be used to audit clinical performance; research; and evaluate clinical practice.⁴⁸

Our pilot data also suggest a multidisciplinary approach to follow-up is required, in keeping with existing evidence.^49,50 Last, further work must also focus on developing services to address needs once they are identified and to maximize the YOU-CANs clinical utility.

Conclusions

A short, valid, and reliable instrument has been developed using robust methods to identify and provide a measure of unmet needs in AYA survivors of a CNS tumor. Future research should include pilot testing YOU-CAN in routine care on a larger scale, examining YOU-CAN’s interpretability, responsiveness, and feasibility, and development guidance for managing unmet needs.

Supplementary Material

Supplementary material is available online at Neuro-Oncology (http://neuro-oncology.oxfordjournals.org/).

Funding

This work was supported by Manchester Cancer Research Centre and The Christie Hospital NHS Foundation Trust. There are no specific grant numbers.

Conflict of Interest Statement

None declared.