The development of scales to measure childhood cancer survivors' readiness for transition to long‐term follow‐up care as adults

Abstract

Purpose

To develop and validate scales to measure constructs that survivors of childhood cancer report as barriers and/or facilitators to the process of transitioning from paediatric to adult‐oriented long‐term follow‐up (LTFU) care.

Methods

Qualitative interviews provided a dataset that were used to develop items for three new scales that measure cancer worry, self‐management skills and expectations about adult care. These scales were field‐tested in a sample of 250 survivors aged 15–26 years recruited from three Canadian hospitals between July 2011 and January 2012. Rasch Measurement Theory (RMT) analysis was used to identify the items that represent the best indicators of each scale using tests of validity (i.e. thresholds for item response options, item fit statistics, item locations, differential item function) and reliability (Person Separation Index). Traditional psychometric tests of measurement performance were also conducted.

Results

RMT led to the refinement of a 6‐item Cancer Worry scale (focused on worry about cancer‐related issues such as late effects), a 15‐item Self‐Management Skills scale (focused on skills an adolescent needs to acquire to manage their own health care), and a 12‐item Expectations scale (about the nature of adult LTFU care). Our study provides preliminary evidence about the reliability and validity of these new scales (e.g. Person Separation Index ≥ 0.81; Cronbach's α ≥ 0.81; test–retest reliability ≥ 0.85).

Conclusion

There is limited knowledge about the transition experience of childhood cancer survivors. These scales can be used to investigate barriers survivors face in the process of transition from paediatric to adult care.

Introduction

While more than 80% of children with cancer survive their initial disease and live well into adulthood,1 the treatments used to cure patients of their cancer place many survivors at risk for developing one or more late effects (e.g. new cancers, cardiac and pulmonary disease etc.).2, 3, 4 The Institute of Medicine has recommended that all survivors of childhood cancer receive long‐term follow‐up care (LTFU) from a practitioner knowledgeable about the survivor's cancer history, their long‐term risks, and their recommended health care and surveillance, to minimize the impact of potential late effects of cancer therapy.5 The LTFU care of adult survivors of childhood cancer varies by locale, but includes specialized LTFU clinics located in paediatric or adult cancer centres, primary care physicians who provide survivorship care, and a shared care approach that involves cancer experts working with primary care physicians.6

Transition has been described as the purposeful, planned movement of adolescents with chronic health conditions from child‐centred to adult‐oriented health‐care systems.7 The goals of transition programmes are to assist adolescents to acquire the knowledge and skills they need to transfer to adult care and to assume independent responsibility for their health care.8 The process of transition to adult services ideally should address medical, psychosocial, educational and/or vocational needs of adolescents,7, 8 and preparation of adolescents for transition should start many years before the transition occurs.9

Survivors of childhood cancer differ from patients with other chronic conditions in that, at the time of transition from child‐centred to adult‐oriented health care, many survivors have not manifested late effects of their therapy, and thus feel well and require no medication or other interventions. Frequently, late effects associated with the intensive treatments used to cure children of their cancer manifest several years after transition, at which point many survivors have been lost to long‐term follow‐up care. Survivors who fail to transition to adult‐oriented long‐term follow‐up care miss out on having care focused on their specific risks resulting in lost opportunities to detect late effects early and placing them at increased risk for late morbidity and early mortality.

There is currently limited knowledge about the transition experience of childhood cancer survivors, and specifically the barriers that they face in continued attendance at LTFU clinics. A study of adult survivors aged 21–51 years identified the following as barriers to obtaining LTFU care as adults: survivors' lack of knowledge about their specific risks, lack of health insurance, and the limited number of health‐care providers equipped to care for childhood cancer survivors.10 Klosky and colleagues studied more than 900 survivors currently aged between 7 and 39 years found that non‐attendance at LTFU clinics was related to ethnicity, lacking medical insurance, and travelling by car (vs. air or bus), with work and school conflicts cited as the main reasons for missed appointments.11

In Canada, to prepare paediatric patients for transition to adult health care, ad hoc checklists (not formally developed or tested) are often used to identify potential barriers (e.g. checklists based on the ON‐TRAC transition program12 are used in the Good‐to‐Go transition program at the country's largest children's hospital in Toronto13). Two recent systematic reviews identified a lack of psychometrically sound published transition measures.14, 15 There is thus a need for scientifically sound and clinically meaningful tools for use in transition programmes to prepare childhood cancer survivors for transition by allowing for the identification of specific barriers to transition, and guiding the application of targeted interventions.

Following internationally accepted guidelines for the development of a new patient‐reported outcome (PRO) instrument,16, 17 our team set out to develop a set of scales to measure constructs that childhood cancer survivors have identified as being important barriers and/or facilitators in the process of transition from paediatric to adult LTFU care. The aim of this paper is to describe the development and psychometric evaluation of three new scales developed by our team. These scales measure the following: (i) cancer worry about cancer‐related issues such as a recurrence or late effects; (ii) self‐management skills that an adolescent needs to acquire in order to manage their own health care; and (iii) expectations about the nature of adult LTFU care.

Methods

Local research ethics board approval was obtained from participating centres prior to starting this study. Internationally recommended guidelines were followed for the development of a PRO measure.16, 17

Phase 1: Development of constructs, scales and items

The first phase involved identifying the key barriers and facilitators to transition faced by childhood cancer survivors, which was accomplished through in‐depth patient interviews. We describe the qualitative phase in detail elsewhere.18, 19 Briefly, 38 Canadian childhood cancer survivors were interviewed to understand their transition experience and identify important barriers and facilitators they encountered. This sample included 10 pre‐transition survivors (adolescent‐aged survivors currently attending a LTFU clinic at a pediatric center), 11 successful transition survivors (attends LTFU for a continuous period of 3 years after either turning 18 or being transferred to an adult LTFU programme), seven survivors who failed to transition (never attended a LTFU appointment in an adult centre or after 18 years of age), and 10 survivors who dropped out of transition (attended at least one appointment in an adult centre or after 18 years of age but failed to attend LTFU for a continuous period of 3 years). Analysis of the data permitted identification of three primary constructs that constitute a framework for a measurement tool covering important barriers and/or facilitators encountered by childhood cancer survivors in the process of transition to adult‐oriented LTFU care. The three constructs are as follows: (i) cancer worry, (ii) self‐management skills and (iii) expectations about adult LTFU care.

The first phase also involved developing items and scales to measure these three key constructs. This step involved using the qualitative data that had been coded in an inductive line‐by‐line manner. (i) All the codes (i.e. key phrases expressed by survivors) associated with each construct in the dataset were cut and pasted from NVivo820 coding reports into Excel along with model of care (i.e. transition from paediatric hospital to new LTFU programme at an adult hospital; or remain at paediatric hospital in combined paediatric‐adult LTFU clinic), and (ii) transition status (i.e. about to transition, transitioned successfully, transitioned but then dropped out of adult care and failed to transition). Including such characteristics is important in item generation as they make it possible to identify potential core items (common across all subgroups) and unique items (specific to a subgroup). The codes were considered one by one, and a total of 1883 preliminary items were generated and assigned a descriptor to capture the essence of what each item measures.

To develop the scales, we examined the item list developed from the coded material both iteratively and interactively to identify items that would together map out a continuum for each of the three constructs. We developed brief instructions and four response options labelled as follows: ‘strongly agree’, ‘agree’, ‘disagree’ or ‘strongly disagree’ to remain consistent with a systematic review reporting that rating scales with complicated question format, a large number of response categories, or unlabelled categories tend to be dysfunctional.21 Flesch‐Kincaid grade level scores (an indicator of comprehension difficulty)22 were examined to reduce items to the lowest possible grade.

Scales were presented to 17 experts in the field including three paediatric oncologists, three parents of childhood cancer survivors, two nurses, two social workers, one childhood cancer survivor, one paediatric neuro‐oncologist, one radiation oncologist, one adult oncologist, one psychologist, one neuropsychologist and one paediatrician. Three experts had substantial research expertise on the topic of transition readiness. Experts provided written feedback on the instructions, response options and items, which was used to revise the scales.

Finally, cognitive interviews were conducted with seven survivors who ranged in current age (range 16–22 years), age at diagnosis (range 4–16 years), and gender (five male, two female). Feedback was sought to identify ambiguities in the instructions, response options, item wording, layout, and to determine acceptability.

Phase 2: Field‐testing, scale construction and psychometric evaluation

A field‐test was conducted to collect data to identify the items that represent the best indicators of each scale based on their performance against a standardized set of psychometric criteria. Data were collected at three Canadian paediatric oncology centres. A convenience sample of survivors aged 15–26 years with any type of cancer was recruited between July 2011 and January 2012. The sample included survivors both pre‐ and post‐transition. We excluded survivors with a cognitive disability that would prevent them completing the questionnaire independently.

Participants were invited to complete a consent form and the questionnaire booklet when they came to the hospital for an oncology appointment. The questionnaire booklet included the three new scales alongside items asking about the participant (e.g. age, gender and ethnicity) and their cancer (e.g. type of cancer, date of diagnosis and types of treatment). To ensure accuracy of questionnaire data, we extracted the same cancer information from the hospital records. For the hospital data, one research assistant was made responsible for collecting the information, and a second research assistant independently checked the extracted data against the hospital charts. Anyone not scheduled for an appointment during the recruitment period was sent a questionnaire booklet in the mail with up to three reminders, as needed. All survivors invited to participate were given a five dollar gift card as a ‘thank you’ for considering participation in the study.

A consecutive sample of survivors who agreed to participate in our study were invited to complete a test retest (TRT) copy of the questionnaire at least 1 week after completing the first copy. Those recruited face‐to‐face were given the questionnaire to take home and complete and return using a prepaid envelope. Participants recruited through the mail, who indicated on their consent form they were willing to complete a TRT questionnaire, were mailed a TRT with a prepaid return envelope. We recruited participants into the TRT study until we had received 50 TRT questionnaires completed between 1 week and 2 months after the initial copy of the questionnaire booklet was provided.

We compared non‐respondents and respondents on the following variables: gender, current age, age at diagnosis and type of cancer. For both respondents and non‐respondents, age at diagnosis and type of cancer was extracted from the hospital charts.

Rasch Measurement Theory (RMT) analysis

We analysed the scales' data using Rasch Measurement Theory (RMT) methods23, 24 within RUMM2030 software.25 RMT methods are being used increasingly in health research.26 RMT analysis examines differences between observed and predicted item responses to determine the extent to which the data accord with (‘fit’) a mathematical model. When data fit the model, the estimates derived from the model are considered appropriate because the measurement theory is supported by the data.27 In the Rasch model, the estimation of item parameters is independent of the sampling distribution of respondents. Sample size calculations for psychometric analyses are a controversial area and there are no widely accepted conventions that cover the many types of analyses conducted during the evaluation of an instrument.28 Rather, the emphasis is upon the degree of precision of the item (and person) estimates. For the current study, we used rule of thumb sample size estimation based on the degree of precision of the item (and person) estimates. Consequently, in terms of targeting, a sample of 108 (best) to 243 (worst) participants will give 99% confidence that item estimates will be within 0.5 logits.29 We therefore aimed to recruit a sample of 250 participants. A set of statistical and graphical tests is used to determine the extent to which responses to scale items fit with responses expected by the Rasch model. Results from these tests are interpreted together to make an overall judgment about the quality of the scale.27 RMT analysis involved an examination of validity and reliability using the following tests and criteria:

Validity

1. Thresholds for item response options: For each item, the use of response categories scored with successive integer scores implies a continuum that increases for the construct of interest. We tested this assumption by examining the ordering of thresholds (i.e. points of crossover between adjacent response categories). When thresholds are disordered, respondents cannot differentiate between the response options for an item. When response options work as expected, evidence supporting the validity of the scale is obtained.30

2. Item fit statistics: The items of a scale must work together as a set both clinically and statistically. When items do not work together (misfit), it is not appropriate to sum item responses to reach a total score. Misfit indicates that an item is not working as intended in a scale. We examined the following three indicators of fit: (i) fit residuals (item–person interaction), (ii) chi‐square values (item–trait interaction), and (iii) item characteristic curves (ICC). It is more meaningful to interpret fit statistics together as an item set rather than separately.31

3. Item locations: The match between two distributions (the range of a construct measured by the items in a scale, and the range of the construct as reported by a sample of patients) provides information about scale‐to‐sample targeting. Examining the spread of person and item locations can highlight problems with a scale (e.g. more than one item at the same location indicates redundancy, and a gap in the continuum is an indication of where new items may be needed).31

4. Stability: The degree to which item performance remains stable across subgroups is known as differential item functioning (DIF). We examined DIF for gender and age at diagnosis. Statistically significant chi‐square values indicate potential DIF (significance interpreted after Bonferroni adjustment).32

Reliability

The Person Separation Index (PSI), a reliability statistic comparable to Cronbach's α, was used to examine person measurements (estimates).33 The PSI quantifies the error associated with the measurements of people in a sample. Higher values indicate greater reliability (>0.70 indicates adequate reliability).34

Traditional psychometric analysis

Traditional psychometric methods used to examine scale reliability and validity included the following: data quality (percent missing data for each item), scaling assumptions (similarity of item means and variances, and magnitude and similarity of corrected item‐total correlations),35, 36 scale‐to‐sample targeting (score means, SD, floor and ceiling effects), internal consistency reliability (Cronbach's α33 and stability (test–retest reliability).37 The test–retest analysis was computed using data provided by a subset of participants who completed a second copy of the questionnaire booklet at least 1 week, and no more than 2 months after the initial questionnaire. A minimum standard for Cronbach's α coefficients and test–retest reliability is 0.70.38

Results

Phase 1: Item generation, preliminary scale formation and pre‐testing

Final versions of the three scales appear in Table 1 and include a 6‐item Cancer Worry scale, a 15‐item Self‐Management Skills scale and a 12‐item Expectations scale. Flesh‐Kincaid Grade Levels were 5.4 for Cancer Worry (four of six items below a grade 6 level; range 2.3–7.3), 4.9 for Self‐Management Skills (12 of 15 items below a grade 6; range 2.3–9.6) and 5.1 for Expectations (seven of 12 items below grade 6 level; range 2.8–8.5). Each scale represents a stand‐alone instrument that receives a score from 0 to 100, with higher scores indicating more cancer worry, more self‐management skills, and more acceptable expectations for adult care.

Table 1.

Transition scales

Cancer Worry scale: These statements are about thoughts and feelings you may have as a cancer survivor. For each question, please circle only 1 answer.
	Strongly disagree	Disagree	Agree	Strongly agree
1. I worry it might be difficult to have children in the future.	0	1	2	3
2. I worry about late effects that might happen to me. (Note: late effects are health problems caused by cancer treatments, e.g. heart problems, hearing loss, learning problems).	0	1	2	3
3. Cancer is always at the back of my mind.	0	1	2	3
4. I worry about getting a new type of cancer.	0	1	2	3
5. I worry my cancer will come back (i.e. relapse).	0	1	2	3
6. I worry about my cancer every day.	0	1	2	3

Self‐Management Skills scale: These questions are about being in charge of your health. For each question, please circle only 1 answer.
	Strongly disagree	Disagree	Agree	Strongly agree
1. I answer a doctor's or nurse's questions.	0	1	2	3
2. I participate in making decisions about my health.	0	1	2	3
3. I make sure I go to all my doctor's appointments.	0	1	2	3
4. I ask the doctor or nurse questions.	0	1	2	3
5. I talk to a doctor or nurse when I have health concerns.	0	1	2	3
6. I talk about my medical condition to people when I need to.	0	1	2	3
7. I am in charge of taking any medicine that I need.	0	1	2	3
8. I know how to contact a doctor if I need to.	0	1	2	3
9. I prefer it when a doctor speaks to me instead of my parent(s).	0	1	2	3
10. I can briefly describe my medical history when asked.	0	1	2	3
11. I prefer to see a doctor or nurse without my parent(s) with me.	0	1	2	3
12. I know how to access medical care when I travel.	0	1	2	3
13. I book my own doctor's appointments.	0	1	2	3
14. I know the type of medical insurance I have. (Note: medical insurance pays for things not paid for by the healthcare system).	0	1	2	3
15. I fill my own prescriptions when I need medicine.	0	1	2	3

Expectations scale: Childhood cancer patients who receive treatment and follow‐up in a children's hospital eventually need to transfer to a new hospital or to a family doctor for long‐term follow‐up care as an adult. Imagine that you are about to go for your first adult follow‐up appointment. What do you expect this appointment will be like? For each question, please circle only 1 answer.
When I transfer to adult care …	Strongly agree	Agree	Disagree	Strongly disagree
1. … I expect the doctor to know my cancer history.	0	1	2	3
2. … I expect my appointment will start on time.	0	1	2	3
3. … I expect to be called if I miss my appointment.	0	1	2	3
4. … I expect to be seen by the same doctor each time I visit.	0	1	2	3
5. … I expect to get a reminder call before my appointment.	0	1	2	3
6. … I expect to be able to call the doctor any time I need to (e.g. if I have questions about late effects).	0	1	2	3
7. … I expect other appointments related to my cancer will be booked for the same day.	0	1	2	3
8. … I expect my parent(s) will be able to see the doctor with me.	0	1	2	3
9. … I expect the doctor will look after all my health‐care needs.	0	1	2	3
10. … I expect the doctor will become like a friend.	0	1	2	3
11. … I expect the doctor will spend a lot of time with me.	0	1	2	3
12. … I expect to like going to cancer follow‐up appointments.	0	1	2	3

Phase 2: Field‐testing, scale construction and psychometric evaluation

A total of 331 survivors were invited to participate, and 250 questionnaires were completed (response rate 75.5%). The response rate for face‐to‐face recruitment (114/118; 96.6%) was significantly higher (P < 0.01 on chi‐square test) than that of mailed surveys (136/213; 63.8%). Non‐respondents were younger (mean 17.2 vs. 18 years; P < 0.01 on t‐test) but did not differ from respondents in terms of gender, age at diagnosis and type of cancer. Characteristics of the 250 participants are shown in Table 2. Fifty participants completed the test–retest booklet between 1 week and 2 months after the initial questionnaire.

Table 2.

Field‐test sample characteristics

Characteristic	n (%)
Gender
Male	135 (54.0)
Female	115 (46.0)
Current age (years)
15–17	134 (53.6)
18–20	62 (24.8)
21–23	42 (16.8)
24–26	12 (4.8)
Parent's marital status
Married/Common‐law	176 (70.4)
Separated/divorced	49 (19.6)
Widowed/single/never married	20 (8.0)
Missing	5 (2.0)
Ethnicity
Caucasian	181 (72.4)
Other	65 (26.0)
Missing	4 (1.6)
Cancer type
Leukemia	100
ALL	86 (86.0)
AML	14 (14.0)
Lymphoma	55
Hodgkins	25 (45.4)
Non‐Hodgkins	30 (54.6)
CNS tumours	15
Astrocytoma	4 (26.7)
Glioma	3 (20.0)
Clival chordoma	1 (6.6)
Medulloblastoma	3 (20.0)
Other	4 (26.7)
Embryonal tumours	20
Neuroblastoma	10 (50.0)
Hepatoblastoma	6 (30.0)
Germ cell tumour	4 (20.0)
Renal tumours	26
Wilms' tumour	26 (100)
Sarcoma	34
Clear cell sarcoma	1 (2.9)
Rhabdomyosarcoma	12 (35.3)
Ewing's sarcoma	10 (29.5)
Oesteogenic sarcoma	11 (32.3)
Age at diagnosis (years)
0–4	110 (44.0)
5–12	93 (37.2)
13–17	47 (18.8)
Relapse status
Did not relapse	232 (92.8)
Relapsed at least once	18 (7.2)
Treatments
Chemotherapy	241 (96.4)
Radiation therapy	116 (46.4)
Surgery	92 (36.8)
Transplant	20 (8.0)
Years since diagnosis
0–8	90 (36.0)
9–16	126 (50.4)
17–26	32 (12.8)
Missing	2 (0.8)
Transition status
Pre	168 (67.2)
Post	82 (32.8)
Hospital
A	134 (53.6)
B	87 (34.8)
C	29 (11.6)

Cancer Worry scale

All 250 participants were included in the analyses for the Cancer Worry scale. Item thresholds were ordered for all items (Fig. 1a). Table 3 demonstrates that all items had acceptable fit residuals within the recommended range of −2.5 to + 2.5, and that item level chi‐square statistics were all non‐significant. The targeting between the distribution of person measurements (top histogram) and the distribution of item locations (bottom histogram) is shown in Fig. 2a. This figure indicates that the scale defined a continuum for cancer worry and that the scale was targeted adequately to the sample. There was no DIF for gender or age at diagnosis. Scale reliability was supported by a PSI of 0.82.

Figure 1.

Ordering of item response thresholds (location order). (a) Cancer Worry scale. (b) Self‐Management Skills scale. (c) Expectations scale.The x‐axis represents the construct with more cancer worry, self‐management skills and ‘realistic’ expectations for adult healthcare increasing to the right. The y‐axis shows each of the items response categories as follows: ‘Strongly Agree’ labeled 0; ‘Agree’ labeled 1; ‘Disagree’ labeled 2; ‘Strongly Disagree’ labeled 3.

Table 3.

Statistical indicators of fit (fit residual; chi‐square) for Cancer Worry scale items

Scale	Item	Location	Standard error	Fit residual	Chi‐square	Probability
Cancer Worry	Every day	−1.26	0.12	0.11	0.41	0.94
	Relapse	−0.09	0.10	−0.97	3.41	0.33
	New cancer	−0.05	0.10	0.13	0.33	0.96
	Back mind	0.18	0.10	−0.32	2.34	0.51
	Have children	0.49	0.09	2.03	3.12	0.37
	Late effects	0.74	0.10	0.95	0.43	0.94
Self‐Management Skills	Answer questions	−1.89	0.14	−0.72	2.96	0.40
	Decision making	−1.69	0.14	−1.22	8.22	0.04
	All appoints	−1.59	0.16	1.93	1.35	0.72
	Ask questions	−0.47	0.12	−0.82	9.20	0.03
	Health concerns	−0.43	0.12	−0.54	5.77	0.12
	Talk about	−0.32	0.11	−0.14	2.72	0.44
	Medicine	−0.26	0.11	−1.75	6.44	0.09
	Contact doctor	−0.20	0.17	−0.72	2.68	0.44
	Speak to me	−0.11	0.11	0.25	5.16	0.16
	Briefly describe	0.01	0.13	0.40	0.86	0.84
	Without parent	0.99	0.10	1.89	9.87	0.02
	Access care	1.12	0.10	1.34	3.33	0.34
	Book appoints	1.26	0.12	−0.09	1.13	0.77
	Insurance	1.48	0.10	1.55	3.12	0.37
	Prescriptions	2.10	0.11	0.41	3.70	0.30
Expectations	Like going	−0.59	0.13	0.66	2.01	0.57
	Spend time	−0.58	0.14	−0.51	3.52	0.32
	Friend	−0.54	0.13	−0.83	6.41	0.09
	All my needs	−0.45	0.12	0.03	1.43	0.70
	Parents	−0.30	0.12	2.68	7.26	0.06
	Other appoints	0.08	0.13	0.97	2.93	0.40
	Call anytime	0.10	0.14	−0.51	7.49	0.06
	Reminder call	0.16	0.12	0.17	1.39	0.71
	Same doctor	0.26	0.13	0.14	0.89	0.83
	Miss appoint	0.35	0.14	−0.35	2.06	0.56
	On time	0.40	0.13	1.36	4.00	0.26
	Know history	1.09	0.15	−0.41	2.16	0.54

Figure 2.

Targeting of scale to sample (person‐item threshold locations spread). (a) Cancer Worry scale. (b) Self‐Management Skills scale. (c) Expectations scale.The x‐axis symbolizes the construct with cancer worry, self‐management skills and ‘realistic’ expectations increasing to the right. The y‐axis shows the frequency of person measure locations (top histogram) and item locations (bottom histogram).

The traditional analyses supported the Cancer Worry scale as a valid and reliable measure (Table 4). Data quality was high (missing data up to 2%; scale scores computable for 96% of respondents) and scaling assumptions were satisfied (similar mean item scores, corrected item‐total correlations range = 0.59–0.70). Scale‐to‐sample targeting was good (scale scores spanned the scale range, were not notably skewed and floor/ceiling effects were negligible) and reliability was high (Cronbach's α = 0.85; TRT = 0.85).

Table 4.

Data quality, scaling assumptions and targeting for each scale

Scales	Items	Data quality	Scaling assumptions					Targeting
		Item missing data (%)	Possible range	Actual score range	Mean score	SD	CITC	Floor/ceiling effects (%)	Skew ness
Cancer Worry	Every day	1	0–3	0–3	2.19	0.72	0.61	2/36	−0.64
	Relapse	2	0–3	0–3	1.79	0.89	0.70	7/24	−0.20
	New cancer	1	0–3	0–3	1.81	0.92	0.64	9/26	−0.27
	Back mind	1	0–3	0–3	1.69	0.93	0.67	10/23	−0.09
	Have children	1	0–3	0–3	1.58	1.02	0.59	18/22	−0.11
	Late effects	2	0–3	0–3	1.47	0.90	0.64	13/15	0.14
	Total	4	0–18	1–18	10.51	4.13	–	0/7	0.02
Self‐Management Skills	Answer questions	4	0–3	0–3	2.46	0.57	0.46	1/49	−0.65
	Decision making	1	0–3	0–3	2.37	0.60	0.50	1/43	−0.51
	All appoints	1	0–3	0–3	2.67	0.51	0.21	1/69	−1.47
	Ask questions	4	0–3	0–3	2.02	0.71	0.57	2/24	−0.33
	Health concerns	4	0–3	0–3	2.14	0.76	0.50	3/34	−0.64
	Talk about	3	0–3	0–3	2.19	0.76	0.43	3/37	−0.80
	Medicine	1	0–3	0–3	2.11	0.83	0.55	4/36	−0.67
	Contact doctor	1	0–3	0–3	2.06	0.80	0.58	4/31	−0.57
	Speak to me	3	0–3	0–3	2.03	0.78	0.42	4/28	−0.55
	Briefly describe	1	0–3	0–3	2.24	0.72	0.35	3/37	−0.94
	Without parent	2	0–3	0–3	1.33	0.83	0.29	14/9	0.26
	Access care	2	0–3	0–3	1.30	0.89	0.35	19/10	0.21
	Book appoints	2	0–3	0–3	0.87	0.78	0.47	36/6	0.88
	Insurance	4	0–3	0–3	1.01	0.90	0.33	33/7	0.60
	Prescriptions	4	0–3	0–3	0.78	0.77	0.36	40/2	0.70
	Total	12.4	0–45	3–42	17.6	5.9	–	0/0	−0.02
Expectations	Like going	1	0–3	0–3	1.19	0.80	0.49	20/4	0.10
	Spend time	0	0–3	0–3	1.26	0.74	0.60	15/3	−0.07
	Friend	1	0–3	0–3	1.19	0.76	.063	18/3	0.11
	All my needs	1	0–3	0–3	1.14	0.86	0.54	26/5	0.16
	Parents	0	0–3	0–3	1.05	0.80	0.39	26/3	0.29
	Other appoints	0	0–3	0–3	0.85	0.75	0.42	35/2	0.53
	Call anytime	0	0–3	0–3	0.90	0.72	0.58	29/2	0.46
	Reminder call	0	0–3	0–3	0.71	0.76	0.44	46/3	0.91
	Same doctor	0	0–3	0–3	0.87	0.75	0.51	34/1	0.41
	Miss appoint	0	0–3	0–3	0.74	0.70	0.53	39/1	0.63
	On time	0	0–3	0–3	0.72	0.76	0.41	45/1	0.69
	Know history	0	0–3	0–3	0.47	0.63	0.46	60/1	1.16
	Total	2	0–36	0–29	11.01	5.40	–	4/0	0.07

CITC, corrected item‐total correlation.

Self‐Management Skills scale

Teenagers (aged up to 19 years; n = 185) were included in the analyses of the Self‐Management Skills scale. The item response option thresholds (Fig. 1b) were ordered for 12 of 15 items. The three items with disordered thresholds were re‐scored to three response options (i.e. ‘disagree’ and ‘strongly disagree’ were merged into one category), which resulted in ordered thresholds for all items. Subsequent RMT analyses used the rescored data. Items had fit residuals within the recommended range of −2.5 to + 2.5 and item level chi‐square statistics were non‐significant (Table 3). The targeting (Fig. 2b) of person measurements and the distribution of item locations defined a continuum providing support that the scale is targeted to the sample. DIF was not detected for gender or age at diagnosis. Scale reliability was supported by a high PSI of 0.81.

The traditional analyses supported the Self‐Management Skills scale as a valid and reliable measure (Table 4). Data quality was high (missing data up to 4% and scale scores computable for 88% of respondents). Scaling assumptions were satisfactory, with mean item scores that ranged from 0.78 to 2.67 and corrected item‐total correlations that ranged from 0.21 to 0.58. Scale‐to‐sample targeting was good (scale scores spanned the scale, scores were not notably skewed and there were no floor/ceiling effects). Reliability was high (Cronbach's α = 0.81; TRT = 0.90).

Expectations scale

Only participants who had not yet transitioned to adult LTFU care and completed the Expectation scale (n = 156) were included in the analyses. Item thresholds were ordered for all items (Fig. 1c). One item (i.e. ‘I expect my parents will be able to see the doctor with me’) had a fit residual marginally outside the recommended range of −2.5 to + 2.5 (Table 3). Chi‐square statistics were non‐significant. Figure 2c shows good targeting between the distribution of person measurements (top histogram) and the distribution of item locations (bottom histogram). DIF was not detected for gender or age at diagnosis. Scale reliability was supported by a high PSI of 0.84.

The results of traditional analysis supported the Expectations scale as a valid and reliable measure (Table 4). Data quality was high (missing data up to 1%, scale scores were computable for 98% of respondents) and scaling assumptions show that mean item scores varied from 0.47 to 1.26 and corrected item‐total correlations ranged from 0.39 to 0.63. In terms of scale‐to‐sample targeting, the scores spanned the lower part of the scale range (0–29), were not skewed and had minimal floor/ceiling effects. Reliability was high (Cronbach's α = 0.84; TRT = 0.86).

Discussion

Since many survivors fail to transfer, or transfer but subsequently drop out of LTFU care, it is important to identify any barriers that survivors face in order to resolve these prior to transition. The Social Ecological Model of AYA Readiness for Transition (SMART) was developed from literature, expert opinion and pilot data collected from a sample of childhood cancer survivors to account for many of the factors important in the process of transition for patients with chronic illnesses.39 This model can be used to guide the study of factors that can act as barriers and/or facilitators to transition from pediatric to adult care, including pre‐existing objective factors (i.e. socio‐demographics/culture, access/insurance, medical status and risk, neurocognitive/IQ) as well as modifiable subjective variables (i.e. knowledge, skills/efficacy, beliefs/expectations, goals, relationships, psychosocial functioning). Our study describes three scales for childhood cancer survivors that measure concepts identified as barriers and/or facilitators to transitioning successfully to adult‐orientated health care within this model. These three scales – Cancer Worry (i.e. about cancer‐related issues such relapsing or getting a new type of cancer), Self‐Management Skills (i.e. skills that adolescents need to acquire to be able to care for their health as adults, such as booking doctor's appointments and filling prescriptions), and Expectations (i.e. about the nature of adult LTFU care, such as expecting to get a reminder call before an appointment) – were demonstrated to be short, easy to understand, valid, and reliable measurement tools that could now be tested in transition programs.

Our scales differ fundamentally from other scales measuring similar constructs14, 15, 40 as we utilized a modern psychometric approach, which involved rigorous qualitative research18, 19 followed by quantitative methods that focus on the relationship between a person's measurement and their probability of responding to an item rather than the relationship between a person's measurement and their observed scale total score. This approach leads to the legitimate summing of items to produce total scores that provide interval‐level data, improving the accuracy with which clinical change can be measured.24 Scales developed using RMT methods are sufficiently valid and reliable to allow their use in clinical practice for patient monitoring and management.27

A particular advantage of RMT analysis is that a range of statistics and graphics can be used to identify a rating scale's strengths and limitations (areas for future improvement). For example, the person‐item threshold distribution (Fig. 2a–c) provides a visual representation of how the items of a scale map out a continuum for a construct (i.e. item hierarchy), including how adequate a scale is for measuring a construct within a sample. Figure 2b shows that, while the items map out a continuum for self‐management skills, the sample scored on the higher end of the scale (evidenced by the skewed distribution to the right). Since one of the goals of transition programmes is to begin the transition process many years before the planned transfer so that adolescents can develop appropriate self‐management skills,8 future research should examine scale‐to‐sample targeting in a younger cohort of survivors who may score lower on the scale.

Most adult survivors of childhood cancer eventually transfer from their paediatric care centre to a new health‐care provider who is initially unfamiliar with their cancer history. Our team reported previously that psychological factors affecting survivors have an impact on whether or not they attend a specialized LTFU clinic as adults.18, 19 Some survivors reported that cancer worry motivated them to attend adult LTFU care while others reported that cancer worry made them reluctant to seek care. In some instances, worry acted as both a barrier and facilitator in the same person. Other researchers have developed scales to measure cancer worry in other cancer patients including those with breast41 and prostate tumours42 and adult survivors of childhood cancer.43, 44 Given that cancer worry in childhood cancer survivors can work as both a barrier and facilitator of transition, our scale may be useful in research that seeks to better understand factors that can account for the bi‐directional relationship between cancer worry and transition. Our scale may also be useful in clinical practice to address cancer worry clinically, regardless of whether it promotes or impedes successful transition.

Our team reported previously that some survivors describe their experience of transitioning to a new facility as a deterrent to continued attendance.18 Differences between paediatric and adult centres in how care is organized and delivered were seen as barriers to LTFU attendance by some survivors and point to the need to prepare adolescents by developing the skills needed to navigate care in the unfamiliar setting of an adult hospital or cancer centre. Exploring expectations about adult health care might help to identify those adolescents who need education about what is most likely to happen in an adult‐oriented health‐care setting (e.g. it is unlikely that someone will call if they miss an appointment).

Our study has several limitations. The response rate to our mailed survey was lower than face‐to‐face recruitment, introducing the possibility of response bias. In addition, our sample was recruited from three of sixteen Canadian paediatric oncology centres. The inclusion of more centres might have increased the heterogeneity of our sample in terms of how they were prepared for transition and different models of adult LTFU care. At the same time, we recognize that our sample is restricted to Canadian childhood cancer survivors. Research using our scales outside of Canada is also warranted. While research has described a range of barriers faced by childhood cancer survivors in the process of transition to adult health care10, 11, 18, 19 our team decided to focus on the three themes that were demonstrated to be important in our qualitative study.18, 19 We recognize that there are other constructs that may be equally as important to transition success and that there may be scope for future research to develop additional scales. We recommend further psychometric work be carried out with our scales to add to the evidence base for the use of the scales and the generalizability of their measurement properties as our study provides the first available evidence for reliability and validity of these scales using modern psychometric methods. Specifically, research using a traditional approach in which additional psychometric properties are examined beyond those reported here is called for (e.g. different forms of validity as well as responsiveness). Finally, the clinical meaning of the scales' scores for subgroups of survivors will be clarified as the scales are taken up and used.

Our scales are now available for researchers to use to investigate barriers and/or facilitators to transition in childhood cancer survivors. Using such tools, research could be conducted to determine the relationship between the scale scores and attendance at adult LTFU appointments and improved health outcomes within the context of the SMART framework of transition readiness.

Conflicts of interest

The authors do not have a conflict of interest to report.