Returning research results: caregivers’ reactions following computerized cognitive training among childhood cancer survivors

Abstract

Background

Few researchers routinely disseminate results to participants; however, there is increasing acknowledgment that benefits of returning results outweigh potential risks. Our objective was to determine whether use of specific guidelines developed by the Children’s Oncology Group (COG) when preparing a lay summary would aid in understanding results. Specifically, to determine if caregivers of childhood cancer survivors found a lay summary comprehensive, easy to understand, and helpful following participation in a computerized cognitive training program.

Methods

In a previous study, 68 childhood survivors of acute lymphoblastic leukemia or brain tumor with identified cognitive deficits were randomly assigned to participate in a computerized cognitive intervention or assigned to a wait list. Following conclusion of this study, participants’ caregivers were contacted and provided with a summary of results based on COG guidelines and survey. Forty-three participants returned the surveys, examining caregivers’ interpretation of the summary, reaction to the results, and information regarding preference for receiving results.

Results

Caregivers reported results as important (93%), helpful (93%), easy to understand (98%), and relevant to their child (91%). They interpreted the results as generally positive, with many caregivers endorsing satisfaction (84%); however, concern of long-term implications was expressed (25%). Most preferred receiving results through postal letter (88%) or email (47%).

Conclusions

Benefits of returning research results to families appear to outweigh potential negative consequences. Returning results may help inform families when making future health care-related decisions. There is a great need to develop and assess the utility of guidelines for returning research results.

There is an increasing acknowledgment that researchers have an obligation to share research study results with participants, with some endorsing the return of results as a component of responsible ethical research practice.¹ Yet, few medical institutions have implemented policies providing recommendations for dissemination of research results to participants.² On average, fewer than 40% of researchers regularly offer research results to participants.^3–5 However, institutional review boards (IRBs) are becoming increasingly aware of the obligation to return results, as more than one-third of IRBs and Research Ethics Boards (REBs) require researchers to address this matter.^3,6,7 In a research study examining United States IRB policy regarding the return of results to participants, it was determined that 56% of research settings do not have a specific policy addressing this issue.³ Further, MacNeil and Fernandez⁷ surveyed Canadian university REB coordinators and found of 22 coordinators contacted, none reported guidelines on specific methods of returning results.⁷ As research in this area continues to grow and researchers routinely disseminate results, there is an increased emphasis on following guidelines in order to provide results in a respectful, time-sensitive manner.^8,9

Recent literature concerning communication of results to participants indicates individuals want to receive any study results available;¹⁰ however, it is important to consider benefits and risks when disseminating this information. Potential benefits include providing information that may be useful for medical decision making and helping the participant feel valued by recognizing their contribution to science.^1,7 More broadly, benefits include promotion of greater public understanding of societal benefits of research and dissemination of more accurate results than may be portrayed in the lay media.^11,12 Some researchers have voiced concern about a uniform requirement to provide results to participants, including creating potential distress by revisiting a diagnosis or death.^11,13 Other concerns include misunderstanding of adverse information, such as a parent learning something they had control over may have increased the likelihood of a specific outcome, thus increasing their guilt. Additionally, risks include anger at results describing unfavorable outcomes or concern when the participant does not show the cited improvements described in the findings.¹¹ Financial, logistic, and time burdens associated with returning research results are also concerning factors to researchers.¹⁴ Further, it is important to consider the appropriateness of providing information that will be inconclusive or difficult to interpret, which highlights the importance of developing specific guidelines for the delivery of research results.

In 2008 a Return of Results Task Force (RRTF) was established by Children’s Oncology Group (COG) in order to develop guidelines for providing research results to participants and their families.¹¹ The RRTF detailed recommendations for providing results in a consistent and accessible manner to participants, outlining 18 specific recommendations, including which results should be offered, when these results should be offered, and in what form the summary should be presented to participants.¹¹ Fernandez and colleagues¹¹ conducted an international, multicenter study examining participant needs and attitudes for the returning of results and found parents generally accept impersonal means of communication (Internet website, postal letter, or email) in order to receive results with positive implications. However, this preference changed if the results were considered negative; specifically, 70% of parents would prefer results be communicated either by phone call or personal visit.¹¹

Previously, St. Jude Children’s Research Hospital (SJCRH) conducted a study investigating an intervention for cognitive late effects among childhood cancer survivors (COGTRN; NCT01217996). The results of this study indicate that computerized cognitive training is feasible for childhood cancer survivors experiencing cognitive late effects and that training appeared to improve working memory, attention, and processing speed, as measured by cognitive assessments.¹⁵ Additionally, caregivers reported improvements in attention and executive functioning. Neuroimaging results displayed a reduction in prefrontal and parietal activation from pre- to post-intervention, suggestive of training-induced neuroplasticity.¹⁵

The current study explores the practice of returning research results to families of childhood cancer survivors that previously participated in the COGTRN study. Objectives were to investigate the clarity and understanding of the results disseminated to families, specifically using COG guidelines as the basis of formulating the summary of research results. Secondarily, we assessed families’ emotional reactions to the summary of results. Finally, we aimed to identify the application of these results to the children who participated and preferred method of returning research results to families given the specific results of the current study. Although previous research has examined returning research results for medical oncology studies, there is value in examining caregiver reactions to a psychosocial oncology study as these findings are more nuanced. The current study allows researchers to focus on the factors that may impact caregiver reactions to study results related to cognitive and quality-of-life outcomes, thus improving the process of returning these results and reducing adverse reactions.

Methods

Background: Computerized Intervention for Cognitive Late Effects Study (COGTRN)

Between December 2010 and December 2013, SJCRH conducted a study investigating an intervention for cognitive late effects among childhood cancer survivors (COGTRN; NCT01217996). Eligible participants included childhood survivors of acute lymphoblastic leukemia (ALL) or brain tumor who received cranial radiation therapy and/or intrathecal chemotherapy and were off treatment for at least 1 year without disease recurrence. Participants had to be English speakers, between 8 and 16 years old, and with IQ ≥ 70. Children were randomly assigned to a computerized cognitive intervention (n = 34) or wait-list (n = 34). Those participants assigned to the intervention group were asked to complete 25 training sessions of Cogmed, a computerized intervention designed to improve working memory, at home with weekly phone-based coaching. Cognitive assessments and fMRI scans were completed pre- and post-intervention.

Participants and Procedure

After obtaining approval from the SJCRH IRB, researchers contacted all 68 participants from the COGTRN study by letter, phone call, and/or patient approach and provided a summary of research results. All 68 families who had completed the COGTRN study were sent a postal letter summarizing the results of the study and a survey to return in a return-addressed, stamped envelope on March 2, 2016. Phone calls were initiated in May 2016 to families who had not yet returned the survey. A second summary of results and survey were then sent to families who requested it, along with a return-addressed, stamped envelope. In July a second round of phone calls were initiated to families that had not returned the surveys, which resulted in the mailing of additional copies of the summary and survey, at the families’ request. Additionally, 15 families were approached while attending appointments at SJCRH. In total, 43 of the 68 (63.2%) families returned the surveys; the process of receiving the results are available in Fig. 1.

Fig. 1.

Return of Results Process.

The lay summary was developed based on the COG recommendations for returning a summary of research results.¹¹ The two-page summary consisted of 5 paragraphs, the first comprised a section thanking the families for their participation, disclosing publications and presentations that resulted from the COGTRN study, and describing the benefits and risks of receiving the research results. Second, the COGTRN study was summarized, including the hypotheses and methods. Following the COGTRN study review, results were presented in a bulleted, easy to read format, then summarized in a brief paragraph, along with future directions. A section offering future neuropsychological assessment opportunities and contact information for researchers was provided. At the end of the summary, a reference to the survey was enclosed, along with publication citations.

Families were also administered a 31-question survey, which consisted of 17 Likert-type questions, 3 True/False questions, 4 open-ended questions, and 7 multiple choice questions. Survey items were developed on the basis of literature relating to returning results and a review of the survey used by Fernandez and colleagues;⁶ potential survey items were then refined by the research team. Based on previous research,^2,6,16 items were developed to include closed- and open-ended questions to elicit responses about attitudes and reactions to the summary of results. There were 4 specific areas included in the survey. First, the meaning and clarity of the summary was assessed through questions inquiring about the importance of receiving the results, ease of understanding the information, and comprehension questions to verify interpretation of provided results. Second, a section gauging the families’ reaction to the findings was comprised of questions that assess the extent to which the summary provided was surprising or applicable to their child. Additionally, families were asked to use a 5-point Likert-type scale to assess their psychological reaction after learning the results of the study (sad, anxious, satisfied, guilty, angry, or relieved). Third, application of the information was determined by caregivers’ indication concerning whether they have others they can talk to about the results, their interest in further cognitive assessment or therapy, and their general reaction to cognitive late effects and similar studies. Lastly, multiple choice questions were administered in order to identify caregiver demographic information. See Supplementary Figure S1 for full survey information.

Statistical Analyses

Descriptive statistics for demographic and clinical variables were calculated to characterize groups of survey respondents and non-respondents. Percentages were calculated to determine overall level of agreement regarding clarity and understanding of the summary, caregivers’ reaction to the summary, and application of the information. Independent samples t tests were used to evaluate differences in demographic information, importance of returning results, and emotional reactions between COGTRN control and intervention groups.

Results

Table 1 presents clinical and demographic information including a breakdown by survey respondents and non-respondents. Among the 68 families that were contacted, 11.7% were unreachable via phone or mail. Participants that returned the survey were diagnosed with acute lymphoblastic leukemia (58.1%) or brain tumor (41.9%). The respondent group was made up of primarily Caucasian (86.0%) individuals, balanced by gender (males = 53.5%), with a socioeconomic status of 41.77, according to the Barratt Simplified Measure of Social Status. Of the 43 caregivers that responded, 22 were assigned to the intervention group; however, 33 caregivers responded that they believed their child was part of the intervention group.

Table 1.

Participant characteristics

			Respondents	Non-respondents	P value
Demographic	Sex	Female	20 (46.5%)	12 (48%)	.91
		Male	23 (53.5%)	13 (52%)
	Race	African American	2 (4.7%)	4 (16%)	.08
		Asian/Pacific Islander	2 (4.7%)	0
		Caucasian	37 (86%)	12 (64%)
		Hispanic	1 (2.3%)	2 (8%)
		Other/multiple races	1 (2.3%)	3 (12%)
	SES (BSMSS)*		41.77	37.14	.18
Clinical	Assigned group	Control	21 (48.8%)	13 (52%)	.81
		Intervention	22 (51.2%)	12 (48%)
	Acute lymphoblastic leukemia		25 (58.1%)	22 (88%)	< .01
	Brain tumor		18 (41.9%)	3 (12%)
	Mean age at diagnosis, years		5.06 ± 2.88	4.58 ± 2.66	.50
	Mean age at enrollment, years		12.16 ± 2.29	11.76 ± 2.69	.53
	Mean time since treatment, years		4.28 ± 0.56	4.01 ± 0.49	.06
	Brain tumor group	Ependymoma	4 (9.3%)	0	.01
		Glioma	2 (4.7%)	0
		Medulloblastoma/PNET	12 (27.9%)	3 (12%)
	Mean Baseline IQ		106.21	98.52	.04

Abbreviations: PNET, primitive neuroectodermal tumor; SES, socioeconomic status.

*Barratt Simplified Measure of Social Status. Scores derived from maternal and paternal education and occupation; scores range from 8 to 66 with high being indicative of higher SES.

Additionally, among the children that participated in the current return of results study, the respondent group had a statistically significantly higher baseline IQ ( = 106.21) than the non-respondent group ( = 98.52; P < .05); however, the difference did not exceed one standard deviation. Surveys were mostly returned by the patient’s mother (81.4%) and there were significantly more caregivers of patients with brain tumors than leukemia that responded (P < .01). Further, children that displayed significant improvement in either working memory (WISC-IV Spatial Span Backward)¹⁷ or executive functioning (Conners’ Parent Rating Scale-3)¹⁸ did not have higher response rates.

When caregivers were asked their preferred means for receiving research results following the conclusion of studies, the majority endorsed preferring a letter in the mail (n = 38). However, other preferences included email (n = 20), direct contact (n = 12), phone call (n = 5), and through magazine or journal (n = 4).

Clarity and Understanding of Summary of Results

Table 2 presents questionnaire information regarding the caregivers’ understanding of the results presented in the summary. The patient’s caregiver perceived the results provided as generally important (93.0%), helpful (93.0%), and appropriately detailed (88.4%). Additionally, caregivers demonstrated understanding of the summary, as 95.3% answered all 3 True-False comprehension questions correctly. Participants’ caregivers found the information relevant to their child (90.7%). In the questionnaire, caregivers were also asked an open-ended question regarding “…what information in the summary of results [they found] most helpful.” Twenty-nine caregivers responded with similar answers indicating: easy-to-read bullet points, satisfaction with cognitive gains comparable to that of stimulant medication, changes in neuroimaging viewed as “solid proof” of change, and applicability of cognitive training on “all areas of life,” including memory, attention, and processing speed. Specifically, one caregiver wrote, “The summary indicated that there were positive benefits to the program, which makes me feel it was worthwhile,” demonstrating a sense of satisfaction and value following the receipt of the research results. Furthermore, of the 11 responses to the open-ended question, “Are there new concerns or questions you have after reading the summary of results,” caregivers conveyed questions or concern regarding lack of improvement in math skills and whether results would be long-term.

Table 2.

Return of results questionnaire

	Disagree* (1 and 2) n (%)	Neutral (3) n (%)	Agree (4 and 5) n (%)	Mean ± SD
1. Important for families to receive results	1 (2.3%)	2 (4.7%)	40 (93%)	4.51 ± 0.79
2. Summary was helpful	0 (0%)	3 (7%)	40 (93%)	4.51 ± 0.63
3. Summary was easy to understand	0 (0%)	1 (2.3%)	92 (97.7%)	4.63 ± 0.54
4. Summary provided enough detail	1 (2.3%)	4 (9.3%)	38 (88.4%)	4.28 ± 0.73
10. Results were surprising	16 (37.2%)	16 (37.2%)	11 (25.6%)	2.84 ± 0.95
11. Results were relevant	2 (4.7%)	2 (4.7%)	39 (90.7%)	4.14 ± 0.71
19. There are others I can talk to	5 (11.6%)	5 (11.6%)	33 (76.7%)	3.84 ± 1.15
20. Interested in cognitive therapy for child	9 (21%)	4 (9.3%)	30 (69.8%)	3.88 ± 1.28
21. Interested in cognitive reassessment	14 (32.5%)	8 (18.6%)	21 (48.9%)	3.3 ± 1.37
22. Interested in learning more about cognitive late effects	5 (11.7%)	2 (4.7%)	36 (83.7%)	4.16 ± 1.11
23. Recommend others participate	0 (0%)	1 (2.3%)	42 (97.7%)	4.63 ± 0.54

*Full scale: 1 = Strongly disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree; collapsed for analysis

Note that numbers are from survey questions.

Reaction to the Summary of Results

Table 3 presents the caregivers’ reactions to the results provided in the summary. Overall, most caregivers’ reactions to the summary were positive, with 83.3% endorsing satisfaction and 93.0% reporting relief or neutrality. Overall, few caregivers reacted with sadness (9.3%), anxiety (11.6%), or guilt (4.7%). Additionally, no caregivers reported reacting with anger. When asked the open-ended question to describe their reactions, most caregivers described satisfaction with the results or importance of cognitive research. However, 6 caregivers explained feeling guilty “because [they would] like to do more for [their] child” or “did not complete the last few sessions” of the cognitive training.

Table 3.

Caregivers’ reaction to results

	Disagree (1 and 2) n (%)	Neutral (3) n (%)	Agree (4 and 5) n (%)	Mean ± SD
12. Sad	27 (62.8%)	11 (25.6%)	4 (9.3%)	1.98 ± 1.07
13. Anxious	25 (58.2%)	12 (27.9%)	5 (11.6%)	2.10 ± 1.10
14. Satisfied	0 (0%)	5 (11.6%)	36 (83.8%)	4.22 ± 0.65
15. Guilty	28 (65.1%)	12 (27.9%)	2 (4.7%)	1.83 ± 1.01
16. Angry	33 (76.8%)	9 (20.9%)	0 (0%)	1.57 ± 0.83
17. Relieved	3 (7%)	22 (51.2%)	18 (41.9%)	3.44 ± 1.01

*Full scale: 1 = Strongly disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree; collapsed for analysis

Note that numbers are from survey questions.

Application of the Information

Caregivers’ interest and potential application of the results that were presented in the summary are presented in Table 2. Most caregivers indicated that they can talk to others about the results (76.7%) and would recommend others participate in similar cognitive studies (97.7%). Further, the majority of caregivers reported interest in future cognitive therapy (69.8%) and learning more about cognitive late effects (83.7%); whereas, 48.9% indicated interest in cognitive re-assessment. Caregivers were also given the opportunity to respond to the following question regarding the applicability of the results, “Do you think you will use the information provided in the summary of results?” Most caregivers (n = 28) responded to this question; 9 indicated use of this information as important for school and future purposes, 7 caregivers reported the continued use of “brain-training type activities,” and 9 plan to use the information in order to “reassure” and “motivate” their child and other families. Further, 1 caregiver reported feeling grateful at the opportunity to participate, stating, “I am satisfied with the results because that’s exactly how I see my son. If the program really worked on him, I am grateful.”

Discussion

The current study supports growing consensus that the benefits of returning research results to patients and their families outweigh any negative consequences. Overall, using the COG guidelines when formulating a summary of research results generated easy-to-understand and appropriately detailed findings that the family found helpful. More specifically, these research results are perceived as important and helpful by families, especially when making health care-related decisions. Caregivers’ reactions to the results were generally positive and included interest in future assessment and cognitive therapy. Further, a postal letter or email were the preferred means of receiving the results; however, it is important to point out that mode of receipt and overall acceptability of information may have been influenced by the generally positive implications of the current findings. Personal visits or phone calls are the preferred means of communicating distressful results with potential negative implications.¹ Additional steps to mitigate potential distress include giving the participant, as part of the consent process, the option of receiving results and asking them in which manner they would prefer receiving these results.

Although the overall results of the COGTRN study were generally positive, many caregivers did express concern regarding the lack of improvement in math skills and long-term implications. Additionally, not all of the participants had personally experienced the favorable cognitive outcomes. The lack of positive outcomes in some children lead to guilt among some caregivers that did not mandate their child complete all Cogmed sessions; however, sharing of results was still appreciated and this finding did not seem frequent or significant enough to discourage dissemination of research results. Further, there were no significant differences in perception of the results or emotional reactions between the control and intervention groups, likely due to the fact that 11 of the control group families participated in Cogmed training following conclusion of the study.

The importance of sharing research results with families has increased substantially, with some researchers endorsing returning results as an ethical responsibility.¹ The disbursement of research results provides patients and families with information that may have direct health care implications affecting patients’ quality of life and future health care-related decisions. Additionally, when results are returned to patients, researchers convey appreciation of the patients’ time and effort. On a broader level, the promotion of a greater public understanding of the societal benefits of research help protect against misleading scientific findings than may be portrayed in the lay media.

Additional considerations when returning research results include resource expenditures. The project timeline may need to be extended to contact participants, prepare the lay summary, and disseminate results. It is important to factor in the possibility of spending additional time making phone calls and locating patients with changed phone numbers, in addition to mailing out results when returned to sender. This burden can partially be mitigated by keeping in contact with the patient. In this instance, the COGTRN study had concluded before steps were taken to return results, consequently the resource burden was significant. It is estimated that approximately $200 was spent on office supplies to disseminate results and approximately 20 hours were spent contacting caregivers to provide survey reminders and receive updated mailing addresses. However, preparing a lay summary alongside the published manuscript could promote a more timely return to caregivers, as well as reduce personnel resources. An additional means to reduce costs includes distributing the lay summary electronically.

Limitations to the current study included an initial low return rate; several participants were completely unreachable likely due to the long lapse in time between conclusion of the study and dissemination of results. Results were initially distributed by postal letter; however, due to an initial low return rate of 17.6%, phone calls were initiated to the families. Additionally, 15 patients were approached while at SJCRH for their medical appointments, which led to an ultimate return rate of 63.2%. Relative to the response rate for strictly postal surveys (26% to 70.5%),^2,6,19 the current study displayed an appropriate rate, especially given the great lengths taken to contact families via postal letter, phone, and/or patient approach. Additionally, the low initial return rate could be attributed to a mean of 4 years between the conclusion of the study and mailing of research results. Returning results more promptly would provide families the ability to utilize the information sooner through re-assessments and educational and medical decisions. While there were differences between the respondent and non-respondent groups in cancer diagnosis and baseline IQ, neither of these variables are suggested to have influenced caregivers’ motivation to return the survey.

Future implications primarily include planning accordingly. Disseminating research results to families is a seamless process when timelines and funding are established ahead of time. COG recommends including the intent to distribute results within the informed consent document and reminding families upon completion of their participation in the study.¹¹ Currently, most medical centers do not have a policy that provides recommendations for disseminating results to families; however, IRBs are becoming increasingly aware of this use and many institutions are now requiring researchers to address this issue. Establishing guidelines within institutions regarding the process of returning results would benefit researchers and participants alike. Further, researchers should include in the informed consent process questions regarding whether families would still prefer to receive results if the patient passes away or relapses. Gauging the education level of the caregivers can allow researchers to tailor the guidelines to that sample. Researchers should continue making an effort to solicit reactions of study participants following conclusion of research studies in order to further inform policies and procedures in the future.

Future research directions include randomizing the mode through which caregivers receive results. This would allow researchers to investigate whether the mode of receipt plays a role in the caregivers’ perception of the summary and reaction to the results. Further, examining the difference between patients who have particularly negative outcomes (eg, relapse, pass away) and those who have generally positive outcomes may prove helpful in deciding mode of response. The current study did not have any patients pass away and had 1 patient relapse in the 4 years following conclusion of the study, precluding examination of reactions of caregivers of children with negative health outcomes. In conclusion, the current study highlights the importance of returning research results to families in a timely manner, as these results can provide helpful information for patients. Research should continue exploring the dissemination of results to participants to further influence institutional policies and procedures.

Supplementary Material

Supplementary material is available at Neuro-Oncology Practice online.

Funding

This work was supported, in part, by the National Cancer Institute at the National Institutes of Health (St. Jude Cancer Center Support [CORE] Grant [P30 CA21765]); the American Cancer Society (RSGPB CPPB-119423 to HC); and the American Lebanese Syrian Associated Charities (ALSAC).

Conflict of Interest Statement: No potential conflicts of interest.