Participants’ perspectives and preferences on clinical trial result dissemination: The TRUST Thyroid Trial experience

Emmy Racine; Caroline Hurley; Aoife Cheung; Carol Sinnott; Karen Matvienko-Sikar; Christine Baumgartner; Nicolas Rodondi; William H Smithson; Patricia M Kearney

doi:10.12688/hrbopenres.12817.2

. 2019 Mar 22;1:14. Originally published 2018 Apr 11. [Version 2] doi: 10.12688/hrbopenres.12817.2

Participants’ perspectives and preferences on clinical trial result dissemination: The TRUST Thyroid Trial experience

Emmy Racine ^1,^a, Caroline Hurley ¹, Aoife Cheung ¹, Carol Sinnott ², Karen Matvienko-Sikar ¹, Christine Baumgartner ³, Nicolas Rodondi ^3,⁴, William H Smithson ⁵, Patricia M Kearney ¹

PMCID: PMC6973522 PMID: 32002507

Version Changes

Revised. Amendments from Version 1

This improved version contains some minor revisions as suggested by peer-reviewers. Throughout the manuscript, the following changes have been made:

“patient- preferred” has been changed to “patient-based”.
“patient-preferred method” has been changed to “patient-based approach”.
“Standard method” has been changed to “standard approach”.

Within the Abstract, the aim of the study has been re-worded to clarify that all TRUST participants were aged 65 and over. Within the Introduction section, additional background information has been provided on the need to evaluate the impact of PPI. This serves as a rationale for doing the study. We have also introduced the recent movement towards transparency in trials including references to the SPIRIT, CONSORT and AllTrials initiatives. Within the Methods section, additional details have been provided on the PPI group and how PPI partners were identified and recruited. Further information has also been provided on the Consensus Oriented Decision Making (CODM) model and how the model was specifically used in this study. We have also provided a clear distinction between adult literacy and health literacy. Within the Results section, a footnote has been added to Table 1 to clarify that only a subgroup of Irish participants were invited to the focus groups. A footnote has also been added to Table 2 to clarify how patient understanding was assessed. Within the Discussion, the section entitled ‘Limitations of the study’ has now been reworded to ‘Strengths and limitations of the study’ and the paragraph that discusses how PPI partners were participants in the trial has been rephrased as a strength of the study.

Abstract

Background: While there is an increasing consensus that clinical trial results should be shared with trial participants, there is a lack of evidence on the most appropriate methods. The aim of this Study Within A Trial (SWAT) is to use a patient and public involvement (PPI) approach to identify, develop and evaluate a patient-based approach to receiving trial results for participants in the Thyroid Hormone Replacement for Subclinical Hypo-Thyroidism Trial (TRUST), a trial of thyroxine versus placebo in people aged 65 years and older.

Methods: Mixed methods study with three consecutive phases. Phase 1 iteratively developed a patient-based approach using semi-structured focus groups and a consensus-orientated-decision model, a PPI group to refine the method and adult literacy review for plain English assessment. Phase 2 was a single-blind parallel group trial. Irish TRUST participants were randomised to the intervention (patient-based approach) and control group (standard approach developed by lead study site). Phase 3 used a patient understanding questionnaire to compare patient understanding of results between the two groups.

Results: Participants want to receive results of clinical trials, with qualitative findings indicating three key themes including ‘acknowledgement of individual contribution’, ‘contributing for a collective benefit’ and ‘receiving accessible and easy to understand results’. Building on these findings, the patient-based approachwas developed. TRUST participants (n=101) were randomised to the intervention (n=51) or control group (n=50). The questionnaire response rate was 74% for the intervention group and 62% for the control group. There were no differences in patient understanding between the two approaches.

Conclusions: We have demonstrated that it is feasible to involve trial participants in the development of result dissemination materials. Although, in this study PPI did not influence patients’ understanding of results, it documents the process of conducting PPI within the clinical trial setting.

Keywords: Patient and public involvement, patient involvement in clinical trials, study within a trial, SWAT, Clinical trial result dissemination, study results, research dissemination, trial results.

Introduction

Patient and public involvement (PPI) is increasingly recognised as an essential component of clinical research. In the UK, the national advisory group supporting active public involvement in health services, public health and social care research (INVOLVE) defines PPI as ‘research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’ ‘about’ or ‘for’ them’ ¹. In clinical trials, PPI has been defined as experimenting with participants instead of experimenting on participants ². PPI may occur at any stage during the research process from priority setting and drafting study protocols right through to conducting the study, interpreting the end results and communicating and disseminating research findings ^3,
4. Research funders increasingly expect that PPI is prioritised and resourced within studies. This increasing expectation has heightened the risk of researchers carrying out ‘tick-box’ PPI rather than ‘meaningful’ involvement ⁵. There are many moral and ethical arguments being made for PPI. Many believe that as citizens and taxpayers, members of the public have a right to influence research that is being funded by public money ⁶. PPI researchers are also making pragmatic arguments for PPI and providing anecdotal accounts about how PPI can make research more relevant, accessible and acceptable to participants ⁷. The ethical arguments are often seen as sufficient regardless of any pragmatic impact. However, PPI costs time and money, therefore pragmatic claims need scrutiny ⁸. More substantive evidence is needed to evaluate the potential impact of PPI on the conduct and outcomes of research ^5,
9. In 2001, the need to establish if PPI leads to actual, rather than merely perceived benefits for research processes and output was identified. Over fifteen years later, this need remains.

In clinical research, the results of clinical trials have not traditionally been shared with clinical trial participants. A recent survey carried out on a large registry of health research participants, found that while 95.6% of respondents said researchers should always or sometimes offer the results to participants, only 33% of respondents actually received the results of studies in which they had participated ¹⁰. An upcoming European Union Clinical Trial Regulation requires sponsors to provide summary results of clinical trials in a format understandable to laypersons, including participants ¹¹. However, there is a lack of evidence on the most appropriate methods of sharing results with participants. Uncertainty persists around what information should be shared, how results should be shared and who should be responsible for sharing the results. Since the findings of clinical research often exist in a complex context of scientific exchange and debate, it is important that the information shared is accessible and relevant to participants ¹². The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols ¹³, the Consolodated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials ¹⁴ and the AllTRials iniative calls for all past and present triasl to be registered and their full methods and summary results reported ¹⁵. Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community ¹³.

The Thyroid Hormone Replacement for Subclinical Hypothyroidism Trial (TRUST) was a multi-centre, double blind, placebo controlled, phase III clinical trial testing the efficacy of thyroxine replacement in subclinical hypothyroidism in older community dwelling adults ¹⁶. The results of the TRUST trial were published in the New England Journal of Medicine on 3 ^rd of April, 2017 ¹⁶. This Study Within A Trial (SWAT) was conducted at the Irish TRUST trial site prior to and after publication of results. The aim of this SWAT was to investigate methods of disseminating trial findings to participants by using a PPI approach to identify, develop and evaluate a patient-based approach of receiving trial results.

Methods

Study design

This was a sequential mixed methods study with three phases. In this study, methods were combined for complementarity, where each method addressed a different aspect of the study aim ¹⁷. The first phase used a qualitative approach to identify and develop a patient-based approach to disseminating the results, the second phase used a SWAT intervention to compare the dissemination approaches and the third phase used a quantitative patient understanding questionnaire to evaluate the patient-based approach. The full study protocol has been published elsewhere ¹⁸, but a summary follows here.

Setting

The study sites for the TRUST trial were the University of Glasgow, Scotland (lead site); Leiden Academy on Vitality and Ageing, The Netherlands; Leiden University Medical Centre, The Netherlands; University of Berne, Switzerland; and University College Cork, Ireland. A total of 738 participants with subclinical hypothyroidism were recruited to the trial over a three-and-a-half year period from 2013–2017 ¹⁶. The trial completed recruitment in November 2016 and the results were published in April 2017 ¹⁶.

This SWAT was conducted at the Irish TRUST site. The hub centre for the Irish TRUST site was located at the Mercy University Hospital, Cork where 38 participants were recruited. A further 77 participants were recruited from five satellite sites.

Population

As this SWAT was embedded in an ongoing clinical trial the study sample was determined by the TRUST Thyroid trial. There were 115 TRUST participants recruited in the Irish site, 11 of these participants withdrew over the course of the trial. Our study sample included all remaining TRUST participants (n=104).

Phase One: Identification and development of patient-based approach (qualitative and PPI phase)

The first phase of the study used a qualitative approach to iteratively identify and develop a patient-based approach to disseminate the results of TRUST trial. This was done in three separate stages: qualitative focus groups, a PPI group and an adult literacy review.

Focus groups

Three semi-structured focus groups were conducted with four to eight TRUST trial participants per group. All Cork-based patients (n = 38) were contacted via letter and invited to participate. A €20 shopping voucher was given to all participants to cover travel expenses. Each session was led by trained qualitative researchers (WHS, ER, CH). A topic guide was used to guide the focus groups. The topic guide was reviewed and refined by all members of the SWAT research team (see Supplementary File 1: Focus group topic guide).

The Consensus-Oriented-Decision-Making (CODM) model was used to guide the group to reach a consensus ¹⁹. The CODM model is accepted as a flexible model for reaching decisions ¹⁹. In this study some of the steps were initiated by the focus group facilitator and others occurred naturally as a follow on from the previous step. Below is an outline of each of the seven steps of the CODM model and how they were used in this study:

1.
Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants.
2.
Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial.
3.
Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used.
4.
Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language.
5.
Choosing a direction: This step occurred naturally as part of the previous step.
6.
Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback.
7.
Closure: This step occurred naturally as part of the previous step.

Analysis. Focus group recordings were transcribed verbatim and entered into NVivo Version 11 for data management during thematic analysis. Braun and Clarke guidelines ²⁰ for conducting thematic analysis were followed. Initial focus group transcripts were analysed independently by two researchers (ER and AC). Each transcript was read multiple times (data familiarisation) and initial codes were identified. These codes were then used to identify emerging themes. Both researchers discussed emerging themes and conducted further refinement. The refined themes were then discussed and agreed upon with other members of the research team (ER, CH, AC, KMS). Researchers (ER, CH, AC) then used the focus group findings to develop an initial draft of a patient-based approach for the dissemination of results (see Supplementary File 2: Draft one of patient-based result letter).

PPI group

A PPI group was established to develop and refine the content of the patient-based appproach for the dissemination of results. During the focus groups, three TRUST trial participants volunteered to take part in the PPI group. In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This individual was keen to learn more about research and expressed an interest in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner. Originally, we intended to conduct these sessions in a group format, due to difficulties with PPI partners’ schedule commitments, one-to-one sessions were conducted. At the one-one session, a researcher (ER) and the PPI partner discussed the layout, content and language of the initial draft of the result method. Researchers and PPI partners worked together to edit different sections of the document. These discussions were not audio recorded but comprehensive field notes were taken by the researcher (ER). These notes were then collated by the researcher and used to further ensure that the results letter reflected PPI partners’ perspectives and preferences.

Adult literacy review

While the PPI group had significant input into the format and language used in the patient-based approach, the research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national “Plain English” standards. These standards ensured that the information presented to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented (health literacy) ²¹. This review was an iterative process with several drafts exchanged for editing. Although the review was taken as an additional step to the published protocol for the study, the research team felt it was helpful to further ensure that the document was accessible and easy to understand.

At the end of the first phase of the study, a final draft of the patient-based result letter was approved by researchers, PPI group and adult literacy experts (see Supplementary File 3: Final draft of patient-based result letter).

Phase Two: Dissemination of trial results (intervention phase)

The second phase of the study used a SWAT intervention to disseminate the results of the TRUST Thyroid Trial to trial participants. This was done using a prospective, randomised, single blind, parallel trial design. It is important to note that when the term randomisation is used, it refers to the allocation of patients to intervention/control within the SWAT and not the TRUST Thyroid trial. Irish TRUST participants were randomised to intervention or control groups using an online random number generator. The intervention group received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter). Participants were blinded to their intervention group. One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial. As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way.

Phase Three: Evaluation of patient –based approach (quantitative phase)

The third phase of the study used a quantitative patient understanding questionnaire to evaluate the patient-based approach to disseminating trial results. The questionnaire was developed in consultation with experts in the area of subclinical hypothyroidism and scale (questionnaire) development (PK and KMS). The early development of the questionnaire was guided by a consultation document, which accompanies the EU Clinical Trials Regulation No 536/2014 ²². This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial. However, initial questionnaire items were modified to allow for psychometric testing. The final questionnaire contained 12 questions; six items were measured on a five point LIKERT scale, there were four multiple-choice questions and two vignettes. The first six items measured patients’ perceived understanding of results, the four multiple choice measured patients’ actual understanding of results by requiring them to select the correct answer. To further test participants’ understanding of the trial results, two vignettes describing two typical patient case studies of older adults with subclinical hypothyroidism were provided with a question on whether a doctor should prescribe thyroxine for the hypothetical patient described. The questionnaire was reviewed by the PPI group to assess content and face validity. It then underwent further review by NALA to ensure adherence to the national ‘Plain English’ standard. The final version of the questionnaire can be seen in Supplementary File 5: Patient understanding questionnaire.

The questionnaire was sent to all Irish TRUST participants (intervention and control group) one week after they received the results of the trial. A reminder questionnaire was sent to non-responders 3 weeks later.

Analysis. The primary outcome was the difference in levels of patient understanding between the intervention and control groups. This measured the impact of PPI on patient understanding of end of trial results. The psychometric properties and construct validity of the questionnaire were examined with exploratory factor analysis. Principal component analysis (PCA) was conducted on the six LIKERT scale items. Internal consistency of the questionnaire was investigated using Cronbach's alpha coefficient. Completed questionnaires were entered into SPSS software (version 24) and analysed using descriptive and inferential (Chi-square test and Fishers Exact) statistics. The researcher carrying out data input and analysis was blinded to the participants' allocation status.

Costs of conducting PPI

The lead researcher (ER) kept a detailed account of all direct costs associated with conducting PPI for the purpose of this study. These costs included researcher salary, travel and expenses for PPI participants, adult literacy review and printing and postage costs.

This paper has been written in adherence to the Guidance for Reporting Involvement of Patients and Public 2 (GRIPP 2) ²³. The GRIPP 2 checklist is a tool, developed to improve the reporting of patient and public involvement in research and guide the development of a transparent, consistent and high-quality PPI evidence base. The Good Reporting of a Mixed Methods Study (GRAMMS) framework was also used to inform the reporting of the findings ²⁴.

Results

Characteristics of the trial participants stratified by participation in the different stages of the study are presented in Table 1.

Table 1. Characteristics of trial participants stratified by participation in the different stages of the study.

	Total Irish TRUST participants (n=104)	Attended SWAT focus groups ¹ (n=19) Total Sample n=38 RR ² =50%	Randomised ³ (n=101)		Returned SWAT questionnaire (n=69) Total Sample n=101 RR ²=68%
	Total Irish TRUST participants (n=104)		Intervention Group (n=51)	Control Group (n=50)	Intervention Group (n=38) RR= 74%	Control Group (n=31) RR=62%
Sex
Male	61 (58.7%)	14 (73.7%)	31 (60.8%)	28 (56%)	26 (68%)	16 (52%)
Female	43 (41.3)	5 (26.3%)	20 (39.2%)	22 (44%)	12 (32%)	15 (48%)
Age
65–74	57 (54.8%)	12 (63.1%)	32 (62.7%)	24 (48%)	25 (66%)	12 (45%)
75+	47 (45.2%)	7 (36.9%)	19 (37.3%)	26 (52%)	13 (34%)	17 (55%)
Education
Primary only	22 (21.2%)	2 (10.5%)	12 (23.6%)	9 (18%)	10 (26%)	8 (26%)
Secondary/Tertiary	47 (45.1%)	12 (63.2%)	24 (47.1%)	22 (44%)	19 (50%)	11 (35%)
Unknown	35 (33.7%)	5 (26.3%)	15 (29.3%)	19 (38%)	9 (24%)	12 (39%)

Open in a new tab

¹A subgroup of Irish TRUST participants (n=38) were invited to focus groups.

²RR=Response Rate

³Total Irish TRUST participants (n=104) excluding PPI partners (n=3)= n=101.

Phase One: Identification and development of patient-based approach (qualitative and PPI phase)

Focus groups

Three focus groups were held with 19 out of 38 participants accepting an invitation to join. Participants who attended the focus groups were similar in age, gender, education level to those who did not attend.

Focus group findings indicate that participants want to receive the results of the trial in which they are taking part. Three main themes emerged in relation to participants’ perspectives of and preferences for receiving trial results: ‘acknowledgement of individual contribution’, ‘contributing for a collective benefit’ and ‘receiving accessible and easy to understand results’.

Acknowledgement of individual contribution

Many participants reported feeling they had made an individual contribution to the trial in terms of their time and personal information while attending the trial study visits. As such, participants felt that receiving the results of the trial would provide an acknowledgement of this individual contribution:

‘Yes, I mean it’s kind of instinctive… when you go into a [clinical trial] and you spend and invest that time in it. I mean okay I had the time to invest but you know at the end of the day, [receiving the result] is kind of like your pay off. ’ (FG2 P3)

Contributing for a collective benefit

While participants spoke about making an individual contribution to the trial, they felt that their involvement contributed to a collective benefit or greater good. Participants reported that receiving the results of the trial would help them to feel that they had contributed to this greater good:

‘I’m not really interested in my own personal results but as the results of the scheme as a whole. You know the idea is, does the study help or hinder old people and that’s what I want to know’ (FG2 P1)

This feeling of contributing for a collective benefit was further reinforced when participants discussed their desire to understand how the results of the trial will be implemented by medical experts and ultimately how it will affect others who have the condition:

‘I would like to know, if they found out, okay, do we treat these people or not. That would be good. Do we treat them or don’t we treat them? I think that is what it’s all about’ (FG3 P4)

Receiving accessible and easy to understand results

Participants expressed a clear need to receive the results of the trial in an accessible and easy to understand way. This preference applied to the format, language and content of the patient-based approach.

The majority of participants said they would like to receive the results in a letter format posted to them directly from the TRUST trial. Participants felt that this method would be accessible to them as they could read the results ‘in text’ (FG3 P4) and keep a ‘hardcopy’ (FG P1). While participants wanted an official statement of the results in a letter format, they also felt it was important to add a personal element to the letter. They suggested this could be done by offering participants a phone number that they could call if they wished to discuss any further issues or concerns with the TRUST study team:

‘Could you attach a helpline on to it? If you know, somebody had some kind of serious medical question or that they thought was a bit personal element or whatever. That they’d like to talk to a medical person or whatever. Instead of just talking to your GP, maybe that would add another dimension of care around the TRUST’ (FG2 P3)

Participants agreed that the format, content and language of the results letter needed to be easy to read and understand. All participants wanted the letter to be no longer than 2–3 pages and presented in a question and answer format. Participants believed the content of the results letter should include ‘pertinent information’ (FG1 P7) relating to the trial itself, the study drug (including side effects) and the results of the trial. They stressed the importance that this information needed to be informed by medical experts and ‘from a good authoritative source’ (FG2 P2) but it should be presented to them in a language that fits their current context and could be easily understood by those who do not have scientific or medical backgrounds.

‘Just in ordinary language that we can understand ourselves, you know that we don’t want big and long explanation or that, just that we can pick it up straight away that it’s without any huge number of pages. Just the bare, to me anyway, answers to the questions.’ (FG3- P2)

It was evident from the focus groups that participants want to receive the results of the trial both to acknowledge their individual contribution to the trial and also help them to feel that they had contributed to a greater good. Participants expressed a clear preference to receive the results in an accessible and easy to understand way. These results were used by the researcher (ER) to develop an initial draft of the results letter (see Supplementary File 2: Draft one patient-based result letter).

PPI group

The initial draft of the results letter was then further iteratively developed by the PPI group. There were four PPI partners in total (three trial participants and one older adult) Each partner toook part in one-to-one session. Each session contained an open discussion between the researcher (ER) and PPI partners on the layout, content and language of the document. Researchers and PPI partners worked together to write, re-write, edit and change different sections of the document.

Health literacy review

This draft was then iteratively reviewed and approved by health literacy experts from the NALA (see Supplementary File 3: Final version patient-based results letter).

Phase Two: Dissemination of trial results (intervention phase)

There were a total of 101 Irish TRUST participants randomised to the SWAT intervention. Trial participants from the PPI group (n = 3) were excluded from randomisation as they reviewed the content of the intervention method prior to the intervention. The intervention group (n=51) received the patient-based letter format (see Supplementary File 3: Final version patient-based results letter) and the control group (n=50) received a copy of the TRUST results press release, which was made available by the lead study site on the TRUST Thyroid Trial Website (see Supplementary File 4: Standard results letter).

Phase Three: Evaluation of patient-based approach (quantitative phase)

The overall response rate for the patient understanding questionnaire was 68% (69/101). The response rate for the intervention group was 74% (38/51) and the response rate for the control group was 62% (31/50). There were no significant differences in age, gender and education between those who returned the questionnaire and those who did not.

Post hoc power calculations showed that the study was underpowered to detect an effect. Power for each of the patient understanding components ranged from .01 to. 58.

Table 2 below shows the results of patients’ perceived understanding of the purpose and context of the TRUST Thyroid Trial. Due to low participant numbers across the five Likert responses, the questionnaire response bands have been contracted from ‘Strongly Agree’ and ‘Agree’ to ‘Yes, ‘Strongly Disagree’ and ‘Disagree’ to ‘No’ and ‘Neither agree nor disagree’ to ‘Neutral’. The results show that patients’ perceptions of understanding are similar between the intervention and control groups. Subgroup analysis showed patient’s understanding was not significantly impacted by age, gender or educational level.

Table 2. Patient perceptions of understanding presented by group ¹.

Item	Group	Yes	No	Neutral	p-value
I understand why the TRUST Thyroid Trial took place.	Intervention (n=38)	37 (97.4%)	1 (2.6%)	0 (0%)	0.584
I understand why the TRUST Thyroid Trial took place.	Control (n=31)	29 (93.5%)	2 (6.5%)	0 (0%)	0.584
I understand why I was invited to the TRUST Thyroid Trial	Intervention (n=38)	38 (100%)	0 (0%)	0 (0%)	0.198
I understand why I was invited to the TRUST Thyroid Trial	Control (n=31)	29 (93.5%)	2 (6.5%)	0 (0%)	0.198
I know why the medicine Levothyroxine is used to treat subclinical hypothyroidism	Intervention (n=38)	32 (84.2%)	2 (5.3%)	4 (10.5%)	0.893
	Control (n=31)	25 (80.6%)	3 (9.7%)	3 (9.7%)	0.893
I am aware of the side effects of Levothyroxine	Intervention (n=38)	30 (78.9%)	5 (13.2%)	3 (7.9%)	0.090
I am aware of the side effects of Levothyroxine	Control (n=31)	17 (54.8%)	7 (22.6%)	7 (22.6%)	0.090
I understand the impact of Levothyroxine on thyroid specific quality of life	Intervention (n=38)	31 (81.6%)	5 (13.2%)	2 (5.3%)	0.281
	Control (n=31)	20 (64.5%)	7 (22.6%)	4 (12.9%)	0.281
I understand how doctors will use the results of the TRUST Thyroid trial to treat people with subclinical hypothyroidism	Intervention (n=38)	33 (86.8%)	2 (5.3%)	3 (7.9%)	0.878
	Control (n=31)	26 (83.9%)	3 (9.7%)	2 (6.5%)	0.878

Open in a new tab

¹Patient perceptions of understanding were assessed using a five point LIKERT scale.

Figure 1 shows patients’ actual understanding of the primary aim, side effect and results of the TRUST Thyroid Trial. Almost 82% (n=31) of the intervention group and 65% (n=20) of the control group correctly understood the primary aim of the TRUST trial (p=0.108). Almost 40% (n=15) of the intervention group and 36% (n=9) of the control group correctly understood the associated side effects of the active drug (p=0.734). In total 50% of the intervention group (n=19) and 58% of the control group correctly understood the results of the trial (p=0.504). There were no differences in patient understanding of trial results between the intervention and control groups.

In terms of patient understanding of hypothetical patient case studies, 43% (n=13) of the intervention group gave the correct answer to Vignette A; this was lower than the control group (62.1%, n=18, p=0.15). In total 77% (n=23) of the intervention group gave the correct answer to Vignette B, this was higher than the control group (66%, n=19, p=0.344).

Psychometric testing

An exploratory principal components analysis (PCA) was conducted on the patient understanding questionnaire to determine its usefulness as a measure of perceived understanding. The Kaiser-Meyer-Olkin (KMO) measure verified the sampling adequacy for the analysis, KMO= .83. Bartlett’s test of sphericity indicated that the correlation matrix was significantly different from an identity matrix, X2 (.852) = 283.92, p<.001. An examination of eigenvalues greater than Kaiser’s criterion of one, suggested the extraction of one factor; this was supported by inspection of Cattell’s scree plot. An examination of the constituent items for this factor structure also indicated that items loaded most highly on a single factor. This single factor represents a measure of perceived understanding of trial results. PCA was then conducted using an oblique (direct oblimin) rotation, specifying the extraction of one factor. This model explained a combined 69.58% of the variance in patients understanding of the TRUST thyroid trial.

Cost of conducting PPI

The total cost of this study amounted to €8,049 (see Supplementary File 6: Costs of conducting PPI).

Discussion

While PPI is increasingly recognised as an important element of clinical research, evidence on optimal methods and potential impact is lacking ^4,
9. Previous research conducted on the impact of PPI has largely focused on the experiences of participants and researchers ²⁵ and on the research process in broad terms ²⁶. In this study, our primary outcome was specific: a quantitative measure of patient understanding of trial results between those who received the patient-based approach and the standard approach. To our knowledge there has been no previous research conducted on the impact of PPI on patient understanding of trial results.

The involvement of clinical trial participants in this study offered insightful perspectives on the information needs of the study population in terms of receiving end of trial results. Study findings show that trial participants want to receive the results of the clinical trial in which they had participated. This is supported by much of the available literature on patients’ preferences of receiving results, with up to 90% of participants in previous studies reporting a desire to receive results ²⁷. Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an ‘ethical imperative or ‘moral obligation’. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism ^12,
28.

Unsurprisingly, findings also show that participants want to receive results that are accessible and easy to understand. In this study, the preferred format of receiving results was a letter posted to them directly from the TRUST trial. This preference is also consistent with the literature on patient preferences of receiving results. A previous study investigating prefrences of individuals taking part in a cardiac rehabilitation trial found that 80% of trial participants preferred to receive the results by post ²⁹. The patient-based approach identified in this study was feasible for researchers to develop with significant involvement from trial participants and adult literacy experts.

Previous studies exploring participants’ reactions found that sharing trial results with participants can cause some negative impacts such as anxiety, anger, guilt, upset and confusion ^30–
32. As far as researchers in this study are aware, providing results did not cause any negative impacts. This may have been due to the fact that the TRUST trial had a low risk of morbidity or mortality compared to some of the other studies citing negative impacts. Both result methods contained the telephone number, email address and postal address of the research team and participants were urged to contact should they have any questions or concerns relating to the study. The research team did not receive any queries.

Previous systematic reviews highlight the lack of evidence on economic analysis of PPI and call for researchers to consider the costs of its implementation ^26,
33. As discussed previously research funders are increasingly demanding that PPI be carried out in research. However, the costs of PPI are often underestimated and can cause a significant financial burden on research project budgets ^26,
33–
35. It is extremely important that researchers plan PPI at the grant proposal stage and estimate the costs appropriately. If these costs are not correctly estimated during the initial stages of developing research proposals, they may cause a financial burden on PPI partners.

Participants in this study were not paid for their time but were provided with a €20 voucher to cover travel expenses. When PPI is not the primary focus of a study, researchers do not consider the cost implications at the beginning of the study and are often tied with limited resources to carry out PPI ^34–
36. INVOLVE, the national advisory group supporting active public involvement in health services, public health and social care research in the UK, have recommended that PPI partners should be paid for their involvement ³⁷. Despite this, existing research suggests that institutional difficulties make negotiating the mechanisms of paying participants very difficult ³⁴. One study reported that in order for participants to be remunerated for their efforts, they needed to be registered as employees, a process that incurred much paperwork and time delays ³⁴. This study outlines the cost of conducting PPI and includes a full breakdown of costs (see Supplementary File 6: Costs of conducting PPI). This breakdown provides a template to other researchers who plan to carry out and evaluate PPI as part of their research. It is important to note that not all costs associated with carrying out the study were included in this amount. For example, the only salary costed was that of the research assistant. The expertise provided by other members of the study team were not included in the total cost as they were being paid by the University or other research grants. The total cost of conducting this study was €8,049 which is not insignificant but should be considered in the context of the cost of large-scale trials.

Strengths and Limitations of the study

While this study provides important insights into patients’ preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that ‘to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population’ ³. All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed to long-term follow up. This is a strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial ³⁸, thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention. Previous research suggests that people that actively choose to engage in research either as research participants or involvement partners are more likely to be middle-class and highly educated ^39,
40. In this study, those that attended the focus groups and PPI group were similar in education level to those that did not attend. This is not surprising considering the entire study sample had already actively volunteered to take part in the TRUST trial.

Secondly, the results of the patient understanding questionnaire show that the levels of patient understanding were similar between the two groups. However, this study was underpowered to detect an effect. As this was a Study Within A Trial (SWAT), the power was limited by the sample size that was available to us from the trial (n=115). Furthermore, validation of the patient understanding questionnaire was limited by the sample size in this study. While validation of the questionnaire was limited, exploratory factor analysis provided some evidence that the questionnaire is a useful tool for measuring patient understanding of trial results. The developed questionnaire can be tailored for use in other trials in future examinations of patients understanding of trial results. This would provide insight into patient understanding and provide further validation data.

Thirdly, all SWATparticipants were aged 65 and over. The layout, format and language of this patient-based approach which was identified and developed may only be relevant for this study population. Other trial populations may prefer to receive the results via email, online or in person from a member of the study team ¹². The evidence on patient preferences of receiving trial results is limited, therefore further research is needed to explore patient preferences of receiving trial results amongst different study populations.

It is also important to point out that the control group in this study received a copy of the trial results in a press release format. Most trial participants do not receive this. While this control method was a step further than normal procedure, the researchers in this study felt this was appropriate. The information presented in the press release was similar to that of the patient-based approach. However, the format and layout of the press release was different. Information was writtern in four long paragraphs separated by individual headings. It was also much shorter (1 page in total) that the patient-based approach (3 pages in total). Given the fact that press releases are written by public relations professionals with a view to communicating effectively and efficiently, this may have potentially minimised differences between the intervention and control conditions. The primary outcome of this study was assessing the impact of PPI on patient understanding of results, however, this was not the only potential impact. In hindsight, we adopted a limited approach to PPI in this study as we did not involve our PPI partners from the outset of the SWAT. Involving PPI partners in the development of core outcome sets for this SWAT could have identified other more appropriate primary outcome measures ⁴¹.

The aim of this SWAT was to investigate methods of disseminating trial findings to trial participants by using a PPI approach to identify, develop and evaluate a patient-based method of receiving trial results. The PPI approach actively involved focus group participants in making decisions about the result method and worked with PPI partners to co-develop the result letter. However, PPI partners were not involved in other aspects of the research process such as research design, data collection or analysis. This is partly due to the fact that PPI is a relatively new concept in clinical trials. As the majority of the literature has only been published in the last 12 months, there is little evidence available on the impact of PPI and no gold standard or comprehensive guidelines for researchers to follow ²⁹. Thornton ² suggests that in order for PPI to develop it is important to record its social and cultural history by collecting comprehensive databases and undertaking ongoing reviews of the impact of PPI. This paper along with the study protocol have been written in adherence with the Guidelines for Reporting Involvement of Patients and the Public ²³, thus providing templates for involving patients and the public in clinical trial design and development. This study is an important step forwards in documenting the process of conducting PPI as part of a SWAT and evaluating its impact. Future research is needed to further develop PPI in clinical trial settings. As there is currently no gold standard or comprehensive guidelines for researchers to follow when evaluating the impact of PPI, further research is needed. This research should involve PPI partners in the development of core outcome sets for evaluating PPI impact. These would significantly enhance the literature in the area.

Conclusion

Patient and Public Involvement (PPI) is advocated for every step of the trial process. We have demonstrated that it is feasible to involve PPI partners in the development of dissemination materials. Sharing clinical trial results with participants in a format understandable to laypersons will soon be a legal requirement ¹¹. However, there is a significant lack of evidence as to the most appropriate methods of sharing results with participants. The study identified and developed a patient-based approach to disseminating clinical trial results for trial participants. Although, in this study PPI did not influence patients’ final understanding of results, it documents the process of conducting PPI within the clinical trial setting. This process may be useful for other trialists interested in conducting and evaluating the impact of PPI in clinical trials.

Ethics approval and consent to participate

The research was approved in Ireland by the Clinical Research Ethics Committee of the Cork Teaching Hospitals, UCC, Ref ECM 4 (t).

All participants provided signed informed consent to take part in the study.

Data availability

The raw data from this study cannot be sufficiently de-identified, and therefore are not publicly available. However, the data from the current study are available for further (collaborative) research purposes on reasonable request. Available datasets include transcripts from focus groups, field notes from PPI sessions and responses from the patient understanding questionnaire. To access the data, please contact the corresponding author ( emmy.racine@ucc.ie) or the Principal Investigator ( patricia.kearney@ucc.ie). Researchers must provide a written proposal on how the data will be used in research before access is granted.

Funding Statement

Health Research Board –Trials Methodology Research Network (HRB-TMRN) under the Study Within A Trial (SWAT) funding programme (TMRN-2014-1). The TRUST Trial was supported by a research grant (grant agreement number 278148) from the European Union FP7-HEALTH-2011 program and by grants from the Swiss National Science Foundation (SNSF 320030-150025 and 320030-172676, to Dr. Rodondi) and the Swiss Heart Foundation (to Dr. Rodondi) and Velux Stiftung (974a, to Dr. Rodondi). Christine Baumgartner was supported by the Gottfried and Julia Bangerter-Rhyner Foundation. Study medication was provided by Merck Darmstadt.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 2; peer review: 2 approved]

Supplementary material

All supplementary files are contained in one PDF document.

Click here for additional data file.^{(1.1MB, tgz)}

Supplementary File 1: Focus group topic guide

Supplementary File 2: Draft One of Patient-Based Result Letter

Supplementary File 3: Final Draft of Patient-Based Result Letter

Supplementary File 4: Standard Results Letter

Supplementary File 5: Patient Understanding Questionnaire

Supplementary File 6: Costs of Conducting PPI

References

1. INVOLVE: What is public involvement in research?2017. Reference Source [Google Scholar]
2. Thornton H: Patient and public involvement in clinical trials. BMJ. 2008;336(7650):903–4. 10.1136/bmj.39547.586100.80 [DOI] [PMC free article] [PubMed] [Google Scholar]
3. Rayment J, Lanlehin R, McCourt C, et al. : Involving seldom-heard groups in a PPI process to inform the design of a proposed trial on the use of probiotics to prevent preterm birth: a case study. Res Involv Engagem. 2017;3(1):11. 10.1186/s40900-017-0061-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. : Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66. 10.1111/hex.12090 [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Buck D, Gamble C, Dudley L, et al. : From plans to actions in patient and public involvement: qualitative study of documented plans and the accounts of researchers and patients sampled from a cohort of clinical trials. BMJ Open. 2014;4(12):e006400. 10.1136/bmjopen-2014-006400 [DOI] [PMC free article] [PubMed] [Google Scholar]
6. Dyer S: Rationalising public participation in the health service: the case of research ethics committees. Health place. 2004;10(4):339–48. 10.1016/j.healthplace.2004.08.004 [DOI] [PubMed] [Google Scholar]
7. INVOLVE: Exploring the impact of public involvement on the quality of research: examples. Eastleigh;2013. Reference Source [Google Scholar]
8. Dudley L, Gamble C, Allam A, et al. : A little more conversation please? Qualitative study of researchers' and patients' interview accounts of training for patient and public involvement in clinical trials. Trials. 2015;16(1):190. 10.1186/s13063-015-0667-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Staniszewska S, Herron-Marx S, Mockford C: Measuring the impact of patient and public involvement: the need for an evidence base. Int J Qual Health Care.Oxford University Press;2008;20(6):373–4. 10.1093/intqhc/mzn044 [DOI] [PubMed] [Google Scholar]
10. Long CR, Stewart MK, Cunningham TV, et al. : Health research participants’ preferences for receiving research results. Clin Trials. 2016;13(6):582–91. 10.1177/1740774516665598 [DOI] [PMC free article] [PubMed] [Google Scholar]
11. Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC.2014. Reference Source [Google Scholar]
12. Fernandez CV, Kodish E, Weijer C: Informing study participants of research results: an ethical imperative. IRB. 2003;25(3):12–9. 10.2307/3564300 [DOI] [PubMed] [Google Scholar]
13. SPIRIT: The SPIRIT Statement.2018. Reference Source [Google Scholar]
14. CONSORT: The CONSORT Statement.2018. Reference Source [Google Scholar]
15. AllTrials: Find out more.2018. Reference Source [Google Scholar]
16. Stott DJ, Rodondi N, Kearney PM, et al. : Thyroid Hormone Therapy for Older Adults with Subclinical Hypothyroidism. N Engl J Med. 2017;376(26):2534–2544. 10.1056/NEJMoa1603825 [DOI] [PubMed] [Google Scholar]
17. O'Cathain A, Murphy E, Nicholl J: Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res. 2007;7(1):85. 10.1186/1472-6963-7-85 [DOI] [PMC free article] [PubMed] [Google Scholar]
18. Racine E, Hurley C, Cheung A, et al. : Study within a trial (SWAT) protocol. Participants' perspectives and preferences on clinical trial result dissemination: The TRUST Thyroid Trial experience. Contemp Clin Trials Commun. 2017;7:163–5. 10.1016/j.conctc.2017.07.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
19. Hartnett T: Consensus-oriented Decision-making: the CODM Model for Facilitating Groups to Widespread Agreement.Gabriola Island. New Society Publishers;2011. Reference Source [Google Scholar]
20. Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
21. Agency NAL: NALA's Plain English Editing and Training service.2017. Reference Source [Google Scholar]
22. European Commission: Summary of Clinical Trial Results for Laypersons.Recommendations of the expert group on clinical trials for the implementation of Regulation (EU) No 536/2014 on clinical trials on medicinal products for human use. Consultation Document. European Commission;2016. Reference Source [Google Scholar]
23. Staniszewska S, Brett J, Simera I, et al. : GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3(1):13. 10.1186/s40900-017-0062-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Cameron R, Dwyer T, Richardson S, et al. : Lessons from the field: Applying the good reporting of a mixed methods study (GRAMMS) framework. Electronic Journal of Business Research Methods. 2013;11(2):53–64. Reference Source [Google Scholar]
25. Brett J, Staniszewska S, Mockford C, et al. : A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387–95. 10.1007/s40271-014-0065-0 [DOI] [PubMed] [Google Scholar]
26. Brett J, Staniszewska S, Mockford C, et al. : The PIRICOM Study: A systematic review of the conceptualisation, measurement, impact and outcomes of patients and public involvement in health and social care research.2010. Reference Source [Google Scholar]
27. Shalowitz DI, Miller FG: Communicating the results of clinical research to participants: attitudes, practices, and future directions. PLoS Med. 2008;5(5):e91. 10.1371/journal.pmed.0050091 [DOI] [PMC free article] [PubMed] [Google Scholar]
28. Kerasidou A: Sharing the Knowledge: Sharing Aggregate Genomic Findings with Research Participants in Developing Countries. Dev World Bioeth. 2015;15(3) :267–74. 10.1111/dewb.12071 [DOI] [PMC free article] [PubMed] [Google Scholar]
29. Dalal H, Wingham J, Pritchard C, et al. : Communicating the results of research: how do participants of a cardiac rehabilitation RCT prefer to be informed? Health Expect. 2010;13(3):323–30. 10.1111/j.1369-7625.2009.00580.x [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Snowdon C, Garcia J, Elbourne D: Reactions of participants to the results of a randomised controlled trial: exploratory study. BMJ. 1998;317(7150):21–6. 10.1136/bmj.317.7150.21 [DOI] [PMC free article] [PubMed] [Google Scholar]
31. Dixon-Woods M, Jackson C, Windridge KC, et al. : Receiving a summary of the results of a trial: qualitative study of participants' views. BMJ. 2006;332(7535):206–10. 10.1136/bmj.38675.677963.3A [DOI] [PMC free article] [PubMed] [Google Scholar]
32. Partridge AH, Wong JS, Knudsen K, et al. : Offering participants results of a clinical trial: sharing results of a negative study. Lancet. 2005;365(9463):963–4. 10.1016/S0140-6736(05)71085-0 [DOI] [PubMed] [Google Scholar]
33. Domecq JP, Prutsky G, Elraiyah T, et al. : Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89. 10.1186/1472-6963-14-89 [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Mockford C, Murray M, Seers K, et al. : A SHARED study-the benefits and costs of setting up a health research study involving lay co-researchers and how we overcame the challenges. Res Involv Engagem. 2016;2(1):8. 10.1186/s40900-016-0021-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
35. Jinks C, Carter P, Rhodes C, et al. : Patient and public involvement in primary care research - an example of ensuring its sustainability. Res Involv Engagem. 2016;2(1):1. 10.1186/s40900-016-0015-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
36. Perkins R, Goddard K: Reality out of the rhetoric: increasing user involvement in a mental health trust. Ment Health Rev J. 2004;9(1):21–4. 10.1108/13619322200400006 [DOI] [Google Scholar]
37. INVOLVE: Payment for Involvement. 2010. Reference Source [Google Scholar]
38. Brandberg Y, Johansson H, Bergenmar M: Patients' knowledge and perceived understanding–Associations with consenting to participate in cancer clinical trials. Contemp Clin Trials Commun. 2016;2:6–11. 10.1016/j.conctc.2015.12.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
39. Wilson P, Mathie E, Keenan J, et al. : ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study.2015. [PubMed] [Google Scholar]
40. Patrick JH, Pruchno RA, Rose MS: Recruiting research participants: a comparison of the costs and effectiveness of five recruitment strategies. Gerontologist. 1998;38(3):295–302. 10.1093/geront/38.3.295 [DOI] [PubMed] [Google Scholar]
41. Williamson PR, Altman DG, Bagley H, et al. : The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. 10.1186/s13063-017-1978-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

HRB Open Res. 2019 Mar 22. doi: 10.21956/hrbopenres.13965.r26546

Reviewer response for version 2

Shaun P Treweek ¹, Hanne Bruhn ²

We are happy with how the authors have addressed our suggestions and we have no further comments.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

HRB Open Res. 2018 May 16. doi: 10.21956/hrbopenres.13878.r26210

Reviewer response for version 1

Thilo Kroll ¹, Marina Zaki ¹

The paper is an important contribution to enhance patient and public involvement in trial research. A very worthy SWAT, one of a few SWATs conducted within the TRUST trial. Please explain SWAT at the outset.

Some suggestions:

While the authors refer to the PPI definition by INVOLVE it would have been helpful to get a clearer understanding of the authors’ specific understanding and conceptualisation of PPI in the context of their study. The approach of involving patients in shaping the dissemination of results is laudable but admittedly quite limiting and appears to be an afterthought rather than an integral element of the trial. The paper very much resides in the traditional experimental (albeit using a preference-based design), researcher-led paradigm (clearly expressed in the ‘randomisation’ to determine effectiveness of modes of dissemination – which somewhat contravenes the notion of engaging with a diverse public and patient population for bespoke dissemination) with limited shift of control over the research process to patients or the public. For example, how were study participants involved at the intersection of analysis, dissemination and knowledge transfer (selection of outcomes that are relevant? Decision on subgroup analysis? Involvement in interpretation of analysis findings? Decisions about outlets for dissemination (audiences, events, settings, format and content?).

It would be helpful to see a more detailed description of the PPI Group. How was this group selected, what is its composition, how frequently did it meet and what was its immediate influence in the research process throughout. Did members of the PPI group participate as co-researchers at any stage of contribute to the facilitation, analysis etc of the focus group results?

Paper mentions qualitative aspect and discusses the need to ensure accurate and relevant information is being shared with participants. Wonder if benefit could be derived from drawing upon the notion of “translational statistics”, where additional statistical information is obtained, analysed and accurately translated to clinically meaningful findings - perhaps with the authors referring to the tension of statistical significance versus clinical relevance and the importance of minimising statistical jargon when presenting trial results. Great to see contribution of NALA – but did the authors experience this barrier of translating statistical results into findings that were clinically relevant, most notably in the context of PPI in understanding this? The suggestion therefore is that including PPI here helps to minimise both statistical and clinical jargon, which can often be challenging, for a concise and accurate report.

Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), “This document highlights the information that trial participants should understand after reading their trial summary.” The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.

It would be important to distinguish between ‘literacy’ and ‘health literacy’. They represent related but substantially different concepts.

In line with other members of the trial community, the paper could benefit from the word ‘transparency’ – paragraph 2 introduction – to put it in context with other papers discussing sharing of trial findings and results. The authors could also consider making reference to the AllTrials initiative in paragraph 2 (not directly relevant to PPI, but important nonetheless in the movement towards transparency in trials).

It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.

Methods section of Phase 2 - “ One member of the research team was un-blinded in order to perform the randomisation and distribute the results of the trial.” – what other parts of the study was this member involved in? Did their unblinding have any impact on any other aspects of the study?

The paper could benefit from briefly mentioning the ‘moral obligation’ trialists have to share results – in this sense, I mean results to be the trial findings – (results in the context of data sharing is subject to new EU GDPR, which is something the authors could consider to mention but unlikely to have a place for discussion in this article; justification for not being able to share the data is already briefly mentioned in the “data availability” paragraph). So the term ‘moral obligation’ of trialists would complement the qualitative quotes the authors have in the results section about patients feeling they contributed to research.

Phase 1 results part: “This feeling of contributing for a collective benefit was further reinforced when participants discussed their desire to understand how the results of the trial will be implemented by medical experts and ultimately how it will affect others who have the condition:” à could mention one aspect, particularly this being a drug trial, of how statistics can explain why one treatment works on some patients and not others etc and that this may be something public and patients would like to know, therefore is important to accurately portray.

“education” – demographic not discussed in detail in other sections of paper- only mentioned in terms of not being statistically significantly different – need more? Relevance to PPI is not explained thoroughly and should be strengthened

Great mention of CT regulation and checklist when reporting PPI – but could also benefit from mentioning the fundamental reporting guidance in trials, such as putting SPIRIT into context and relevance, if any, to layman’s terms and similar to using CONSORT as a reference and explaining this to PPI? Also refer to www.clinicaltrials.gov reporting section for public

Clearly, it is a strength of the SWAT having participants already from the trial - see the trial through from beginning to end so their help is important in terms of accurate context/background when interpreting and disseminating findings

Generally, well-written, well laid out, concise and important paper.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

HRB Open Res. 2019 Jan 11.

Emmy Racine ¹

Firstly, thank you for your comprehensive comments.

To address this, we have inserted the following sentence into the abstract:

“The aim of this Study Within A Trial (SWAT) is to use….” (Abstract)

We have reflected on our approach to PPI in this Study Within a Trial and have added the following text to the manuscript to address this comment:

A more detailed description of the PPI group has been added to the manuscript to address this comment:

A PPI group was established to develop and refine the content of the patient-based approach for the dissemination of results. During the focus groups, three TRUST trial participants volunteered to take part in the PPI group. In addition to these three PPI partners, an additional partner was identified from a previous qualitative research study undertaken by the research team. This individual was keen to learn more about research and expressed an interested in being involved in future projects. While this individual had previous experience of taking part in research (as an interview participant), she had no experience of taking part in a clinical trial or being involved as a PPI partner.(Methods: PPI group)

There were four. PPI partners in total (three trial participants and one older adult). Each partner took part in one one-to-one session (Results: PPI group)

Members of the PPI group did not participate as co-researchers in the facilitation, analysis etc. of focus group results. We have addressed our approach to PPI in the previous comment and have made necessary additions to the main manuscript.

While we believe this is an important point, we believe it was not a major contributor in our study. As there were no statistically significant or clinically relevant results, the results of the trial were simple and straightforward (ie. no effect), and so we did not have this challenge.

Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), “This document highlights the information that trial participants should understand after reading their trial summary.” The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.

The authors do not mean to sound prescriptive in their approach. This sentence has been changed to the following:

‘This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial’. (Methods: Evaluation)

It would be important to distinguish between ‘literacy’ and ‘health literacy’. They represent related but substantially different concepts.

Both concepts were important in this study. We collaborated with the National Adult Literacy Agency to ensure that the material in the results letter to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented to them (health literacy).

To ensure accuracy, we have changed the wording throughout the document to ‘adult literacy’ and described our approach as follows:

Adult literacy review

While the PPI group had significant input into the format and language used in the patient-based approach. The research team felt that it would be of additional benefit to collaborate with the National Adult Literacy Agency (NALA) to ensure the document adhered to national “Plain English” standards. These standards ensured that the information presented to trial participants was sufficiently easy to read and understand (literacy). This would help to ensure that trial participants were able to make sound health decisions based on the information presented (health literacy) (15). This review was an iterative process with several drafts exchanged for editing. (L-P-)

Yes we agree, this paragraph would benefit from a reference to the recent movement towards transparency in trials. The following text has been added:

The increasing understanding of the importance of sharing research results with study participants is somewhat linked to a wider movement towards transparency in trials. This movement is largely promoted by initiatives such as SPIRIT, CONSORT and AllTrials. The SPIRIT Statement provides guidance to researchers to improve the completeness and quality of trial protocols (11), the Consolidated Standards of Reporting Trials (CONSORT) Statement is an evidence based, minimum set of recommendations for reporting randomised trials (12) and the AllTrials initiative calls for all past and present trials to be registered and their full methods and summary results reported (13). Some of these initiatives also include recommendations for disseminating results to research participants. For example, the SPIRIT statement states that study results must be released to participating physicians, referring physicians, patients and the general medical community (11). (Introduction, Paragraph 2).

It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.

The following text has been added to the manuscript to describe how the CODM model was specifically used in this study:

The Consensus-Oriented-Decision-Making (CODM) model was used to guide the group to reach a consensus. The CODM model is accepted as a flexible model for reaching decisions (14). In this study some of the steps were initiated by the focus group facilitator and others occurred naturally as a follow on from the previous step. Below is an outline of each of the seven steps of the CODM model and how they were used in this study:

1. Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants.

2. Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial.

3. Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used.

4. Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language.

5. Choosing a direction: This step occurred naturally as part of the previous step.

6. Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback.

7. Closure: This step occurred naturally as part of the previous step. (Methods: Phase 1)

This member of the research team (AC) was involved in the design of the study, preparing study materials, performing data collection and developing the result material. After they were un-blinded in order to perform the randomisation and distribute the results, they were not involved in the data analysis or interpretation in any way. The following sentence has been added to the manuscript to provide clarity on this:

“As they were un-blinded to perform these two important tasks, they were not involved in the data analysis or interpretation in any way.” (Methods: Phase 2)

We have inserted the following text into the discussion to address this:

“Focus group findings showed that participants felt that receiving results would provide an acknowledgement of their individual contribution to the trial. This finding complements previous commentaries about result sharing being an ‘ethical imperative or ‘moral obligation’. Fernandez et al. points out that many participants place their trust in science and researchers owe a debt to participants to fulfil their trust and recognise their altruism (10, 23).” (Discussion, Paragraph 2)

We have not mentioned the EU GDPR in this paper as we feel it is not directly relevant to sharing trial results with participants. The upcoming EU Clinical Trial regulation is mentioned in the introduction which will be the main EU regulation which requires investigators to share clinical trial results with participants in a format understandable to laypersons.

We acknowledge that understanding how statistics can explain why one treatment works on some patients and not others is an important aspect and may be something that patients would like to know. However, this aspect was not brought up by the trial participants in this study (either during the focus groups or PPI sessions). As this paper was about listening to trial participants and giving them the information they wanted to know, we feel it would be inappropriate to add it in to the paper at this late stage and presume that it is something that they wanted to know. Therefore, while it is an important aspect, we have not made any changes to address it in our paper.

We have added the following text to the discussion to address this:

Previous research suggests that people that actively choose to engage in research either as research participants or involvement partners are more likely to be middle-class and highly educated (34, 35). In this study, those that attended the focus groups and PPI group were similar in education level to those that did not attend. This is not surprising considering the entire study sample had already actively volunteered to take part in the TRUST trial. (Discussion: Strengths and Limitations, Paragraph 1)

The following text has been inserted into the introduction to address this comment:

We have made the following changes to the manuscript to address this:

This section of the discussion has been reworded from ‘Limitations of the study’ to ‘Strengths and limitations of the study’ (Discussion)

This paragraph in question has been rephrased as a highlight of the SWAT and moved to the first paragraph in the ‘Strengths and limitations of the study’ section. It now reads as following:

“While this study has provided important insights into patients’ preferences of receiving trial results, it is not without limitations. Firstly, existing PPI literature states that ‘to understand the research needs and challenges, PPI has to engage people who are able to offer perspectives from the study population’ (3). All PPI partners in this study were active members of the research community as they had taken part in the TRUST trial and had agreed for long-term follow up. This is a major strength of the SWAT as they were able to offer perspectives from the study population, however it does have an important implication for their reporting of understanding the results of the trial. They may be more inclined to rate their understanding as high because of their investment in the trial (31), thus potentially minimising differences between the intervention and control conditions and minimising inferences that can be drawn about the intervention.” Discussion: Strengths and Limitations, Paragraph 1)

Generally, well-written, well laid out, concise and important paper.

HRB Open Res. 2019 Jan 11.

Emmy Racine ¹

Firstly, thank you for your comprehensive comments.

To address this, we have inserted the following sentence into the abstract:

“The aim of this Study Within A Trial (SWAT) is to use….” (Abstract)

We agree that our approach to PPI in this study was limited. We involved trial participants in shaping the dissemination of results and did not involve them in other stages of the research process. We agree that there was a limited shift of control over the research process to PPI partners. We have reflected on our limited approach to PPI and have added the following text to the manuscript to address this comment:

This study is an important step forwards in documenting the process of conducting PPI as part of a SWAT and evaluating its impact. (Discussion: Strengths and Limitations, Paragraph 5)

A more detailed description of the PPI group has been added to the manuscript to address this comment:

There were four. PPI partners in total (three trial participants and one older adult). Each partner took part in one one-to-one session (Results: PPI group).

Members of the PPI group did not participate as co-researchers in the facilitation, analysis etc. of focus group results. We have addressed our limited approach to PPI in the previous comment and have made necessary additions to the main manuscript.

Also, some of the language used appears to be a departure from the PPI approach. For example (in Section Phase 3), “This document highlights the information that trial participants should understand after reading their trial summary.” The intention of the authors is clear but the wording very much suggests a prescriptive approach on the part of the academic team.

The authors do not mean to sound prescriptive in their approach. This sentence has been changed to the following:

‘This document highlights the information which should be presented to trial participants in the trial summary at the end of a trial’. (Methods: Evaluation)

It would be important to distinguish between ‘literacy’ and ‘health literacy’. They represent related but substantially different concepts.

To ensure accuracy, we have changed the wording throughout the document to ‘adult literacy’ and described our approach as follows:

Adult literacy review

Yes we agree, this paragraph would benefit from a reference to the recent movement towards transparency in trials. The following text has been added:

It would be helpful to clarify how the CODM was specifically used in the process and to provide evidence for the step by step approach.

The following text has been added to the manuscript to describe how the CODM model was specifically used in this study:

Framing the topic: The focus group facilitator introduced the idea of sharing results with participants and provided some context on the reasons why results are/ are not shared with participants.
Open discussion: The facilitator asked the group whether or not they think results should be shared with trial participants and whether or not they would like to receive the results of the TRUST trial.
Identifying underlying concerns: The previous discussion naturally followed on to participants asking questions and expressing concerns about the result method, content and language that would be used.
Collaborative proposal building: The group worked together to agree on the important elements of the results in terms of result method, content and language.
Choosing a direction: This step occurred naturally as part of the previous step.
Synthesizing a final proposal: The facilitator re-iterated the proposal the group had agreed upon and asked the group for feedback.
Closure: This step occurred naturally as part of the previous step. (Methods: Phase 1)