Abstract
This study assessed the subjective experience of participating in a clinical trial, specifically positive and negative experiences and the experience of audio recording assessment sessions. The study was cross-sectional from a single blinded randomised controlled trial. Forty participants with a primary diagnosis of non-organic psychosis completed baseline and 12-week follow-up questionnaires assessing their experiences. Participants rated research interviews as moderately helpful in facilitating their therapy and talking to the interviewer as moderately helpful at baseline and 12-week follow-up. Self-report ratings of the degree of self-realisation promoted by the research questionnaires were significantly higher at 12-week follow-up compared to baseline. Participants adjusted quickly to being audio recorded and rated interviews as not at all disruptive and not at all to slightly intrusive. On average there were neutral emotional reactions, positive gains and minimal inconveniences as a result of participation. The main reasons for taking part were: ‘To help myself’, ‘I was curious’ and ‘To help others’. The findings offer support to previous research reporting that individuals with mental health problems find participating in clinical trials a beneficial experience. This may alleviate concerns that participation in similar studies may be personally intrusive or harmful.
Keywords: positive, negative, participate, research, mental health, audio recording
1. Introduction
Research involving human participants has been a longstanding topic for ethical, legal and methodological discussion (Braunack-Mayer, 2002; Cassileth et al., 1982; Taylor et al., 2010). However, there has been limited research documenting the effects of psychiatric research procedures on participants (Kassam-Adams and Newman, 2002) as well as the perceptions of the participants involved (Grant, 2015; Marshall et al., 2001; Schäfer et al., 2008). There is an identified need for the development of new instruments assessing participants’ experiences of treatment research in order to ensure the protection of participants and the integrity of the research methods employed (Marshall et al., 2001).
Recruiting eligible participants for clinical trials has often proven to be very difficult (Campbell et al., 2007). However, trial participants play a major role in the development of effective new treatments (Harris et al., 1996) and provide evidence that helps inform treatment options for patients (Tallon et al., 2011). These studies rely on recruiting large numbers of participants (Marshall et al., 2001) and are dependent on timely recruitment and retention of participants (Tallon et al., 2011). Moreover, the way in which research participants experience associated benefits and risks can be a complex issue (Braunack-Mayer, 2002). Individuals may decide to participate in research for a number of reasons such as feeling a sense of obligation, hoping to receive some benefit or wanting to help others (Braunack-Mayer, 2002).
Therefore, it is important to understand the perceptions of potential participants (Marshall et al., 2001) and their readiness to participate (Roberts et al., 2006) when designing and conducting successful trials that are acceptable to the target population (Tallon et al., 2011). Furthermore, it is important to document their experience of research which may also help to assess the quality and validity of research findings (Newman et al., 2001).
The AVATAR Clinical Trial aims to test the clinical efficacy of the AVATAR therapy in reducing the frequency and severity of auditory verbal hallucinations (Craig et al., 2015). It involves completing a number of measures at baseline, 12 weeks and 24 weeks follow-up. The assessment sessions are audio-recorded with the participant’s consent and participants are randomly allocated to be invited to attend seven weekly therapy sessions in both arms of the trial. Main results have been recently published, showing that AVATAR therapy resulted in a rapid and substantial fall in the frequency, omnipotence and power of voices, and associated distress that was significantly superior to supportive counselling at 12 weeks follow-up. At 24 weeks follow-up the differences between the two arms were no longer statistically significant. Reductions in the AVATAR therapy scores on primary outcome were sustained and the absence of statistically significant between group differences at 24 week follow-up relates to the supportive counselling group continuing to improve between 12 and 24 week follow-up. There was no evidence of any adverse events attributable to either therapy (Craig et al., 2017).
It is known that clinicians tend to overestimate the negative impact of research participation and underestimate the potential positive benefits (Marshall et al., 2001; Newman et al., 2001). It has also been noted that negative reactions from participants tend to be rare (Jorm et al., 2007; Taylor et al., 2010), with psychosocial research assessing the role of traumatic events more likely to be approved by patients rather than biological research (Schäfer et al., 2008). Although some participants may find it distressing to talk about the experience of voices and previous traumatic events, they may also find it beneficial to address these topics in a supportive environment (Braunack-Mayer, 2002). It is therefore of interest to assess the impact of participating in the AVATAR Clinical Trial and to understand the associated benefits by asking the participants involved. Little is also known about the impact of audio recordings in research and practice despite its widespread use (Briggie et al., 2016).
1.2. Aim
To assess the subjective experiences of participants involved in the AVATAR Clinical Trial, specifically the positive and negative experiences of the research as well as the experience of audio recording assessment sessions. The study also aims to ascertain participants’ motivation for taking part and to establish the internal consistency of the measures used.
2. Methods
The AVATAR (Audio Visual Assisted Therapy Aid for Refractory auditory hallucinations) study is a single blinded randomised controlled trial assessing the effectiveness of a new computer assisted therapy to help reduce the frequency and severity of auditory hallucinations (Craig et al., 2017). Participants who met the eligibility criteria were randomly allocated to receive AVATAR therapy (intervention group) or supportive counselling (control group). Both therapies were delivered over 7 sessions (1 introductory session plus 6 therapy sessions) lasting approximately 45 minutes. Participants were also required to complete weekly in-session measures with the therapists. Therapy sessions were audio-recorded and participants were provided with an MP3 player and given instructions to listen to their sessions in between therapy.
The research element of the AVATAR study involved participants completing assessments at three time points: at baseline (before randomisation), 12 weeks and at 24 weeks follow-up. These assessments included a number of interview-based and self-report measures to assess the impact of the interventions on specific outcomes. Research assessments were also audio-recorded (with participant’s permission), lasted approximately 1.5 hours and participants were paid £20 for each assessment as a reimbursement for their time.
2.1. Participants
Participants were eligible for the present study if they completed the baseline assessment, attended at least one session of either intervention, and the 12-week follow-up assessment. They were not given any additional reimbursement for completing the questionnaires in this study.
2.2. Measures
The Assessing the Impact of Research Questionnaire (AIR; (Marshall et al., 2001)) is a 14-item (baseline evaluation) and 19-item (follow-up evaluation) self-report measure employing a Likert-type scale to assess positive and negative emotional and cognitive appraisals of experiences of completing structured interviews, questionnaires and having sessions audio-recorded. There were minor adaptations to the wording to make the questionnaire relevant to the specific time points being assessed in this study. One item assessing the specific number of audio-recorded sessions it took for participants to feel comfortable (“How many tape-recorded sessions did it take for you to feel as comfortable as you usually do?”) was not applicable and therefore omitted. The final version of the AIR used at baseline consisted of 13 items; four assessing positive impacts (e.g. promoting self-realisation - where participants realise something new about themselves), four assessing negative impacts (e.g. disruptiveness or intrusiveness of interviews) and five items assessing difficulty adjusting to audio recording. (Note: Impact refers to participants’ appraisals of experiences rather than inferring a causal role). The final version of the 12-week follow-up evaluation consisted of 20 items with the addition of an item about keeping the researchers ‘blinded’ about the type of therapy received. Participants were also given the opportunity to leave free text comments about their overall experience. The AIR was scored by calculating the means and frequencies of individual items and required interpretation on an individual basis. The internal consistency of this measure has not yet been published and will be established in the present study.
The Reactions to Research Participation Questionnaire (RRPQ; (Newman et al., 2001)) is a 23-item self-report measure rated on a 5 point Likert scale (from 1 - ‘Strongly disagree’ to 5 - Strongly agree). It was developed to assess participants’ evaluation of research participation and was designed to be applicable to a wide range of research areas. There are five subscales: participation factor, personal benefits, emotional reactions, perceived drawbacks and global evaluation. Scores for the RRPQ were calculated by computing the averages of the items loading on to each subscale. The emotional reactions subscale was used at the baseline evaluation to assess initial reactions, whilst the full scale was used at the 12-week follow-up evaluation to assess the overall impact of the study. Good psychometric properties have been reported for this measure (Kassam-Adams and Newman, 2002; Murphy et al., 2011; Newman et al., 2006). A cross-sectional study of women’s reactions to body image and eating disorder research reported good reliability for the emotional reactions (α = 0.86), personal benefits (α = 0.82), perceived drawbacks (α = 0.73) and global evaluation (α = 0.77) sub-scales (Murphy et al., 2011). The internal consistency of the participation factor subscale was 0.6 (Murphy et al., 2011). The RRPQ also includes a checklist which asks participants to rank their top three reasons for taking part in the study out of a possible nine reasons. This list included reasons such as “I was curious”, “I felt I had to” and “For the money”.
2.3. Procedure
The 13-item baseline AIR evaluation together with the emotional reaction subscale of the RRPQ and the checklist of reasons for participating were completed after the baseline assessment. The 20-item 12-week follow-up AIR evaluation and 23-item RRPQ were completed after the 12-week follow-up assessment. The questionnaires primarily focused on participants’ experience of the research assessments rather than experience of therapy sessions. However, there were questions in the 12-week follow-up evaluation which asked participants whether the research experience was discussed in therapy sessions and the helpfulness of discussing this experience as part of assessing the impact of the research.
In order to prevent ‘unblinding’ of the researchers, the trial coordinator reviewed the returned questionnaires before passing them on for data entry.
The study data collection took place between January 2015 and January 2016, concurrently with recruitment for the AVATAR study (from November 2013 to February 2016). Participants who completed the baseline and 12-week follow-up assessments during this period (n =75) were asked at the end of each session whether they would be interested in completing a short questionnaire about their experience of taking part in the research. If agreeable, they were provided with a copy of the relevant questionnaire (baseline or 12-week follow-up) with a stamped addressed envelope to be completed at home without the researcher present and returned within one week. The aim was to obtain an adequate sample size of baseline and follow-up responses, with the possibility of some participants completing both questionnaires.
2.4. Data analysis
Data were analysed using SPSS version 21. The positive and negative impacts of the research at baseline and 12-week follow-up were reported using means, standard deviations, frequencies and percentages. RRPQ scores were also reported descriptively. Percentages for the reasons for participating were calculated by summing the frequencies of the top three reasons on the checklist. Given the small sample size and the non-normal distribution of the data, the non-parametric Wilcoxon Signed Ranks test was used to investigate any differences between baseline and 12-week follow-up for those participants who completed surveys at both time points (n = 10).
2.4.1. Internal consistency
In the present study, the internal consistency of the AIR was assessed using Cronbach’s alpha, with a criterion of 0.70 taken as indicating good internal consistency (Streiner et al., 2014). The reliability of the five subscales of the RRPQ was also calculated and reported using the same criterion.
3. Results
A total number of 54 participants agreed to take baseline questionnaires and 41 agreed to take 12-week follow-up questionnaires for completion in their own time out of the 75 participants approached (72% and 55% response rate respectively). Of these, 31/54 baseline questionnaires (57%) and 19/41 12-week follow-up evaluations (46%) were returned. All respondents had a primary diagnosis of non-organic psychosis including paranoid schizophrenia, schizoaffective disorder and bipolar disorder. The sociodemographic and clinical characteristics of all participants are shown in Table 1. A majority of the respondents were male (65%), single (72.5%) and not currently in employment (85%). The mean age was 44.34 years (S.D. = 9.87) and most (75%) had at least secondary (O Level/GCSE equivalent) qualifications. There were no differences between participants in the present study and those in the main AVATAR Clinical Trial (n = 150; Craig et al., 2017) in terms of the sociodemographic and clinical characteristics reported below indicating a representative sample.
Table 1. Participants’ sociodemographic and clinical characteristics*.
| Range | Mean (SD) | ||
|---|---|---|---|
| Age | 30-72 | 44.34 (9.87) | |
| % | N | ||
| Gender | Male | 65 | 26 |
| Female | 32.5 | 13 | |
| Ethnicity | White British | 30 | 12 |
| Black British | 20 | 8 | |
| Black Caribbean | 10 | 4 | |
| Black African | 7.5 | 3 | |
| Asian Indian | 5 | 2 | |
| Asian Chinese | 2.5 | 1 | |
| Other | 20 | 8 | |
| Diagnosis | Paranoid Schizophrenia | 67.5 | 27 |
| Schizoaffective Disorder | 15 | 6 | |
| Bipolar Disorder | 5 | 2 | |
| Unspecified non-organic psychosis | 2.5 | 1 | |
| Schizophrenia unspecified | 2.5 | 1 | |
| Depression with psychotic symptoms | 2.5 | 1 | |
| Marital Status | Single | 72.5 | 29 |
| Divorced or Separated | 17.5 | 7 | |
| Married/cohabiting | 5 | 2 | |
| Employment | Working full-time | 2.5 | 1 |
| Working part-time | 7.5 | 3 | |
| Not working | 85 | 34 | |
| Education | Higher education qualification/degree | 17.5 | 7 |
| Vocational qualification | 17.5 | 7 | |
| A Levels | 15 | 6 | |
| GCSE/O level/CSE | 25 | 10 | |
| No formal qualifications | 20 | 8 |
Note: Data missing for two participants (5%) on all characteristics (except gender); data missing for one participant (2.5%) for gender due to questionnaires being returned anonymised.
3.1. Internal consistency
The internal consistency of the 13-item baseline AIR employed in this study was 0.61. The 12-week follow-up measure demonstrated good reliability with an alpha of 0.7. The emotional reactions subscale employed at baseline had good internal consistency (α = 0.83). The internal consistencies for the five subscales at 12-week follow-up were: participation factor (α = 0.65), personal benefits (α = 0.82), perceived drawbacks (α = 0.75), global evaluation (α = 0.84) and emotional reactions (α = 0.85).
3.2. Impact of the research
Table 2 displays the means and standard deviations for each item assessing the positive and negative impacts at baseline and 12-week follow-up. Categorical endorsements for each item are shown in Table 3. On average, participants rated completing the research questionnaires at baseline as slightly to moderately helpful in realising something new about themselves and moderately helpful at 12-week follow-up. Additionally, participants at both baseline and 12-week follow-up rated the research interviews as moderately helpful in facilitating their therapy in some way and found it moderately helpful to talk to research interviewers about themselves. Overall, participants rated that they were moderately personally helped by taking part in the research.
Table 3. Baseline and 12-Week Follow-up Endorsements for each item.
| Baselime Endorsements (%) | 12-Week Follow-up Endorsements (%) | |||||||
|---|---|---|---|---|---|---|---|---|
| AIR Item |
Not
at all |
Slightly | Moderately | Considerably |
Not
at all |
Slightly | Moderately | Considerably |
| Positive Impacts | ||||||||
| Self-realisation promoted by questionnaire |
20.7 | 17.2 | 27.6 | 34.5 | 17.6 | -- | 29.4 | 52.9 |
| 3.6 | 3.6 | 57.1 | 35.7 | 20 | 33.3 | 13.3 | 33.3 | |
| 7.1 | 17.9 | 50 | 25 | 6.7 | 20 | 33.3 | 40 | |
| Therapy facilitated by questionnaire | 6.5 | 12.9 | 41.9 | 38.7 | 12.5 | 18.8 | 12.5 | 56.3 |
| Therapy facilitated by interviews |
-- | -- | -- | -- | 25 | 31.3 | 12.5 | 31.3 |
| Helpfulness of interviews |
-- | -- | -- | -- | 5.9 | 23.5 | 11.8 | 58.8 |
|
Overall positive impact
of research |
70 | 16.7 | 6.7 | 6.7 | 61.1 | 16.7 | 11.1 | 11.1 |
| 61.3 | 12.9 | 12.9 | 12.9 | 64.7 | 11.8 | 11.8 | 11.8 | |
| Therapy facilitated by research participation |
82.8 | 10.3 | 6.9 | -- | 93.8 | 6.3 | -- | -- |
| 36.7 | 33.3 | 10 | 20 | 52.9 | 17.6 | 17.6 | 17.6 | |
| Personally helped by research participation |
— | — | — | — | 77.8 | 11.1 | 5.6 | 5.6 |
| — | — | — | — | 76.5 | 5.9 | 11.8 | 5.9 | |
| Negative Impacts | ||||||||
| Negative experience of questionnaire |
70 | 20 | 6.7 | 3.3 | — | — | — | — |
| 50 | 6.7 | 16.7 | 26.7 | — | — | — | — | |
| Irrelevance of interviews |
43.3 | 26.7 | 20 | 10 | — | — | — | — |
| 14.3 | 25 | 25 | 35.7 | — | — | — | — | |
| Disruptiveness of interviews |
||||||||
| Intrusiveness of interviews |
||||||||
|
Overall negative
impact of research |
||||||||
| Therapy interfered by research participation Intrusiveness and harmful effects | ||||||||
| Impact of audiotaping | ||||||||
| Difficulty adjusting to audiotaping Comfort level with audiotaping vs. note taking Time to adjust to audiotaping a |
||||||||
| Experience as helpful or reassuring |
||||||||
Rated as 1 = right away; 2 = very soon; 3 = most of the session; 4 = never did feel comfortable.
The research questionnaires and interviews at baseline and 12-week follow-up were rated as not at all to slightly irrelevant to commencing therapy and not at all to slightly a negative experience. The research interviews at baseline were rated as slightly intrusive whilst at 12-week follow-up they were rated as not at all to slightly intrusive. At both time points the research interviews were rated as not at all disruptive to the therapy received (82.8% endorsements in ‘Not at all’ category at baseline; 93.8% at 12-week follow-up). Overall, participants reported that completing the research assessments did not interfere with their therapy at all and rated their overall experience as not at all personally intrusive or harmful.
Adjustments to the audio-recording occurred right away to very soon (70% endorsement in these categories). There was little difficulty adjusting to being audio recorded, with a majority (90%) endorsing the ‘not at all’ and ‘slightly’ categories. Participants also found it slightly to moderately helpful or reassuring to have the sessions audio recorded. Of particular note, the presence of the recorder did not seem to interfere with thoughts shared with the researcher, with 80.6% reporting no interference at all.
For those participants who completed questionnaires at both time points (n = 10), self-report ratings of the degree of self-realisation promoted by the questionnaires were significantly higher at 12-week follow-up (Mean = 3.4; S.D. = 0.97) compared to baseline (Mean = 2.75; S.D. = 1.17) (Z = -2.06; p = 0.039). Further investigation revealed that 9/10 of these participants completed all therapy sessions (n = 7) and a majority (60%) reported at 12-week follow-up that discussion of their research experience during therapy sessions promoted new insight about themselves. There were no other statistically significant changes between baseline and 12-week follow-up evaluations on any other variable examined.
Some participants (52.6%) did not go on to discuss their research experience during therapy sessions and some (57.9%) also reported that there was nothing that was revealed in the research assessments that was not otherwise discussed in therapy. A majority (71.4%) went on to eventually discuss things revealed in research during therapy.
Overall participants found it slightly difficult to conceal their therapy allocation in order to keep researchers ‘blinded’ (Mean = 2; S.D. = 1.31). When asked about informed consent, almost all participants (17 of 19) felt they were well informed about the research study prior to agreeing to participate. A majority of participants (73.7%) reported that they would definitely or probably participate in this research if they had to do it over again. Examination of the free texts comments indicated good experiences overall. Participants found the research to be helpful, interesting and beneficial. Some appreciated being given the opportunity to talk to someone about current experiences and distress in their lives in a comfortable, safe and non-judgemental environment. Others reported that the research helped to ease worries, helped with their illness or with moving on with their lives in general. Examples of comments related to the research interviews include:
“I felt comfortable doing the research. The interviewer has a nice trustworthy attitude which made me feel safe and able to talk.”
“The person I spoke with in sessions was easy to get along with and I felt she didn’t judge me, which allowed me to be open and honest.”
“I found it slightly stressful but helpful”
3.3. Participant reactions
From the checklist provided, participants indicated that they took part in the clinical trial for the following reasons: “To help myself” (65%), "I was curious” (45%), “To help others” (25%), “Thought it might improve my access to health care” (20%), “I don’t know” (15%), “For the money” (7.5%), “I didn’t want to say no” (7.5%), and “Felt I had to” (5%). (Percentage sums are greater than 100 as participants could select more than one reason.)
Scores on the emotional reactions subscale at baseline were around 3 (Mean = 2.84; S.D. = 1.06; n = 31) indicating neutral agreement with the statements pertaining to negative emotional reactions caused by the study. Descriptive statistics for the five subscales of the RRPQ at 12-week follow-up are described in Table 4.
Table 4. Descriptive statistics for the RRPQ subscales at 12-Week Follow-up (N = 18*).
| RRPQ Subscale | Mean | SD | Possible range |
Sample range |
|---|---|---|---|---|
| Participation | 4.25 | 0.66 | 1 - 5 | 2.75 - 5 |
| Personal Benefits | 4.13 | 0.84 | 1 - 5 | 1.5 - 5 |
| Global Evaluation | 4.51 | 0.5 | 1 - 5 | 3.6 - 5 |
| Perceived Drawbacks | 2.04 | 0.85 | 1 - 5 | 1 – 3.33 |
| Emotional Reaction | 3.12 | 1.13 | 1 - 5 | 1 - 5 |
One case omitted due to missing data.
Scores on all three positive factors (participation, personal benefits and global evaluations) at 12-week follow-up were greater than 4 indicating agreement with statements pertaining to positive gains and experiences from the study. Scores on the perceived drawbacks factor were around 2, indicating disagreement with statements relating to inconveniences that may have been caused by the study. Finally, scores on the emotional reaction subscale at 12-week follow-up were around 3 (Mean = 3.12; SD = 1.13) indicating neutral agreement with the statements pertaining to negative emotional reactions.
4. Discussion
This study aimed to investigate the subjective experiences of participants in a clinical trial assessing the efficacy of a new computer assisted treatment (AVATAR therapy). It also aimed to describe the positive and negative impacts, participants’ motivation for taking part and their reactions to research participation. The findings indicate minimal negative impacts from participating in the trial and a moderate positive impact. These findings mirror those from another clinical trial in which similar impacts were reported (Marshall et al., 2001). Self-report ratings of the degree of self-realisation promoted by completing research questionnaires were significantly higher at 12-week follow-up compared to baseline for those who completed this study at both time points. Additionally, most of these participants reported that they had realised something new about themselves through discussing their research experience during therapy sessions.
Concealing therapy allocation from researchers was identified as a challenge for some participants in the AVATAR Clinical Trial. There was little difficulty in adjusting to being audio-recorded, with participants rating the presence of the recorder as slightly to moderately reassuring. It is important to note that initial assessments included questions about childhood trauma (Bernstein et al., 2003) and voice experiences including verbatim content. In most instances recording did not affect the level of information disclosed or thoughts shared with the researcher (although in a minority of cases where the person was reluctant to discuss, for example, distressing voice content it was agreed that this could be done directly with the therapist). The experience of paranoid thoughts and persecutory beliefs are a frequent feature of psychosis (Freeman, 2007), and it has been noted that recording in research may increase inhibition (Gelso, 1974). Clinicians and researchers often worry about issues such as ‘unblinding’ and audio recording sessions. However, these were not found to be problematic in this study. This is in line with previous research in which participants expressed no or slight concerns to being recorded and the implementation of audio recording in research, practice and clinical training was supported (Briggie et al., 2016).
Consistent with other research, the main reasons for participating in this study were to improve one’s chance of recovery (‘To help myself’), to help others (Schafer et al, 2008) as well as being curious about the research. On average there were neutral emotional reactions, positive gains and minimal inconveniences as a result of participation. The overall findings appear to echo those of previous research reporting that individuals with mental health problems may find participating in a clinical trial a beneficial experience (Rosen et al., 2007). Although not all participants felt they were well informed about the research, a majority were willing to participate again, very few were motivated by monetary gains and participants rated the experience as not at all personally intrusive or harmful.
4.1. Strengths and limitations
This study involved individuals with psychosis and contributes to an area of research that is not adequately covered in the existing literature (Grant, 2015; Schäfer et al., 2011). It also employed the use of a measure that was specifically designed to assess subjective experiences of treatment research and sought to report information on its reliability. The addition of an item assessing the consequences of trial ‘blinding’ for the participant addresses an important and relevant issue for future trials; something which is currently missing in other studies assessing the impact of being involved in a clinical trial.
However, the study is limited by its relatively small sample size and low response rate for the 12-week follow-up questionnaires (55%); therefore these findings may be considered tentative. Participants were asked to complete the additional questionnaires for this sub-study in their own time and without the researcher present to avoid bias or coercion. As such participants may have been less likely to return them, particularly after completing a lengthy research interview and going through the entire research process. This may account for the low response rate for 12-week follow-up data and is a factor which needs further consideration in future similar studies. The number of eligible participants who could be approached for this study was also reduced due to delays in the ethical approval process.
The results also reflect the subjective experiences of a selection of participants in the main trial. The experience of those who opted not to participate or were not approached for this study remains unknown. However no demographic differences were found between participants in the present study and those in the larger AVATAR Clinical Trial which helps to demonstrate representativeness in this sample. A further caveat is that the sample consists of individuals who were willing to take part in this additional study, and therefore might be more willing to accept conditions such as being audio-recorded than those individuals who did not agree to take part. Nine out of ten of the individuals who completed both baseline and 12-week follow-up questionnaires in this study attended all therapy sessions which can be seen as a further indication of willingness to participate and to engage in all aspects of the study.
The study design ensured participants’ subjective experiences were collected at the earliest opportunity following intervention and participants received no further therapy between 12 weeks and 24 weeks follow-up. While the main avatar trial indicated potential differences in trajectory of treatment response the purpose of this study was to assess the subjective experiences of taking part in a clinical trial, including the experience of research assessments and exploring benefits of talking about experiences across the recruited sample rather than the experience of therapy sessions between arms. A full discussion of the limitations of the main AVATAR clinical trial including the absence of a treatment as usual condition and the ‘augmented’ form of supportive counselling delivered can be found in the main paper (Craig et al., 2017).
4.2. Implications and conclusions
Considering the substantial commitment required for participating in the AVATAR Clinical Trial and the initial concerns of the research team that the assessments and procedures might be somewhat intensive, minimal negative impacts were reported overall. The evidence that participation in a clinical trial can be a positive and beneficial experience for individuals with mental health problems is important to note in light of possible reservations from some clinicians and researchers. Increased self-realisation between baseline and 12-week follow-up evaluations suggests that there might be specific personal gains and insight for an individual simply through engaging in the research process. Detailed clinical summaries prepared for the therapists by the research team were helpful in facilitating this process and may be good practice for similar trials.
The findings also support the view that audio recording of sessions may offer reassurance to some participants in that the content and process of the research is being captured objectively (Briggie et al., 2016). This has implications for its continued use in clinical practice and research. Though difficult and traumatic experiences were discussed, negative emotional reactions were rarely elicited and participants felt supported throughout the research process. Researchers ensured that the participants felt in control of the amount of detail they were comfortable sharing while completing the assessments. These findings may help to alleviate concern that participation in similar studies may be personally intrusive or harmful to the individuals involved. Discussions around trauma in research assessments might differ in sensitivity to more open explorations of abuse occurring in clinical assessments. Therefore the implementation of audio recordings requires further exploration in future studies to explore potential differences in its acceptability in research and clinical practice.
Keeping researchers ‘blinded’ to treatment allocation is also a relevant issue for future single blinded randomised controlled trials. This issue was approached in the main study by sensitively explaining the rationale of ‘blinding’ at each stage of the process to participants (i.e. with researchers at baseline and subsequently with the therapist) while recognising that it is inevitable that ‘unblinding’ may occur. It was also important to ensure that this did not become a source of concern to the participants in the trial. Although ‘unblinding’ occurred in 28 participants (of 150) in the main clinical trial, 25 of those were still happy to meet with a different researcher to complete the assessments (Craig et al, 2017).
In summary, the subjective experiences of individuals in the AVATAR Clinical Trial indicate that participants found taking part in the trial a beneficial experience with minimal evidence of a negative impact. The findings offer support to previous research and may help to alleviate concern from some clinicians and researchers that participation in similar studies may be personally intrusive or harmful.
Highlights.
Positive and beneficial experience for people with mental health problems
Specific personal gains and insight through engaging in the research process
Audio recordings may offer reassurance - content and process captured objectively
Potential to explore audio recording acceptability in research & clinical practice
Consequences of trial ‘blinding’- an important and relevant issue for future trials
Table 2. Baseline and 12-Week Follow-up Ratings by Participants on the Impact of Participating in the AVATAR Clinical Trial.
| AIR Item | Baseline | 12-Week Follow-up | ||
|---|---|---|---|---|
| Positive Impacts a | Mean | SD | Mean | SD |
| Self-realisation promoted by questionnaire | 2.76 | 1.16 | 3.18 | 1.31 |
| Therapy facilitated by questionnaire | 3.25 | 0.7 | 2.6 | 1.18 |
| Therapy facilitated by interviews | 2.93 | 0.86 | 3.07 | 0.96 |
| Helpfulness of interviews | 3.13 | 0.89 | 3.13 | 1.15 |
| Overall positive impact of research a | ||||
| Therapy facilitated by research participation | -- | -- | 2.5 | 1.21 |
| Personally helped by research participation | -- | -- | 3.24 | 1.03 |
| Negative Impacts a | ||||
| Negative experience of questionnaire | 1.5 | 0.9 | 1.72 | 1.07 |
| Irrelevance of interviews | 1.77 | 1.12 | 1.71 | 1.1 |
| Disruptiveness of interviews | 1.24 | 0.58 | 1.06 | 0.25 |
| Intrusiveness of interviews | 2.13 | 1.14 | 1.88 | 1.11 |
| Overall negative impact of research a | ||||
| Therapy interfered by research participation | -- | -- | 1.39 | 0.85 |
| Intrusiveness and harmful effects | -- | -- | 1.47 | 0.94 |
| Impact of audiotaping | ||||
| Difficulty adjusting to audiotaping a | 1.43 | 0.77 | -- | -- |
| Comfort level with audiotaping vs. note takinga | 2.2 | 1.32 | -- | -- |
| 1.97 | 1.03 | -- | -- | |
| Time to adjust to audiotaping b | 2.82 | 1.09 | -- | -- |
| Experience as helpful or reassuring a | ||||
Rated on a scale from 1 to 4 (1 = none/not at all; 2 = slightly; 3 = moderately; 4 = considerably)
Rated on a scale from 1 to 4 (1 = right away; 2 = very soon; 3 = most of the session; 4 = never did feel comfortable)
Acknowledgements & funding
This work was supported by the Wellcome Trust Translation Award (grant number 09827). The clinical trial is based at King’s College London while further development of the AVATAR computer-based system is based at the Speech, Hearing & Phonetic Sciences Department at University College London. The trial team express their gratitude to Prof. Julian Leff for training advice on the delivery of therapy, and to Dr. Mark Huckvale and Dr. Geoffrey Williams for their continued technical support of the AVATAR system. PG is part funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. The funding source had no input in the preparation of this manuscript.
Footnotes
Authors’ contributions
JH designed and led the study, drafted the versions of the questionnaires, collected, analysed and interpreted the data. MR contributed to study design and questionnaire development. LOA assisted with the data collection. All other authors critically revised the draft manuscript, and read and approved the final manuscript.
References
- Bernstein DP, Stein JA, Newcomb MD, Walker E, Pogge D, Ahluvalia T, et al. Development and validation of a brief screening version of the Childhood Trauma Questionnaire. Ch Ab Neg. 2003;27(2):169–190. doi: 10.1016/s0145-2134(02)00541-0. [DOI] [PubMed] [Google Scholar]
- Braunack-Mayer A. The ethics of participating in research. Med J Aus. 2002;177(9):471–472. doi: 10.5694/j.1326-5377.2002.tb04912.x. [DOI] [PubMed] [Google Scholar]
- Briggie AM, Hilsenroth MJ, Conway F, Muran JC, Jackson JM. Patient comfort with audio or video recording of their psychotherapy sessions: relation to symptomatology, treatment refusal, duration, and outcome. Prof Psychol Res Prac. 2016;47(1):66. [Google Scholar]
- Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, et al. Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study Hea Tech Ass. 2007;11(48):1–126. doi: 10.3310/hta11480. [DOI] [PubMed] [Google Scholar]
- Cassileth BR, Lusk EJ, Miller DS, Hurwitz S. Attitudes toward clinical trials among patients and the public. Jama. 1982;248(8):968–970. [PubMed] [Google Scholar]
- Craig TK, Rus-Calafell M, Ward T, Fornells-Ambrojo M, McCrone P, Emsley R, et al. The effects of an audio visual assisted therapy aid for refractory auditory hallucinations (AVATAR therapy): study protocol for a randomised controlled trial. Trials. 2015;16(1):1. doi: 10.1186/s13063-015-0888-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Craig TK, Rus-Calafell M, Ward T, Leff J, Huckvale M, Howarth E, et al. AVATAR therapy for auditory verbal hallucinations in people with psychosis: a single-blind, randomised controlled trial. Lancet Psych. doi: 10.1016/S2215-0366(17)30427-3. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Freeman D. Suspicious minds: the psychology of persecutory delusions. Clin Psychol Rev. 2007;27(4):425–457. doi: 10.1016/j.cpr.2006.10.004. [DOI] [PubMed] [Google Scholar]
- Gelso CJ. Effects of recording on counselors and clients. Coun Ed Sup. 1974;14(1):5–12. [Google Scholar]
- Grant C. The experiences of research participants living with schizophrenia. Soc Wk Men Hea. 2015;13(6):532–552. [Google Scholar]
- Harris Y, Gorelick PB, Samuels P, Bempong I. Why African Americans may not be participating in clinical trials. J Nat Med Ass. 1996;88(10):630. [PMC free article] [PubMed] [Google Scholar]
- Jorm AF, Kelly CM, Morgan AJ. Participant distress in psychiatric research: a systematic review. Psychol Med. 2007;37(07):917–926. doi: 10.1017/S0033291706009779. [DOI] [PubMed] [Google Scholar]
- Kassam-Adams N, Newman E. The reactions to research participation questionnaires for children and for parents (RRPQ-C and RRPQ-P) Gen Hos Psych. 2002;24(5):336–342. doi: 10.1016/s0163-8343(02)00200-1. [DOI] [PubMed] [Google Scholar]
- Marshall RD, Spitzer RL, Vaughan SC, Vaughan R, Mellman LA, MacKinnon RA, et al. Assessing the subjective experience of being a participant in psychiatric research. Amer J Psych. 2001;158(2):319–321. doi: 10.1176/appi.ajp.158.2.319. [DOI] [PubMed] [Google Scholar]
- Murphy MJ, Edwards KM, Merrill JC, Gidycz CA. Undergraduate women's reactions to body image and eating disorder research. Account Res. 2011;18(1):55–69. doi: 10.1080/08989621.2011.542686. [DOI] [PubMed] [Google Scholar]
- Newman E, Risch E, Kassam-Adams N. Ethical issues in trauma-related research: a review. J Emp Res Hum Res Eth. 2006;1(3):29–46. doi: 10.1525/jer.2006.1.3.29. [DOI] [PubMed] [Google Scholar]
- Newman E, Willard T, Sinclair R, Kaloupek D. Empirically supported ethical research practice: the costs and benefits of research from the participants’ view. Account Res. 2001;8(4):309–329. doi: 10.1080/08989620108573983. [DOI] [PubMed] [Google Scholar]
- Roberts LW, Warner TD, Hammond KG, Hoop JG. Views of people with schizophrenia regarding aspects of research: study size and funding sources. Schizophr Bull. 2006;32(1):107–115. doi: 10.1093/schbul/sbj022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosen C, Grossman LS, Sharma RP, Bell CC, Mullner R, Dove HW. Subjective evaluations of research participation by persons with mental illness. J Ner Men Dis. 2007;195(5):430–435. doi: 10.1097/01.nmd.0000253785.81700.7d. [DOI] [PubMed] [Google Scholar]
- Schäfer I, Burns T, Fleischhacker WW, Galderisi S, Rybakowski JK, Libiger J, et al. Attitudes of patients with schizophrenia and depression to psychiatric research: a study in seven European countries. Soc Psy Psychia Epid. 2011;46(2):159–165. doi: 10.1007/s00127-010-0181-7. [DOI] [PubMed] [Google Scholar]
- Schäfer I, Gschwend C, Karow A, Naber D. Attitudes of patients with schizophrenia to psychiatric research. Int J Psychia Clin Prac. 2008;12(3):165–170. doi: 10.1080/13651500701636502. [DOI] [PubMed] [Google Scholar]
- Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford University Press; New York: 2014. [Google Scholar]
- Tallon D, Mulligan J, Wiles N, Thomas L, Peters TJ, Elgie R, et al. Involving patients with depression in research: survey of patients' attitudes to participation. Brit J Gen Prac. 2011;61(585):134–141. doi: 10.3399/bjgp11X567036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taylor PJ, Awenat Y, Gooding P, Johnson J, Pratt D, Wood A, et al. The subjective experience of participation in schizophrenia research: a practical and ethical issue. J Nerv Men Dis. 2010;198(5):343–348. doi: 10.1097/NMD.0b013e3181da8545. [DOI] [PubMed] [Google Scholar]
