Abstract
Background:
Attention control is an important part of clinical trials, allowing the influence of attention to be parsed from the effects of the intervention. Poor control group design can lead to unintended outcomes, overestimate the effectiveness of an intervention, or threaten a potentially beneficial treatment. The aims of this paper were to describe the development of a novel attention control using TED Talks and provide recommendations for the design of an attention control condition.
Methods:
Twenty-four TED Talks were selected for inclusion in the control condition. The talks ranged from 6.0 to 19.1 minutes for a total length of 4 hours and 12 minutes (mean = 10 minutes, 30 seconds). Topics included education, technology, health, space, human behavior, sanitation, parenting, cliff climbing, public speaking, happiness, obituaries, procrastination, and age diversity. We then developed 3 to 4 simple questions reflecting the content of the Talk. Responses ranged from true/false to 4 multiple-choice answers. Four to five videos were selected for each attention control session and bundled to create an approximately 90-minute attention control condition to match the intervention (acupuncture) session (45 minutes × 10 sessions).
Discussion:
We developed an attention control condition that met crucial requirements of (1) equivalent attention to that of the experimental group, (2) a design that can hold the participant’s interest, (3) standardization to avoid potentially confounding outcomes, and (4) the evaluation of expectancy.
Conclusion:
Investigators can develop a rigorous attention control condition using TED Talks guided by our protocol.
Trial registration
This study is registered with Clinicaltrials.gov (NCT06311461).
Keywords: randomized controlled trials, attention control, placebo effect, TED Talks, acupuncture
A well-designed clinical trial is critical for determining intervention efficacy and effectiveness. Attention control groups allow researchers to examine whether the intervention being tested is efficacious/effective beyond the benefits of social interaction alone and ensure that participants receive social attention equivalent to that provided by the intervention in both dose and timing.1,2 Inappropriate control conditions may threaten internal validity and make drawing meaningful conclusions from the trial intervention impossible. Poor control group design can lead to unintended effects on the outcomes, overestimate the effectiveness of an intervention, or threaten a potentially beneficial treatment.3 An attention control condition is an important part of clinical trials in which the influence of attention is parsed from the effects of the intervention. However, there needs to be more consistency in the design and selection of control conditions.4 Many trials use attention control groups, but more studies need to provide more detail about the attention control group for other investigators to evaluate the rigor of the method.2 The aims of this paper were to describe the development of a novel attention control using TED Talks and provide suggestions for the design of an attention control condition.
Indications for Attention Control
In an experimental study, investigators considering an attention control condition should consider several factors. There should be an acknowledgment that attention or some aspect of social support is a factor that may impact the outcome of interest.5,6 Pagoto et al suggested that previous data should support the impact of attention on the outcome before including attention control in the study design.5 However, if there is a need for more research in the area of interest, primarily when the outcome of interest measures a physical characteristic rather than a behavioral symptom, an attention control may be unnecessary or even problematic. For example, if a control group participant believes that a complementary therapy may be superior to medication, they may discontinue the medication without the clinician’s knowledge. Lindquist importantly suggested that the design should be driven by a conceptual model, where attention is identified as a confounding variable.6 The model will drive the design, identifying the proposed therapeutic components and mechanisms contributing to the treatment effects.7 Importantly, the benefit of the attention control condition should be independent of the outcome of interest.2 Once the conceptual model has been developed and there is a clear indication to include an attention control condition, careful consideration should be given to the actual control condition design.1
Designing an Attention Control Condition
Informed by the study hypothesis, the components of an attention control condition must include attention, social support, and expectancy.3,6 Freedland et al8 proposed 2 main ingredients for designing an attention control intervention: clinical attention and the expectation of a therapeutic benefit. In addition, the treatment and attention control groups are designed as structurally equivalent (ie, time, attention, setting) to obtain accurate effect sizes when analyzing the data.9 This includes equal time and attention given to both groups. The qualifications of the staff providing the intervention should also be equivalent if there is more than 1 staff assigned to the experimental and control groups.6 Furthermore, the attention control group activity must not be related to the treatment intervention to avoid contamination.1,6 Contamination in randomized controlled trials (RCTs) occurs when participants who were not intended to, receive an intervention inadvertently.10 Contamination of control participants has at least 2 risks. First, it reduces the point estimate of an intervention’s effectiveness. Second, this may lead to a type II error, rejecting an effective intervention because the observed effect size was not statistically or clinically significant.11
Contamination is especially salient in experiments using behavioral interventions. Therefore, the attention control activity should be interesting and enjoyable, which will help decrease attrition.1 LaFave also recommends the use of choice in attention control activities.1 Choice also addresses the ethical consideration of not burdening the participant with tasks that they view as having little or no value.6 This seems obvious, but a final consideration is to monitor and collect data on the control group.2 This ensures the control group participant receives an equal dose of attention as the treatment group participants. Given the pros and cons of using attention control conditions in RCTs, this paper aimed to (1) describe the development of a novel attention control condition using TED Talks and (2) provide recommendations for designing an attention control condition.
For our trial of acupuncture for the symptoms of stable angina, we adopted a thorough decision-making process examining a variety of potential control conditions, including double-blind needles, single-blind needles, cross-over designs, and attention control designs. Double-blind needles are designed to blind the participant as well as the acupuncturist.12,13 There are 2 needles; the first penetrates the skin, and the second skin-touch placebo needle touches but does not penetrate the skin. Single-blind needles are designed to blind the participant but not the acupuncturist.14 It has not yet been determined whether both double-blind and single-blind needles may contain the potential for participant and/or acupuncture bias. Cross-over designs may not be suitable for acupuncture studies since the length of a suitable washout period has yet to be determined.15 Hence, we sought to design an effective attention control condition for an RCT of acupuncture.
Organizing Framework
Attention control research designs have been used more frequently in intervention trials that measure behavioral symptoms than physical symptoms.7 However, any clinical trial involving interactions with an investigator has the potential to confound the measurement of the study outcomes. In our RCT, an attention control design was chosen to control the nonspecific confounders that may accompany acupuncture treatment. These include the time and attention given by research staff and acupuncturists, the setting, compensation, expectancy, and completion of the study instruments. Controlling for these variables will help isolate the effect of acupuncture intervention on the primary outcomes of pain and quality of life. Choosing an interesting control activity that is similar in time, feasible in our research setting, and involves attention from the study personnel can be challenging. TED Talks are a readily available resource for interesting topics that can serve as an attention control activity. The use of TED Talks as the control condition was chosen because of the variety of non-pain-related topics, availability, and brevity of each Talk. The concepts of interest included the interventions (experimental and control), potential confounding variables, and outcome variables. The organizing framework used to guide our attention control design is depicted in Figure 1.
Figure 1.
Organizing framework for EASE trial.
Results
Aim 1: The Development of a Novel Attention Control Condition Using TED Talks
Our pilot study used videos from the PBS “Nova” television series. We received some feedback from participants that the videos were long, and some could have been more interesting.16 Therefore, we decided to review TED Talks, which are shorter and vary in length and topics. Two authors (H.A.D. and L.R.) logged on to TED Talks independently.17 Each chose approximately 25 Talks for discussion. Available categories were education, psychology, leadership, artificial intelligence, sleep, mental health, business, motivation, communication, personal growth, sports, health, and language. Search criteria were any topics thought to be of general interest, including health, but excluded any topics related to pain or heart disease. Our goal was to prevent bias related to the intervention, which would be a threat to the internal validity of the study findings. We searched “Newest Talks” first to offer contemporary topics to study participants. A total of 48 TED Talks were reviewed.
After reviewing all TED Talks, 24 talks were selected for inclusion in the control condition (see the Supplemental Table). The talks ranged from 6.0 to 19 minutes and 8 seconds. The total length of all 24 talks was 4 hours and 12 minutes (mean 10 minutes, 30 seconds). The topics included education, technology, health, space, human behavior, sanitation, parenting, cliff climbing, public speaking, happiness, obituaries, procrastination, and age diversity.
After selecting the titles, we sought to develop 3 to 4 simple questions reflecting the content of the Talk. The goal was to see how closely participants were engaging in the content of each Talk. We did not frame the questions as a quiz for participants because our goal was not to test knowledge or recall but to pay attention to the talks. Responses ranged from true/false to 4 multiple-choice answers. For example, the TED Talk “Your social media ‘likes’ expose more than you think” included the following questions: (1) Which company sent a family an ad for diapers, predicting that a girl was pregnant? (2) What Facebook page likes are most suggestive of high intelligence? And (3) what does the author say is the product being sold by social media companies?
The study investigators felt that the participants randomized to the TED Talk attention control group needed to come to the research suite to view the videos. As noted in our conceptual model, the setting and the attention of the research personnel are important components of the design. Hence, all participants in the control group come to the same research suite as the participants in the acupuncture group to view videos with a member of the research staff. The staff then remain in the room while the participant completes the questions. Since coming to campus is a burden for participants living in a congested urban area, they are only asked to come in once per week. Therefore, the time for each attention control session needed to be approximately double that of the twice-weekly acupuncture sessions for those randomized to the experimental group.
Study data, including TED Talk questions, were collected and managed using REDCap (Research Electronic Data Capture).18,19 REDCap is a secure, web-based software platform designed to support data capture for research studies. A TED Talk questionnaire form was designed in the REDCap database for completion at 5 time points for the 5-week intervention period. The videos were embedded in the form using YouTube links, and the questions associated with each Talk were added and appeared after each video. All participants met in person with the research coordinator who engaged minimally with them except for the completion of data collection, like the acupuncture group. Additional questionnaires and research procedures were also completed at the first and last sessions. We initially planned for participants to select which of the 24 videos they would like to watch during each in-person session. In weighing the pros and cons of this strategy, we realized that this was logistically difficult, as the total number of videos selected had to be equivalent to the amount of time of the acupuncture intervention (~45 minutes for each of 10 sessions). If the participant chose the videos, it is possible they would not have added up to the required time. Therefore, 4 to 5 videos were selected for each session and bundled to create an approximately 1 hour attention control condition. Total video time ranged from 45 to 56 minutes per session.
Aim 2: Suggestions for Designing an Attention Control Condition
Grounded by our pilot study and a comprehensive review of the literature, our evaluation of the pros and cons of various factors associated with attention control designs, and the development of our TED Talks control condition, we compiled suggestions that may aid other investigators in the design and use of an attention control condition. Suggestions include conceptualization and consideration of equivalence, placebo effects, and threats to validity (see Table 1).
Table 1.
Suggestions for Designing an Attention Control Design.
| Item | Suggestion | Pros | Cons |
|---|---|---|---|
|
| |||
| 1 | Employ a conceptual model or organizing framework to guide the selection of an attention control design | • Especially useful in behavioral research • Useful in trials that measure physical symptoms when confounding variables such as expectancy or placebo effect can affect outcomes |
• If no conceptual or organizing is used to ground the control condition, it may overlap with the experimental conditions and contaminate study findings |
| 2 | Use equivalent attention (dose, time, and setting) for the control and experimental groups | • Removes the influence of attention given to participants from the investigators • Increases the validity of the findings • Increases accuracy of effect sizes |
• Not necessary if the intervention does not involve interaction with the research team |
| 3 | Design a control condition that is interesting | • Interesting content is likely to hold the participant’s attention and not feel their time is wasted (more ethical) • Reduces the risk of attrition which is more common in the control group • Interesting content helps the participant focus, so there are less distractions |
• Can be time-consuming to design an interesting control condition • What is interesting to one participant may not be interesting to another • If the control condition is not standardized, the outcomes may be a result of chance |
| 4 | Consider the effects of the intervention beyond social interaction alone | • TED Talks can be interesting and informative while unrelated to the content of the intervention | • Care must be taken to avoid a control condition that becomes another intervention arm due to content overlap |
| 5 | Design a control condition that is independent of the intervention | • Prevents contamination from design overlap with the intervention • Reduces threats to internal validity • Improves point estimate of the intervention’s effectiveness • Reduces chance of a Type II error |
• Control group unnecessary if the intervention can be compared with usual care or evidence-based practice guidelines |
| 6 | Avoid weak control designs | • Strong design will strengthen the validity of study findings • Strong designs may also increase participant interest and retention |
• Weak control designs may make it impossible to draw meaningful conclusions from the study • Can overestimate the effectiveness of an intervention • Threatens a potentially beneficial treatment |
| 7 | Include choice in the design of the control condition | • More likely to hold the participants’ attention • Likely to increase retention • Does not burden participant with low value tasks |
• Choice can contribute to uneven attention defeating the purpose of measuring the effect of attention on study outcomes |
| 8 | Consider expectations of the participants | • Expectations may have a therapeutic benefit | • If expectations are not measured, the investigator cannot be certain that any effect is a result of the control condition alone |
Discussion
Based on our organizing framework, the literature, and findings from our pilot study, we developed the TED Talk control condition for our multisite trial of an acupuncture intervention for stable angina. While we agreed with most of the information found in the literature, we disagreed with some points. We did not agree with or follow the suggestion that the attention control design should result in a therapeutic effect.8 Our goal was not to demonstrate that the attention control had a beneficial effect but to provide the participant with attention equal to the time spent with the experimental group and to control for potential effects of attention on the study outcomes: pain and quality of life. Part of our study protocol includes offering a session of acupuncture to participants randomized to the control group on study completion. Therefore, it is possible that they may develop an expectation of a “reward” after the TED Talks viewing and thus experience a placebo effect.
Bundling multiple TED Talks into units approximately equal to the time it takes to complete 1 acupuncture session enabled us to offer a strong attention control condition with minimal burden to our participants. Bundling also allowed us to standardize the control condition, as the goal was to control for time and not introduce other confounding variables that could account for a change in the outcome variables.
There is little research on whether an attention control group produces a positive expectancy effect or an effect on the targeted outcome.5 The expectancy effect,20 also known as the placebo effect or the Hawthorne effect, refers to the phenomenon where participants’ expectations and beliefs about an intervention influence their perceived outcomes. For our study, we measure participants’ expectancy using the Treatment Expectation Questionnaire (TEX-Q). The TEX-Q is a generic, 35-item, multidimensional self-report scale that measures patients’ expectations of medical and psychological treatments. Items are measured on a 10-point Likert-type scale with zero indicating no expected change.21 We will then conduct statistical analyses in the attention control group to determine whether expectancy influenced pain scores. The paired t-test will allow us to assess whether there are statistically significant changes in angina pain and associated symptoms, thereby determining if the attention or expectancy effects elicited by the TED Talks have any measurable impact on the severity of the symptoms. This method effectively controls interindividual variability, enhancing the sensitivity of the analysis to detect potential effects of the intervention.
The placebo effect, where participants experience improvements due to their belief in the intervention’s potential benefits, can also occur in attention control groups. Participants who receive additional attention may develop heightened expectations that their situation will improve, potentially leading to perceived or actual improvements in their condition. If significant changes in expectancy are observed in the attention control group, it may indicate that the attention itself might be driving some of the effects. Therefore, by assessing the expectancy effect in an attention control group over time, researchers can better understand the role of placebo effects and ensure more accurate interpretations of intervention efficacy in future studies.
Some authors have suggested that a control group is unnecessary to determine behavioral studies’ efficacy. Baskin et al9 stated that no control condition is needed to differentiate components of the intervention if the goal is simply to determine efficacy. Mohr et al3 also suggested that an attention control is unnecessary for early trials (such as Phase II). However, for our intervention trial to reduce physical symptoms of angina, we concluded that there was a need to design a control condition to account for potential confounders, expectancy, and placebo effects.
We intended to offer our acupuncture participants refreshments following each session. We found nothing in the literature about the potential distribution of drinks and snacks to the experimental and control groups. Initially, our discussion focused on keeping acupuncture participants comfortable, but we determined that offering the control group participants the same refreshments would also be necessary. Many of our participants have a long commute to the research suite and will feel more comfortable and stronger with some hydration and nourishment.
Our team developed an attention control condition that met crucial requirements of (1) equivalent Attention to that of the experimental group, (2) a design that can hold the participant’s interest, (3) standardization to avoid potentially confounding outcomes, and (4) the evaluation of expectancy.
Supplementary Material
Supplemental material for this article is available online.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This publication was made possible in part by Grant Number R01 NR020376 from the National Institutes of Health, National Institute for Nursing Research (NINR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NINR. The final peer-reviewed manuscript is subject to the National Institutes of Health Public Access Policy.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics Approval
This study received ethics approval from the UCLA IRB (approval #22–1111) on September 9, 2022. All participants gave written informed consent.
References
- 1.LaFave SE, Granbom M, Cudjoe TKM, Gottsch A, Shorb G, Szanton SL. Attention control group activities and perceived benefit in a trial of a behavioral intervention for older adults. Res Nurs Health. 2019;42(6):476–482. doi: 10.1002/nur.21992 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Aycock DM, Hayat MJ, Helvig A, Dunbar SB, Clark PC. Essential considerations in developing attention control groups in behavioral research. Res Nurs Health. 2018;41(3):320–328. doi: 10.1002/nur.21870 [DOI] [PubMed] [Google Scholar]
- 3.Mohr DC, Spring B, Freedland KE, et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother Psychosom. 2009;78(5):275–284. doi: 10.1159/000228248 [DOI] [PubMed] [Google Scholar]
- 4.Popp L, Schneider S. Attention placebo control in randomized controlled trials of psychosocial interventions: theory and practice. Trials. 2015;16:150. doi: 10.1186/s13063-015-0679-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pagoto SL, McDermott MM, Reed G, et al. Can attention control conditions have detrimental effects on behavioral medicine randomized trials? Psychosom Med. 2013;75(2):137–143. doi: 10.1097/PSY.0b013e3182765dd2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lindquist R, Wyman JF, Talley KM, Findorff MJ, Gross CR. Design of control-group conditions in clinical trials of behavioral interventions. J Nurs Scholarsh. 2007;39(3):214–221. doi: 10.1111/j.1547-5069.2007.00171.x [DOI] [PubMed] [Google Scholar]
- 7.Barkauskas VH, Lusk SL, Eakin BL. Selecting control interventions for clinical outcome studies. West J Nurs Res. 2005;27(3):346–363. doi: 10.1177/0193945904271446 [DOI] [PubMed] [Google Scholar]
- 8.Freedland KE, Mohr DC, Davidson KW, Schwartz JE. Usual and unusual care: existing practice control groups in randomized controlled trials of behavioral interventions. Psychosom Med. 2011;73(4):323–335. doi: 10.1097/PSY.0b013e318218e1fb [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Baskin TW, Tierney SC, Minami T, Wampold BE. Establishing specificity in psychotherapy: a meta-analysis of structural equivalence of placebo controls. J Consult Clin Psychol. 2003;71(6):973–979. doi: 10.1037/0022-006X.71.6.973 [DOI] [PubMed] [Google Scholar]
- 10.Keogh-Brown MR, Bachmann MO, Shepstone L, et al. Contamination in trials of educational interventions. Health Technol Assess. 2007;11(43):iii, ix-107. doi: 10.3310/hta11430 [DOI] [PubMed] [Google Scholar]
- 11.Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322(7282):355–357. doi: 10.1136/bmj.322.7282.355 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Schlaeger JM, Takakura N, Yajima H, et al. Double-blind acupuncture needles: a multi-needle, multi-session randomized feasibility study. Pilot Feasibility Stud. 2018;4:72. doi: 10.1186/s40814-018-0265-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Steffen AD, Burke LA, Pauls HA, et al. Double-blinding of an acupuncture randomized controlled trial optimized with clinical translational science award resources. Clin Trials. 2020;17(5):545–551. doi: 10.1177/1740774520934910 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lim SM. Validation of a new sham acupuncture needle for double-blind trials: a study protocol. Perspect Integr Med 2024;3(1):57–60. doi: 10.56986/pim.2024.02.008 [DOI] [Google Scholar]
- 15.Mills EJ, Chan AW, Wu P, Vail A, Guyatt GH, Altman DG. Design, analysis, and presentation of crossover trials. Trials. 2009;10:27. doi: 10.1186/1745-6215-10-27 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.DeVon HA, Uwizeye G, Cai HY, et al. Feasibility and preliminary efficacy of acupuncture for angina in an underserved diverse population. Acupunct Med. 2022;40(2):152–159. doi: 10.1177/09645284211055754 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.TED. Ted talks: Discover ideas worth spreading. TED. Accessed October 21, 2024. https://www.ted.com/talks [Google Scholar]
- 18.Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Pardo-Cabello AJ, Manzano-Gamero V, Puche-Canas E. Placebo: a brief updated review. Naunyn Schmiedebergs Arch Pharmacol. 2022;395(11):1343–1356. doi: 10.1007/s00210-022-02280-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Alberts J, Löwe B, Glahn MA, et al. Development of the generic, multidimensional Treatment Expectation Questionnaire (TEX-Q) through systematic literature review, expert surveys and qualitative interviews. BMJ Open. 2020;10(8):e036169. doi: 10.1136/bmjopen-2019-036169 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.