Abstract
Objective
Behavioral weight management trials are traditionally conducted in‐person. The COVID‐19 shutdown halted in‐person operations, forcing investigators to develop new methods for remote treatment and assessment delivery without additional funding for website development or remote equipment. This study examined the feasibility and acceptability of remote procedures from an ongoing weight management trial impacted by COVID‐19.
Methods
Using a quasi‐experimental longitudinal design, in‐person (pre‐COVID) and remote (COVID) treatment and assessment procedures were used. Attendance at in‐person versus remote (videoconference) treatment sessions was compared. Acceptability of treatment modalities (in‐person vs. remote) was examined via self‐report. Validity and reliability were assessed on bathroom scales. Attendance at remote (videoconference + mailed, scales) versus in‐person assessment sessions was compared. Finally, exploratory analyses were conducted to determine whether participant characteristics moderated the effects.
Results
Remote treatment attendance was significantly better than in‐person. Overall, there was no significant difference in modality preference. However, Hispanic (vs. non‐Hispanic) individuals had greater preference for remote options and attended more remote treatment sessions. Bathroom scales demonstrated excellent validity and reliability. Adherence to remote and in‐person assessment sessions was similar.
Conclusions
COVID‐19 has provided an opportunity to rethink how we conduct research. Results herein establish an evidence‐base to support a paradigm shift to remote clinical trial procedures. Such a shift may enhance diversity in clinical trials.
Keywords: clinical trials, ethnicity, research design, treatment
COVID‐19 has provided an unprecedented opportunity to rethink how we conduct research. Results from this study establish an evidence‐base to support a paradigm shift to remote clinical trial procedures for behavioral weight management using widely available technology. Such a shift may enhance diversity in our clinical trials and generalizability of treatment results.Online behavioral weight loss coach
Behavioral weight management trials are traditionally conducted in‐person in academic research settings. 1 In March 2020, the COVID‐19 shutdown forced investigators to abruptly halt in‐person trials and pivot to remote interventions and assessments, while maintaining trial integrity and data quality. Little prior research was available to guide solutions within budgetary and time constraints (e.g., no funding or time to build intervention websites or apps). Moreover, while costly behavioral weight loss websites have been built and shown to produce clinically meaningful outcomes, 2 , 3 the COVID‐19 pandemic further necessitated online intervention. However, research on best‐practices for how to deliver treatment without building expensive websites is sparse. To our knowledge, only one study, a preliminary investigation, has examined the feasibility of delivering adult behavioral weight loss treatment via videoconference and results show promising attendance rates. 4 In addition to delivering treatment remotely, given the unexpected nature of the pandemic, investigators were confronted with how best to accurately obtain remote objective weight data without funds to purchase validated WiFi/Bluetooth enabled e‐scales (cost $50–80). 5 , 6 Identifying feasible and inexpensive methods for remote treatment delivery and valid and reliable remote measurement of weight is paramount as COVID‐19 challenges remain and new preferences for remote options have been formed. 7
This investigation aimed to create an evidence‐base for inexpensive remote clinical trial methods in behavioral weight management using data from an ongoing weight loss maintenance trial. The COVID‐19 shutdown occurred during the trial, necessitating transition from in‐person to remote procedures. Treatment attendance collected from in‐person group sessions prior to shutdown was compared to remote group sessions delivered via videoconference following shutdown. Participants who transitioned from in‐person to remote treatment due to the pandemic reported the acceptability of delivery modalities. In addition, reliability and validity data were collected on inexpensive (∼$25) bathroom scales, and adherence to remote assessment procedures (videoconference weigh‐in with mailed scale) was examined.
1. METHODS
1.1. Overview
Data were from the weight loss maintenance phase of an ongoing two‐phase trial (Phase I: weight loss; Phase 2: maintenance; NCT03396653). Participants (age 18–75 years, BMI 25–50 kg/m2) were recruited for Phase I (16‐week online weight loss program 8 ) via mass mailings and electronic media. Exclusion criteria were inability to walk two blocks without stopping, bariatric surgery, recent ≥5% weight loss, pregnancy, no Internet access, and medical condition that jeopardizes safe participation. Participants who achieved a ≥5% weight loss in Phase I were eligible for Phase II and randomized to 18‐month of either a peer‐ or professional‐delivered intervention. The trial is ongoing; Cohorts 1–3 (n = 160) have completed the 18‐month treatment. Given the time‐sensitive nature of this paper, data are reported on these cohorts only with a sole focus on in‐person versus remote trial procedures.
1.2. In‐person procedures
Cohort 1 completed the 18‐month weight loss maintenance treatment in‐person prior to shutdown (In‐Person; N = 50) and is, thus, the comparator. Cohort 1 included 24 group face‐to‐face maintenance sessions over 18 months. Weight assessment occurred in‐person at baseline and at months 6, 12, and 18 using a research‐grade Tanita scale (Model WB800H) and gold‐standard weigh‐in procedures (one layer of clothing, removal of shoes and heavy items ‐ e.g., wallet, jewelry, keys).
1.3. Remote procedures
Cohorts 2 and 3 (Remote; N = 110) transitioned to remote procedures partway through the maintenance intervention (C2: month 13; C3: month 5) due to COVID‐19. Participants were informed of the transition via email. Zoom and WebEx were used for session delivery. Participants were emailed a link to join the meeting and instructions were provided for audio and video connection. Those uncomfortable with the platform were offered a “trial‐run” prior to the first online group session. No participants discontinued due to the transition. Group treatment was delivered via videoconference by intervention staff. Lesson handouts were sent via email. Virtual group participation was fostered by encouraging participants to use the raise hand function to speak and/or use the chat. For privacy, participants entered the virtual room with first name, last initial only. A staff member helped leaders manage technical aspects of virtual sessions (attendance, recording, troubleshooting).
It was not economically feasible to purchase e‐scales ($50–80); thus, we report on the validity and reliability of inexpensive digital bathroom scales (HealthOMeter: HDM171DQ60, $25). Validity and reliability measurements were taken at two timepoints: before scales (n = 151) were mailed and after they were returned from participants (n = 110) to ensure calibration was not impacted during shipping. Note, validity and reliability were evaluated on n = 151 scales (and not n = 110 scales) prior to mailing as these scales were being evaluated and prepared for subsequent cohorts; all available data (n = 151 scales before mailing; n = 110 returned scales) are reported on in the present study. At both timepoints, validity was assessed using a 15‐lb calibration weight. The 15‐pound weight was placed first on a research‐grade Tanita scale and then on each bathroom scale twice for both validity and test‐retest reliability purposes; bathroom scale readouts were recorded to the nearest tenth. Scales were then mailed to participants and videoconference assessments were conducted via WebEx or Zoom (months 6, 12, and 18). A script was used to ensure uniformity. Using video capability, research staff visually confirmed the scale was placed on a hard, flat surface and participants were in one layer of clothing (no shoes, heavy jewelry, etc.). The participant was then asked to step onto the center of the scale and show the assessor the scale's read‐out. Consistent with in‐person weigh‐ins, three measurements were taken, and, if discrepant, the process was repeated until three identical, consecutive weights were obtained. These procedures took ∼5 min.
1.3.1. Measures
Masked staff conducted in‐person and virtual assessments. Participants were compensated $25 for completing 6‐ and 12‐month assessments and $50 for completing the 18‐month assessment.
1.3.2. Demographics
Participants reported sex, age, race, ethnicity, marital status, and education.
1.3.3. Session attendance
Treatment and assessment attendance were measured by staff at each in‐person or videoconference session.
1.3.4. Treatment preferences
Participants who experienced both in‐person and remote meetings due to the shutdown were asked to report treatment modality preferences assuming COVID‐19 is no longer an issue (“Assuming COVID‐19 is not an issue, which types of meetings would you prefer?” In‐person, virtual/no preference) and engagement with digital versus paper lessons (“Which lesson material is more engaging?” Hardcopy, digital/no preference) at their final assessment (month 18). Given that Cohort 1 had no experience with both remote and in‐person treatment, they were not asked these questions.
1.3.5. Bathroom scale validity & reliability
Validity and reliability of bathroom scales were assessed as detailed above. Validity was assessed by examining difference scores between scale readouts and the 15‐pound weight. Internal consistency and test‐retest reliability were assessed on scale read‐outs via Cronbach's alpha and examining correlations between the two scale readouts.
1.4. Statistical analyses
Demographic characteristics and differences between Remote versus In‐Person participants were examined using t‐tests or chi‐square tests for continuous or categorical variables, respectively. Given that adherence decreases with time, treatment and assessment attendance adherence was compared between In‐Person versus Remote participants over the same timeframes; that is, percentage of sessions attended and assessments completed were compared for the two groups from sessions 13–24 and at the 6‐, 12‐, and 18‐month assessment visits, respectively. Differences in treatment preferences were examined with chi‐square tests. Given the lack of pragmatic difference between “virtual/digital” and “no preference” selections, those selections were collapsed and compared to “in‐person” or “hardcopy.” Moderation was examined using Analyses of Variance for categorical moderators (sex, race, ethnicity, marital status, education) and session attendance and regression analyses for continuous moderators (age, BMI) and session attendance and treatment preferences. Simple descriptives (mean, median, mode, and min/max) were conducted for bathroom scale validity, and Cronbach's alpha and correlations were used for reliability.
2. RESULTS
2.1. Participant characteristics
Participants (N = 160) in the weight loss maintenance trial were majority female, non‐Hispanic White, and married/partnered (Table 1). In‐Person versus Remote participants differed on education, marital status, ethnicity, and race (p's < 0.04; Table 1).
TABLE 1.
Participant characteristics
| Total sample % or M (SD) (n = 160) | In‐person % or M (SD) (n = 50) | Remote % or M (SD) (n = 110) | In‐person versus Remote (p‐value) | |
|---|---|---|---|---|
| Sex | ||||
| Female | 81.3 | 76.0 | 83.6 | 0.25 |
| Male | 18.7 | 24.0 | 16.4 | |
| Age | 53.4 (10.7) | 53.9 (8.8) | 53.2 (11.5) | 0.68 |
| Race/ethnicity | ||||
| American Indian/Alaskan Native | 1.3 | 2.0 | 0.9 | 0.57 |
| Asian | 3.1 | 6.0 | 1.8 | 0.16 |
| Black/African American | 13.8 | 22.0 | 10.0 | 0.04 |
| White | 78.8 | 66.0 | 88.2 | 0.001 |
| More than one race | 4.4 | 8.0 | 3.6 | 0.24 |
| Hispanic/Latino | 10.0 | 25.0 | 5.5 | 0.004 |
| Education | ||||
| College degree or higher | 76.9 | 62.0 | 83.6 | 0.003 |
| Married/partnered | 69.4 | 58.0 | 74.5 | 0.04 |
| BMI at start of phase I (kg/m2) | 34.4 (5.6) | |||
| Phase I weight loss (%) | 10.0 (3.6) | |||
| BMI at start of phase 2 (kg/m2) | 31.0 (5.3) | |||
2.2. Treatment session attendance & treatment preferences
Treatment attendance was significantly better with remote versus in‐person delivery (65.8% vs. 49.8%; p = 0.02). Hispanic ethnicity moderated this effect (p = 0.02); compared to non‐Hispanic individuals, Hispanic individuals attended more treatment sessions when delivered remotely (non‐Hispanic: 59.4% vs. 56.0%; Hispanic: 94.4% vs. 13.9%). Overall, there was no difference in modality preference (49.2% vs. 50.8%, p = 0.90). However, Hispanic ethnicity and single marital status were associated with stronger preference for remote options (75.0% of Hispanics, 15.3% non‐Hispanics; 73.3% single, 41.7% married; p's < 0.03). While preference for intervention material was not significant (60.3% digital options vs. 39.7% paper, p = 0.10); single participants preferred digital options compared to married/partnered participants (86.7% vs. 52.1%, p = 0.02). Remaining moderation effects were non‐significant (p > 0.34).
2.3. Scale reliability and validity
Bathroom scales demonstrated excellent validity and reliability (Table 2). Prior to being sent to participants and after being returned, scale read‐outs were nearly identical to the 15lb calibration weight (Pre: 14.9 ± 0.1lbs; Post: 15.0 ± 0.2lbs) with little variability in readouts (pre range = 1.4lbs; post range = 1.6lbs). Reliability was also excellent at both timepoints (e.g., α = 0.99, r = 0.98).
TABLE 2.
Bathroom scale validity and reliability data before being mailed to participants and after being returned from participants
| Pre | Post | |
|---|---|---|
| Validity: 15‐lb weight measurement | ||
| Mean (SD) (lbs) | 14.9 (0.1) | 15.0 (0.2) |
| Median (lbs) | 15.0 | 15.0 |
| Mode (lbs) | 15.0 | 15.0 |
| Min (lbs) | 14.2 | 14.4 |
| Max (lbs) | 15.6 | 16.0 |
| Reliability | ||
| Internal consistency (α) | 0.99 | 0.99 |
| Test‐retest (r) | 0.98 | 0.99 |
2.4. Assessment attendance
Assessment attendance was high and similar to remote and in‐person sessions (Month 6: Remote 95.4%, In‐Person 96.8%; Month 12: Remote 93.8%, In‐Person 86.3%, Month 18: Remote 93.6%, In‐Person 92.0%, p's > 0.16). There were no moderator effects for assessment attendance (p's > 0.90).
3. DISCUSSION
This study collected objective data on treatment and assessment attendance both before (in‐person) and during (remote) the COVID‐19 pandemic, allowing for an unprecedented comparison of adherence between in‐person and remote treatment and assessment procedures that are feasible, reliable, valid, inexpensive, and easy to implement in a randomized trial for behavioral weight management. Remote procedures were demonstrated to be feasible, acceptable, and rigorous. In fact, remote treatment session attendance was better than in‐person. Further, preferences for remote delivery options were similar to in‐person, which is striking given that participants were recruited and chose to enroll in an in‐person weight loss maintenance treatment. Finally, assessment attendance was similar between remote and in‐person sessions, with inexpensive bathroom scales demonstrating excellent validity and reliability. These results suggest that remote procedures are a rigorous and cost‐effective strategy for conducting behavioral weight management trials and may be a particularly effective approach for improving treatment and assessment adherence in long‐term weight loss maintenance trials.
Excellent remote intervention attendance has been found in prior studies, 2 , 4 perhaps due to elimination of treatment barriers. 1 Harvey‐Berino and colleagues (2010) developed a behavioral weight loss website that included group chat for session delivery; results showed that attendance at website sessions was similar to in‐person. Our study adds to this literature by demonstrating that free, ubiquitous videoconference platforms produce better attendance than in‐person offerings. Further, in the present study, better remote treatment session attendance and a stronger preference for remote sessions was most pronounced in Hispanic and single individuals, two groups that tend to have reduced incomes compared to their peers, 9 , 10 lending some support to the notion that remote delivery eliminates financial barriers. Given known health disparities, 11 , 12 and lack of racial, ethnic, and economic diversity among clinical trial participants, 11 it is critical that online interventions are accessible to all. Such an approach will eliminate barriers associated with in‐person treatment thereby improving investigators' and practitioners' ability to reach those with transportation difficulties, childcare needs, and/or time constraints due to multiple jobs and life demands. 13 Further, our acceptability data are consistent with previous findings that patients prefer remote options and wish to continue using these modalities after pandemic restrictions are eased. 7 These findings, coupled with data suggesting greater access to mobile technology among racial and ethnic minorities, 13 suggest more work is needed to determine how best to effectively leverage technology in individuals from diverse backgrounds to improve health. Remote assessment adherence was also excellent and no different from in‐person. Further, inexpensive bathroom scales were valid and reliable, with substantially smaller error rates than from more expensive e‐scales. 5 , 6 This minimization of error variance is critical for maintaining statistical power in remote clinical trials moving forward.
COVID‐19 has shifted how we work, socialize, and live, as well as provided an opportunity to rethink how we conduct research. Data from this study provide a foundation for a paradigm shift from in‐person to remote clinical trial procedures for the treatment of obesity. Remote trial procedures may also allow for recruitment without geographical limitations, eliminating treatment barriers and facilitating recruitment of underrepresented populations. Such a shift may enhance diversity in clinical trials, allowing us to reach and generalize our findings to those in greatest need.
CONFLICT OF INTEREST
The authors declare no conflicts of interest.
AUTHOR CONTRIBUTIONS
Conceptualization: Leahey & Gorin; Data curation: Pham, Denmat, Jenkins, Harris‐Starling, Gilder; Analyses: Leahey, Pham; Funding: Leahey, Gorin; Methodology: Leahey; Original draft: Leahey, Gorin, Denmat, Jenkins; Review & Editing: Pham, Denmat, Jenkins, Harris‐Starling, Gilder.
ACKNOWLEDGMENT
This research was supported by the National Institutes of Health through an award administered by the National Institute of Diabetes and Digestive and Kidney Diseases (R01DK111232). The sponsor had no role in the study design or in the writing of this report. We would like to acknowledge our participants; without them this research would not be possible.
Leahey TM, Pham J, Denmat Z, et al. Feasibility of online behavioral clinical trials: The future of weight management randomized clinical trials? Obes Sci Pract. 2022;8(6):811‐815. 10.1002/osp4.615
REFERENCES
- 1. Wadden TA, Tronieri JS, Butryn ML. Lifestyle modification approaches for the treatment of obesity in adults. Am Psychol. 2020;75(2):235‐251. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Harvey‐Berino J, West D, Krukowski R, et al. Internet delivered behavioral obesity treatment. Prev Med. 2010;51(2):123‐128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Tate DF, Jackvony EH, Wing RR. Effects of Internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial. JAMA. 2003;289(14):1833‐1836. [DOI] [PubMed] [Google Scholar]
- 4. Hu L, Illiano P, Pompeii ML, et al. Challenges of conducting a remote behavioral weight loss study: lessons learned and a practical guide. Contemp Clin Trials. 2021;108:106522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Ross KM, Wing RR. Concordance of in‐home "smart" scale measurement with body weight measured in‐person. Obes Sci Pract. 2016;2(2):224‐228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Pebley K, Klesges RC, Talcott GW, Mehmet K, Krukowski RA. Measurement equivalence of E‐scale and in‐person clinic weights. Obesity. 2019;27(7):1107‐1114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Lohnberg JA, Salcido L, Frayne S, Mahtani N, Bates C, Hauser ME. Rapid conversion to virtual obesity care in COVID‐19: impact on patient care, interdisciplinary collaboration, and training. Obes Sci Pract. 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Leahey TM, Gorin AA, Wyckoff E, et al. Episodic future thinking, delay discounting, and exercise during weight loss maintenance: the PACE trial. Health Psychol 2020;39(9):796‐805. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Mora M. The Hispanic‐White Wage Gap Has Remained Wide and Relatively Steady. Economic Policy Institute; 2018. [Google Scholar]
- 10. Pew Research Center . Rising Share of U.S. Adults Are Living without a Spouse or Partner; 2021. [Google Scholar]
- 11. Brown JC, Carson TL, Thompson HJ, Agurs‐Collins T. The triple health threat of diabetes, obesity, and cancer‐epidemiology, disparities, mechanisms, and interventions. Obesity. 2021;29(6):954‐959. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Hales CM, Carroll MD, Fryar CD, Ogden CL. Prevalence of obesity and severe obesity among adults: United States, 2017‐2018. NCHS Data Brief. 2020(360):1‐8. [PubMed] [Google Scholar]
- 13. Perrin A. Smartphones Help Blacks, Hispanics Bridge Some ‐ but Not All ‐ Digital Gaps with Whites. Pew Research Center; 2019. [Google Scholar]