How accurate are recalls of self-weighing frequency? Data from a 24-month randomized trial

Abstract

Background

Self-weighing is an important component of self-monitoring during weight loss. However, methods of measuring self-weighing frequency need to be validated. This analysis compared self-reported to objective weighing frequency.

Methods

Data came from a 24-month RCT. Participants received 12 months of a behavioral weight loss program and were randomly assigned to: 1) daily self-weighing, 2) weekly weighing, or 3) no weighing (excluded from analysis). Objective weighing frequency was measured by Wi-Fi enabled scales and self-reported weighing frequency was assessed every 6 months by questionnaire. Objective weights were categorized to match the scale of the self-report measure.

Results

At 12 months, there was 80.8% agreement between self-reported and objective weighing frequency (weighted kappa = 0.67, p < 0.001). At 24 months, agreement decreased to 48.5% (kappa = 0.27, p < 0.001). At both time points where disagreements occurred, self-reported frequencies were generally greater than objectively assessed weighing. Both self-reported and objectively assessed weighing frequency were associated with weight loss at 12 and 24 months (p’s < 0.001).

Conclusions

Self-reported weighing frequency is modestly correlated with objective weighing frequency; however, both are associated with weight change over time. Objective assessment of weighing frequency should be used to avoid overestimating actual frequency.

Introduction

Self-monitoring is a central recommendation for behavioral weight loss interventions [1, 2]. In many cases, this includes regularly recording dietary intake, physical activity, and body weight [3]. While self-monitoring of physical activity and diet are recommended each day, or multiple times per day, the recommendations for self-weighing (hereafter referred to as “weighing”) are less clear. Many interventions provide recommendations for weighing, although these vary and are often not included in published descriptions of interventions. As an example, weighing is mentioned in the description of the Look AHEAD trial but the frequency of weighing is not explicit [4]. In the publicly available intervention materials, weighing frequency is left up to the individual with the instructions “Weigh yourself at the same time of day. We think you should weigh yourself at least once per week on the same day each week. More often is fine if you’d like” [5].

Greater frequency of self-monitoring weight has been associated with greater weight loss across multiple studies [e.g., 6, 7, 8]. A systematic review of these studies concluded that although the association between weighing and weight loss is present in most studies, the effect sizes and measurement of weighing frequency are inconsistent [9]. Due to these inconsistencies, it is currently unknown which frequency of weighing is required to achieve significant weight benefits, and therefore a precise recommendation for weighing frequency to produce the greatest weight loss outcomes cannot be made.

One challenge to determining the appropriate recommendation for weighing is in measuring how often people weigh themselves when given specific weighing recommendations. Most studies relating weighing to weight loss have relied on self-report of weighing frequency (i.e., 9 of the 12 cited in Zheng et al., 2015). This is potentially problematic for a number of reasons. First, the recall of weighing frequency often occurs after significant time has passed, typically included as part of assessments three to six months after the start of a weight loss attempt. Second, weighing habits may change over time, so reports of average frequencies may not be meaningful. The challenges of using self-reported behaviors in other areas of weight loss, including diet [10] and physical activity [11] have been well documented.

Weighing frequency can be assessed more accurately by utilizing Internet connected scales that automatically record and transmit weights and frequency data to servers. These scales have been used in prior weight loss programs [12–14] but none of these studies have compared objective weighing frequency to self-reported frequency. Because of the additional cost of these scales (approximately $100 per scale), it is important to know for measurement purposes whether these scales are needed or if self-reported weighing frequency is sufficient to assess the behavior accurately.

The purpose of this analysis was to compare self-reported weighing frequency to objective measures of weighing frequency over 24 months. First, we examined the degree of agreement between self-reported and objective measures. Second, we examined the relationships between self-reported and objective weighing frequency and weight loss to examine the measures’ validity over time.

Methods

Data for this analysis came from the Tracking Study: a randomized controlled trial evaluating the effect of self-weighing frequency on weight loss with randomized comparisons of daily, weekly, and no weighing conditions (PI: Linde; R01DK093586; NCT01646086). All participants in the Tracking Study received 32 sessions of the same behavioral weight loss program delivered via six months of weekly group sessions and six months of sessions tapered to bi-weekly and then monthly sessions over the course of one year. All groups received the same recommendations for changing diet and exercise. A broader description of the intervention has been published elsewhere [15].

Participants randomized to daily and weekly self-weighing conditions were provided with Withings Wi-Fi enabled body weight scales (Model WS-30) and instructions for weighing at home. Both groups were encouraged to weigh themselves consistently at the same time of day (preferably upon waking) in either light clothing or without clothes. The daily weighing group was instructed to weigh themselves each day while the weekly weighing group was instructed to weigh themselves once per week on the same self-selected day. During the group intervention sessions, participants were encouraged to adhere to their assigned weighing frequency. The no weighing group was encouraged to use alternative methods to monitor progress (e.g., clothing size, feedback from others, energy levels) and were discouraged from weighing themselves. The no weighing group did not receive the Wi-Fi enabled scales until they completed the study at 24 months. Because the goal of this analysis was to compare objective to self-reported weighing frequency, data from the no weighing condition were not included here.

Participants and recruitment

Participants for this study were recruited from a mid-sized Midwestern city in 2012 to 2013 using a combination of print advertisements, radio advertisements, flyers, and at community events. To be eligible for the study, participants needed to have a BMI 25-40 kg/m², be 18-64 years old, have wireless internet at home, and be free of significant health conditions. Participants were excluded if they had significant depressive symptoms, binge eating disorder, or a history of an eating disorder. A full listing of inclusion criteria are available elsewhere [15]. All participants provided informed consent prior to participating in the study and all procedures for this study were reviewed and approved by the University of Minnesota Institutional Review Board.

Measures

Self-reported weighing frequency was assessed at baseline, 12, and 24 months via the question: “How often do you weigh yourself?” Response options were: never, once a year or less, every couple of months, once a month, once a week, once a day, or more than once a day [6]. Due to low response for the least and most frequent categories, these responses were recategorized as Daily, Weekly, or Less than Weekly.

Objective weighing frequency was measured using data collected from the Withings scales. Weights were automatically transmitted from the scale to the Withings website. Research staff downloaded data for study participants each week of the intervention (49 weeks for cohort 1; 50 weeks for cohort 2; 51 weeks for cohort 3) and each week of the follow-up period (52 weeks for cohort 1; 50 weeks for cohort 2; 48 weeks for cohort 3). Weights that were recorded within a 15-minute period were counted as one weighing event. The scale automatically detected whether weights were from the study participants (i.e., the registered user) or from other users. Only the weights from study participants were transmitted to the Withings website and were downloaded by study staff.

Objective weighing frequency was classified to match the self-report categories of Daily, Weekly, or Less than Weekly, based on approximately 70% or more of expected weighing frequency. This level of adherence was selected in order to allow for some missing days (e.g., unreported data capture errors, participants traveling without access to the Withings scale) while still indicating a high level of consistency with the self-report categories. With an average of 50 weeks between assessments, complete adherence to daily weighing would have yielded an average of 350 weights, therefore participants with 245 weights or greater were classified as “Daily”. Similarly, as there were 50 weeks between assessments, those with 35 to 244 weights were classified as “Weekly”, and less than 35 weights were classified as “Less than Weekly”. Other cut points were also considered and yielded similar results (see Supplemental File).

Data Analysis

Analyses were conducted using SAS 9.4. Missing data were handled using pairwise deletion where analyses used all available data for that time point and variables used. Sample sizes (n) are provided separately for each analysis. Agreement between self-reported and objective weighing frequency were assessed using weighted kappa statistics [16]. A Bland-Altman plot was considered to graphically represent the similarity of the two measures [17]. However, because our data are categorical and therefore have few points on the graph (i.e., 7 possible points), histograms showing the frequency of the difference scores are presented instead [18]. Objective and self-reported weighing frequency were tested as predictors of weight loss using linear regression models, controlling for treatment condition.

Results

Demographic characteristics of the participants are shown in Table 1. Most randomized participants took part in the 12-and 24-month assessments (90.6% and 83.0%, respectively); there was no statistically significant difference in assessment completion rates between study groups at 12 months (p = 0.74) or 24 months (p = 0.57). Participants who took part in the 24-month assessments were older (p = 0.01; Table 1), had a lower starting BMI (p = 0.02), were more likely to be non-Hispanic white (p = 0.04), and were more likely to be married (p = 0.001) than those who did not take part. At baseline, 57.4% (n = 128) of participants reported weighing less than once per week, 34.1% (n = 76) weighed once per week, and 8.5% (n = 19) reported weighing daily.

Table 1.

Baseline Demographic Characteristics by Study Completion Status

	Total Sample	Did not Complete 24-Mo. Assessment	Completed 24-Mo. Assessment	p-value
	n = 223	n = 38	n = 185
	M ± SD	M ± SD	M ± SD
Baseline BMI, kg/m2	33.0 ± 3.6	34.3 ± 3.1	32.8 ± 3.6	0.02
Age, years	46.6 ± 10.2	42.6 ± 11.0	47.5 ± 9.9	0.01
	n (%)	n (%)	n (%)
Women	142 (63.7)	67 (63.2)	118 (63.8)	0.94
Non-Hispanic white	189 (84.8)	28 (73.7)	161 (87.0)	0.04
College degree or higher	147 (65.9)	22 (57.9)	125 (64.6)	0.25
Married/living with partner	160 (71.8)	19 (50.0)	141 (76.2)	0.001
Baseline weighing frequency				0.74
Daily	19 (8.5)	3(7.9)	16 (8.7)
Weekly	76 (34.1)	15 (39.5)	61 (33.0)
Less than weekly	128 (57.4)	20 (52.6)	108 (58.4)

A similar proportion of participants in both the weekly (5.6%, n = 6) and daily (6.1%; n = 7) treatment arms did not use the Wi-Fi scales (p = 0.84); therefore, no objective weights were recorded during the study for these participants and they were excluded from the analysis. An additional four participants in each treatment group changed their Withings account password post-intervention and thus their weighing data during months 12-24 were not available (available sample size: Daily group n = 103; Weekly group n = 99). Forty-seven (23.3%) of participants did not weigh using the Wi-Fi scales during the 12 to 24 month period. These people with included in the analysis as having zero objective weights.

As shown in Figure 1A, between baseline and 12-months, average objective weights per week varied by self-reported weighing frequency assessed at 12 months (F = 23.96; p < 0.001), controlling for treatment group. Each self-reported category was significantly different the others on weighing frequency (all p’s < 0.05). Between the 12- and 24-month assessments (Figure 1B), average objective weights per week also varied by self-reported weighing frequency assessed at 24 months (F = 20.31, p < 0.001). Again, each self-reported category was significantly different from the others on objective weighing frequency (all p’s < 0.05).

Figure 1.

A. Average weights per week by self-reported weighing frequency at 12 months

B. Average weights per week by self-reported weighing frequency at 24 months

Comparison of Self-Report to Objective Self-Weighing

As shown in Table 2, there were matched classification for 80.8% and 48.5% of participants at 12 and 24 months, respectively. The weighted kappa for the agreement between self-reported and objective weighing frequency was 0.67 at 12 months and 0.27 at 24 months; both kappa values were significantly different than zero (p < 0.01). Figure 2 visualizes the percentage of agreement between self-reported and objective weighing frequency. For participants for whom self-report and objective categorization differed, self-report was greater (indicated by a positive difference score; 17.5% of sample). Similarly, at 24 months (Figure 2), 46.1% of the sample had a positive difference score indicating a self-report category greater than the objective category. There were no differences between those who over reported their weighing frequency versus those who correctly reported on age, baseline BMI, gender, race/ethnicity, education, marital status, or baseline weighing frequency (all p’s > 0.05).

Table 2.

Agreement between self-reported weighing frequency and objective weights at 12 and 24 months

		Self-Reported Weighing: 12 Monthsa (n = 182)				Self-Reported Weighing: 24 Monthsb (n = 167)
Objective Weighing Frequency		Monthly or less	Weekly	Daily	Weighted Kappa	Monthly or less	Weekly	Daily	Weighted Kappa
	Monthly or less	3.3	3.3	0.6		18.6	25.8	7.2
	Weekly	2.2	45.6	13.2		6.0	23.4	12.6
	Daily	0.0	0.0	31.9		0.0	0.0	6.6
					0.67***				0.27***

Notes.

^aFor comparison with 12-month self-report, objective weighing covers period from baseline to 12-month assessment.

^bFor comparison with 24-month self-report, objective weighing covers period from 12 month assessment to 24 month assessment. Values are the percent of total sample for that time period.

^***p < 0.001

Figure 2.

Frequency distributions of difference scores between self-reported and objective weighing frequency

Note. Self-reported and objective weighing frequency values are 1 = less than weekly; 2 = weekly; 3 = daily. Difference scores are self-report minus objective. Positive scores indicate self-report greater than objective frequency.

Comparison of weighing frequency and weight loss

To test the predictive validity of the self-reported and objective measures of weighing, we compared the measures’ associations with weight change during the subsequent assessment. For example, objective weighing frequency between baseline and 12 months used to predict weight change at 12 months. Self-reported weighing frequency assessed at 12 months was used to predict weight change between baseline and 12 months. As shown in Table 3, both self-reported and objective self-weighing were associated with weight loss at both time points with similar effect sizes. In each case, weight change was negatively associated with weighing frequency in all cases (i.e., more frequent weighing was associated with greater weight loss).

Table 3.

Association between weighing frequency and weight loss.

Weighing Frequency	N	Observed Weight Change, kg (Mean ± SD)	F	p	Model R2
	12 months
Self-report			9.19	0.0002	0.09
Monthly or less	10	0.9 ± 5.0
Weekly	92	−8.3 ± 8.5
Daily	84	−9.4 ± 8.0
Objective			26.66	<.0001	0.22
Monthly or less	17	−1.3 ± 8.3
Weekly	121	−6.8 ± 7.5
Daily	59	−11.7 ± 8.0
	24 months
Self-report			9.90	<.0001	0.10
Monthly or less	42	−1.4 ± 6.0
Weekly	88	−4.4 ± 7.9
Daily	45	−7.9 ± 7.8
Objective			10.39	<.0001	0.11
Monthly or less	92	−2.7 ± 7.5
Weekly	73	−6.6 ± 7.7
Daily	12	−10.9 ± 8.9

Note:

^aWeighing frequency between baseline and 12 months used to predict weight change at 12 months. Weighing frequency between 12 and 24 months used to predict weight change at 24 months. Self-reported weighing frequency was measured concurrent with weight. Analyses control for randomized treatment group.

Discussion

In this analysis, we compared the self-reported frequency of weighing to objective measures of weighing collected using Wi-Fi connected scales. The self-reported measure agreed with the objective measurements moderately to poorly, based on interpretation of the kappa statistics [19]. Where disagreement occurred, self-report frequency was generally higher than the objective measure. However, both self-reported weighing frequency and objective weighing frequency were inversely associated with weight change, as expected based on prior studies where greater weighing frequency was associated greater weight loss [e.g., 6, 8, 9].

This study is the first to compare self-reported weighing frequency to objective weighing frequency. To date, most studies that have investigated the effects of weighing on weight loss have using the same single-item measure of self-weighing frequency used in this analysis. Similar to these other studies, self-reported weighing frequency was assessed at dispersed intervals (e.g., every 6 months). This approach has yielded positive associations with weight loss [6, 9], as confirmed in this study. Unfortunately, this self-reported measurement is likely overestimating weighing frequency, for daily weighing in particular.

The self-report measure asks participants to categorize their weighing behavior over the past several months into one of seven categories (i.e., “never” to “multiple times per day”). These categories may be insufficient to capture the true frequency of weighing and forces participants to either over- or under-report their weighing behavior. As an example, of participants who weighed between 1 and 2 times per week between the 12 and 24-month assessments (n = 28), 32% self-reported weighing daily while 54% reported weighing weekly, as shown in Figure 1B. These participants did not have an option to select ‘1-2 times per week’ on the survey and were forced to choose between ‘weekly’ and therefore discount their more frequent weighing or ‘daily’ and overestimate how frequently they were weighing themselves. While this study brings to light the limitations of this measure, the implications of misreporting self-weighing are certainly of interest for behavioral weight loss intervention development. That is, a better understanding of the impacts of over or under reporting of self-weighing is a necessary next step in this research.

An important consideration of measures of weighing frequency is the association with objectively assessed weight change. Both objective weighing frequency and self-reported weighing frequency were associated with weight change at 12 and 24 months, though to varying degrees. At 12 months, objective weighing frequency was a stronger predictor than self-report whereas at 24 months, both measures of weighing frequency were similarly and more weakly associated with weight change. While we can only speculate why neither measure was strongly associated with weight change during this later period, one potential explanation is the overall decrease weighing observed during that period (using either the self-report or objective measures). It is possible that without consistent weighing, frequency is less associated with weight loss, although future analyses will be needed to test this hypothesis. The poorer predictive value of self-reported weighing frequency at 12 months may have been influenced by the intervention study from which these data were collected. Participants were assigned to either daily or weekly weighing as part of a comprehensive weight loss intervention. Participants may have felt compelled to report weighing frequency that matched their assigned treatment group rather than accurately reflect their weighing frequency. This would have diluted the predictive association between self-reported weighing frequency and weight loss.

To overcome the shortcomings of the current self-report measure, several options are possible. Based on these results of this analysis, it is recommended that future studies interested in examining the direct effects of weighing frequency on weight loss use an objective assessment of weighing frequency, such as that provided by readily available “smart” scales. When objective measures are not possible, a second approach is to modify the self-reported measure to provide more options for weighing frequency by allowing choices between “daily” and “weekly.” Few participants (12.6%) reported weighing less often than monthly so removing those options and instead using a more granular assessment of frequent weighing (e.g., 1-2 times per week; 3-5 times per week) during a weight loss program may be appropriate. This new approach will need to be validated prior to use to ensure that recall of this behavior is possible at this level. In particular, it will be important to assess whether participants can accurately recall from memory their behaviors over the prior 3-6 months.

This study addresses an important need to validate an often-used measure in weight loss research. Other strengths of the study include: a long follow-up period (24 months), the inclusion of treatment and no-contact follow-up intervals to examine the measures during both phases of weight control, and a sufficiently large sample to address the research questions. A notable limitation of this study is the inability to separate self-report of weighing frequency from reporting of self-weighing frequency to match the assigned treatment group. This study design likely increased the overall frequency of weighing from baseline, limiting the range of weighing frequency that could be examined (i.e., most participants reported weighing at least weekly). Another limitation is related to the reduced sample used in this analysis: because self-reported weighing frequency was assessed only at study assessment visits, the approximately 18% of participants who did not complete the study assessment questionnaires were excluded from the sample. Finally, the sample used in this study was comprised primarily of non-Hispanic, white women, though representation of men in this study (36.3%) is greater than average across weight loss studies [20]. Future studies investigating self-weighing will need to use a more diverse sample to enhance generalizability of results. Additional studies are also needed that investigate the relationships between self-reported self-weighing and objective self-weighing that do not explicitly provide weighing recommendations or a structured weight loss curriculum, to better understand these relationships in a more naturalistic setting.

In order to better understand the frequency of weighing that is associated with the greatest weight loss, an accurate assessment of participant behavior is needed. Because self-reported weighing overestimates weighing frequency and is open to error due to social desirability, memory of participants, and other shortcomings, it will be important for researchers interested in studying the effects of self-weighing frequency on weight loss to use objectively measured self-weighing frequency when possible. For those unable to use objective measures, self-reported frequency may be sufficient for some research questions, but its limitations need to be considered.

What is already known about this subject

• Self-weighing is an important component of self-monitoring during weight loss.

• Self-reported and objectively assessed self-weighing frequency have been associated with weight loss during a weight loss program.

• No studies have been conducted that compare self-reported weighing frequency to objective measures.

What this study adds

• This study adds evidence that self-reported weighing frequency as measured in this study is only modestly related to objectively measured self-weighing frequency.

• However, both self-report and objective frequency measures were associated with weight loss, indicating that the self-report measure has some utility.