Using Instrumented Paper Diaries to Document Self-Monitoring Patterns in Weight-Loss

Abstract

Self-monitoring of eating is associated with successful weight loss, but adherence is imperfect and deteriorates over time. Moreover, intentionally or not, many individuals have difficulty keeping faithful records. We used instrumented paper diaries (IPDs) to study self-monitoring in randomly chosen participants in the PREFER trial, a behavioral treatment for weight-loss study. The diaries they used to self-report eating were periodically replaced with IPDs at various times during an 18-month weight-loss program, consisting of three successive phases: intense treatment (n = 35), less-intense treatment (n = 13), and maintenance (n = 16). We compared electronically documented self-monitoring data, showing when and how often IPDs were used, with self-reported data, then compared the electronically validated adherence and weight loss. Self-reported diary usage exceeded IPD-documented usage while the electronic data demonstrated a significant decline in self-monitoring over time. Diary recording often was not timely. Percentage weight lost correlated significantly with frequency of IPD use (p = .001) and the number of diary entries made within 15 minutes of opening the IPD (p = .002). This is the first study to document patterns of self-monitoring among participants in a weight-loss program, which demonstrated that individuals may falsify the times and frequency of self-monitoring. Furthermore, our results showed that adherence to self-monitoring and the timeliness of recording significantly correlate with improved weight loss.

Introduction

Two-thirds of Americans are overweight or obese [1], and those who lose weight often regain it [2]. Self-monitoring, the systematic observation and recording of target behaviors [3], is integral to current behavioral treatment for excess weight [4]. It heightens self-awareness, which in turn can affect the target behaviors (such as eating and physical activity in the case of weight loss).

Baker and Kirschenbaum found a consistent relationship between self-monitoring and success in both losing weight and maintaining weight loss [5, 6]. Ideally, self-monitoring is continuous and concurrent, e.g., as a person eats throughout the day, the person records what they ate. However, research shows that adherence to self-monitoring is less than ideal [5] while others reported earlier that adherence deteriorates over time,[7] and we recently confirmed these findings [8].

Self-monitoring has not received the attention it warrants; it is viewed more as a process that mediates weight loss than as a central component of a behavioral intervention. Little methodological work has been done to investigate self-monitoring, and some researchers have concerns about the validity of self-reported data. The few studies that have examined self-monitoring have relied on participant-reported patterns of self-monitoring, that is, they trust subjects to report their own behavior faithfully [9, 10]. Although self-reports are usually considered reliable methods to collect data (apart from the rare case of intentional deception [11]), they are less so when they require recall.

Human memory can be unreliable [12]. Autobiographical memory entails four important processes: encoding the event (or experience), storing the event’s memory, retrieving the event from memory, and reconstructing the event [13, 14]. Because reconstruction of an event can unintentionally introduce biases and inaccuracies, the accuracy of self-monitoring improves with timely recording.

The diet diary is a beneficial, real-time method of self-monitoring intake if used correctly [15], that is, if the user records all foods consumed as soon as possible after consumption. However, conventional paper-and-pencil diaries provide no objective verification of the recording time, making it difficult to determine if entries were actually logged in real time. Studies of paper-and-pencil diaries reveal several limitations, the most common being honest forgetfulness to record or inability to record because the diary was not in the user’s immediate possession. These problems are potentially compounded by retrospection error, which can occur when the user needs to reconstruct the events of missed entries [16]. Observational evidence suggests that subjects often record in their diaries long after an eating or exercise event, often waiting until their next appointment or treatment session. When this happens, they log entries as if they were completed at the time of the event, deceiving the investigator, defeating the main benefit of diaries - to provide accurate real-time data - and subjecting the data to the very recall biases that the diary method was intended to avoid. Moreover, recording food intake in batches (e.g., at the end of the afternoon and end of the day), may dilute the positive effects of self-monitoring by reducing the opportunity to take corrective action throughout the day if dietary constraints are violated.

Until recently, patterns of self-monitoring could not be examined objectively. Today a paper-and-pencil diary is available that registers the actual time of self-monitoring behaviors by stamping the date and time that the user opens and closes the diary. (invivodata, inc.) This instrumented paper diary (IPD) differs from other electronic diaries, such as those based on personal digital assistants, because it appears to users as a standard paper-and-pencil diary, so that they are not aware of the surveillance function.

Stone and colleagues used this technology in a 3-week study of patients with chronic pain who were told to record their pain episodes three times a day and found that subjects did not comply with this protocol [11, 17]. Subjects’ pain diaries indicated that their rate of adherence to the protocol was 90.5%, but actual adherence according to the electronically detected IPD openings was 10.9%. Moreover, although the IPDs were not even opened on 32.4% of the days, entries appeared for most of those days.

This technology seemed well suited to obtain previously unknown information about the actual patterns of self-monitoring of eating behaviors and their effect on weight loss, with the ultimate goal of finding strategies to address deficits in self-monitoring. We therefore conducted a study of individuals in a weight-loss program to: (1) to describe the electronically recorded patterns of self-monitoring of eating behaviors by weight-loss study participants over the course of the 18-month study; (2) to compare reported vs. electronically recorded self-monitoring; and (3) to examine the relationship between the electronically validated adherence to self-monitoring goals and success in weight loss in a sub-sample of participants in the PREFER study. We hypothesized that (1) over time adherence to self-monitoring would deteriorate, (2) individuals would report that they were recording more often and in a timelier manner than they actually were, and (3) that there would be a relationship between percent of time the individual recorded and weight loss success.

Methods

Design

This study is ancillary to a prospective, randomized controlled trial called the PREFER study, which has been described previously [18]. In brief, subjects (N=182) in the PREFER study indicated their preferred calorie- and fat-restricted diet (standard or lacto-ovo-vegetarian) and then were randomly assigned either to receive their preferred diet or to be randomly assigned to one of the two diets. The intervention (32 sessions) was delivered over 12 months that comprised two phases: In Phase 1 (months 1–6), subjects attended weekly treatment sessions and used paper diaries to record their food intake and activity on a daily basis. In Phase 2, treatment sessions were biweekly (months 7–9) and then monthly (months 10–12). Regardless of the interval between sessions, subjects recorded daily and submitted their diaries at each session for review by the investigators. Phase 3 (months 13–18) was the maintenance phase; no treatment sessions were held, and self-monitoring was encouraged but not required.

The present study used a subsample from the PREFER study to examine self-monitoring during the three phases of the parent study. The design of the present study was detailed previously. [19] Briefly, we used a sequential, mixed-methods approach. In the first part, we used a quantitative design and randomly chose 35 subjects from the third cohort of the parent study to electronically record their patterns of self-monitoring (Figure 1). This subsample of 35 had an equivalent representation of each of the four treatment groups (standard and vegetarian diets by choice and standard and vegetarian diets by assignment). In the second part of the study, we used qualitative methods and interviewed 16 subjects about their experience with the process of self-monitoring. This report concerns the quantitative portion of the ancillary study. We analyzed the subsamples from the four treatment groups together and thus are reporting on them without consideration of their random treatment assignment.

Figure 1.

Participant assignment for IPD use over three phases of the PREFER study

Selection of participants for IPD use

To obtain our subsample, we stratified subjects in the parent study by gender and ethnicity and used a computer-generated list to randomly select those who would use the IPD in each of the three study phases, since there was an insufficient number of IPDs to sample the entire cohort. Approximately one-third of the PREFER sample comprised racial minorities, and because we do not know if their adherence to self-monitoring differs from the larger population, we sampled at a ratio of 1 minority:1 non-minority on ethnicity. Subjects who used the IPD during the first phase were eligible to use it in subsequent phases. If a participant who was selected to use the IPD the following week was absent from the treatment session and thus could not receive it, he or she was replaced by the next person on the computer-generated list who matched the gender and ethnicity of the absentee. Because only six IPDs were available for the study, six participants were chosen to use the IPD at one time: one from each of the four treatment groups and a second subject from two of the groups (see Figure 1).

In order to sample a wide distribution of self-monitoring patterns during Phase 3, we stratified participants based on their level of adherence to self-monitoring during the first 12 months of the study. Adherence was computed as the percentage of times participants turned in a diary, with the number of treatment sessions as the denominator and the number of times a diary was submitted as the numerator, multiplied by 100, thus providing a range of 0% to 100% for adherence. We divided this measure of adherence into tertiles - high (67–100%), medium (33.1–66.9%) and low adherers (0–33%) - so that in Phase 3, the 16 participants who used the IPD represented each of the adherence tertiles.

Self-monitoring protocol

Implementation of the behavioral change strategy in the PREFER trial included collection of several facets of the eating experience with the paper diary: the time the individual ate, the time the individual was recording, whether eating was at home or away from home, a brief description of the food and amount eaten, and the food’s calorie and fat gram content. Participants were instructed to turn in a diary at each treatment session and encouraged to self-monitor all weeks even when there was no treatment session.

Since participants no longer came to the center for treatment sessions after the 12^th month, the diaries were exchanged by mail. Initially, participants were given a postage-prepaid, padded envelope to use for the return of the IPD. For participants selected later in the 6-month period, a large padded envelope was used for sending the IPD to the participant with a smaller padded envelope enclosed to use for its return. Participants were called prior to sending the diary and informed that it was being mailed and then were called a month later to remind them to return the binder.

Period for Use of the IPD

As noted previously, the study was divided into three 6-month phases: the first phase included weekly treatment sessions and diary exchanges, the second phase included bi-weekly sessions and diary exchanges for months 7 through 9 and monthly sessions and diary exchanges for months 10 through 12; the third phase had no treatment sessions but the IPDs were mailed to participants to use for one month. Because we had the use of only six IPDs, we continually rotated their use, distributing them to six participants for their use according to which phase they were in, e.g., during the first phase, every week the IPDs were returned and re-assigned to new participants and in the second phase, the same was done except every two weeks or every month. Because the IPDs had to be recalibrated during the second phase due to the increased number of pages being used, the IPDs were out of service for 12 weeks and thus limited the assessment of the entire group (n = 35) that was sampled in the first phase. During the third phase, there was some delay between the uses of the IPDs because of their being distributed through the postal mail system, and also because two of the IPDs were lost. Thus, as noted in Figure 1, of the 35 participants who used the IPD during the first phase, 13 used it during the second phase and 16 during the third phase. Four of the original group of 35 from the first phase were lost to follow-up from the parent study after the 6-month assessment.

In accordance with the Institutional Review Board (IRB)-approved protocol, the consent form signed by all PREFER subjects included self-monitoring in a paper-and-pencil diary but did not disclose the use of the IPD. Also, with IRB approval, we did not debrief participants at the end of the study about the use or true purpose of the IPD to monitor their recording behaviors, to avoid embarrassing participants and to protect the relationship we established with them.

The IPD and Data Collection

Instrumented Paper Diary

The IPD is housed in a standard black canvas ring binder that measures 17×11×3.5 cm and weighs approximately 250 g, including instrumentation. A zipper closes the binder and prevents entry of light. We inserted preprinted diary pages into the binder rings, to which we secured plastic ties at each end to prevent subjects from opening the rings and removing pages. The binder’s spine is unobtrusively instrumented with a circuit board embedded with photosensors that detect the change in light level when the binder is open or closed and provide a date and time stamp with a 1-minute resolution (patent pending, invivodata, inc., Scotts Valley, CA, USA) [19]. The device records a tamper signal if someone removes the circuit board, which includes the processor that records the opening and closure events of the binder. To discourage users from leaving binders open, we told them that we would be scanning the diary pages for data entry; therefore, it was important to keep the binders closed to protect the pages from spills when not in use. For the same reason, they should not remove the pages. Because usage intervals increased with each phase of the study, we had to increase the number of pages included in the binder accordingly. The IPD then was recalibrated for accuracy in detecting opening and closing of the binder, since a greater number of pages could interfere with the light detected by the photosensors. Consequently, the IPDs were not available for a 12-week period during Phase 2. The staff of invivodata extensively tested each IPD to validate its ability to detect openings and closures of the binder under different light conditions, including low-light, and over a range of ambient temperatures. They also field tested the device by comparing its recorded openings and closures with a simultaneous independent record of openings and closures. Although the IPD does not record with certainty when the diary pages were completed, it provided certainty about when the diary pages were not completed [11].

The IPD data are recorded and stored on the circuit board housed inside the right side pocket of the binder. After the ring binder and attached circuit board are removed from the binder, the board is placed in a card reader so the data can be uploaded to a computer [19]. All participants had the same kind of zippered ring binder as the one used for the IPD so that they did not know when they were using a standard (placebo) diary or the IPD; even the staff could not distinguish between them. A piece of black poster board inside the right pocket of the placebo binders mimicked the weight and feel of the small circuit board, and, like the IPD circuit board, was held in place with Velcro buttons. When it was time for a randomly selected participant to use the IPD, the same zippered binder was used, but the metal ring apparatus was exchanged for the one with embedded photosensors in the binder spine and the circuit board inside the right pocket.

IPD Data

Self-monitoring data for one week was obtained from the IPD at least once for each of the 35 participants who participated in the first phase of the study. During Phase 2, a smaller number of participants were sampled since the inter-treatment interval was longer. For the same reason, the number of diary pages needed to be increased and the IPDs had to be recalibrated to ensure that the photosensors still detected the change in light; this removed the IPDs from use for a period during the second phase. A small sample was able to use the IPD during the final phase since each participant had the IPD for a month and because of using the mail for the exchange process; moreover, not all participants returned them at the end of their one-month usage.

Exchange of IPD and standard diary

IPDs were substituted for placebo diaries just as the paper diary pages were exchanged for all subjects, that is, out of sight of participants during the treatment session. The tie rings were clipped and the diary pages removed. The ring binder with photosensors and attached circuit board was removed from the binder of the participant who had just completed using the IPD, and the standard un-instrumented ring apparatus and poster board were inserted in the participant’s zippered binder for the next period; new pages were inserted, and the tie rings replaced before the binder was returned to the subject. The IPD circuit board was subsequently inserted in another subject’s binder.

Missed visits

Every effort was made to retrieve the diary when it was due to be returned. If for some reason the participant was unable to come to the research center for the regularly scheduled treatment session and return of the diary was delayed beyond the time when there were sufficient pages for recording, alternative arrangements were made, such as picking it up from the participant if the individual worked or lived in the near vicinity or providing a padded envelope for its return by mail. This process was to avoid the cost of a lost or damaged IPD. To avoid making the participant suspect of the procedure, all binders that were not returned in this time frame were handled similarly.

Data management

Personnel from invivodata, inc., the IPD manufacturer, trained our project staff to set up and upload the data from the IPDs. The uploaded data were in ASCII file format. The paper diary pages were scanned electronically and saved in a JPEG file, which was later converted to a Word document for each subject; this text then was coded into numerical categories. The data manager prepared files for analysis according to the statistician’s specifications. IPD ASCII files were cleaned of any unnecessary information, and headers, labels, and new variables were added (i.e., subject identification number, sample number for when the participant used the IPD, and number of times the same subject used the IPD); event time (open/close time per the IPD) was converted to military time.

Data from the paper diaries (self-reported data) were entered into a Microsoft Access database so the self-reported data could be compared to the IPD data. Participants’ reported times eaten and time recorded were converted into military time, ID #, date eaten and date recorded, and session numbers were entered in the database as well. Proper subject tracking was necessary for adherence analysis (length of time IPD kept vs. length of time IPD actually used). For example, a few important variables to track included ID #, date and time the IPD was given to a participant, date and time IPD returned, and the date and time the IPD was uploaded.

Definitions of adherence

We used two definitions of adherence based on the binder openings and the self-reported recording times indicated by the participant. By the conservative definition, participants were adherent if they recorded a diary entry within 2-hours after an eating episode, (e.g., if a person opened the binder and recorded the 7:30 AM breakfast by 9:30 AM). The liberal definition considered participants adherent if they recorded within 6 hours of eating, (e.g. if the person recorded a 7:30 AM breakfast by 1:30 PM that same day). Because there may be some variation in the participant’s clock and the IPD clock, we allowed a 15-minute window on either side when comparing the times of binder opening and the reported times of recording.

Based on the original work of Stone et al., we used these definitions to determine three types of adherence [20]. Reported adherence was based on the time the recording occurred as reported by the participant. Thus, an individual who wrote that she recorded her 7 AM breakfast at 8:30 AM would be considered adherent by the conservative and liberal definitions; if she wrote that she recorded the same meal at 1:30 PM, she would be nonadherent by both definitions. To determine actual adherence, we applied the definitions of adherence to the objective measure, i.e., the time that the diary was opened and closed as recorded by the IPD. Finally, we derived falsified adherence by calculating the difference between the self-reported time of recording and the IPD-recorded time of opening and closing of the binder and then applied the two definitions of adherence to the time difference. Thus, a 2-hour difference would be considered falsified adherence by the conservative definition. Hoarding, a type of falsified adherence, refers to days on which diary pages are reported as being completed, but the IPD does not record any binder openings and closings. Because participants in this study never should have abstained from eating for 24 hours, we assumed that diary entries for such days were recorded on a later day and considered adherence for these days to be 0%.

To calculate falsified adherence, we needed to define a fixed denominator for the number of times a person should record in the diary every day, a number on which there is no consensus. We decided to use three recordings per day based on instruction to participants to record as soon as possible after eating, to consume at least 1000 kilocalories per day, not to skip breakfast, and not to go for extended periods (8–10 hours) without eating. Consequently, a person who recorded three times per day, and each time recorded within 2 hours of eating, was considered adherent to self-monitoring that day, according to the conservative adherence definition.

Data Analysis

Data were analyzed using SAS (version 9.1.3, SAS Institute, Inc., Cary, NC). Initially, the data from the IPD and from the diary (frequency and timing of occurrence) were summarized on a daily basis. In addition the frequency of concordance (agreement) between the electronic recording and the self-reported recording time were summarized over time. Following data reduction from the diary and IPD, adherence to the self-monitoring protocol was determined and descriptively analyzed using exploratory data analysis methods. First, we compared the times the participant reported recording each day and the times the IPD processor recorded binder openings. Second, we compared when the participant reported eating and when in relation to the eating episode the participant reported recording the eating episode. Third, we compared the time the binder was opened to the reported time of eating to determine how closely to eating the actual recording took place.

Random coefficient modeling was used in an exploratory fashion to examine temporal patterns of self-reported eating and recording as well as IPD use over the three phases of the PREFER study. This more modern repeated measures method was employed rather than the more traditional repeated measures analysis of variance since random coefficient modeling allows for data missing at random (MAR), consideration of possibly unequal spacing between assessments, and the modeling of trends over time for the total sample and individual participants. To explore the relationships between the indices of self-monitoring adherence and changes in weight measured at 6, 12, and 18 months, correlational analyses were performed for each phase. Correlational analyses were also used to examine the association between self-monitoring adherence indices when using the IPD and those derived for each 6-month phase based on an examination of the participant’s diary. Due to the small sample size, Spearman rank-order correlations were used in these analyses. Lastly, adherence indices based on the IPD and diary self-report were dichotomized using the same cut point of 66.7%, as reported by Wadden [21] to investigate whether weight change in each phase of treatment differed between participants who were non-adherent (< 66.7%) and those who were adherent (≥ 66.7%) using Wilcoxon rank-sum procedures using exact estimation methods. Significance level was set at p < 0.05.

Results

Sample Description

The sample for the first phase (n = 35) was 85.7% female and 65.7% white, with a mean age of 43.1 (SD = 10.0) years and mean years of education completed of 15.1 (SD = 2.0). The mean body mass index (BMI), measured as kg/m², for the group was 34.8 (SD = 5.2). Participants experienced a mean weight loss of 24.8 (SD = 14.9) pounds (11.2 kg [SD = 6.77 kg]) over the first six months (Phase 1). The composition of the sub-sample (n = 13) for Phase 2 was slightly, yet not significantly different than the total sample in that whites comprised only 53.8% of the group and females made up 76.9% of the sub-sample. The mean BMI was lower; however, the group showed a nonsignificant gain of 1.2 (SD = 5.1) lbs (.55 [SD = 2.3 kg]) from the 6- to the 12-month assessment. In Phase 3, the sub-sample numbered 16; whites represented 56.3 % and females 75% of the group. The two smaller sub-groups were not significantly different from the total sub-group of 35, nor were they different from the overall sample of 182 that completed the parent study on the key socio-demographic variables of age, gender, ethnicity and years of completed education. Thus, the data provided by this sub-sample appear to be representative of the total sample.

Self-reported adherence to self-monitoring over time

Testing the first hypothesis that adherence to self-monitoring would deteriorate over time, we found the mean number of self-reported diary entries made per day was 4.0 (SD = 2.1) during the first phase, which declined during the second phase to 3.7 (SD = 2.2) entries per day. Refer to Table 1. Further reduction occurred in self-reported diary entries during the third phase, 2.2 (SD = 2.3). Using random coefficient modeling, we examined the change in number of self-reported diary recordings over time and found no significant change from the first to the second phase (the predicted mean or estimate was 4.0 [SE = .36] to 3.6 [SE = .52], t = .89, p = .379). However, we did find a significant change from the second to the third phase, 3.6 (SE = .52) to 1.8 (SE = .48) in self-reported diary entries per day, t = 2.98, p = .005. See Table 2.

Table 1.

Descriptive Data on Weight and Self-Monitoring Adherence by Study Phase

	Phase 1		Phase 2		Phase 3
	(1–6 mo)		(6–12 mo)		(12–18 mo)
	(n=35)		(n=13)		(n=16)
Characteristic	M	SD	M	SD	M	SD
Weight measures
Weight, lbs	208.8	35.1	188.4	42.5	191.3	20.7
Weight change from previous clinic visit, lbs*	−24.8*	14.9	2.3	6.2	6.6	8.9
% weight change from previous clinic visit*	−11.7*	6.3	1.3	3.4	3.8	4.9
BMI	34.3	3.9	30.4	5.3	30.9	4.4
Adherence measures
# of IPD openings/day	2.5	1.6	1.5	1.4	1.2	1.7
# of diary entries/day	4.0	2.1	3.7	2.2	2.2	2.3
Time between eating & recording, hours	6.4	17.1	6.0	10.5	2.7	2.5
Time IPD was open, hours	0.6	0.7	0.9	0.5	0.8	0.7
% of days IPD used	63.1	25.9	46.9	33.2	35.2	29.1
% of days IPD was opened at least 3 times	33.8	23.7	21.4	24.2	12.2	17.8
% of days recording time was reported	68.4	28.9	52.1	38.5	30.6	33.6
% of days recording time was reported at least 3 times	57.0	27.6	43.0	34.5	25.4	29.9
% of days eating time was reported	78.6	19.9	76.7	22.9	40.3	37.9
% of days eating time was reported at least 3 times	68.4	20.6	60.8	25.7	35.7	35.0
# of recorded entries/day w/in 2 hours of eating	2.0	1.5	1.4	1.2	0.7	0.8
% of recorded entries/day w/in 2 hrs of eating	51.1	29.5	41.4	33.4	40.5	28.8
# of recorded entries/day w/in 6 hrs of eating	2.7	2.0	1.9	1.6	1.1	1.3
% of recorded entries/day w/in 6 hrs of eating	67.1	30.3	53.0	37.8	51.1	31.3
% of recording of eating w/in 15 min IPD opening	20.7	22.2	11.6	13.5	18.2	26.0
% of diary recording of eating w/in 15 min IPD opening relative to minimum # expected entries (3 entries)	25.4	24.2	14.9	19.8	14.6	24.0

^*For Phase I, 6-month weights were compared to baseline values for weight.

Table 2.

Changes over Time in Self-Reported and IPD Recordings (n=35) using Random Coefficients Modeling

	Phase 1		Phase 2		Phase 3		F-value	t-value (p-value)
	(1–6 mo)		(6–12 mo)		(12–18 mo)		(p-value)	Differences between Phases
Adherence measures	M	SE	M	SE	M	SE	Overall	1 & 2	1 & 3	2 & 3
# of IPD openings/day	2.5	0.3	1.4	0.4	1.2	0.4	8.48 (.001)	2.77 (.009)	3.69 (<.001)	0.53 (.600)
# of diary entries/day	4.0	0.4	3.6	0.5	1.8	0.5	11.27 (<.001)	0.89 (.380)	4.73 (<.001)	2.98 (.005)
Time between eating & recording, hours	6.4	3.5	8.4	3.5	8.4	3.5	0.63 (.541)	−2.00 (.342)	−2.00 (.342)	0.00 (.999)
Time IPD was open, hours	0.6	0.1	0.9	0.2	0.8	0.2	1.05 (.364)	−0.24 (.196)	−0.15 (.362)	0.09 (.686)
# of recorded entries/day w/in 2 hours of eating	2.0	0.2	1.5	0.3	0.6	.3	8.63 (<.001)	1.37 (.179)	4.15 (<.001)	2.10 (.042)
% of recorded entries/day w/in 2 hrs of eating	51.1	5.3	48.4	7.5	37.4	7.7	1.75 (.193)	0.39 (.701)	1.87 (.072)	1.20 (.239)
# of recorded entries/day w/in 6 hours of eating	2.7	0.3	1.9	0.5	0.8	0.4	10.49 (<.001)	1.70 (.097)	4.55 (<.001)	2.13 (.039)
% of recorded entries/day w/in 6 hrs of eating	67.1	5.5	59.3	8.1	46.6	8.4	3.20 (.056)	0.98 (.337)	2.48 (.019)	1.24 (.224)
% of diary recordings of eating w/in 15 minutes of IPD opening	20.7	3.8	14.7	5.3	19.2	4.9	0.80 (.458)	1.27 (.215)	0.35 (.727)	−0.82 (.421)
% of diary recording of eating w/in 15 minutes of IPD opening relative to expected minimum # entries (3 entries)	25.4	4.0	15.4	5.6	12.7	5.2	4.29 (.022)	1.92 (.064)	2.65 (.012)	0.44 (.665)

Electronically-recorded adherence to self-monitoring over time

Compared to the IPDs being opened on average 2.5 (SD = 1.6) times per day during the first phase, IPDs were opened significantly less often over time, on average 1.5 (SD = 1.4) times per day during the second phase and 1.2 (SD = 1.7) times per day during the third phase. The slope of the decline in IPD openings over time was different from that of the self-reported entries. There were significant differences in the average number of IPD openings between the first phase, 2.5 (SE = 0.3) and second phase, 1.4 (SE = .40), t = 2.77, p = .009, and between the first and third phases, t = 3.69, p < .001. No significant differences were observed between the second and third phases, t = .53, p = .60.

Differences between self-reported and electronically documented self-monitoring

When we tested the second hypothesis, that individuals would report that they were recording more often and in a timelier manner than they actually were, we found that the electronically recorded openings were fewer in number than the self-reported openings during each phase, also shown in Table 1. As noted in Table 2, the mean (SE) of the time between a self-reported eating behavior and recording that behavior was mean 6.4 (SE = 3.0) hours during Phase 1. This period increased during the second and third phases to 8.4 (SE = 3.5). The difference between the phases was not significant, ps > .05.

Looking at how closely the self-reported diary recordings and self-reported eating behaviors were aligned with the IPD opening, we found 39.5% (SD = 34.0%) of recordings were within 15 minutes of the IPD opening while only 20.7% (SD = 22.2%) of the eating episodes occurred within 15 minutes of the IPD opening, suggesting that only one-fifth of the time recordings were made soon after eating. During the second and third phases of the study, 12% and 18% of the IPD recordings were within the 15-minute window, respectively.

Changes over time in self-reported and IPD recordings

When we examined the average number of diary entries per day within liberal and conservative windows of time from the eating, we found the average number of entries made within two hours was 2.0 (SD = 1.5), and the average number of entries made within six hours was 2.7 (SD = 2.0) in the first phase of the study. As noted in Table 2, there was a decline in the number of entries within both time windows with the changes between the first and the third and the second and third phases being significant (p = .0002 and p = .04) respectively, but not between the first and second phases. The number of self-reported diary entries was always higher than the number of IPD openings. The estimated mean (SE) number of self-reported diary entries was 4.0 (SE = 0.4) during the first phase, 3.6 (SE = 0.5) during the second phase, and 1.8 (SE = 0.5) in the third phase; the difference between the first and third phase was significant (p <.0001) as was the difference between the second and third phase (p = .005). The estimated mean (SE) number of IPD openings per day was 2.5 (SE = 0.3) during the first phase, 1.4 (SE = 0.4) during the second phase and 1.2 (SE = 0.4) during the third phase; the difference between the first and second phase and between the first and third phases was significant, p = .001 and p = .0008, respectively. The estimated mean percent of recordings of eating within 15 minutes of an IPD opening was 20.7 (SE = 3.8) in the first phase, 14.7 (SE = 5.3) in the second phase, and 19.2 (SE = 4.9) in the third phase, differences over time were nonsignificant (p > .05).

Association between self-monitoring and weight change

To test the third hypothesis, that there would be a relationship between percent of time the individual recorded and weight loss success, we examined the association between recording and percent weight change from baseline to 6 months, 6 to 12 months and 12 to 18 months. See Table 3. At six months, significant Spearman correlation coefficients were revealed between percent weight loss and percent of days that the IPD was used (r = −0.45, p = .007), percent of recording within 15 minutes of the IPD opening (r = −0.43, p = .009), and percent of diary recordings of eating within 15 minutes of the IPD opening (r = −0.53, p = .001). At 12 and 18 months, no significant correlations were found between percent weight loss and recording variables.

Table 3.

Spearman Correlations of Recording and Percentage Weight Change in Phase 1

Variable	r	p-value
# of IPD openings/day	−0.51	.001
% days IPD used	−0.45	.007
% of days IPD was opened at least 3 times	−0.56	<.001
% diary recordings w/in 15 minutes of IPD opening	−0.43	.009
% diary recordings of eating w/in 15 minutes of IPD opening	−0.53	.001

We also examined weight change from baseline to 6 months, 6 to 12 months and 12 to 18 months among those who were most adherent to self-monitoring over the entire 18-month period and those who were least adherent, using the self-reported diary data. During the first 6 months, 91.4% (n = 32) of the sample of 35 was adherent ≥ 66.7% of the time and only 8.6% (n = 3) was adherent to self-monitoring < 33.3% of the time. The mean weight loss for these groups was −11.52 (SD = 6.99) kgs vs. −8.57 (SD = 3.09) kgs with the most adherent losing more weight although the difference was not significant (p = .48). During the second 6-month phase, the weight change was the same for both groups; in the final 6-month or maintenance phase, those who continued to self-monitor and were the most adherent (n = 7) to this strategy regained a mean of 0.86 (SD = 4.72) kgs vs. those who were adherent less than 33% of the time (n = 8) regained 3.58 (SD = 2.59) kgs, differences not statistically significant. When we used the IPD adherence measures (as listed in Table 1) and compared those who were adherent ≥ 66.7% of the time to those who were < 33.3% adherent, similar, yet non-significant, differences were observed between the two groups.

Discussion

In this first study to use the IPD in a weight loss trial, we tested three hypotheses. We found that electronically documented adherence to self-monitoring declined over time, that individuals reported that they recorded their eating behaviors more often than the electronically recorded data indicated, and that there was a significant relationship between the amount and time of self-monitoring and success at weight loss during the first 6-month phase of the study. These findings are consistent with earlier studies of self-reported self-monitoring [5, 6, 9, 10] as well as the more recent study by Wadden et al. that found that those who were in the highest third of adherence to self-monitoring lost significantly more weight than those in the lowest tertile of adherence [21].

The results of our study confirm the findings of others, that the veracity of self-reported data collected using paper diaries is suspect. The wide discrepancy found by Stone and colleagues between self-reporting of pain symptoms in a paper diary (91%) and IPD verification (11%) are consistent with the results of studies that compared objective and self-report measures of adherence to medication regimens [11, 17]. That medication adherence research also showed consistent inflation of self-reported adherence compared to the objective electronic measures [22]. We found patterns of adherence similar to what Stone and others have reported: some participants never open the diary for the entire period of interest, while others open them frequently, but not at the times they alleged that they were recording. Our confirmation that adherence to self-monitoring deteriorated over time is reason for increasing concern about the value of self-monitoring as the self-monitoring period is extended.

The ideal technology to verify self-monitoring adherence has not been developed for widespread use. Several studies measure medication adherence with the medication event monitoring system, which uses a medication bottle containing an unobtrusive microchip in the cap. Reports of these studies indicate that individuals remove pills from the medication bottle and place them in pill boxes or in their pocket, often referred to as “pocket dosing.”[23] Similarly, anecdotal comments by a few of our subjects revealed that sometimes they kept notes during the day and then recorded in the binder diary at the end of the day. Thus, according to their self-reported behavior, the subjects were adherent, but the IPD did not register such behavior. The underlying problem in both of these situations is the need to house the monitoring microprocessor in a bulky apparatus, making it difficult for some individuals to carry the device with them throughout the day. However, other subjects in our study preferred the zippered binder to the small paper pocket diary that is the convention in weight-loss studies, and most used the binder for direct recording.

Although the IPD does not document the actual recording on the page, it does provide a rare and unique view of the patterns of self-monitoring performed by participants in a weight-loss study. This is the first study to use this technology in weight management, and moreover, to monitor adherence for the full duration of the intervention and maintenance phases. For the first time, we have been able to document the actual pattern of the recording of eating behaviors among participants in a weight-loss study and relate it to weight change. Approximately 50% of our subjects made diary entries within 2 hours of eating during the study. This finding suggests that participants in a weight-loss program may eat foods they do not record without considering the relationship between their current intake and their daily goal; thus they cannot take timely corrective action if they are nearing their maximum calories or fat grams. This situation is magnified when we consider our more liberal definition of adherence: initially, 67% of our participants recorded within 6 hours of eating, which declined to 51% in the latter phase of the study, suggesting that they participate in a series of eating episodes without considering the bigger picture of daily intake.

Association between self-monitoring and weight change

The studies done by others who have examined self-monitoring and weight loss clearly demonstrated that there was a relationship between self-monitoring and weight loss [5, 6, 9, 10, 21]. However, these studies were not able to establish that a relationship exists between the timing of self-monitoring and weight loss. Our findings showed that the frequency of recording and the interval between eating and self-monitoring was significantly related to weight change, with the association between recording within 15 minutes of eating and weight change being highly significant.

Limitations

Our study had a few limitations. First, we could not determine if adherence to self-monitoring was the sole driving force for success in weight loss. Those who adhered to self-monitoring might have adhered to the diet and lost weight without self-monitoring. Ruling out this possibility would require a randomized controlled trial testing the presence of self-monitoring compared to different levels of self-monitoring. Anecdotally, some of our participants proclaimed success at weight loss due to diligent self-monitoring. Another limitation was the limited number of IPDs available for our use, and thus the relatively small sample, which may have accounted for the absence of significant correlations between the percentage of weight loss and recording variables at 12 and 18 months. Nonetheless, because our sub-sample was representative of the parent study’s total sample of 182, our results most likely reflect those we would have obtained had the total sample used the IPD. Lastly, not all subjects were assessed in all three phases using the IPD. Although a smaller number of subjects (n=22) had some follow-up data (either assessment in the second and/or third phase) and only 7 subjects had all three phases assessed, the findings based on random coefficient modeling for these sub-samples were similar to that reported for the full sample of 35, both in terms of the actual direction of the effect across the three phases and the statistical significance of the findings. This sensitivity analysis suggests that the results reported for the full sub-sample of 35 are fairly robust.

Clinical implications

Our findings confirm that participants are not adherent to the standard self-monitoring protocol that is part of conventional behavioral treatment for weight management and that they often falsify their diary entries such that they do not accurately reflect their adherence to self-monitoring. Our findings also show the beneficial effect of timely recording and suggest that participants in a weight-loss program are not benefiting as much as possible from the key component of self-monitoring. Moreover, the recall required when entries are delayed can hinder the accuracy of diaries as a record of eating behaviors. Strategies are needed to improve adherence to self-monitoring and to shorten the interval between eating and recording so that corrective action can be taken if a participants exceeds his or her limitations on calories or fat grams.

One potential strategy that may reduce the burden of self-monitoring and is being tested by researchers is the use of an electronic diary or a personal digital assistant [24–26]. Such devices can lessen the inconvenience of looking up and recording the fat and calorie content of foods. However, most commercially available software programs for dietary self-monitoring do not include a date and time stamp and therefore prevent the timing of self-monitoring to be included in therapy, such as counseling regarding the importance of timely self-monitoring and then noting clinical improvement as self-monitoring improves. Again, using the model of medication adherence, Burnier found that adherence improved when subjects were shown their patterns of inadequate adherence and the impact on clinical outcome [27, 28]. There is potential for the application of these findings to the broader behavioral domain of self-monitoring. However, first we need more widely available devices and software that permit us to document self-monitoring in a manner that enables sharing the information with patients so we can counsel them to improve.