Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Apr 1.
Published in final edited form as: Appetite. 2011 Jan 18;56(2):465–468. doi: 10.1016/j.appet.2011.01.014

Project TwEATs: a feasibility study testing the use of automated text messaging to monitor appetite ratings in a free-living population

Susan M Schembre a, Jessica Yuen b
PMCID: PMC3055950  NIHMSID: NIHMS270091  PMID: 21251941

Abstract

There are no standardized methods for monitoring appetite in free-living populations. Fifteen participants tested a computer-automated text-messaging system designed to track hunger ratings over seven days. Participants were sent text-messages (SMS) hourly and instructed to reply during waking hours with their current hunger rating. Of 168 SMS, 0.6-7.1% were undelivered, varying by mobile service provider, On average 12 SMS responses were received daily with minor variations by observation day or day of the week. Compliance was over 74% and 93% of the ratings were received within 30-minutes. Automated text-messaging is a feasible method to monitor appetite ratings in this population.

Keywords: mobile phones, social network services, technology, hunger ratings, measurement

Introduction

Communication-based technology is being rapidly infused into several areas of behavior-based research primarily because it is widely available, inexpensive, and virtually instantaneous (Cole-Lewis & Kershaw, 2010). Mobile phones, particularly the feature of short message service (SMS; text-messaging), which has demonstrated large increases in use since 2007, is one such technology (Pew Internet & American Life Project, 2010). Researchers acknowledge that SMS has been especially useful in a wide variety of behavioral intervention studies that utilize support (e.g. weekly SMS reminders/cues) and positive reinforcement strategies to effect behavior change or maintenance of treatment gains (Armstrong, et al., 2009; Haapala, Barengo, Biggs, Surakka, & Manninen, 2009; Hanauer, Wentzell, Laffel, & Laffel, 2009; Hou, Hurwitz, Kavanagh, Fortin, & Goldberg, 2010; Lim, Hocking, Hellard, & Aitken, 2008; Patrick, et al., 2009; Prabhakaran, Chee, Chua, Abisheganaden, & Wong; Riley, Obermayer, & Jean-Mary, 2008; Shapiro, et al., 2009; Shapiro, et al., 2008). In this way, SMS offers a unique ability to communicate with participants in a real-time manner that is convenient and relatively unobtrusive.

Of particular interest is the application of SMS technology to appetite monitoring in free-living populations. Currently, there are no standardized methods for collecting appetite-related data in free-living populations. Appetite sensations have traditionally been measured using visual analogue scales (VAS) or by category scales that are divided into a number of distinct categories numerically labeled (Mattes, Hollis, Hayes, & Stunkard, 2005). Typically, these scales have been administered in a paper and pen (P&P) format, which introduces issues of accuracy and reliability of the data. Particular concerns with the traditional P&P method include the lack of a guarantee the scales are being completed at the specified time of day (faked compliance) and the time-consuming nature of data processing (Stratton, et al., 1998). More recently, electronic appetite rating systems (EARS) have been developed to overcome limitations of the P&P method (Stubbs, et al., 2000) including a desktop computer system (Yeomans, Gray, Mitchell, & True, 1997) and two portable, hand-held devices (PDA) (Almiron-Roig, et al., 2009; Stratton, et al., 1998). Alarms are set on the PDA to prompt participants to complete the VAS at desired time-intervals and all entries are time/date stamped. Also, PDA are equipped with an application that measures the VAS, making data processing faster and less prone to human error. These technological advances in appetite monitoring are a vast improvement over the P&P method.

The purpose of this study was to test the feasibility of using automated SMS technology to monitor appetite ratings hourly over seven consecutive days. Because a majority of research studies use SMS technology to send messages and not to receive responses, particular attention has been paid to the response data. The aims of this paper are to 1) to explain the design of the automated system, 2) to evaluate the quality of the collected data with respect to the number of SMS delivered, the number of SMS responses received, and the timeliness in which the responses were received, and 3) assess compliance to responding to hourly SMS during waking hours.

Methods

Participants

A convenience sample of 15 male (n=2) and female (n=13) young adults (18-24 years) were recruited for the current study; Project TwEATs (Text with Ease Appetite Tracking System). Participants were screened for eligibility via on-line survey supported by SurveyMonkey.com that included demographic questions as well as questions about weight and height (self-reported). All participants reported being healthy, free from chronic diseases that affect eating patterns (e.g. diabetes), had no reported history of a diagnosed eating disorder, and were not pregnant or lactating and provided informed consent prior to study initiation. Study protocols were approved by the Institutional Review Board.

Automated text-messaging protocol

The automated SMS system tested in this study integrated two web-based technologies: a social media dashboard (Hootsuite.com; HootSuite Media, Inc.) and a social networking service (Twitter.com; Twitter Inc.). HootSuite™ is a website that can be used by commercial organizations to submit messages to social networking services such as, Twitter™. HootSuite™ was specifically used to schedule automated SMS to be sent to the mobile phones of participants actively “following” Project TwEATs on Twitter™ (explained below). All possible reminder times were scheduled through HootSuite™ and the “following” feature was used to manage which participants received reminder SMS. This allowed participants to begin the seven-day monitoring period on different days.

Twitter™ is a free web service that enables users to send and receive text-messages (140-character notifications referred to as “tweets”). Twitter™ users can send and receive tweets through the Twitter™ website, other compatible external applications (such as those for Apple™ iPhone), or choose to have them forwarded as SMS. Tweets are publicly visible to individuals who subscribe to receive tweets from another user (referred to as “following”). However, private messages (direct messages demarked with a “D”) are shared between Twitter™ users who mutually elect to follow each other. For the purpose of confidentiality, participants were set-up to mutually follow Project TwEATs only. In other words, they were restricted to receiving and responding to SMS from Project TwEATs, and were unable to communicate with each other. Additionally, Twitter™ account usernames and passwords were pre-assigned and not shared with the participants to prevent access their accounts. Personal information (e.g. phone numbers) is not made visible to followers.

For the purposes of this study, only hunger ratings were collected. Participants were prompted hourly by SMS to respond with their (current) perceived hunger rating as follows: “9:00 pm reminder: Reply with your appetite rating with “D projecttweats” (Rate 1-10)”. Reports of undelivered SMS were logged. All participant SMS responses, were recorded on the Project TwEATs’ Twitter™ account and automatically forwarded to an email account created for the Project for organizational purposes. All data were manually transferred into a working database and verified for accuracy.

Study protocols

Participants met with the study coordinator for a one-time visit for eligibility screening, a review of the study protocols, and signing of informed consent. An explanation of the 10-point hunger rating scale (1=Not hungry at all; 10=Extremely hungry) was provided. Personal Twitter™ accounts were created and a practice test of the SMS system was conducted. Additional instructions were to reply to Project TwEATs SMS during waking hours at their earliest convenience, and contact information was provided in the event any problems arose with their mobile phone or mobile service leading to missed SMS.

Statistical Analyses

Descriptive statistics were used to describe the number of undelivered SMS and average number of daily responses (both collectively and by mobile service provider) as well as the frequency of SMS responses received within 15- and 30-minutes and compliance with responding to hourly SMS during waking hours. Waking hours were identified for each participant based on the time of their first and last SMS response each day during the observation period; this often extended past midnight for some individuals. Analyses were conducted using SPSS version 16.0 and significance was set at p<0.05.

Results

A total of 15 young adults (19-24 years; 20.6±1.4 years) enrolled in the study. Thirteen participants were female and 12 had a normal body mass index (18.5-24.9 kg/m2), which was calculated from self-reported weight and height. Three nationwide mobile service providers were represented (Providers A, 60%; B, 20%; C, 20%) and all participants reported having an unlimited text-messaging plan.

A total of 168 SMS were scheduled for delivery. Between 0.6% and 7.1% SMS were undelivered (missed SMS). The mean frequency of missed SMS varied between mobile service providers. (Providers A, 4.5%±2.4%; B, 1.4%±0.9%; C, 0.8±0.3%). There was also some variation in the timeliness of SMS responses received within 15-minutes (Providers A, 80.8±11.7%; B, 91.6±7.5%; C, 76.6±9.8%), but less of a difference within 30-minutes after the hour (Providers A, 91.8±7.5%; B, 95.1±3.8%; C, 92.5±5.1%). However, the mean number of daily SMS responses was similar by Provider (Providers A, 12.1±3.1; B, 11.5±4.5%; C, 13.9±2.9). All SMS responses were successfully transmitted to the study’s Twitter™ account [Table 1].

Table 1.

Project TwEATs SMS details by participant (168 total SMS)

Participant Provider % Missed

SMSa 		% Problem

datab 		Average

daily responsesc 		Complianced Responses received
% within

15-min.c 		% within

30-min.c
1 A 2.4 0.6 13.4 74.4 78.4 89.6
2 A 1.8 0.6 11.6 70.6 76.6 88.2
3 A 7.1 0.6 11.7 71.3 89.7 97.1
4 A 2.4 0.6 11.7 70.8 68.9 82.5
5 A 7.1 0.0 14.9 84.3 89.1 97.9
6 A 6.5 1.2 15.9 96.0 89.3 97.4
7 A 2.4 0.0 5.1 37.7 57.1 78.6
8 A 3.6 1.2 13.9 86.6 85.5 98.8
9 A 7.1 0.6 10.7 70.4 92.4 96.4
10 B 0.6 0.6 15.7 90.3 98.2 99.1
11 B 2.4 0.0 6.7 47.7 93.5 94.5
12 B 1.2 0.0 12.3 81.0 83.3 91.7
13 C 0.6 3.0 16.4 87.9 79.9 97.0
14 C 0.6 3.0 15.5 88.2 65.5 86.9
15 C 1.2 3.6 10.7 60.7 84.3 93.5
Mean±SD 3.1±2.6 1.0±1.1 12.4±3.2 74.5±16.2 82.1±11.3 92.6±6.3
Open in a new tab
a

Missed SMS were not sent to participants due to mobile service complications (total number for seven-day period).

b

Problem data were SMS responses were received within 10-minutes of each other or were multiples of ratings received in the same hour.

c

Not including problem data.

d

Compliance was defined as the percentage of SMS responses received during waking hours per individual and counts missed SMS towards non-compliance.

Overall compliance with instructions to respond to all text-messages sent during waking hours over the seven day observation period was 75% or 12 (mean daily responses) out of 16 possible responses (average waking hours). Only two people had fewer than 10 daily responses (<60% compliance). The number of responses per day ranged between 9.7±3.7 (Day six) and 13.3±3.8 (Day four) with the lowest number of responses being received on Thursday (9.5±4.4). However, this had little impact on compliance rates during week days (74.5±15.4%) compared to weekend days (75.1±16.8%). [Table 1].

Discussion

The findings of this study (Project TwEATs) suggest the use of automated SMS is a feasible approach to monitoring appetite ratings in a free-living sample. Using free, web-based services, SMS can be scheduled for delivery to an unlimited number of participants with a mobile phone without additional resources (other than a computer and internet access). An average of 12 SMS responses per day were provided over seven days of monitoring with the lowest mean number of responses occurring on the sixth day of observation and on Thursdays. Despite issues with undelivered SMS, which varied by Provider, there was little variation in the number of SMS responses (hunger ratings) or in the timeliness of their receipt. Participant compliance with responding to hourly SMS during waking hours was exceptional for the frequency of monitoring and there was virtually no difference in compliance between week days and weekend days.

Mobile service interruptions, resulting in undeliverable SMS, were experienced by all participants at a rate of 3.1%. One other SMS-based feasibility study addressed similar issues of undelivered SMS reporting an average of 2.5% (114/4500) undelivered SMS to six of 95 participants over a four-month period (Gerber, Stolley, Thompson, Sharp, & Fitzgibbon, 2009). The two most common reasons for undelivered SMS were a full message inbox and disconnected phone service. In the current study, there were no reports of full inboxes or disconnected phones. Undelivered SMS appeared to result from Provider errors. Unsuccessful attempts were made to confirm mobile service interruptions with the various Providers. Interestingly, five participants provided several hunger ratings though the SMS prompt was undelivered, suggesting that the expectation of receiving hourly SMS may encourage compliance. Future protocols applying the use of SMS technology should include contingency plans to deal with undelivered SMS, such as preparing to collect an additional day of data, reminding participants to delete prior SMS, instructing participants to contact the research staff in the event of undelivered, and encouraging unprompted responses if possible.

Compliance with responding to timed-prompts, either in using SMS or PDA, in free-living populations appears to vary greatly with the frequency and length of the study. This study monitored hunger ratings frequently (hourly), but for a relatively short-period of time (seven days) with an overall compliance of 74.5% (range: 38% to 96%). Based on a review of other studies using SMS technology, where SMS were both sent and received, compliance can range between 16% and 87% (Hanauer, et al., 2009; Shapiro, et al., 2009; Shapiro, et al., 2008). Published data were not found that explicitly addressed participant compliance using other methods of monitoring appetite in free-living individuals. However, whether it be P&P or PDA, 100% compliance may not be feasible. Participants are requested to respond to timed-prompts every hour during waking hours. Resulting analyses utilize data collected during average waking hours (calculated for the group). In two studies, these times were reported to be between 6am and 12pm (Mattes, 1990) and between 7am and 9pm (Stratton, et al., 1998). The Stratton et al. (1998) study also reported an average of 11 observations per day suggesting an average compliance of 78% (11 observations/14 hours), which is comparable to the compliance rate observed in this study.

Of note, explicit directions about how quickly to respond to delivered SMS prompts were not given. Even without this direction, a high proportion of responses (82%) were received within 15-minutes of receiving the SMS. While these data are not specifically provided in similar research, it may be of particular interest going forward with developing protocols for monitoring appetite ratings in free-living individuals as appetite ratings are not stable for extended periods of time. Currently, single appetite ratings are used to represent an entire hour (Mattes, 1990; Stratton, et al., 1998). When creating and analyzing appetite profiles using time-stamped data, it will likely be important to ensure responses are received in a consistent manner (equally-spaced in time) to ensure accurate representation. For this reason, when monitoring appetite ratings in a free-living population, participants should be instructed to respond to prompts (SMS or PDA alarms) in a timely fashion, at least until research suggests otherwise.

There are noted limitations of this study and methodology. The most limiting factor was that all of the appetite ratings collected in this study were manually transferred from individual emails to a working database and verified. One of the more attractive aspects of using PDA is that they are equipped with computer software that allows data to be downloaded directly into a working database without the concern of transcription errors. It is expected that this technology is already available or will soon be available and will be incorporated into future applications of the SMS methodology for appetite monitoring. Also, undelivered SMS were self-reported by study participants and may not have been accurately logged. For this reason, undelivered SMS may be underrepresented in this study. However, undelivered SMS did not reduce response rates, partly due to participants sending hunger rating without being prompted. Another potentially concerning point could be that using mobile phones to collect appetite data (at this time) does not allow for using VAS. Rather, participants were trained to report appetite ratings on a categorical scale that was not immediately visible. In a recent review (Blundell, et al., 2010) it was recognized that research shows little difference in results from line (VAS) versus category scales so long as there are an adequate number of categories and that different types of scales often produce quantitatively similar results. While it is acknowledged that further validation of monitoring appetite using SMS technology will be needed, particularly in populations whose lifestyle is different from those of college students (e.g. adults and older adults), it is likely that this method will be comparable with other data collection methods that utilize VAS. Furthermore, future research should include larger sample sizes, which have a greater proportion of males and are adequately powered to detect differences in response patterns by mobile service provider as well as to explore differences in compliance by gender and weight status. Lastly, requiring the use SMS/mobile phones for monitoring purposes may be considered a recruiting bias based on socio-economic status (SES). However, research indicates that individuals of low SES typically have mobile phones and use texting more frequently than those of higher SES (Pew Internet & American Life Project, 2010). In this feasibility study, we chose to include only those with mobile phones and an unlimited text plan in an effort to avoid SMS charges. However, mobile phones and service plans could be purchased inexpensively for a price that is likely less than purchasing a PDA.

This Project TwEATs demonstrates that automated text-messaging is a feasible method of monitoring appetite ratings in this young adult population and is supported by the ease with which the system was created, the low frequency of undelivered SMS and untimely responses as well as the high level of compliance despite the intensity of the SMS schedule. Improvements, including more detailed participant instructions and computerized data transfer will encourage more researchers to utilize this ubiquitous technology in future appetite monitoring studies. Future research testing the feasibility of using text-messaging to monitor appetite ratings should be done in other populations, particularly those whose lifestyles are different from this sample of college students.

Acknowledgement

This study’s authors were supported by the Nutritional and Behavioral Cancer Prevention in a Multiethnic Population postdoctoral fellowship (R25 CA 90956; SMS) and a National Cancer Institute funded Continuing Umbrella of Research Experiences (CURE), an internship for high school and college students from underrepresented minority populations (JY).

References

  1. Almiron-Roig E, Green H, Virgili R, Aeschlimann JM, Moser M, Erkner A. Validation of a new hand-held electronic appetite rating system against the pen and paper method. Appetite. 2009;53(3):465–468. doi: 10.1016/j.appet.2009.09.014. [DOI] [PubMed] [Google Scholar]
  2. Armstrong AW, Watson AJ, Makredes M, Frangos JE, Kimball AB, Kvedar JC. Text-message reminders to improve sunscreen use: a randomized, controlled trial using electronic monitoring. Archives of Dermatology. 2009;145(11):1230–1236. doi: 10.1001/archdermatol.2009.269. [DOI] [PubMed] [Google Scholar]
  3. Blundell J, de Graaf C, Hulshof T, Jebb S, Livingstone B, Lluch A, et al. Appetite control: methodological aspects of the evaluation of foods. Obesity Reviews. 2010;11(3):251–270. doi: 10.1111/j.1467-789X.2010.00714.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Cole-Lewis H, Kershaw T. Text messaging as a tool for behavior change in disease prevention and management. Epidemiologic Reviews. 2010;32(1):56–69. doi: 10.1093/epirev/mxq004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Gerber BS, Stolley MR, Thompson AL, Sharp LK, Fitzgibbon ML. Mobile phone text messaging to promote healthy behaviors and weight loss maintenance: a feasibility study. Health Informatics Journal. 2009;15(1):17–25. doi: 10.1177/1460458208099865. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Haapala I, Barengo NC, Biggs S, Surakka L, Manninen P. Weight loss by mobile phone: a 1-year effectiveness study. Public Health Nutrition. 2009;12(12):2382–2391. doi: 10.1017/S1368980009005230. [DOI] [PubMed] [Google Scholar]
  7. Hanauer DA, Wentzell K, Laffel N, Laffel LM. Computerized Automated Reminder Diabetes System (CARDS): e-mail and SMS cell phone text messaging reminders to support diabetes management. Diabetes Technology & Therapeutics. 2009;11(2):99–106. doi: 10.1089/dia.2008.0022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Hou MY, Hurwitz S, Kavanagh E, Fortin J, Goldberg AB. Using daily text-message reminders to improve adherence with oral contraceptives: a randomized controlled trial. Obstetrics and Gynecology. 2010;116(3):633–640. doi: 10.1097/AOG.0b013e3181eb6b0f. [DOI] [PubMed] [Google Scholar]
  9. Lim MS, Hocking JS, Hellard ME, Aitken CK. SMS STI: a review of the uses of mobile phone text messaging in sexual health. International Journal of STD & AIDS. 2008;19(5):287–290. doi: 10.1258/ijsa.2007.007264. [DOI] [PubMed] [Google Scholar]
  10. Mattes R. Hunger ratings are not a valid proxy measure of reported food intake in humans. Appetite. 1990;15(2):103–113. doi: 10.1016/0195-6663(90)90043-8. [DOI] [PubMed] [Google Scholar]
  11. Mattes R, Hollis J, Hayes D, Stunkard AJ. Appetite: measurement and manipulation misgivings. Journal of the American Dietetic Association. 2005;105(5 Suppl 1):S87–97. doi: 10.1016/j.jada.2005.02.029. [DOI] [PubMed] [Google Scholar]
  12. Patrick K, Raab F, Adams MA, Dillon L, Zabinski M, Rock CL, et al. A text message-based intervention for weight loss: randomized controlled trial. J Med Internet Res. 2009;11(1):e1. doi: 10.2196/jmir.1100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Pew Internet & American Life Project Cell phones and American adults. 2010 Available from www.pewinternet.org/~/media//Files/Reports/2010/PIP_Adults_Cellphones_Report_2010.pdf.
  14. Prabhakaran L, Chee WY, Chua KC, Abisheganaden J, Wong WM. The use of text messaging to improve asthma control: a pilot study using the mobile phone short messaging service (SMS) Journal of Telemedicine and Telecare. 16(5):286–290. doi: 10.1258/jtt.2010.090809. [DOI] [PubMed] [Google Scholar]
  15. Riley W, Obermayer J, Jean-Mary J. Internet and mobile phone text messaging intervention for college smokers. Journal of American College Health. 2008;57(2):245–248. doi: 10.3200/JACH.57.2.245-248. [DOI] [PubMed] [Google Scholar]
  16. Shapiro JR, Bauer S, Andrews E, Pisetsky E, Bulik-Sullivan B, Hamer RM, et al. Mobile therapy: Use of text-messaging in the treatment of bulimia nervosa. International Journal of Eating Disorders. 2009 doi: 10.1002/eat.20744. [DOI] [PubMed] [Google Scholar]
  17. Shapiro JR, Bauer S, Hamer RM, Kordy H, Ward D, Bulik CM. Use of text messaging for monitoring sugar-sweetened beverages, physical activity, and screen time in children: a pilot study. Journal of Nutrition Education & Behavior. 2008;40(6):385–391. doi: 10.1016/j.jneb.2007.09.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Stratton RJ, Stubbs RJ, Hughes D, King N, Blundell JE, Elia M. Comparison of the traditional paper visual analogue scale questionnaire with an Apple Newton electronic appetite rating system (EARS) in free living subjects feeding ad libitum. European Journal of Clinical Nutrition. 1998;52(10):737–741. doi: 10.1038/sj.ejcn.1600636. [DOI] [PubMed] [Google Scholar]
  19. Stubbs RJ, Hughes DA, Johnstone AM, Rowley E, Reid C, Elia M, et al. The use of visual analogue scales to assess motivation to eat in human subjects: a review of their reliability and validity with an evaluation of new hand-held computerized systems for temporal tracking of appetite ratings. British Journal of Nutrition. 2000;84(4):405–415. doi: 10.1017/s0007114500001719. [DOI] [PubMed] [Google Scholar]
  20. Yeomans MR, Gray RW, Mitchell CJ, True S. Independent effects of palatability and within-meal pauses on intake and appetite ratings in human volunteers. Appetite. 1997;29(1):61–76. doi: 10.1006/appe.1997.0092. [DOI] [PubMed] [Google Scholar]

RESOURCES