Abstract
Objective To test the effectiveness of multiple interventions on increasing the response rate of text messaging for longitudinal data collection.
Methods Our cohort included 283 caregivers of children aged 6–12 months who were participating in an anemia program in rural China. Using text messages to collect data on anemia medication adherence, we conducted a delayed randomized controlled trial to test multiple interventions (an additional four reminders; a ¥5.0 (US$0.79) credit reward for replying; and a feedback text message). After a 6-week pilot study with week 7 as the baseline measurement, we randomly allocated all participants into two groups: group 1 (n = 142) and group 2 (n = 141). During weeks 8–11, we introduced the interventions to group 1, and in weeks 12–15 the intervention was introduced to both groups. We compared the response rates between groups and explored factors affecting the response rate.
Results During weeks 8–11, the response rates in group 1 increased and were significantly higher than in group 2 (p<0.05). During weeks 12–15, the response rate increased significantly in group 2 (p>0.05) and slightly decreased in group 1. Younger participants or participants who had children with lower hemoglobin concentration were more likely to reply (p = 0.02). Sending four reminders on the second day contributed to only 286 (11.7%) extra text messages.
Discussion Our study showed that multiple interventions were effective in increasing response rate of text messaging data collection in rural China.
Conclusions Larger multi-site studies are needed to find the most effective way of using these interventions to allow usage of text messaging data collection for health research.
Keywords: randomized controlled trial, program evaluation, nutrition surveys
BACKGROUND AND SIGNIFICANCE
The number of mobile phone subscribers has increased rapidly worldwide over the past decade. It has been estimated that there were 6.8 billion mobile phone subscriptions by the end of 2013.1 In China, during this time, there were more than 1.1 billion mobile phone subscriptions.2 In 2009, text messaging was used by 4 billion mobile users globally, who sent 5 trillion messages, and the number of text messages sent has been estimated to have doubled to 10 trillion by 2013.3 Data collection shows that the number of text messages sent by Chinese people was an estimated 431 billion during 2013.4
With the growing number of mobile phone subscribers, the use of text messages in medical research has received a growing interest and could be an innovative way to collect data.5 Previous studies, which have evaluated the use of text messaging for data collection, have indicated that it is accessible for many people regardless of time, place, or setting. Moreover, text messaging allows data collection in real-time and removes interviewer bias.6,7 In certain cases, text messaging data collection enables collection of longitudinal data8–10 and can give access to a migrating population and to people that are difficult to reach.11,12 Using text messaging for data collection can be an innovative method which is cheap and easy to conduct.5
However, using text messaging data collection in health research is restricted by a low response rate. In past studies, the response rate of text messaging data collection has varied considerably, from 15% to 100%.5,7,9,13–18 Kew et al.5 conducted a study with 38 highly motivated IT-proficient students and obtained a response rate of 100%, which would be difficult to achieve in ‘real-life’ settings. Bexelius et al.14 randomly selected 2400 mobile phone numbers from the Swedish population registry and achieved a response rate of 15%. Haberer et al.19 conducted a 2-week text messaging data collection study of anti-retroviral therapy adherence and had weekly completion rates of between 0 and 33%. We achieved a response rate of 38% in a previous study.20 Therefore, exploring interventions and assessing their effectiveness on increasing the response rate is a key step for text messaging data collection.
Many factors can affect the response rate. The initial contact with participants,15 an advance introduction letter,21and calling at an anticipated time,21 can increase the response rate to a survey. A written or telephone reminder can increase the response rate to mail surveys.22 In addition, a review reported that the responses to postal and electronic questionnaires were at least doubled by using monetary incentives.23 Factors negatively influencing the response rate for a text messaging survey were a foreign phone number15 and sending messages at an inappropriate time.24 However, few of these interventions aiming to increase survey response rate were tested in text messaging data collection.
OBJECTIVE
This study aimed to test the effectiveness of multiple interventions on increasing the response rate of text messaging data collection in rural China.
METHODS
We used a delayed randomized controlled study design to assess the effectiveness of three interventions on increasing the response rate of text messaging data collection of anemia medication adherence. The study reported in this paper was a sub-study of an anemia intervention program and was conducted at the same period of the program.
Study setting
This study was conducted as part of a community trial of interventions on nutritional anemia for children aged 6–11 months in two townships in Zhao County, Hebei Province, China. Zhao County has 16 townships and 281 villages, with a total population of 580 000 and a total area of 675 km2 (859 people per km2).25 Shahedian township (township 1) has 17 villages, with a total population of 35 345 and surface area of 61 km2 (579 people per km2)26; Wangxizhang (township 2) has 16 villages, with a total population of 27 953 and surface area of 26 km2 (1075 people per km2).27 Both of these townships are near to the downtown area of the county and people work and live mainly through farming activities. These two townships were selected for the anemia program because they had high and comparable anemia prevalence, a comparable number of constituents, and high level of cooperation from local people. More information on the study setting can be found elsewhere.28,29
Overview of the anemia program
Our cross-sectional survey from 2011 showed that the anemia prevalence was 66.2% for children aged 6–11 months and 48.2% for children aged 12–23 months in rural Zhao County (unpublished data), which was much higher than the national average (28.2% and 20.5%, respectively).30 Therefore, we implemented a research program aiming to test the effectiveness of interventions on reducing children’s anemia. Children aged 6–11 months were selected because this age group had the highest anemia prevalence and was at the key stage of transition from exclusive breastfeeding to complementary feeding.
The anemia program was a community intervention trial conducted in May 2012. The trial tested the effectiveness of giving medication to children and providing dietary counseling to caregivers on reducing child anemia (unpublished), which was registered through the Chinese Clinical Trial Registry (http://www.chictr.org/cn), registration number: ChiCTR-ONRC-12002179. Shahedian was the intervention township and Wangxizhang was the control township. For both townships, we sent a letter to every caregiver to explain the trial. In addition, village doctors gave recommendations to caregivers to take anemia medications as soon as they were recruited. We provided exclusively to participants from township 1 a complementary food recipe booklet,28 a food blender, face-to-face group counseling on anemia and infant feeding, and text messages about feeding knowledge. The text messaging data collection took place after the trial started. The primary outcome was anemia prevalence, with the measurement of hemoglobin at baseline, 3 months after initiating the intervention, and at the end (6 months after initiating the intervention).
Study design of randomized controlled trial
Text messaging data collection took place over 15 weeks and was divided into two stages: (1) a pilot study from week 1–6; and (2) a delayed intervention randomized controlled study from weeks 7–15. We tested and chose a Chinese text messaging platform during the pilot study. In weeks 7–15, we conducted a delayed intervention randomized controlled study.31 We introduced three interventions to increase the response rate. We did a baseline assessment in the 7th week. During weeks 8–11, we only exposed participants in group 1 to the interventions, and during weeks 12–15, we exposed participants in both groups to the interventions.
Interventions
The response rate of text messaging data collection could be affected by a number of factors, such as demographic characteristics of participants,5,32–34 length of data collection,32 outcomes measured by studies,32,34 giving introduction letters to participants in advance,23 text message format,5 sending and receiving time,5 reminders,5 and incentives.35 Some of these factors, including participants’ demographic characteristics, length of data collection, study outcomes, and introduction letters to participants, were restricted by the anemia program, and therefore could not be changed in this study. In our previous study of text messaging data collection, we finalized the appropriate text messaging format and time for sending and receiving text messages.20 We chose additional reminders as our first intervention. Given that participants could be irritated by too many reminders, we decided to send only four additional reminders in the next day. The second intervention involved giving ¥5.0 (US$0.79) mobile phone credit as a reward every 2 weeks for participants who completed data collection. We usually provided token gifts at the value of ¥5.0 to participants as an incentive. This was affordable and included in the study budget, and it seemed enough to be attractive to caregivers to reply our text messages. Moreover, our pilot study indicated that providing caregivers with feedback of hemoglobin test results and anemia medication intake might contribute to increase the response rate. Therefore, providing feedback to caregivers was also used as an intervention.
Participants
In the anemia program, we included caregivers whose children were aged 6–12 months in the two townships. The inclusion criteria for the randomized controlled study were as follows: (1) being a participant in the anemia program; (2) having provided a mobile phone number in the baseline survey of the anemia program; and (3) having agreed to receive and reply to text messages. We excluded caregivers if their mobile phone number was wrong, or if they refused to participate in this study.
Sample size
Before the study, we did not have any knowledge of the potential effect size of our interventions on the response rate. Therefore, we did not calculate the sample size beforehand but recruited all participants of the community trial who met our inclusion criteria. Our study indicated that the response rate increased from 36% to 60% (the average response rate from the 8th week to 11th week). With 142 participants in each group and level of confidence of 95%, the power was calculated as 0.97.
Recruitment
The baseline survey for the anemia program was conducted on May 19 and 20, 2012; we collected demographic information and caregivers’ mobile phone numbers, and asked caregivers to participate in our text message study. After caregivers signed the informed consent form, we gave them a printed letter sealed by the Zhao County Maternal and Child Health Hospital. In the anemia program, we included 243 children in township 1 and 195 in township 2. We excluded 83 children in township 1 (69 without anemia, 4 with wrong phone numbers, 2 without phone numbers, 1 unable to send text messages, 7 with twins) and 72 in township 2 (59 without anemia, 2 with wrong phone numbers, 7 without phone numbers, 2 unable to send text messages, 1 with twins, 1 refused). Therefore, 160 children in township 1 and 123 in township 2 were eligible for the randomized controlled study on text messaging monitoring.
Randomization and allocation
The strata were based on the township. We randomly allocated all participants into two groups considering that the anemia interventions were different in the two townships. A person outside the research team generated the random sequence with Microsoft Excel 2010. The allocation result was directly sent to the researcher in charge of sending text messages. In addition to the 160 eligible children in township 1 and 123 in township 2, we included 4 more caregivers in township 1, and 5 more caregivers in township 2 during the first 6 weeks, because we updated their mobile phone numbers during the anemia program's follow-up. At the same time, we excluded 4 caregivers (3 unable to send text messages, 1 with wrong number) in township 1 and 5 caregivers (2 unable to send text messages, 1 with wrong number, 2 refused) in township 2.
We randomized 142 caregivers (83 from township 1 and 59 from township 2) to group 1, and 141 (77 from township 1 and 64 from township 2) to group 2. Figure 1 shows the flow diagram of the recruitment and pilot stage of the study, and figure 2 shows the flow diagram of the randomized controlled study.
Figure 1:
Flowchart of the recruitment and pilot stage of the study.
Figure 2:
Flowchart of the delayed intervention randomized controlled study.
Text message questions
We developed and tested four questions to assess caregivers’ adherence to dispensing the anemia medication to their child. As shown in box 1, the first two text messages were about children's anemia medication. These messages were sent to both townships, whereas the subsequent two text messages (about children's meat intake) were only sent to caregivers in Shahedian township. Dietary counseling as an intervention was not provided in Wangxizhang township. Therefore, it might potentially bias the anemia program if we had sent meat intake messages to caregivers in Wangxizhang township because the messages contained information on the benefit of meat intake.
Box 1: Text message questions.
Message 1. Drugs taken yesterdayTo cure anemia, you need to give your child medication as prescribed by your doctor. How many bottles of anemia medication did your child have yesterday (month/day)? (Zhao County Maternal and Child Health Hospital)Message 2. Drugs taken last weekPlease count how many bottles of anemia medication your child had this week (month/day to day)? (Zhao County Maternal and Child Health Hospital)Message 3. Meat intake yesterday*Giving meat to your child every day can help with anemia. Did your child eat meat (such as pork, beef, mutton or chicken), fish, liver or blood tofu† yesterday (month/day)? (Zhao County Maternal and Child Health Hospital)Message 4. Frequency of meat intake last week*How many times did your child eat meat (such as pork, beef, mutton or chicken), fish, liver or congealed blood tofu from last Saturday till yesterday (month/day to day)? (Zhao County Maternal and Child Health Hospital)*We did not send messages 3 and 4 in township 2 to avoid potential bias in the community trial†Blood tofu is made of coagulated chicken, duck, goose or pig blood.
Pilot study: weeks 1–6
We conducted a pilot study from July 21 until August 27, 2012 to test the text messaging software and the four text message questions. The population to whom we sent text messages was the same as the population in the formal study. We sent text messages on a fixed day each week. A team member sent messages from 10:00 to 23:00 using a computer and checked messages frequently. The first text message was sent at 10:00. When we received a response, we sent the next text message. When we did not receive a response, we sent that text message again as a reminder at 13:00, then 16:00 and 19:00 on the same day.
The delayed intervention randomized controlled study: weeks 7–15
During weeks 7–15, we conducted the delayed intervention randomized controlled trial using the 7th week as baseline. The content of text messages, message sending method, and the sending time (10:00, 13:00, 16:00, and 19:00) were the same as in the pilot.
We introduced three interventions to participants in group 1: (1) sending text messages on a second day with four extra text message reminders; (2) giving a ¥5.0 (US$0.79) reward; and (3) providing feedback to caregivers via text messages. First, we sent a text message to inform caregivers that if they replied to all four or two text messages in the following 2 weeks, they would receive a reward of ¥5.0 (US$0.79) mobile phone credit. Second, we prolonged the message data collection time by one day and sent four extra reminders on the second day. Third, we sent feedback text messages to caregivers, giving our advice based on adherence data collected by the text messages and hemoglobin test. We told caregivers whose children had good medication adherence to continue giving medicine to their child. We emphasized the importance of medication to caregivers whose children had sub-optimal drug adherence and encouraged them to give medication to their children. We implemented mobile credit reward and the sending of four reminders on a second day to group 1 during weeks 8–11 and to both groups during weeks 12–15. We only implemented text message feedback to group 1 on the 8th week and to group 2 on the 12th week, because we only had one opportunity to test the hemoglobin levels. Table 1 lists when we introduced the interventions in this study.
Table 1:
Interventions in group 1 and group 2 over time
| Group 1 | Group 2 | |
|---|---|---|
| Week 7 | No intervention | No intervention |
| Week 8 |
|
No intervention |
| Weeks 9–11 |
|
No intervention |
| Week 12 |
|
|
| Weeks 13–15 |
|
|
Telephone interview
We conducted telephone interviews to learn about caregivers’ reasons for non-response. After text messaging data collection finished at the end of the 8th week (September 10, 2012), we conducted the telephone interviews with caregivers who did not respond in group 1. A female interviewer conducted the interviews in Mandarin. After a short introduction of the program, she obtained oral informed consent. The interviewer asked open-ended questions to explore the reasons by prompting caregivers to explain reasons in detail and recorded the interview on paper.
Outcomes
The primary outcome was the weekly response rate, which was defined as the proportion of participants who answered the question clearly.
The secondary outcomes were the cumulated percentage of text messages received after every reminder, reasons of caregivers’ non-response, and factors affecting the response rate.
The contribution of four reminder text messages on a second day was the proportion of text messages received on the second day in group 1 during weeks 8–15 and in group 2 during weeks 12–15. We calculated the cumulated percentage of text messages collected after sending four message reminders on the second day. Reasons for non-response were reasons given by participants during the telephone interviews in group 1 who did not reply to the text messages after the 8th week (the first time we implemented the three interventions).
There were no known demographic factors that affected the response rate. Therefore, we considered a number of factors which might have had the potential to influence the response rate: whether the interviewee was the primary caregiver, number of children in the household, annual income, mother’s education, whether the mother started going out for work, whether the mother was anemic during pregnancy, and the baseline hemoglobin level that we tested in May 2012.
Data cleaning and analysis
We exported the text message data into Excel files. Two team members cleaned the text message information and then coded the text message answers independently. All messages received from participants were ‘yes/no/don’t know’ or number of times. We coded ‘yes’ as ‘1’, ‘no’ as ‘2’, ‘don’t know’ as ‘8’, and number of times as the exact number. The two team members compared the coded text messages and resolved the discrepancies by checking the original text messages.
We expressed continuous variables as median and quartiles (Q25–Q75). We present discrete variables as proportions. We used the χ2 test, Fisher's exact test, and Mann–Whitney U/Wilcoxon W to compare the two groups. We explored the factors influencing the response rate for the first text messages in each of the 8 weeks during the trial with generalized linear models, given the situation that the response or non-response in each week were correlated within each participant. We used the factors differing for the responders versus non-responders at baseline (week 7) as covariates in the model. We used the Glimmix procedure specifying the link function of logistic for analysis in SAS V.9.2. We presented all probability values for two-tailed tests and considered p<0.05 as statistically significant.
RESULTS
There were 142 caregivers in group 1 and 141 in group 2. There were no significant differences in the township, children's gender, and children's age between the two groups. Table 2 shows a comparison of the characteristics of children and their caregivers and the response rate of the text messaging survey at baseline in the two groups.
Table 2:
Characteristics of children and their caregivers and response rate in the 7th week
| Total | Group 1 | Group 2 | χ2 | p Value | ||||
|---|---|---|---|---|---|---|---|---|
| Characters | N | % | n | % | n | % | ||
| Township | ||||||||
| 1 | 123 | 43.5 | 59 | 41.6 | 64 | 45.4 | 0.43 | 0.52 |
| 2 | 160 | 56.5 | 83 | 58.5 | 77 | 54.6 | ||
| Child gender | ||||||||
| Male | 157 | 55.5 | 83 | 58.5 | 74 | 52.5 | 1.02 | 0.31 |
| Female | 126 | 44.5 | 59 | 41.6 | 67 | 47.5 | ||
| Child's age in days | 283 | 373.3 (322.0–424.0)* | 142 | 373 (320.0–427.0)* | 141 | 373.6 (327.0–417.0)* | 0.00† | 0.98† |
| Hemoglobin | 282 | 99 (90–104)* | 142 | 96.1 (91–104)* | 140 | 96.7 (89–104)* | 0.40† | 0.69† |
| Participants' age | 140 | 28.1 (25.0–30.0)* | 141 | 29.1 (25.0–31.0)* | 281 | 28.6 (25.0–30.0)* | 1.08† | 0.30† |
| Primary caregiver | ||||||||
| Mother | 269 | 95.1 | 136 | 95.8 | 133 | 94.3 | – | 0.59‡ |
| Grandmother | 11 | 3.9 | 4 | 2.8 | 7 | 5 | ||
| Other | 3 | 1.1 | 2 | 1.4 | 1 | 0.7 | ||
| Mother's education level | ||||||||
| Primary school | 14 | 5 | 3 | 2.1 | 11 | 7.8 | 0.07 | 0.95 |
| Junior high school | 217 | 76.7 | 110 | 77.5 | 107 | 75.9 | ||
| Senior high school | 36 | 12.7 | 21 | 14.8 | 15 | 10.6 | ||
| College and above | 16 | 5.7 | 8 | 5.6 | 8 | 5.7 | ||
| In your mind, is anemia common for infant and young child | ||||||||
| Yes | 87 | 30.7 | 47 | 33.1 | 40 | 28.4 | 0.79 | 0.68 |
| No | 141 | 49.8 | 69 | 48.6 | 72 | 51.1 | ||
| Do not know | 55 | 19.4 | 26 | 18.3 | 29 | 20.6 | ||
| In your mind, how harmful is the impact of anemia on health | ||||||||
| No problem | 27 | 9.5 | 11 | 7.8 | 16 | 11.4 | –0.25 | 0.80 |
| A little harmful | 43 | 15.2 | 22 | 15.5 | 21 | 14.9 | ||
| Harmful | 142 | 50.2 | 76 | 53.5 | 66 | 46.8 | ||
| Serious | 37 | 13.1 | 18 | 12.7 | 19 | 13.5 | ||
| Very serious | 8 | 2.8 | 3 | 2.1 | 5 | 3.6 | ||
| Do not know | 26 | 9.2 | 12 | 8.5 | 14 | 9.9 | ||
| Received any anemia information | ||||||||
| Yes | 35 | 12.4 | 21 | 14.8 | 14 | 9.9 | 3.15 | 0.21 |
| No | 236 | 83.4 | 113 | 79.6 | 123 | 87.2 | ||
| Do not know | 12 | 4.2 | 8 | 5.6 | 4 | 2.8 | ||
| Received anemia information for question 1 | ||||||||
| Yes | 143 | 50.5 | 73 | 51.4 | 70 | 49.7 | 0.09 | 0.77 |
| No | 140 | 49.5 | 69 | 48.6 | 71 | 50.4 | ||
| Received anemia information for question 2 | ||||||||
| Yes | 109 | 38.5 | 58 | 40.9 | 51 | 36.2 | 0.65 | 0.42 |
| No | 174 | 61.5 | 84 | 59.2 | 90 | 63.8 | ||
*Continuous variables were expressed as median and quartiles (Q25–Q75).
†Median (Q25–Q75), Mann–Whitney U/Wilcoxon W test was used to compare differences between group 1 and group 2.
‡Fisher exact test was used to compare differences between group 1 and group 2.
Response rate during the pilot
Figure 3 demonstrates that there was a decrease in response rate over time during the pilot. From week 1 to week 6, the first text messages’ response rates in township 1 were 54.4%, 50.3%, 46.6%, 41.1%, 1.8%, and 51.2%, respectively. Meanwhile the response rates of township 2 were 42.3%, 24.4%, 31.5%, 34.6%, 0%, and 29.1%, respectively. The response rate in the 5th week dropped to nearly zero due to the problem with the text messaging platform. In the 6th week, the response rate increased to a similar level as the response rate in the first week when we changed to the new text message platform. In the 2nd week, the response rate dropped significantly in township 2, because we sent the text messages only twice for those in township 2, while we sent text messages four times to those in township 1.
Figure 3:
Response rates from the 1st to the 6th week; (a) first text message; (b) second text message.
Response rate during the randomized controlled trial
Figure 4 shows the response rate for each week during the randomized controlled trial. At baseline (the 7th week), the response rates for the first question were 49.3% in group 1 and 48.9% in group 2. The response rates for the second question were 57.8% in group 1 and 51.9% in group 2; there was no significant difference between the two groups (χ2 = 0.09, p = 0.77 for response rate of text message 1, and χ2 = 0.65, p = 0.42 for response rate of text message 2). After introduction of the interventions in group 1, the response rates increased dramatically in the 8th and 9th weeks. Subsequently, the response rate decreased in the 10th and 11th weeks, but was still higher than in the 7th week. The response rate in group 2 continued to decrease during weeks 8–11. The response rate in group 1 was higher than in group 2 during weeks 8–11 (when the interventions were being undertaken) and the differences were significant (for the 8th week to the first message, p = 0.01; and for other text messages, p<0.001). In the 12th week, we applied the same interventions to both groups, and the response rate to the first question for group 2 increased and was maintained steadily until the 15th week for both groups. For the second question, the response rate for groups also increased rapidly and continued to rise until the 15th week for both groups. The differences between the two groups from the 12th to the 15th week were not statistically significant.
Figure 4:
Response rate from the 7th to the15th week; (a) first text message; (b) second text message.
Contribution of text messages sent during the second day
Figure 5 shows that on the first day of sending text messages, we obtained 2171 (88.3%) responses of the total 2457 messages received in 2 days. On the second day when we sent four text message reminders, we received only 286 (11.7%) responses of the total 2457 messages received.
Figure 5:
The proportion of responses gained on the second day of sending text messages.
Reasons for non-response
We planned to interview 61 caregivers via telephone, but 12 (20%) of them failed to be contacted (one refused, two were answered by other caregivers, three did not answer the phone, three kept their mobile phones switched off, and three stopped mobile phone services completely). Therefore, we interviewed 49 caregivers and obtained 128 reasons for non-response (one caregiver could provide multiple answers). The main reason reported by caregivers for not responding was that they did not send text messages often (n = 45, 92%), they did not receive the text message (n = 27, 55%), and they did not want to reply (n = 19, 39%) (figure 6). Caregivers expressed their unwillingness to reply by complaining that messages were too much effort or not useful, that children were not taking their anemia medication any more, that they worried about the strange phone number or the fees associated with messages, and that they refused to receive those messages. The reasons for caregivers not responding to messages were that their mobile phones were powered off, out of credit, or broken, or that they just did not receive those messages.
Figure 6:
Distribution of different reasons of non-response.
Factors affecting the response rate
Apart from variables like township, week, and group, other individual-level factors might also affect caregivers’ response to text messages. Therefore, we established a regression model for the baseline (7th week) to select potential factors. There were 20 variables in the model as listed in table 3; we found that caregivers who were younger (p = 0.02) and whose children had a lower hemoglobin test value (p = 0.02) were more likely to reply to text messages. We incorporated these two variables into the model for the longitudinal study, and found that the response rate in group 1 was significantly higher (OR = 2.41, 95% CI 1.37 to 4.27, p = 0.002), while the response rate in the 4th week of intervention was significantly lower (OR = 0.53, 95% CI 0.33 to 0.83, p = 0.006). Table 4 lists the factors influencing the response rate in the study.
Table 3:
Comparison between the two groups at baseline (7th week)
 |
Table 4:
Factors impacting response or non-response to the 1st text message
| Items | OR | 95% CI for OR | p Value |
|---|---|---|---|
| Group | |||
| 1 | 2.41 | 1.37 to 4.23 | 0.002 |
| 2 (ref) | 1.00 | – | . |
| Week | |||
| 15th | 1.34 | 0.82 to 2.20 | 0.24 |
| 14th | 1.25 | 0.77 to 2.05 | 0.37 |
| 13th | 1.39 | 0.88 to 2.19 | 0.16 |
| 12th | 1.42 | 0.90 to 2.24 | 0.13 |
| 11th | 0.53 | 0.33 to 0.83 | 0.01 |
| 10th | 0.96 | 0.61 to 1.51 | 0.86 |
| 9th | 1.57 | 1.00 to 2.46 | 0.05 |
| 8th (ref) | 1.00 | – | . |
| Township | |||
| 1 | 0.79 | 0.45 to 1.40 | 0.43 |
| 2 | 1.00 | – | . |
| Caregivers’ age | 0.97 | 0.93 to 1.02 | 0.30 |
| Hemoglobin test | 0.98 | 0.95 to 1.00 | 0.11 |
DISCUSSION
Principal results
Our study showed that the three interventions (prolonging the time of text messaging survey, giving mobile phone credit as a reward, and providing feedback) were effective in increasing the response rate of text messaging data collection in rural China. However, sending text messages during a second day only contributed marginally to the response rate. The main reasons for not responding were that caregivers were not used to sending messages often, followed by the fact that they did not receive text messages, and they did not want to reply. Younger caregivers and caregivers whose children had a lower hemoglobin value were more likely to reply.
Comparison with prior work
The response rate is a key issue for text messaging data collection. Studies indicated that response rates varied from 15% to 100%.5,7,13–18 In our previous study, the response rate was 38.4% for the first message among caregivers in the same study area.20 In this study, we succeeded in achieving a higher response rate: in group 1 from the 8th week to the 15th week the response rate was more than 50%, and in the 9th week, the highest response rate of 69.1% was attained.
We tested interventions that were potentially useful to increase the response rate, such as sending another four text messages on the second, consecutive day. In a double blind randomized controlled trial, Kew et al.5 obtained an encouraging response rate of 100% with three text message reminders to participants and phone call reminders. Reminders can prompt caregivers to reply and raise participants’ awareness of the survey.5,15 However, our study showed that while sending text messages on the second day did increase the response rate, it only contributed 11.7% of the total amount of information we collected. Taking into consideration that this potentially could have annoyed participants and only made a small contribution, it was postulated that sending messages on a second, consecutive day may not be useful.
Providing feedback
We sent text messages to tell caregivers who had good drug adherence to continue giving children anemia medication. For caregivers whose children did not have good medication adherence, we sent messages to emphasize the importance of medication and encouraged them to give medication to their children. This intervention aimed to make caregivers feel that they benefited from receiving and replying to our text messages and hence motivated caregivers to respond. This intervention might be a reason for the increased response rate, but it might not be the reason for maintaining a high response rate, as the text message feedback was provided to group 1 only in the 8th week and to group 2 only in the 12th week.
Giving mobile phone credit as a reward
Previous studies showed that money incentives have been shown to be effective in increasing survey response rate.36,37 We provided ¥5.0 (US$0.79) mobile phone credit as a reward to those who replied to our text messages for the following 2 weeks. This may explain the sustained increase in the response rate over a period of time. Since the effectiveness of the previous two interventions was questionable, this intervention seemed to be the most promising way to increase the response rate.
Other factors that may increase the response rate
During the anemia trial, there were more interventions than we could test. Those interventions may also have played a part in increasing the response rate, including the complementary food recipe booklet, the food blender, face-to-face group counseling on anemia and feeding, and text messages about nutritional knowledge. They may have gained trust and thereby increased the response rate. In addition, younger mothers may have been more likely to keep the mobile phone on all the time or may have been more used to text messaging. For caregivers whose children have a lower hemoglobin value, caregivers may pay more attention to children’s anemia and thereby may have been more likely to respond.
Strengths and limitations
To our knowledge, this is the first study to test interventions influencing the response rate of longitudinal text message data collection. We used a delayed intervention method and demonstrated the effectiveness of interventions. However, our study had some limitations. First, we were restricted to the anemia program, in which we already provided a complementary food recipe booklet, a food blender, face-to-face group counseling on anemia and feeding, and text messages about feeding knowledge for township 1, which kept the response rate in the 7th week at a high level. We had no way to ascertain how the anemia program interventions contribute to the highest response rate. Second, because the three interventions were introduced together, we were unable to determine the effect of each intervention separately. Third, the type of mobile phone contracts or service models used by the study population might have influenced their participation and response rates. However, we did not collect such information and therefore could not analyze this aspect.
CONCLUSION
Text messaging could be an innovative way of data collection and has significant potential for improving data collection in health research. However, the response rate is the bottleneck for text messaging data collection in different settings, including rural China. We confirmed a way to increase the response rate of text messaging data collection, which can be used in subsequent studies. Larger multi-site studies are needed to find the most effective way of using these interventions and to explore standard guidance or principles in undertaking text messaging surveys.
ACKNOWLEDGEMENTS
We thank all the caregivers for participating in our study. We also thank our colleagues in the Zhao County Maternal and Child Health Hospital for coordination and support of the fieldwork.
CONTRIBUTORS
The study was initiated, conceptualized, and supervised by YZ. YL and XD collected and coded the data. YL and WW performed data analysis. YL, WW, LC, and MHvV participated in the explanation and discussion of the results. The manuscript was drafted by YL, reviewed and revised by WW, LC, YZ, MHvV, IR, and JC. All authors read and approved the final manuscript.
FUNDING
The study was funded by the Capital Institute of Pediatrics (Project No. Fangxiang -14-05) and the Ministry of Health of China (Project No. 201002006).
COMPETING INTERESTS
MHvV is funded by Imperial Global eHealth Unit studentship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
PATIENT CONSENT
Obtained.
ETHICS APPROVAL
We obtained ethics approval from the Ethics Committee of the Capital Institute of Pediatrics in Beijing. All participants gave their informed consent through a text message for text messaging survey or verbally for telephone interviews prior to their inclusion of the study. We omitted all details that might disclose the identity of the participants.
PROVENANCE AND PEER REVIEW
Not commissioned; externally peer reviewed.
REFERENCES
- 1.2013. United Nations. The Millennium Development Goals Report. United Nations. http://www.un.org/millenniumgoals/pdf/report-2013/mdg-report-2013-english.pdf.
- 2.2013. mobiThinking Global mobile statistics 2013 Part A: Mobile subscribers; handset market share; mobile operators. http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/a#chinasubs.
- 3.Montan M, Short M, Barbulescu R, et al. The insiders’ guide to mobile marketing using SMS: the golden rules and top tips. 2013. http://mobithinking.com/guide-to-SMS-marketing
- 4.Ministry of Industry and Information Technology of the People's Republic of China. 2013 operations report for the telecommunications industry. 2014. http://www.miit.gov.cn/n11293472/n11293832/n11294132/n12858447/15861120.html
- 5.Kew S. Text messaging: an innovative method of data collection in medical research. BMC Res Notes 2010;3:342. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lagerros YT, Sandin S, Bexelius C, et al. Estimating physical activity using a cell phone questionnaire sent by means of short message service (SMS): a randomized population-based study. Eur J Epidemiol 2012;27:561–6. [DOI] [PubMed] [Google Scholar]
- 7.Johansen B, Wedderkopp N. Comparison between data obtained through real-time data capture by SMS and a retrospective telephone interview. Chiropr Osteopat 2010;18:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kuntsche E, Labhart F. Investigating the drinking patterns of young people over the course of the evening at weekends. Drug Alcohol Depend 2012;124:319–24. [DOI] [PubMed] [Google Scholar]
- 9.Axén I, Bodin L, Kongsted A, et al. Analyzing repeated data collected by mobile phones and frequent text messages. An example of low back pain measured weekly for 18 weeks. BMC Med Res Methodol 2012;12:105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bopp JM, Miklowitz DJ, Goodwin GM, et al. The longitudinal course of bipolar disorder as revealed through weekly text messaging: a feasibility study. Bipolar Disord 2010;12:327–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Eyrich-Garg KM. Mobile phone technology: a new paradigm for the prevention, treatment, and research of the non-sheltered “street” homeless? J Urban Health 2010;87:365–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Maher JE, Pranian K, Drach L, et al. Using text messaging to contact difficult-to-reach study participants. Am J Public Health 2010;100:969–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Whitford HM, Donnan PT, Symon AG, et al. Evaluating the reliability, validity, acceptability, and practicality of SMS text messaging as a tool to collect research data: results from the Feeding Your Baby project. J Am Med Inform Assoc 2012;19:744–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bexelius C, Merk H, Sandin S, et al. SMS versus telephone interviews for epidemiological data collection: feasibility study estimating influenza vaccination coverage in the Swedish population. Eur J Epidemiol 2009;24:73–81. [DOI] [PubMed] [Google Scholar]
- 15.Moller M, Attermann J, Myklebust G, et al. Injury risk in Danish youth and senior elite handball using a new SMS text messages approach. Br J Sports Med 2012;46:531–7. [DOI] [PubMed] [Google Scholar]
- 16.Anhøj J, Møldrup C. Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): response rate analysis and focus group evaluation from a pilot study. J Med Internet Res 2004;6:e42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Schembre SM, Yuen J. Project TwEATs. A feasibility study testing the use of automated text messaging to monitor appetite ratings in a free-living population. Appetite 2011;56:465–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Roberts A, Gorman A. Short message service for outpatient data collection. Br J Anaesth 2009;102:436. [DOI] [PubMed] [Google Scholar]
- 19.Haberer JE, Kiwanuka J, Nansera D, et al. Challenges in using mobile phones for collection of antiretroviral therapy adherence data in a resource-limited setting. AIDS Behav 2010;14:1294–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Li Y, Wang W, van Velthoven MH, et al. Text messaging data collection for monitoring an infant feeding intervention program in rural China: a feasibility study. J Med Internet Res 2013;15:e269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.O'Toole J, Sinclair M, Leder K. Maximising response rates in household telephone surveys. BMC Med Res Methodol 2008;8:71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997;50:1129–36. [DOI] [PubMed] [Google Scholar]
- 23.Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009;(3):MR000008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Brick JM, Brick PD, Dipko S, et al. Cell phone survey feasibility in the U.S.: sampling and calling cell numbers versus landline numbers. Public Opin Q 2007;71:23–39. [Google Scholar]
- 25.The government of Zhao County. The brief overview of Zhao County. 2004. http://www.zhaoxian.gov.cn/summary.jsp?id=360&code=00073484-8/2010-01498&name=%E8%B5%B5%E5%8E%BF%E6%A6%82%E8%BF%B0
- 26.The information center of Zhao County government. The overview of Shahedian township. 2012. http://www.zhaoxian.gov.cn/content.jsp?code=000735170/2012-00002
- 27.The information center of Zhao County government. The overview of Wangxizhang township. 2013. http://www.zhaoxian.gov.cn/content.jsp?code=00073462-9/2012-00005
- 28.Wu Q, van Velthoven MH, Chen L, et al. Improving the intake of nutritious food in children aged 6–23 months in Wuyi County, China—a multi-method approach. Croat Med J 2013;54:157–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Zhang S, Wu Q, van Velthoven MH, et al. Smartphone versus pen-and-paper data collection of infant feeding practices in rural China. J Med Internet Res 2012;14:e119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Ministry of health of the people’s republic of China. Report for nutrition development in children 0–6 years old in China. 2012. http://news.xinhuanet.com/edu/2012-06/01/c_123220070.htm
- 31.Higginson IJ, Vivat B, Silber E, et al. Study protocol: delayed intervention randomised controlled trial within the Medical Research Council (MRC) Framework to assess the effectiveness of a new palliative care service. BMC Palliat Care 2006;5:7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Broderick CR, Herbert RD, Latimer J, et al. Feasibility of short message service to document bleeding episodes in children with haemophilia. Haemophilia 2012;18:906–10. [DOI] [PubMed] [Google Scholar]
- 33.Macedo LG, Maher CG, Latimer J, et al. Feasibility of using short message service to collect pain outcomes in a low back pain clinical trial. Spine 2012;37:1151–5. [DOI] [PubMed] [Google Scholar]
- 34.Axén I, Bodin L, Bergström G, et al. The use of weekly text messaging over 6 months was a feasible method for monitoring the clinical course of low back pain in patients seeking chiropractic care. J Clin Epidemiol 2012;65:454–61. [DOI] [PubMed] [Google Scholar]
- 35.Kuntsche E, Robert B. Short message service (SMS) technology in alcohol research—a feasibility study. Alcohol Alcohol 2009;44:423–8. [DOI] [PubMed] [Google Scholar]
- 36.Whiteman MK, Langenberg P, Kjerulff K, et al. A randomized trial of incentives to improve response rates to a mailed women’s health questionnaire. J Womens Health (Larchmt) 2003;12:821–8. [DOI] [PubMed] [Google Scholar]
- 37.Olsen F, Abelsen B, Olsen JA. Improving response rate and quality of survey data with a scratch lottery ticket incentive. BMC Med Res Methodol 2012;12:52. [DOI] [PMC free article] [PubMed] [Google Scholar]