Validation of Mobile Messages for an mHealth Intervention for Smokeless Tobacco Cessation in India

Abstract

Background:

With the growth in use of mobile messages for behaviour change, the need to incorporate personal needs and cultural characteristics of target users has been promoted. The study aimed to describe the findings of content validation of mobile messages designed to promote smokeless tobacco cessation in primary care.

Methods:

This study used a concurrent mixed-method approach with 13 patients who were tobacco users at urban primary care clinics. The clarity and appeal of 32 messages were rated on a Likert scale from 1 to 10. A mean clarity and appeal score per message was generated. A 5-item discussion guide was used for in-depth interviews and data was analysed using framework analysis.

Results:

Participants found the content of the messages useful, and preferred shorter and audio formatted messages. The clarity scores for the messages ranged from 7.9 to 9.4 with an average score of 8.7 (SD 0.5). The appeal scores ranged from 7.3 to 9.2, with an average score of 8.5 (SD 0.6).

Conclusions:

Twenty-six from a total of 32 messages were found appropriate and finalised for use. This methodology can be used when developing contextually relevant mobile message interventions in other low resource settings.

Introduction

The World Health Organization (WHO) has identified mobile health (mHealth) technology as an important step in improving health access, knowledge and behaviours across a range of contexts and target groups (World Health Organization, 2017). Low- and middle-income countries (LMICs) like India are increasingly testing and using mobile technology-supported healthcare to improve health outcomes, including tobacco cessation (Akanbi et al., 2019).

In India, a variety of smokeless tobacco (SLT) products are consumed with associated health risks (Siddiqi et al., 2020). Brief advice from healthcare providers is effective in promoting smoking cessation (Stead et al., 2013). However, behaviour changes from such advice are difficult to sustain and need reinforcement. mHealth technology, in particular mobile messages can be used to achieve this (Cobos-Campos et al., 2021). India has the second-highest mobile phone connectivity in the world with 1.147 billion subscribers (Telecom Regulatory Authority of India, 2022). The most basic mobile phones can receive text messages that can be accessed at any time without the need for constant connectivity or internet access (Fjeldsoe et al., 2009; Pfammatter et al., 2016).

When developing mobile messages, one must consider personal needs and how to adapt the message to ensure clarity and appeal to the users (Chahar et al., 2018; Diez-Canseco et al., 2015). The users’ opinions and evaluations on the message so developed can help determine how well the messages are received by the target population (Diez-Canseco et al., 2015; Khandpur et al., 2021). The main aim of this study is to describe the findings of the content validation of a set of mobile messages that were developed as a behavioural change intervention for SLT cessation in an urban primary healthcare setting in Odisha, India.

Materials and Methods

Study design

A validation study was done prior to conducting an exploratory randomised controlled CERTAIN (Counselling intErvention foR smokeless Tobacco cessAtion in INdian primary care) trial (Panda et al., 2022). The mobile messages were a key component of the intervention designed to promote behaviour change. We used a concurrent mixed-method approach where quantitative and qualitative data allowed for an interactive analysis (Creswell and Clark, 2017).

Validation of the mobile messages

The messages were developed from formative research based on the steps of the Trans theoretical model (TTM) (Prochaska & Velicer, 1997) and have been described in a separate paper (Panda et al., 2023). In addition to text, messages also had links that could be used for viewing graphic messages and others had audio messages embedded in it.

Setting and participants

We collected face-to-face data from 13 patients who were tobacco users (SLT or dual users) presenting at the urban primary health centres (UPHCs) in Berhampur city in Odisha where the intervention was delivered. Data was collected from November 2020 to January 2021. A purposive sample ensured maximum variation among participants across gender, age, education and type of tobacco use. All those 18 years or older, in possession of a mobile phone with a valid contact number were eligible to take part and provided verbal and written consent.

Data collection tools and procedures

A survey was administered to capture feedback on message content clarity and appeal. The participants heard the audio messages and were also given a print out of the messages. For each message, participants had to score, on a Likert scale from 1 to 10, statements that capture their ease of understanding [e.g., ‘this message is clear’] and likeability [e.g., ‘this message is appealing’] of the message. In-depth interviews (IDIs) were also conducted using a 5-item discussion guide (Supplementary file, Appendix 2) for collecting information on their habit of tobacco and preferences for the messages. All IDIs were conducted in the local language Odia, recorded using a digital recorder, and then translated and transcribed verbatim for analysis.

Analysis of Validation

A mean score for clarity and appeal of each mobile message was generated and used to rank the messages in each stage of behaviour change. Framework analysis (Hackett, 2019) was used to analyse the qualitative data. Two researchers (SL, RRP) familiarized themselves with two randomly selected transcripts, and independently coded them using pre-developed and newly emerging codes. The codes were discussed between the authors, discrepancies were resolved before the penultimate codebook was finalised. After analysis, the themes were identified with quotes (i.e., the respondents’ exact words) and presented in the results. Atlas.ti (version 8) software was used for the analysis. Based on the quantitative and qualitative findings, the final set of messages were reviewed and approved by the research team.

All ethical approvals were received from the appropriate university institutional review boards.

Results

A total of 13 participants took part in the study, of which 11 (85%) were males and 2 (15%) were females.

Clarity and Appeal of mobile messages

The scores for clarity of the messages ranged from 7.9 to 9.4 (average score of 8.7 SD 0.5) and the scores for appeal ranged from 7.3 to 9.2 (average score of 8.5 SD 0.6) out of 10 (Supplementary file, Appendix 1). Based on the TTM of change, messages relating to preparation phase were given the highest score on clarity as well as appeal (Table 1).

Table 1.

Mean Clarity and Appeal Score (1-10) of Developed Messages

Domains of Messages	No of Messages	Clarity Score (Mean ± SD)	Appeal Score (Mean ± SD)
Pre-contemplation	10	8.61 ± 0.40	8.40 ± 0.64
Contemplation	10	8.48 ± 0.51	8.18 ± 0.57
Preparation	4	9.21 ± 0.30	9.07 ± 0.24
Action	3	8.54 ± 0.21	8.40 ± 0.54
Maintenance	5	8.62 ± 0.59	8.5 ± 0.60

Pre-contemplation phase

A graphical message that presented pictures of various oral diseases occurring from consuming SLT received the lowest scores on both clarity and appeal (SMS #5). The message that received the highest score was one in which the participant was asked if they were aware of the information shared in the message (#4).

Contemplation phase

The highest rated message addressing this phase was an audio message of a conversation offering help and support to the user to quit (SMS #17).

Preparation stage

A conversation between two friends of one motivating the other to quit the habit (#21) and messages offering help to do this (#22, #23) received the highest rating for both clarity and appeal.

Action stage

An audio message that encouraged users to quit SLT for the sake of their family’s well-being was rated high on clarity and appeal (#25). A message encouraging users to change their habits by offering alternative suggestions (#26) was also rated high on both clarity and appeal.

Maintenance stage

A message describing how a former tobacco user was able to refuse SLT (#32) was rated high on both, clarity and appeal (Supplementary file, Appendix 1).

Based on these findings six messages with scores of eight points and below on the clarity or appeal scale, (two from the pre-contemplation and the contemplation stage each, one from the action and maintenance stage each) were found not suitable and thus not used in the final intervention. A total of 26 messages were considered suitable for use in the intervention (Table 2).

Table 2.

Stages of Change, Type of Content, and Number of Mobile Messages Per Stage

Stage of Change	Type of content of the message	Example of the SMS	No. of messages included for validation	No. of messages eliminated after validation
Stage 1: Pre-contemplation	Information about the harmful effects of consumption	Tobacco is the main cause for cancer	10	2
Stage 2: Contemplation	Benefits of quitting and support to overcome the addiction	“Remember - how beautiful your family is. ‘Tobacco’ consumption hurts your family”	10	2
Stage 3: Preparation	Encouragement for users to quit SLT	“If you want to quit tobacco, and you cannot tell anyone, or cannot ask anyone, don’t worry about all this, contact us, let us know your wishes, send us a SMS on mobile. Why are you not leaving? Tell us freely. Your problem to quit tobacco, it will be solved.”	4	0
Stage 4: Action	Urging users to actively change their habits	“If you know that the end result of tobacco is ' death', then stop consuming it today. Yes, it would be difficult to leave it in the first go, your desire to eat again and again will be there. Then do it like this 1. First divert your attention from tobacco to another work. 2. Think about why you wanted to quit tobacco. find out why. 3. Drink as much water as possible. 4. Instead of tobacco, you can eat something without sugar. Gradually you will be able to quit tobacco.”	3	1
Stage 5: Maintenance	Positive reinforcement of their quit attempt	If you have quit tobacco for 24 hours, you have taken the first step to success. Yes, sir praise yourself. Applaud yourself Because you kept yourself away from tobacco for a full 24 hours. You tried – you should move forward.	5	1

Total final mobile messages, 26

Participants’ preferences regarding different aspects of the messages

Perception of the mHealth intervention

Most participants were of the opinion that the proposed intervention would be useful and could facilitate constant motivation that is required to quit SLT.

“If messages are sent repeatedly to the mobile phones, then there is a chance of any change in thinking” (SLT user 11).

Intention to quit using the mHealth intervention

Participants found messages to be of use. They felt that the pictorial messages had the potential to make them quit SLT

“I think after watching pictures in mobile, I can try to quit it (SLT)” (SLT user 11).

Content of the messages

Participants were of the opinion that the content of most of the messages was useful and beneficial.

“No useless things are there in the messages, all the information is useful” (SLT user 3)

One of the participants was of the opinion that the content should focus more on how to quit SLT

“The content should be as how to leave the habit of tobacco use” (SLT user 10).

Format of the messages

One of the participants mentioned that audio messages would be preferable.

“I can’t read properly. Listening to the audios is suitable” (SLT user 13).

Length of the messages

Participants were generally satisfied with the length of all the messages.

“It (length of messages) is okay” (SLT user 10)

One of the participants mentioned that short messages would be preferable.

“Short cut for today as people lack time” (SLT user 9).

Discussion

The content of most messages was valid and useful for SLT cessation. Participants preferred audio messages. They favoured messages that had information on specific methods for quitting. Five messages received a score between seven and eight on clarity and 27 messages received a score between eight and 9.5. On appeal of messages, nine messages received a score between seven and eight and 23 messages received a score between eight and 9.5. Overall, the participants rated some of the longer messages lower on appeal and preferred shorter messages. Based on these findings we finalised 26 messages which received high scores on clarity and appeal.

In our study, short messages were considered most helpful while longer messages repetitive (Müssener et al., 2016). The audio messages were consistently highly rated on both, clarity and appeal. Participants also described their preference for audio messages in the interviews. A study delivering maternal and child health communication in rural Malawi found that the use of both, text and voice messages extended access to health information to illiterate phone owners (Crawford et al., 2014). In our study, the majority of messages (80%) that had conversations were consistently rated high for both clarity and appeal. For instance, a conversation between two friends encouraging quitting SLT received more than nine points on both clarity as well as appeal. Messages offering support and motivation for participants’ quit attempts were also well-received. The messages that emphasize the benefits of quitting tobacco are known as gain-framed messages while those that emphasize the disadvantages of use are loss-framed (O’Keefe and Jensen, 2007). Previous studies have shown that loss-framed messages are less effective and persuasive for behaviour change (O’Keefe and Jensen, 2007; Toll et al., 2007). In our study too, the loss-framed messages that emphasised the costs and ill-effects of SLT use were rated poorly by users.

Previous studies have demonstrated the importance of social context while tailoring messages for enhancing the acceptability and effectiveness of interventions (Graham et al., 2016; Skov-Ettrup et al., 2014). We tailored text messages for situations that user could easily relate to. For instance, we included a conversation taking place at a local betel (paan) store which discouraged the user to buy tobacco and opt for healthier alternatives that are sold at such local stores. Local dialect was used while formulating the messages. We also used local SLT products that are easily available and commonly used so that the tobacco users could easily identify with the messages related to the product.

Work pressure and peer influence were cited as most common reasons for relapse. Peer influence has been identified as an important determining factor for tobacco use in India (Ghose et al., 2019; Hasan et al., 2020). Our study included messages that target not only individual behaviour but also social influence and group behaviour. The messages that aimed at developing confidence to overcome peer pressure were received well by the participants.

Most studies usually focus on the effectiveness of message-based interventions for smoking cessation and there is limited information on validation of the content of mobile messages for SLT products. The validation process investigates the degree to which the content of the messages is adequate for effecting behaviour change and maximizing intervention effectiveness (Sharma et al., 2020). Overall, participants found the messages motivating and reported their intention to quit after receiving messages. The technique utilised in this study involved inputs from potential recipients, which helped finalise a set of messages that were clear and appealing for the potential users.

This study used an evidence-based approach to develop and validate a set of mobile messages developed from a strong theoretical perspective for SLT cessation. This approach can be used by other researchers to develop context-specific messages for mobile interventions in the future.

Strengths and Limitations

The available project resources combined with the COVID-19 pandemic limited the study to only 13 SLT users. Hence, we were unable to analyse any associations between the message scores and sociodemographic characteristics of users. Despite the limited number of participants all 32 messages in the local language (Odia) were reviewed for both, clarity and appeal.

Authors contribution Statement

All authors contributed to the manuscript. RP (PI of the study—India) and IN (PI of the study—UK) conceptualised the study, reviewed and finalised the paper. SL supported the curation of data and was involved in data analysis, writing the original draft, review and editing. RRP reviewed and assisted in writing the early drafts of the paper. SD oversaw the data collection and ensured data quality, reviewed and gave comments to improve the paper. SKP trained the data collection team, collected data, reviewed and gave comments to improve the paper.

Acknowledgements

The authors would like to thank primary care physicians and senior residents of MKCG Medical College for their support and dedication in data collection.

Funding statement

This research work is funded by the Department of Health and Social Care (DHSC), the Foreign, Commonwealth & Development Office (FCDO), the Medical Research Council (MRC), Wellcome Joint Global Health Trials Initiative (ref: MR/P021166/1).

The study was registered at Clinical Trials Registry India (reference number CTRI/2019/05/019484) dated May 31, 2019.

Ethics approval

Ethics approval was obtained from the Institutional Ethical Committee at Public Health Foundation of India (PHFI) (ref: TRC-IEC-391/19; dated May 29, 2019). At the national level, ethical clearance was obtained from the Health Ministry’s Screening Committee (HMSC), led by the Indian Council of Medical Research (ICMR) (ref: 2019-3581; dated December 11, 2019). Also, local level approval was obtained from the Odisha State Ethics Board (ref: 191/PMU/187/17; dated November 14, 2019). In the UK, ethical clearance was obtained by the UCL Research Ethics Committee (ref: 5686/001, dated October 1, 2019). The study was registered at Clinical Trials Registry India (reference number CTRI/2019/05/019484) dated May 31, 2019.

Availability of data

The data is available with the author (SL) and will be provided on reasonable request.