Tracking intervention delivery in the ‘Tobacco-Free Teachers/Tobacco-Free Society’ program, Bihar, India

Abstract

In health education and behavior change interventions, process tracking monitors the delivery of an intervention and its receipt to the intended audience. A randomized controlled trial in the state of Bihar, India was conducted to help school teachers become tobacco free through appropriately designed intervention program and delivery system. We describe the results from process tracking of this intervention delivery. The intervention program was centred on six topics delivered in each school through 12 sessions over 6 successive months. The program deliverers recorded the process measures as total number of sessions and program-components implemented (fidelity); time spent conducting sessions (dose) and proportion of teachers attending at least one session (reach). The outcome measures (teachers’ exposure to intervention messages and tobacco policy adoption) were assessed post-intervention. All 12 sessions were delivered in 33 out of 36 schools. Thirty-one schools implemented all six program components. In 18 schools, ≥95% of the teachers participated in one or more sessions. Thirty-three schools received 12 or more hours of dose. In 29 schools, 100% teachers reported exposure to all program messages. Tobacco policy was adopted by all schools. Thus, the intervention was generally delivered as planned and it had a positive impact on teachers and schools.

Introduction

Reports of intervention trials mostly focus upon outcomes. However, a systematic record and analysis of intervention delivery is also important in order to provide further insight into why a particular intervention was or was not found to be effective [1, 2]. Process tracking has gained particular importance in public health interventions in the last two decades as it helps in documenting and describing the intervention delivery process [1, 3]. For example, when a study finds that an intervention did not significantly improve the expected outcomes, findings from process tracking can help to explain the results by demonstrating whether or not the intervention was delivered as planned or participation rates were adequate. Conversely, when an intervention is shown to be effective, process evaluation results can point to the variations in the potential effects of different intervention components by documenting variations in intervention delivery and participation. These findings are critical to replicating and disseminating the tested intervention. The behavior change interventions are often complex due to the involvement of multiple sites, levels and components. In such interventions, process tracking helps to ensure that the intervention is delivered and received as intended and it is implemented equally across all sites [1, 4].

Tobacco use is one of the most important preventable causes of deaths and diseases [5]. In India, nearly one million tobacco-related deaths occur each year, adding significantly to the global burden of disease attributable to tobacco [6]. Unfortunately, very few people in India quit tobacco use due to the lack of awareness of harm, ingrained cultural attitudes and the lack of widely available cessation support resources [7]. A limited number of tobacco use cessation centers have been established in the country [8], primarily serving urban populations. Considerable data are available from few community-driven interventions implemented in the past, with varying levels of success [7, 9–15]. However, there is a lack of literature that describes the process of delivering the interventions [10], making it difficult to interpret which aspects may have contributed to the success of intervention.

This article describes the findings from the implementation process of a school-based tobacco use cessation intervention for school teachers in the Indian state of Bihar. Tobacco use prevalence was found to be high (78%) among teachers in Bihar according to the Global School Personnel Survey, 2000 [16]. Teachers have been identified as a high priority audience for tobacco control efforts in India, because they serve as role models for students, conveyors of tobacco prevention curricula, key opinion leaders for school tobacco control policies, and informed resource persons in the community [17, 18]. Other evidence indicates that schools with tobacco control policies in place have lower rates of tobacco use than those without [19].

Given these facts, the Bihar School Teachers Study (BSTS) was initiated to test the efficacy of a comprehensive tobacco control program (known as the ‘Tobacco-Free Teachers/Tobacco-Free Society’ program [TFT/TFS]) in increasing tobacco use cessation among teachers and promoting the adoption and implementation of school tobacco policies. The program resulted in higher quit rates in the intervention schools (50%) relative to the control schools (15%) and the quit rates were maintained even 9 months post-intervention (19% versus 7%, respectively) [20]. Understanding the process followed for this program delivery will underscore pertinent features of this highly successful teacher-focused, school-based program. We believe that results from this process evaluation might help in further contextualizing the study for other communities.

This article presents results from the process evaluation of the TFT/TFS program. It specifically examines the feasibility of delivering the intervention, measured as: (i) the extent to which intervention was delivered as planned (‘fidelity’); (ii) the ‘dose’ of intervention delivered; and (iii) the ‘reach’ (participation rate); and the outcomes, measured as (iv) the extent of teachers’ exposure to the intervention messages; and (v) the adoption of school tobacco control policy.

Methods

Study design

The BSTS was a cluster randomized control trial (RCT). The study was implemented over 2 academic years (‘2009–10 and 2010–11’) with half the schools randomly assigned to the intervention group receiving the intervention each year. The study received approval from Institutional Review Boards in India and the United States.

Study population

Ten districts out of 38 total districts in Bihar were selected to participate in this study. To be eligible for the study, schools needed to have at least eight teachers and be located in non-flood-prone areas, since flooding causes schools to close, which would restrict the intervention from being implemented as planned. From the selected districts, we procured a list of 2833 schools, representing grade levels 8–10, that were governed or aided by the Bihar state government. A total of 72 eligible schools were randomly selected from the sampling frame. Schools were then stratified by rural or urban and randomly assigned within each stratum to either intervention (n = 36) or control (n = 36) condition.

The TFT/TFS Program

The intervention program was based on the Social Contextual Model of Health Behavior Change [21] and on the findings from the formative research [22] as well as a pilot test (conducted in two schools). Health education for school personnel, cessation support, program materials and recommendations for policy-level changes at schools were provided as a part of the program.

Program deliverers

Health educators

The TFT/TFS intervention was delivered by six local Health Educators (HEs). They participated in a 4-day training workshop that reviewed the program components, explained their role in the intervention delivery, gave them an opportunity to practice using protocols and providing cessation support, and oriented them to the national legislation (National Tobacco Control Policy and Cigarettes and Other Tobacco Products Act). HEs were provided with continued support from the study team via regular phone check-ins and a refresher training conducted midway through the intervention.

Lead teachers

Each HE was assigned a set of five to six schools in which to deliver the intervention, with assistance from a designated teacher at each school, termed as Lead Teacher (LT). The LT played the role of a liaison between the HE and other school personnel at his/her school and was nominated by their school principal according to guidelines provided (being a non-user/quitter of tobacco, being respected among teachers, possessing good leadership qualities and having willingness to give personal time for the program). LTs, along with their principals, were asked to participate into a 2-day training workshop, which included a description of program components, an introduction to national tobacco legislation, an overview of their roles and responsibilities, and an orientation to program protocols that they were expected to implement. After the training, they were offered ongoing technical and programmatic support by the HEs through monthly school visits, ongoing phone supports and a mid-year refresher meeting.

Health education sessions

Health education sessions were conducted twice per month at each school in the format of group discussions and were centred on six topics (themes) [18]: (i) teachers as role models for tobacco control; (ii) health effects of tobacco; (iii) motivations to quit tobacco; (iv) skills to quit tobacco; (v) dealing with withdrawal symptoms; and (vi) maintaining abstinence from tobacco. The first session of each month was facilitated by the HE and the second session was facilitated by the LT ∼2 weeks later, with the HE present as an observer. These sessions were usually conducted during lunch-breaks (to avoid interference with teaching schedules) and engaged both tobacco users as well as non-users. In-depth protocols were created to ensure standardization of intervention delivery of both HE and LT sessions. Twelve sessions total were offered to each school.

Program materials

Schools were provided with supporting educational and programmatic materials including: (i) posters (one per theme); (ii) a calendar (displaying the monthly theme); (iii) a notice board to display materials; (iv) a suggestion box for the HE/LT to receive feedback/questions; and (v) self-help quit booklets (a step-by-step guide to quitting, available for each teacher). The program information (in English) and materials (in English and Hindi [local language]) are available for free access and download at http://sorensenlab.org/index.php/tobacco-free-teachers.

Cessation support

HEs provided cessation support to teachers during health education sessions. If teachers required support aside from the times that the HE was present, the LT provided assistance by providing them with the self-help quit booklet and directing them to the HE for further support.

School tobacco policy

A tobacco control policy draft was offered to each school with the intent that they would adopt it after making any desired modifications. To facilitate its implementation, schools were encouraged to write this policy on their letterhead. In addition, ‘No Tobacco’ signs were provided to the schools and the study offered to paint the policy on the school wall, as is a common practice in India for communicating messages. Additionally, a tobacco policy workgroup was formed in every school, with the responsibility of regularly announcing the policy and monitoring its implementation in school, thus building organizational support for quitters by creating a tobacco-free school campus.

Data collection and measures

The data were collected from three sources: (i) process tracking forms; (ii) a teachers’ survey; and (iii) a tobacco policy checklist (an observation checklist that recorded indicators of the presence/absence of the policy).

Process-related data and process evaluation indicators

The data pertaining to process of program delivery was recorded on total 12 process tracking forms, one unique form for each health education session.

Process tracking forms

The HEs were trained to record information on the forms in Hindi. Forms were checked for completeness after each session by the study team. These forms documented information on: (i) time spent by each HE and LT conducting the session with the teachers (in hours; excluding time spent on logistics); (ii) the number of school teachers participating in each session; (iii) the number of school personnel present on the school roster; (iv) whether or not specific topics were discussed as per the protocol (recorded as Y/N); (v) types of materials used in each session (recorded as Y/N); and (vi) queries raised by teachers along with responses given by HE or LT. The HEs recorded information about each health education session on the respective form at the end of the day. Based on the recorded data, we developed three process indicators: fidelity, dose and reach (Table I).

1. (i) ‘Fidelity’ of the intervention is defined in two ways:

   1. ‘Fidelity for sessions’ is defined as ‘the percentage of health education sessions conducted per school out of the total number of planned sessions (12)’.
   2. ‘Fidelity for program components’ is a ‘composite score per school’. Each school was expected to implement total six program components: (i) a notice board; (ii) a suggestion box; (c) six posters; (iv) a calendar; (v) quit booklet for each teacher and (vi) discussion on six program topics. Implementation of each of the first five program components was scored as ‘1’ if completed and ‘0’ otherwise. For the sixth component (discussion on six program topics), each program ‘topic’ was scored separately using the same method. So if a topic was discussed at least once in a school (as generating discussion on topic was more important than the frequency of discussion) then the score given to that particular topic was ‘1’. Hence, the maximum score for the sixth component was 6. The resulting score in total (for all six components) had a possible range of 0–11.

2. (ii) ‘Reach’ is defined as the ‘proportion of teachers who attended at least one health education session’. For each session in a school, we computed the participation rate based on the number of teachers who attended the session and those present on the roster. As we did not record the names of teachers attending sessions (to maintain confidentiality), it wasn’t possible to calculate the exact reach per school. Therefore, we considered the highest participation rate among all 12 sessions as reach. It is expressed as percentage and the maximum possible value for this measure is 100%.

3. (iii) ‘Dose’ is defined as ‘the total amount of time spent (‘in hours’) by the HE and LT leading discussions with teachers’ during their respective sessions at each school. This was obtained by summing up all the hours of the sessions conducted. As per the protocol, each school was expected to receive a total dose of 12 hrs over the entire intervention (or 1 hr per health education session).

Table I.

Process indicators

Process indicator	Definition	Measure
(i) Fidelity
(a) ‘Fidelity for sessions (in %)’	‘The percentage of health education sessions conducted per school out of the total number of planned sessions (n = 12)’	(Number of sessions conducted per school/12) * 100
(b) ‘Fidelity for program components (score out of 11)’	‘The composite score attained per school for implementing each of the six program components’	Program component	Score if implemented	Score if not implemented
		1.A notice board	1	0
		2.A suggestion box	1	0
		3.Six posters	1	0
		4.A calendar	1	0
		5.Quit booklet for each teacher	1	0
		6.At least one discussion on
		(i) Topic 1	1	0
		(ii) Topic 2	1	0
		(iii) Topic 3	1	0
		(iv) Topic 4	1	0
		(v) Topic 5	1	0
		(vi) Topic 6	1	0
		Maximum possible score (total) per school	11`	0
(ii) Reach (in %)	‘The proportion of teachers per school who attended at least one health education session’	In each school, participation rate was calculated for each session; where, participation rate per session = (Number of teachers present for the session / Total numbers of teachers available on the roster sheet) * 100.
		Session with the maximum participation value was considered as ‘reach’ for that school.
(iii) Dose (in hours)	‘The total amount of time spent per school (‘in hours’) by the HE and LT in the discussion of six topics with teachers’	Total time spent by the HE and the LT on discussing all six topics during their respective sessions per school.

Outcome-related data and outcome evaluation indicators

The outcomes of this article are: (i) teachers’ exposure to the intervention messages; and (ii) adoption of the tobacco control policy by schools. These data were collected from the teachers’ survey and a tobacco policy checklist, both administered post-intervention in each school.

1. ‘Teacher’s exposure’: A self-administered post-intervention survey with teachers and clerical staff at all schools was conducted in Hindi immediately after the program delivery was over (response rate = 72%, n = 684) [18]. Verbal informed consent was taken prior to the survey. The questions were asked about the respondents’ tobacco-related behavior, knowledge and demographics. A specific section of the survey collected information about respondents’ exposure to the program messages. It contained eight questions that asked whether or not the respondent heard, saw or read about the following topics in the past 6 months: (i) teachers as role models for tobacco control; (ii) health effects of tobacco; (iii) tobacco harms not just yourself but also your family; (iv) benefits of quitting tobacco use; (v) how to quit tobacco use; (vi) how to say no to tobacco when it is offered; (vi) dealing with withdrawal symptoms from tobacco; and (viii) how to support someone in quitting tobacco use. Responses and their respective scores given were: 3 = ‘heard, read or saw inside my school’; 2 = ‘heard, read or saw outside my school’ or 1 = ‘did not hear, read or see/not aware.’ Missing responses were scored as ‘0’. A summary score was calculated by summing the responses to the eight individual items (‘minimum “0”; maximum “24”’) and a score for each school was calculated by averaging the scores of the teachers who completed the survey in that school. The average score per school represented ‘the exposure of teachers in that school to the intervention’.

2. ‘Policy adoption’: A copy of the tobacco policy was requested from each school at follow-up. The presence of ‘No Tobacco’ warning signs and a wall painting displaying school’s tobacco control policy were measured using an observation checklist that was filled out by the field investigators during the post-intervention survey. Schools that recorded the tobacco control policy on their letterhead, painted the policy on the wall, and posted at least one ‘No Tobacco’ sign were considered to have adopted the policy.

Data analysis

Using the participating schools as the unit of analysis, descriptive statistics for each indicator were calculated. All analyses were carried out using Statistical Package for Social Sciences (‘S.P.S.S. version 13.0’) software.

Results

Sample characteristics

Schools were stratified by urban versus rural before randomization, so there were equal numbers (n = 18) of each in the intervention group. The average number of teachers per school was 14 (minimum: 7; maximum: 49). Over half of the rural schools had 7–9 teachers whereas half of the urban schools had 16–49 teachers (Table II). The majority of the teachers were male, and rural schools had more male teachers compared with urban schools. The median distance of schools from Patna (Bihar’s capital and the site of the research office) was 96 km (minimum: 7 km; maximum: 400 km) (data not shown in the table). Rural schools were located furthest from Patna as compared with urban schools.

Table II.

Sample characteristics

Rural/Urban (n = 36)		No. of schools falling in each category	Median percentage of male teachers	Median distance from Patna city (in kilometers)
Rural (n = 18)	7–9 teachers	10	88%	80
	10–15 teachers	6	81%	116
	16–49 teachers	2	57%	113
Urban (n = 18)	7–9 teachers	2	57%	15
	10–15 teachers	7	64%	35
	16–49 teachers	9	74%	107

Feasibility of intervention delivery/process

1. ‘Fidelity’: In all schools (n = 36), a total of 405 sessions were conducted out of 432 planned sessions for an overall fidelity of 94% for the sessions. Thirty two out of the 36 schools conducted all 12 sessions. In the remaining four schools, the number of sessions delivered ranged from 6 to 11.Similarly, 31 of the 36 schools implemented all six components (a notice board, a suggestion box, six posters, a calendar, quit booklets, and discussion on six topics) of the program and had the maximum fidelity score of 11. The remaining five schools implemented the first five components fully whereas the sixth component was implemented partially.

2. ‘Reach’: In nearly half (n = 18) of the schools, over 95% of the teachers participated in at least one health education session (Table III). In 16 out of 36 total schools, all teachers participated in a minimum of one session.

3. ‘Dose’: Even though the intervention was to be delivered over a total of 12 hrs per school (or 1 hr per health education session) as per the protocol, the process data indicated that in many schools (n = 33) more intervention hours were logged, indicating that the program deliverers spent more time on conducting sessions than was planned. Thirty-three schools received 12 or more hours (minimum: 14; maximum: 29) of health education sessions, while the remaining three schools received <12 hrs (minimum: 7; maximum: 11). A dose of 17 hrs (minimum: 7; maximum: 29) was delivered in over half of the schools.

Table III.

Descriptive statistics of process and outcome indicators

Process and outcome indicatorsa	Median	Range (minimum– maximum)
(i) Fidelityb for	—	—
(1) sessions ‘(in %)’	100	50 (50–100)
(2) program components ‘(score: 0–11)’	11	3 (8–11)
(ii) Reachb ‘(in %)’	95	33 (67–100)
(iii) Doseb ‘(in hours)’	17	22 (7–29)
(iv) Exposurec ‘(score: 0–24)’	24	3.6 (20.4–24)

^aPolicy adoption was recorded for each school on Yes/No scale and all 36 schools adopted the policy. Therefore, it is not included under outcome indicators.

^bProcess indicator.

^cOutcome indicator.

Outcome

1. ‘Exposure’: In 29 out of total 36 schools, all teachers reported that they had seen/read/heard about all eight program messages in their schools in the past 6 months (exposure score: 24 out of 24). On average, teachers in all schools reported that they had seen/read/heard about at least seven out of eight program messages during the intervention (exposure score: 23.3 out of 24). On the contrary, the exposure score among control schools was lower (14.9, not shown in the table).

2. ‘Tobacco policy adoption’: As per the criteria, all 36 schools (‘100%’) adopted the tobacco control policy.

Discussion

These results demonstrate the feasibility of delivering a tobacco control intervention for school-teachers in India. The results of the process evaluation showed that the intervention was generally delivered as planned. The average fidelity of the intervention (for sessions and program components) was high. Almost all schools received either close to maximum (12 hrs) or a higher ‘dose’ than targeted during the program. The two schools that had lower fidelity (50%) for sessions received a ‘dose’ of 7–8 h, which still represents delivery of a large proportion of the program. A high proportion of the teachers participated in the intervention (‘reach’); on average, >90% teachers in each school participated in at least one session. However, our estimates of reach could be underestimates since we did not record individual identity. It is quite possible that all teachers from a school may have attended at least one session. The teachers reported a high level of exposure (23.1 out of 24) to the intervention messages, indicating their exposure to almost all eight program topics. The slight differences in ‘dose’ and ‘reach’ did not affect the ‘exposure’ (as teachers reported based on what they learned through the displayed materials during the intervention) which again supports the possibility that the reach could be higher in reality. We also found that all intervention schools adopted the school tobacco control policy. This article however, doesn’t analyse the level of policy implementation.

A variety of factors may have contributed to the uniform delivery of the intervention across all schools. Full-fledged trainings were provided to the intervention deliverers along with intermittent refresher meetings to garner enthusiasm and provide continued technical support. Phone calls were conducted by the program coordinators with the HEs after each session they conducted/attended to obtain general feedback about the session and implement problem solving as necessary. Furthermore, LTs were instructed to call their respective HEs if their fellow teachers approached them for cessation support, which provided an extra opportunity for programmatic support and knowledge transfer. Because the HEs would directly call the teachers in need of support, it took an extra burden off of the LTs. All of these factors in their entirety equipped the program deliverers with the required knowledge, skills and support to deliver the intervention.

Apart from adherence to the protocols, the HEs took extra efforts to make the intervention delivery possible. The average distance of the schools from the study office in Patna was 96 km (7–400 km). To ensure the delivery of the sessions in the furthest schools, HEs traveled a day in advance of the session and stayed in the schools (with the prior permission of the school Principal), while for the inaccessible schools they walked the distance of anywhere from 1 to 6 km to reach to the schools. Climate conditions were another challenge that the HEs were dedicated to overcoming. For example, schools in Bihar generally are closed during winters due to extreme cold. Thus, the extraordinary commitment of the program deliverers to not only adhere to the protocols, but also to overcome any challenges with intervention delivery, along with support from the schools made the intervention delivery feasible. Future efforts to implement similar interventions may need to consider strategies to embed programs into existing school structures (e.g. capacity building and empowerment of the peer leaders from the schools e.g. LTs) to make the program less dependent on external staff and mitigate some of the structural barriers.

Our findings of process evaluation are in line with those from a study conducted for ‘students’ (sixth to ninth grade) in 32 Indian schools [10]. In this study, 13 out of 16 intervention schools fully implemented the classroom sessions, posters, postcards and inter-school components (rigor of implementation). Classroom sessions had high (97%) average participation rate. Both, the ‘rigor of implementation’ and ‘reach’ were associated with the decreased susceptibility to tobacco-use among students, further adding to the evidence that schools are a realistic venue for implementing tobacco control programs for both students and personnel.

Like schools, worksites are considered as a potential venue for tobacco control where social support system can be used for bringing in behavior change [23–28]. Process evaluation of another study (RCT) in worksites has found double quit rates (31% versus 15%) in the intervention arm where average participation rate (46% versus 34%) and mean minutes of person exposure to intervention (33 m versus 15 m) were higher compared with control [28]. In our study, high quit rates (50% versus 15%) were found immediately post intervention and a high effect size was maintained (19% versus 7%) even 9 months post-intervention [20].

We found no relationship between process indicators (fidelity, dose and reach) and outcome indicators (exposure and policy adoption) as there was little variability within each indicator across schools (hence, not reported). Our results indicate that high levels of fidelity, dose and reach together perhaps have had a synergistic effect on the exposure. Therefore, it is imperative that major efforts should be focused upon complying with the protocols (‘fidelity’ and ‘dose’) for future such interventions.

Evidence suggests that level of awareness and participation in activities may lead to behavior change [4, 10, 26, 28]. In our program, the participation in the sessions (reach) was high in addition to improved exposure to the messages (awareness) in intervention schools (Intervention: 23.1 versus Control: 14.9). Our intervention sessions were unique in that they engaged both users as well as non-users of tobacco. Due to this, the required knowledge (harms of tobacco and benefits of quitting) and skills to quit/ support quitting were imparted to even those who were non-users. In addition, the adoption of a tobacco control policy created school-wide norms and social support for being tobacco-free which played an important role in promoting cessation among teachers as found in other studies [19].

There are limitations to the interpretation of data we presented here. First, the data collected through forms and surveys were self reported and were subject to self report bias. However, we took several measures to mitigate the effect of this bias such as, collecting data on process from both intervention deliverers (HEs and LTs) as well as receivers (teachers); using multiple methods of data collection (i.e. documentation by project staff and survey of teachers); and training and regular check-ins with the data collection staff to ensure accuracy of the data. Second, the data on the process tracking forms were initially supposed to be recorded by HEs and LTs for their respective sessions. However, LTs couldn’t assume this responsibility due to competing time priorities. Because of this, HEs completed the forms of LTs’ sessions with the LTs’ input to ensure the completeness and validity of the information. Third, the exact reach of the program couldn’t be measured since we did not record individual identities of teachers. Future school-based programs may need to identify a strategy to link individual participation with the outcome.

On the whole we were able to track the intervention using a robust process tracking mechanism. Other studies have similarly used process tracking to document intervention implementation, with varying degrees of success [29, 30]. Findings from these studies add to the evidence that school-based programs are often complex and it is generally difficult to achieve high levels of fidelity, dose and reach in the studies that involve multiple sites, levels, and components.

Our findings showed that the intervention delivery was uniform and close to maximum in almost all of the schools. This study demonstrates the utility and significance of process evaluation in interpreting findings from outcome evaluation. Our intervention had a positive impact on tobacco control at schools (in terms of policy adoption and thus creating supportive organizational environment) as well as on teachers (by exposing them to different intervention messages, thus providing an opportunity for knowledge and skills transfer necessary for tobacco control). Understanding program implementation helps us better interpret our significantly high tobacco cessation rates that were sustained up to 9 months post-intervention.

Our results have important implications for future studies in similar settings. They indicate that focusing on increasing fidelity to the intervention protocols while targeting maximum coverage may increase the efficacy of the intervention. At the same time, it is especially important to accommodate the intervention within the organization’s schedule as seamlessly as possible. Ensuring that the program deliverers are adequately trained and dedicated to the program's messaging seems to be particularly important. Finding individuals within the community may help overcome logistical barriers, but this study demonstrates that external personnel as well as peers can be effective at delivering programs within a school community as well.

Funding

This work was supported by the National Cancer Institute at the National Institutes of Health (5R01 CA120958, 5K05 A108663).

Conflict of interest statement

None declared.