A Lexicon for Measuring Maintenance of Behavior Change

Abstract

Objectives

To establish a workgroup within the NIH-funded Health Maintenance Consortium (HMC) to examine how “maintenance” of behavior change was conceptualized and measured across and within behaviors.

Methods

Multiple meetings were held by the workgroup to reach consensus definitions of maintenance and maintenance-related constructs across diet/nutrition, tobacco, substance abuse, and physical activity. Once consensus was reached, a survey assessed how maintenance was operationalized across 16 HMC intervention studies.

Results

Seventy-five percent of 16 studies assessed are using a criterion to assess maintenance and are tracking maintenance as a continuous measure. Eighty-one percent are assessing facilitators and barriers, and conceptualizing maintenance as both an intermediate and primary outcome measure. All 16 studies are assessing maintenance at the individual level with fewer at the organizational (N=3), environmental (N=3), and policy levels (N=1).

Conclusions

This survey found similarities and differences in measurement across behaviors that have important implications for advancing the quality of transbehavioral research.

Great strides have been made over the past decades designing behavioral and social interventions to change an array of key lifestyle and other health-related behaviors.¹ Despite these advances, understanding the long-term sustainability of behavioral changes remains a major challenge.^2–5 The National Institutes of Health (NIH)-funded Health Maintenance Consortium (HMC) provided a forum for the promotion of cross-site synergies through an examination of commonalities in research questions, methods, and measurements across site-specific studies. In that way, the HMC could extend research findings beyond single studies to compare multiple behaviors in different populations and settings. These synergies were facilitated by supplemental funding to examine cross-site definitions of adherence/maintenance behaviors and to develop common definitions, measures, and analytical approaches. In this paper, we present findings from the efforts of the Maintenance Workgroup of the HMC, including its consensus concerning the definition of maintenance and maintenance-related constructs across multiple target behaviors and cross-site data from 16 NIH-funded intervention studies detailing ways in which the studies operationalized these constructs. The paper examines the applications and operationalizations of maintenance-related constructs within a major research consortium dedicated to advancing knowledge of behavior-change processes and change strategies. The findings provide a common nomenclature, a description of current measures, and an understanding of challenges that remain to be addressed with respect to measurement in the emerging field of behavior maintenance.

METHODS

A maintenance workgroup composed of HMC investigators focusing on different and often single health behaviors (eg, physical activity, or eating, or smoking, etc) was formed at the first HMC grantee meeting to foster collaboration across sites and promote synergies across them. The workgroup used multiple face-to-face meetings and structured conference calls to reach consensus on a common typology to define maintenance and other constructs associated with maintenance that could be applied across the different health behaviors studied. Recognizing that substantial differences could occur across HMC studies, the group developed a survey using the agreed-upon terminology and operational definitions to investigate commonalities and differences further. The survey was designed to elicit detailed information about the ways in which HMC investigators collected data on maintenance of behavior change across and within multiple different types of behaviors. The initial intent of the consensus definitions was to reduce ambiguity about the meaning of the constructs among survey respondents and maximize the validity of the responses.

Definitions of Maintenance and Maintenance-Related Constructs

HMC collaborators agreed on the following common definitions of maintenance and maintenance-related constructs displayed in Table 1.

Table 1.

Definitions of Maintenance and Maintenance-Related Constructs

Construct	Definition
Maintenance	sustained behavior during the period of observation and after the intervention has stopped that meets a threshold believed to be necessary to improve health or well being within a given population
Adherence	protocol-related behaviors, such as attendance and participation in intervention-related activities (ie, individual or group intervention sessions, use of automated interventions, motivational interviewing, meetings with counselors, etc)
Grace Period	a window of time during which lack of adoption of the behavior was not counted as a failure
Relapse	a period of interruption of regular sustained behavior after its initiation and maintenance
Reactivation	the resumption of the sustained behavior following a period of relapse

Data Collection

After the workgroup finalized the survey definitions and items, the HMC Resource Center staff created a Web-based format for the survey, and the link was e-mailed to all the HMC principal investigators (PIs). Five notices were sent to the PIs reminding them to complete the survey. Responses were obtained for the domains described in the following paragraphs.

Maintenance assessment

Respondents were asked to indicate how maintenance was conceptualized for each behavior that their study addressed: (a) behavior meets a predetermined criterion or threshold necessary for achievement of a health or quality-of-life benefit; (b) behavior meets some proportional increment over that exhibited at baseline; (c) behavior is being assessed along a continuum of low, moderate to high maintenance; and (d) behavior assessed using other means that respondents were requested to specify.

Operational definition of maintenance

The survey also asked how investigators planned to represent maintenance in analyses (eg, whether it would be represented as a dichotomous, ordinal, or continuous measure).

Other items concerning the measurement of maintenance

Respondents were asked to indicate whether they viewed the maintenance variable as an intermediate or primary outcome or both. In addition, the survey required respondents to indicate the level(s) at which they assessed maintenance: individual, family, organizational, environmental, policy, or other. Finally, the survey asked investigators to identify the time intervals during which the maintenance variable was assessed for participants.

Details were also asked about the following maintenance-related concepts:

Grace period

The survey assessed the extent to which grace periods were used across studies.

Relapse

The survey asked investigators to indicate whether or not their study assessed relapse. If yes, methods for assessing relapse were solicited, and the point in time and frequency with which this information was collected were documented.

Reactivation

Investigators were asked to indicate whether or not the study assessed reactivation. If yes, methods for assessing reactivation were solicited, and the time points at which this information was collected were documented.

Facilitators and barriers

Investigators were asked to indicate whether their study was collecting information on facilitators and barriers to maintenance of the targeted behavior change. If yes, methods for assessing facilitators and barriers and the time points at which the information was collected were documented.

Attrition

The survey asked respondents to indicate how attrition was being handled by the study, ie, classifying nonresponders as nonmaintenance, nonresponders excluded from analyses, or other. If respondents chose other, they were asked to describe how attrition was addressed in analyses.

Analysis

Of the 21 HMC studies, 3 HMC investigations are not intervention studies, and investigators did not complete the survey. All 18 HMC intervention study investigators completed the Web-based maintenance survey. A descriptive analysis of their responses by type of behavior targeted was conducted.

Findings

Two of the 18 completed surveys were excluded because the PIs on these studies were examining unique behaviors (cancer screening and safe sex practices, respectively), which did not allow comparison with other studies. Survey results from the remaining 16 studies are presented by targeted behavior followed by a description of the similarities and differences within and across the behaviors. Table 2 provides a brief description of the 16 studies included in this paper.

Table 2.

Health Maintenance Consortium Intervention Studies

Site	Population	Physical Activity	Diet/Nutrition	Smoking Cessation	Substance Abuse
Drexel University	N=238 obese adults		x
The Miriam Hosptial	N=201 overweight and obese adults are primary participants; all participants also have an overweight/obese partner in their home — in one arm of the study, partners attend treatment with the participants; in the standard arm, partners attend assessments		x
University of Washington	N=40 worksites, 2900 adults		x
Boston University School of Medicine	N=405 adults who successfully improved fruit and vegetable consumption (voter registration list)	x	x
Oregon Health & Science University	N=440 adult firefighters	x	x
Oregon Research Institute	N=279 women at risk for coronary heart disease	x	x	x
Health Partners Research Foundation	N=1000, older adult MCO members	x
Lawrence Berkeley National Laboratory	N=40,000 adult runners	x
University of Illinois Chicago	N=600, older adults with lower extremity osteoarthritis	x
Stanford University	N=301 adult smokers			x
University of Tennessee Health Science Center	N=400 adult smokers			x
University of Rochester	N=950, adult current smokers			x
Friends Research Institute (1)	N=120, methamphetamine dependent treatment-seeking adults				x
Friends Research Institute (2)	N=120, methamphetamine dependent treatment-seeking adults				x
Treatment Research Institute	N=131, cocaine dependent or abusing, stabilized on methadone				x
University of Pennsylvania	N=252 adult substance abusers			x	x

Physical activity

Six of the 16 studies are examining the maintenance of physical activity as an outcome (Table 3), and most characterized maintenance as a continuous measure. Four of the 6 are assessing whether participants met a predetermined criterion for level of physical activity (criterion), and 4 are examining a percent increase over baseline levels. The majority of studies are examining maintenance as both a primary and intermediate outcome. There was substantial variety even within this small group of studies in the way that attrition was being categorized (nonadherence vs participants excluded from analyses, vs some other form of categorization). All studies are measuring maintenance at the level of the individual, with one each also assessing this outcome at the organizational and environmental levels.

Table 3.

Maintenance Measures by Behavior

	Total Studies N=16	Physical Activity N=6	Diet/Nutrition N=6	Smoking Cessation N=5	Substance Abuse N=4
Tracking Maintenance
Yes	22	6	6	5	4
No	0	0	0	0	0
Conceptual definitiona
Criterion	12	4	2	5	1
% increase	10	4	3	2	1
Continuum	10	0	4	1	4
Other definition	1	0	1	0	0
Operationalization of Maintenance Variableb
Dichotomous	6	2	2	2	0
Ordinal	6	2	2	1	1
Continuous	19	5	6	3	4
Type of Outcomec
Intermediate	2	1	1	0	0
Primary	6	1	2	3	0
Both	13	4	3	2	4
Other	1	0	0	1	0
Level of Maintenance Variable
Individual	22	6	6	5	4
Family	0	0	0	0	0
Organizational	3	1	2	0	0
Environmental	3	1	2	0	0
Policy	1	0	1	0	0
Other	1	1	0	0	0

Note.

Descriptive analyses (symbols do not indicate statistical significance)

^aDifferences within and across behaviors

^bDifferences within behaviors

^cDifferences across behaviors

Most studies are measuring physical activity maintenance at baseline, 6 months, 12 months, and 24 months; and only a few are measuring at shorter, more continuous intervals (ie, 2, 3, 6, 9, 12, 15, 18 months) (Table 4). With respect to data analysis, 50% (N=3) plan to model maintenance trajectories whereas half do not. None of the physical activity studies included a grace period before which maintenance would be counted. Fifty percent (N=3) of the studies are assessing relapse of activity, but only one is assessing reactivation following relapse. The clear majority (N=5) are assessing barriers to activity, but only 3 are also examining facilitators.

Table 4.

Maintenance Measurement Issues by Behavior

	Total Studies N=16	Physical Activity N=6	Diet/Nutrition N=6	Smoking Cessation N=5	Substance Abuse N=4
Assessment Time Intervalsa
Baseline	14	4	4	2	4
2 months	2	1	0	0	1
3 months	6	1	2	1	2
6 months	11	3	2	4	2
9 months	4	1	0	1	2
12 months	17	5	5	5	2
15 months	4	1	0	1	2
18 months	8	2	2	2	2
21 months	4	1	0	1	2
24 months	14	4	4	4	2
Other	9	3	2	1	3
Model Maintenance Trajectories?a
Yes	13	3	3	3	4
No	9	3	3	2	0
Grace Periodb
Yes	3	0	0	3	0
No	19	6	6	2	4
Attritionc
Nonadherence	9	2	1	2	4
Participants excluded	5	2	2	1	0
Other	12	2	3	2	4
Assessing Relapse?a
Yes	15	3	3	4	4
No	7	3	3	1	0
Assessing Reactivation?c
Yes	9	1	0	3	4
No	13	5	6	2	0
Assessing Barriers?c
Yes	13	5	5	3	0
No	9	1	1	2	4
Assessing Facilitators?a
Yes	13	3	3	2	4
No	9	3	3	3	0

Note.

Descriptive analyses (symbols do not indicate statistical significance)

^aIndicates differences within behaviors

^bIndicates differences across behaviors

^cIndicates differences within and across behaviors

Diet/nutrition

Six studies are examining nutrition. They include 3 that are assessing the maintenance of weight loss; 2 lifestyle programs that are examining the durability of changes in physical activity, nutritional outcomes, and weight loss; and a work site program that is attempting to alter fruit and vegetable consumption. The majority of the studies (5 of 6) planned to assess maintenance as a continuous variable. All studies also planned to examine maintenance at the level of the individual participant, although interventions with a clustered design also anticipated analysis at an organizational or environmental level. Dietary studies uniformly lacked a grace interval. Half are assessing relapse, and none are measuring reactivation. The majority of studies (5 of 6) are examining barriers, and half are examining facilitators of maintenance of changes in diet/nutrition.

Tobacco

Five of the 16 HMC studies reported intent to measure the maintenance of abstinence from tobacco in their trials. Current definitions of abstinence^6,7 require that smokers not use any form of tobacco, because the health effects of tobacco are avoided only when tobacco is completely stopped.⁸ All 5 of the studies will use a criterion measure of abstinence that represents at least 6 months of prolonged abstinence from tobacco. This is the recommended primary outcome of intervention trials in the tobacco dependence literature.^7,9 A secondary outcome criterion outcome is a 6-month 7-day point prevalence.⁷ This latter measure indicates that smokers have reported that they have not smoked even a puff of a cigarette (nor used other tobacco products) for the past 7 days. This outcome typically is assessed 6 months from the start of the intervention (for cessation induction trials) or 6 months from the quit date for cessation trials. Two HMC studies will report percent decrease of cigarette use, and one will report continuous number of days not using tobacco. Three of the studies will incorporate a 2-week grace period after the end of the intervention, and 3 will not.

All 5 studies will assess maintenance of abstinence and adherence to medications for cessation at the individual level. Two studies will report a dichotomous, one will report an ordinal, and 3 will report a continuous maintenance variable. Of the 5 studies reporting on tobacco, 4 will report 6-month outcomes, 5 will report 12-month outcomes, and 4 will report 24-month outcomes dated from randomization. Three studies will assess trajectories of change and barriers; 2 will assess facilitators. Finally, 4 studies will report relapse, and 3 will report reactivation.

Substance abuse

Four of the 16 studies are addressing outcomes related to illicit substance abuse (cocaine, methamphetamines). All 4 are measuring variables such as attendance to sessions (adherence) in addition to alcohol and drug abstinence (maintenance) as continuous variables at the level of the individual. None have a grace period before assessing substance use. Additionally, one is assessing whether participants meet a predetermined criterion related to maintained abstinence and is examining percent change over time. Protocol adherence is considered both a primary outcome and an intermediary outcome (or moderating variable) for other outcomes. All 4 studies are assessing maintenance frequently and monitoring continuous maintenance for the duration of the study in at least one of the 2 following ways: 3 are assessing maintenance through urinalysis tests conducted 2 to 3 times weekly during the active intervention portion of the study; 2 are conducting quarterly assessments that ask about drug use on each day since the previous assessment. All 4 studies are capable of assessing relapse and reactivation, but do not necessarily plan to analyze these as outcomes. All 4 are assessing facilitators by monitoring variables that might moderate the effects of the interventions and also plan to model maintenance trajectories. None are assessing barriers. Attrition occurs when participants can no longer be located for follow-up assessments or when they directly indicate that they wish to withdraw from the study.

Commonalities/differences across behaviors

It is important to note that although investigators reached consensus on definitions of maintenance behavior in order to enable survey participants to respond to the survey in a standardized fashion, investigators did not necessarily change their research protocols to reflect the group consensus definitions. Tables 3 and 4 present findings from descriptive analyses comparing the 16 studies’ approach to and use of maintenance and maintenance-related constructs. The symbols denote differences within, across, and across and within behaviors that were not tested for significance due to the small sample size. The majority of studies (12 of 16, 75%) are using a criterion to assess maintenance of behavior change, consistent with the consensus that the group reached regarding an ideal definition. However, it is important to note that no diet/nutrition studies are using a criterion measure, and only one of 4 substance abuse studies is assessing maintenance this way. The majority of studies (12 of 16, 75%) are also tracking maintenance using a continuously measured variable, and the majority conceptualize maintenance to be both a primary and intermediate outcome. Finally, all studies are assessing maintenance at the level of the individual participant with only 3 assessing this outcome at the organizational level, 3 assessing it at the environmental level, and one at the policy level. Those studies that are examining multiple levels of maintenance are addressing physical activity and diet. None of the smoking cessation or substance abuse studies are examining maintenance at levels beyond that of the individual participant.

As data in Table 4 demonstrate, we found substantial variation within and across targeted behaviors with respect to assessment time intervals. Studies targeting PA, diet, and smoking cessation tended to measure maintenance most frequently at 12 months following participation in an intervention, followed in frequency by 24 months and 6 months.

Across behaviors, roughly half of the studies intended to model maintenance trajectories, but 4 of 5 substance abuse studies intended to model this outcome. We found a substantial amount of variation within and across behaviors with respect to the intended treatment of attrition in planned analyses, suggesting that a more standardized way of approaching this issue might be helpful in the future.

Findings indicate that the majority of smoking cessation and substance abuse studies are examining relapse, compared to 50% of the PA and diet studies. The majority of smoking cessation and substance abuse studies are also assessing reactivation in contrast to only one of the PA studies and none of the diet studies. This finding is perhaps explained by differences in the nature of these behaviors and the history of research in each field.

Finally, across behaviors a majority of studies are assessing barriers to maintenance with the single exception of the substance abuse studies. However, although only half of the studies are examining facilitators to maintenance, substance abuse studies are almost universally examining this issue.

DISCUSSION

Individuals are complex beings who engage in multiple behaviors that are sometimes complementary and synergistic and sometimes contradictory with respect to the achievement of a common outcome like good health. This complex array of risk characteristics and behaviors presents challenges to clinicians and to public health practitioners with respect to impacting multiple conditions simultaneously. This survey of conceptual and measurement similarities and differences occurring across 16 studies examining behavioral maintenance of multiple behaviors found similarities and differences within and across behaviors that have implications for advancing the quality of transbehavioral research.

First, researchers examining the maintenance of different types of behaviors were able to reach consensus about a definition of maintenance that applied equally well to all behaviors. The definition of maintenance adopted by the group was “sustained behavior during the period of observation that meets a threshold believed to be necessary to improve health or well being within a given population.” We hope that a major contribution of this paper will be the utility of these common definitions of maintenance and other aspects of its measurement that contribute to maintenance and/or better describe maintenance as a process. It is important to note that agreement on a definition does not necessarily mean that all fields of behavior-change research can achieve this outcome. For example, tobacco dependence researchers have achieved a clear working consensus on standardized maintenance outcomes. In contrast, investigators conducting research on physical activity agree that our consensus definition is an ideal but recognize the fact that the dose response required to achieve improved health or quality of life is not well understood at this time, especially for certain populations like older adults.¹⁰

The HMC Maintenance Workgroup was also able to reach consensus regarding definitions of relapse and reactivation that may help the measurement of these phenomena in future research to be more precise and standardized across studies.

One limitation of this study is that a limited number of studies were available. Therefore, our ability to conduct within behavior comparisons was limited. The findings presented in this paper are tentative with regard to the state of measurement of maintenance of behavior change for these behaviors. However, we hope that these findings can be used to guide decision making by researchers and funders interested in long-term maintenance of behavior change.

Findings clearly indicate that there is an urgent need in the field to measure maintenance of behaviors and barriers and facilitators to them at levels other than that of the individual. For example, it is reasonable to assume that individuals’ ability to maintain healthy diets and physical activity and to abstain from substance abuse would be substantially mediated by the neighborhoods in which they reside. By the same token, a co-intervention in some of the tobacco dependence trials could be local or state tax policy. In this latter case, the examination of maintenance at the population level would be appropriate.

Our findings indicate how a group of premier behavioral scientists were conceiving and measuring maintenance of behavior change variables for 4 specific health behaviors being targeted for change. Substantial efforts remain to be made with respect to future research targeting maintenance, including the examination of barriers and facilitators to maintenance and predictors of both relapse and reactivation. These would appear to be areas that are particularly ripe for future research.