Practical and Relevant Self-Report Measures of Patient Health Behaviors for Primary Care Research

Abstract

PURPOSE With increasing evidence for the value of behavior change counseling, there is a need for health behavior measurements that can be implemented in primary care research. This article discusses criteria for and reviews self-report measures to briefly assess cigarette smoking, eating patterns, physical activity, and risky drinking across the life course. It then proposes pragmatic measures for use in practice-based research.

METHODS Drawing from literature reviews, previous multisite studies, personal communications with experts in the field, and guidance from an expert panel, we identified self-report behavior change measures and gave priority to items that addressed Healthy People 2010 goals, as well as those that were practical (ie, shorter, and easier to score and use for intervention), were sensitive to change, and produced results that could directly inform primary care intervention.

RESULTS Separate recommendations are described for measures for adults and for children/adolescents. We recommend a set of 22 items for adults and 16 items for adolescents to track succinctly their status on the 4 health behaviors above. Perfected measures remain elusive: newly developed measures of physical activity and eating patterns are recommended, and in general, the brief measures for adults are currently better validated than are the child measures.

CONCLUSIONS A set of totally satisfactory practical instruments for measuring behavior change in primary care settings does not yet exist. There is sufficient progress to encourage use of and further research on the proposed items. Use of a common set of items across different interventions and projects will help to advance clinical and behavioral research in primary care settings.

INTRODUCTION

Health behavior counseling in primary care settings holds great potential for disease prevention and management.^1–3 Emerging evidence points to the importance of the physician as a change agent,⁴ the efficacy of health behavior change interventions in primary care settings,⁵ and the value of behavioral change interventions for high-risk populations.⁶

It is timely to emphasize health behavior counseling in primary care given the large potential impact of personal behavior change on premature death and avoidable suffering.^1,7,8 The underperformance of the US health care system is acknowledged and accompanied by keen interest to improve.⁹ The 3 primary care medical specialties, as well as primary care nurse practitioners and physician’s assistants, exist in sufficient numbers to make feasible greater attention to the health needs of primary care patients.^10–12 Also, all 3 of the primary care physician specialties—pediatrics, family medicine, and general internal medicine—are now calling for revisions in training and practice congruent with a greater emphasis on the health behaviors of their patients.^13,14

Evaluation of behavioral interventions in primary care research requires appropriate assessments of targeted health behaviors. Valid assessment methods are needed to identify patients who could benefit from interventions, to inform the design of intervention strategies, and to evaluate intervention success. Whereas many instruments are available to assess health behaviors,¹⁵ few have been designed for or used in primary care settings.

Challenges for primary care settings include the “tyranny of competing demands.”¹⁶ Lengthy measures are not realistic to administer in practice-based research.¹⁷ Practical measures are needed that are broadly applicable, able to be self-administered, and appropriate for a range of age and cultural groups, and that have sufficient validity and reliability. Also, measures should be congruent with national public health goals and sensitive to intervention effects, because the goal is to produce improvements in individual health as well as affect population health.

Measures for primary care research need to be brief because of the lack of time or supervision to complete lengthy instruments. Additionally, because primary care is comprehensive^18,19—concerned with improving a variety of risk factors and conditions—focusing on 1 or 2 areas should not result in worse care in other areas. Unfortunately, almost all “gold standard” assessment procedures have focused on 1 health behavior in isolation and attain part of their precision by being relatively lengthy, burdensome, or expensive.

In this article, we focus on assessment of 4 health behaviors that collectively account for the majority of illness and mortality in the United States: cigarette smoking, eating patterns, physical inactivity, and risky drinking.^8,20 This work was conducted as part of Prescription for Health (P4H),²¹ a program of the Robert Wood Johnson Foundation and the Agency for Health-care Research and Quality. In its first round, P4H funded 17 practice-based research networks to field test evidence-based strategies to improve the delivery, feasibility, and reach of behavior change counseling for these 4 health behaviors in primary care practices. A major goal of the second round is to assess the extent to which interventions are effective in attaining patient behavior change. A common set of brief measures is necessary to compare outcomes across settings, interventions, and populations.

Our purposes in this article are to (1) discuss issues in selecting measures of physical activity, risky drinking, cigarette smoking, and eating patterns that are appropriate for adult or pediatric primary care research; (2) identify specific, practical behavioral assessments that could be implemented in primary care research without adding undue burden to patients or clinicians; and (3) discuss implementation, conditions of administration, and directions for future research.

METHODS

Context

To understand the current state of measurement in primary care studies, we reviewed assessments proposed by grantees in the first round of P4H funding. Among the 17 sites, 15 focused on physical activity, 7 on risky drinking, 11 on cigarette smoking, and 14 on eating patterns. All projects addressed at least 2 health behaviors. Despite the common health behaviors addressed, there were few commonalities in the instruments selected to measure health behaviors. The initial P4H projects proposed the following behavioral measures.

Physical Activity

Physical activity logs, recalls, weekly assessments, and screening questions were all used. Referenced questionnaires or sources included Physical Activity Scale for the Elderly (PASE),^22,23 PACE project assessment,²⁴ Summary of Diabetes Self-Care Activities,²⁵ Healthy People 2010,³ National Health Interview Survey (NHIS),²⁶ Behavioral Risk Factor Surveillance System—BRFSS,²⁷ and Guidelines for Adolescent Prevention Services (GAPS).²⁸

Risky Drinking

Number of alcoholic drinks per week, quantity of drinks, social aspects of drinking, and alcohol exposure and risk (adolescents only) assessments were all used. Referenced questionnaires or sources included the Five Shot questionnaire,²⁹ Alcohol Use Disorders Identification Test (AUDIT),^30,31 NHIS,²⁶ BRFSS,²⁷ and GAPS.²⁸

Cigarette Smoking

Questions included whether the patient ever smoked, was a current smoker, number cigarettes smoked and quit attempts. Referenced questionnaires or sources included Summary of Diabetes Self-Care Activities,²⁵ NHIS,²⁶ BRFSS,²⁷ and GAPS.²⁸

Eating Patterns

Food diaries, logs, 24-hour dietary recall, and nutrition intake questionnaires were used. Referenced sources included the Summary of Diabetes Self-Care Activities,²⁵ Block Brief 2000 Questionnaire,³² “Rate Your Plate,”³³ NHIS,²⁶ BRFSS,²⁷ Healthy People 2010,³ and GAPS.²⁸

The length of the proposed assessment instruments was surprising. Subsequent conversations with P4H grantees suggested that it was difficult to implement these measurement tools in practice. Although many of the proposed instruments had demonstrated validity for screening, their ability to detect changes in health behaviors is uncertain.

Procedures and Criteria Used in Making Recommendations

To identify practical measures for assessing and tracking health behaviors, we needed to specify selection criteria. Greatest weight was given to 5 primary criteria: practicality, clinical and public health relevance, validity, sensitivity to change, and relation to behavioral intervention. The 5 secondary criteria listed at the bottom of Table 1 ▶ were also considered. Because child and adolescent risk behaviors differ in frequency and nature from those of adults, separate assessments were generally necessary.

Table 1.

Criteria Used to Select Health Behavior Change Assessments

Criteria	Rationale/Description
Primary
1. Practicality	A composite criterion including length; ease of administration, scoring, and interpretation; appropriateness to a wide range of settings and populations; and in the public domain, to reduce costs
2. Relationship to clinical and public health goals	Assessment of success toward achieving quantifiable public health goals linked to health behavior change objectives. For example, preference was given to results that could be linked to achievement of Healthy People 2010 goals
3. Validity	Agreement with reference standard measures. We gave moderate emphasis to convergent and divergent validity (correlations with other established measures of the same behavior and lack of correlation with other constructs)
4. Sensitivity to change	Ability of a measure to detect improvements resulting from effective interventions was given the largest weight. To achieve sensitivity to change, an instrument must demonstrate other characteristics, such as test-retest reliability. The primary purpose of including behavioral measures was to evaluate whether significant improvements are produced
5. Relation to behavioral interventions	Degree to which instruments were behaviorally based and thus directly related to P4H interventions21 (eg, it is less valuable to have a recall that produces detailed dietary nutrients, than measures that assess changes in eating behavior)
Secondary
6. Cultural and literacy issues	Extent to which measures were appropriate for and had been translated into and validated in other languages, especially Spanish
7. Norms and breadth of use	Having data on national or regional norms. Data from instruments with well-established norms allow comparison of interventions and populations to larger reference groups
8. Age appropriateness	Appropriate across the full age range and functional levels. Given special attention for children and adolescents
9. Reliability	Given the brevity required, we did not place strong emphasis on internal consistency (α coefficients), but gave considerable weight to test-retest reliability
10. Usability for feedback and interventions	Scales that can be scored immediately or can identify areas for improvement to help with patient goal setting

Candidate measures were identified from several sources, including instruments implemented in the first phase of P4H, review of published literature, suggestions of professional contacts (because many instruments are under development) and consensus statements and guidelines for recommended assessments. Searches were conducted on PubMed and National Institute of Health (NIH) Web sites, using the search terms “nutrition,” “diet,” “eating,” “smoking,” “cigarette smoking,” “alcohol consumption,” “risky drinking,” “exercise,” and “physical activity” with “self-report,” “measurement,” “primary care,” and “assessment.” In particular, reviews from 2 major NIH-funded health behavior change consortia, the Behavior Change Consortium³⁴ and the Health Maintenance Consortium (http://hmcrc.srph.tamu.edu/default.aspx), provided excellent information on potential instruments.

An initial subset of measures was identified by the first 3 authors, with exclusion of measures that were longer than 10 items or that were not appropriate for self-administration. Candidate measures were initially reviewed to establish their suitability, and tables were developed that outlined information on the various selection criteria.

Summary information and tables were presented to a consensus panel of P4H program representatives and invited experts. The panel included 12 experts in the areas of primary care, practice-based research, and quantitative and qualitative analyses. The panel agreed that priority for further recommendations should be based on assessments demonstrated to have (1) sensitivity to change or intervention effects; (2) broad applicability, especially to minority populations or validated in Spanish; (3) normative data; and (4) usefulness for intervention planning. Final revised recommendations were based on discussion by this consensus panel.

The following key issues emerged from the panel discussion: (1) the number of items needed to be even briefer than originally proposed if the expectation was that all practices would assess multiple behaviors; (2) the wording of several items did not apply to primary care; (3) response formats of several items needed to be changed to fit time frames relevant to practice-based research; (4) a recommended list of brief items should be provided, as well as a secondary recommendation where warranted, of slightly longer instruments if they had superior qualities; and (5) sensitivity to change was very important, and several widely used screening measures did not meet this criterion.

RESULTS

For each target behavior, we offer both a primary recommendation, and in most cases, a secondary recommendation, which often involves an elaboration of the primary measure that might be applicable for sites wishing to devote more time or priority to that behavior. Our primary recommendations are summarized in Figure 1 ▶^34–36 and Table 2^27,37–41 (which are available online at http://www.annfammed.org/cgi/content/full/3/1/73/DC1); secondary recommendations are in Appendixes 1^{38,39,42–44,45} and 2^{25,27,30,31,36} (which are available online at http://www.annfammed.org/cgi/content/full/3/1/73/DC1). For adults, 22 items are recommended as a minimum assessment across all 4 behaviors. The corresponding number for adolescents, ages 12 to 17 years is 16, and 12 items for younger children. Although many of the items recommended are from validated instruments, seldom was it possible to include entire instruments. Several of our recommendations endorsed new instruments, and potential users should review these measures for updates before implementing. Table 3^{25,27,34,36–45} (available online only at http://www.annfammed.org/cgi/content/full/3/1/73/DC1) summarizes the characteristics of recommended measures.

Figure 1.

Adult Measures of Behavior Change

Scoring - see Web site: http://hmcrc.srph.tamu.edu/Measurement%20folder/PA/R\ APA%20fnl%201004.doc

Physical activities are activities where you move and increase your heart rate above its resting rate, whether you do them for pleasure, work, or transportation.

The following questions ask about the amount and intensity of physical activity you usually do. The intensity of the activity is related to the amount of energy you use doing these activities.

Examples of physical activity intensity levels:

Recommended Measures for Adolescents and Children

Only a few measures of health behaviors in children or adolescents were located that fit most of the desired criteria, and none were identified that reported sensitivity to change. Some instruments with acceptable validity and reliability were identified, but they were too long or cumbersome to be feasible in primary care settings. Given the paucity of available measures, recommended items were sometimes based on items from adult samples or that had face validity but no established measurement properties in adolescents and children.

The following measures are recommended for assessing adolescent and child behavior. Table 3 presents the actual items.

Physical Activity

Adolescents.

Three items are recommended, including 2 items from the PACE+ project³⁷ and 1 item from the Youth Risk Behavior Surveillance System (YRBSS).³⁸ The PACE+ items, which assess frequency of physical activity in the last 7 days and in a typical week, have been validated in a diverse population of middle-school adolescents. The items demonstrated acceptable measurement properties (sensitivity 71%, specificity 63%; significantly correlated with accelerometer readings r = .40, P<.001).³⁷ In addition, our panel believed it was important to assess sedentary behavior, so an item from the YRBSS was recommended to evaluate hours of television watched on an average school day. All recommended items can be used to address the Healthy People 2010 goals. A secondary recommendation was to assess sedentary behaviors of computer and video game use via an adaptation of the YRBSS item that assesses television viewing. Measurement characteristics and normative data are not currently available for this item.

Younger Children.

Obtaining reliable and valid self-reported physical activity in younger children is problematic.^46,47 No feasible and reliable self-report instruments were identified for children, and objective measures such as activity monitors were not considered practical. Therefore, no recommendation is made for this age-group.

Risky Drinking

Adolescents.

The 3 items recommended are based on questions from the 2003 BRFSS. While our panel recognized that overall prevalence of alcohol use would be lower in adolescents compared to adults, these items were recommended to identify current problems with alcohol consumption. They are also useful for calculating Healthy People 2010 goal attainment.

As a secondary assessment of risky drinking, 6 items from the CRAFFT screening test are recommended.⁴² This measure has been validated in clinical settings with diverse populations of adolescents aged 13 to19 years and has demonstrated reliability and validity.^42–44 An adapted version of the instrument that screens only for alcohol, instead of alcohol and drug use, was believed to be most practical. Normative data are available on the longer CRAFFT instrument, but cutoffs for specific alcohol-related problems are not.

Children.

Risky drinking in children younger than 12 years is infrequent,⁴⁸ and validated measures of susceptibility are not currently available.

Smoking

Adolescents.

Two items from the Smoking Uptake Continuum³⁹ assessment are recommended. These items assess susceptibility to smoking, with the first item asking about experimentation with cigarettes and the second asking how many days, in the last 30, that an adolescent smoked. The second item can be used to calculate attainment of Healthy People 2010 goals.

The secondary recommendation for adolescent smoking is to include 8 additional items from the Smoking Uptake Continuum (Appendix 1). This 10-item measure³⁹ categorizes youth into 7 levels of smoking behavior from committed never smoker to current established smoker. The instrument is validated with adolescents aged 12 to 18 years and has demonstrated good predictive validity (<6% committed never smokers progress to established smokers over a 3- to 4-year period, susceptible never smokers are 2 to 3 times more likely to initiate smoking over a 3- to 4-year period than are committed never smokers).

Children.

A primary recommendation is to assess children’s susceptibility to future smoking based on 4 items from Jackson’s Susceptibility to Smoking scale.⁴¹ This measure has been validated in children in grades 3 to 5 and found to be predictive of smoking initiation 2 years after assessment.

Eating Patterns

Adolescents.

The literature review identified a few measures that assessed nutrient intake and had been validated in clinical settings with adolescents and children.^49,50 These instruments were considered too long, however, to be feasible in primary care settings. The panel also emphasized the importance of assessing dietary behaviors and food groups, such as fruits and vegetables, rather than nutrients, for use in behavioral counseling.

We recommend an 8-item instrument⁴⁰ developed by Ammerman and colleagues (Alice Ammerman, personal communication, May 15, 2004) that assesses dietary patterns, such as frequency of fast-food intake, fat snack intake, and sweetened beverage consumption. The instrument was designed to be completed by adolescents on their own and together with parents for younger children. It was designed using literature review, expert input, and feasibility testing, and measurement properties are currently being evaluated.

A secondary recommendation is a 2-item assessment of fruit and vegetable intake developed by the PACE+ project.⁴⁵ These items are consistent with Healthy People 2010 goals and have been validated in a diverse population of middle school-aged adolescents. They have demonstrated acceptable measurement properties (significant correlations with 3-day food record; sensitivity 81%, specificity 47%, positive predictive value 79%). Although the items have not been validated in younger children, the panel believed that younger children could complete these items with assistance from a parent or guardian.

Recommended Measures in Adults

Our review indicated that there are now several practical lifestyle measures being tested for use in adults, 2 of which are primary recommendations. We attempted to select instruments that could bridge the vast diversity of adult populations (Figure 1 ▶).³⁵

Physical Activity

Primary measure.

The Rapid Assessment Physical Activity Scale (RAPA).³⁵ This scale involves 9 yes/no items assessing the type and amount of physical activity in which adults engage. Clinicians and measurement experts at the University of Washington developed this scale for primary care settings. In preliminary validation studies it compares well with longer scales. A major advantage is that it enables respondents to visualize differences in activity intensity. Pictures and examples can be made culturally and setting relevant. Other advantages are a quick and easy scoring sheet, and availability in different languages (eg, Spanish and Vietnamese).

Secondary measure.

Seven items from the BRFSS.²⁷ The main advantage of the BRFSS is that it permits comparison with national norms, and Spanish translations are available. Although separating types of activity may be helpful, the original purpose of the BRFSS was for surveillance, not intervention tracking. Additionally, responses are often ambiguous because of respondent failure to differentiate among intensity levels. We recommend RAPA over this scale, for likelihood of more accurate reporting, and the use of graphics to clarify different levels of activity. Both scales can be mapped to Healthy People 2010 recommendations.

Risky Drinking

Primary measure.

Three items from the BRFSS.²⁷ These items offer a fast, simple way to detect risky drinking and to examine changes in binge drinking. Findings can be compared with national norms and related to Healthy People 2010 goals.

Secondary measure.

AUDIT.^30,31 This 10-item instrument is often used in national and international studies to screen for risky drinking, and it is available in Spanish. It is less relevant for intervention studies and assessing change over time than it is for screening. Time frames of many of the AUDIT items prevent precise change estimates.

Cigarette Smoking

Primary measure.

Three items from national health surveys.^27,34,36 These items (Figure 1 ▶) are recommended to assess whether respondents ever smoked, their current smoking status, and extent of smoking. Individual items are taken from national surveys, and thus there are normative data; however, with the exception of the second item, these are not standard measures from major trials to evaluate smoking interventions. There was concern about wording of some of the lengthier and more complex items used in smoking cessation trials. The recommended items form an efficient condensed set (if persons answer “no” to initial items, they skip the remaining questions) and allow tracking on Healthy People 2010 goals.

Secondary measure.

Six items recommended by the Society for Research on Nicotine and Tobacco³⁶ (embedded within 3 questions in Appendix 2) to measure cessation in cessation trials. These items reflect expert consensus assessment of abstinence parameters (eg, temporal intervals, slips). They are recommended for those emphasizing smoking behaviors. This approach recognizes that there will be relapses and also allows survival analyses.

Eating Patterns

Primary measure.

Starting the Conversation (STC) – Diet. (Alice Ammerman, personal communication, May 15, 2004) This 7-item scale comes out of a program of nutrition research by Ammerman and colleagues. The advantage of this scale is its simplicity and relative brevity. It also assesses food patterns vs nutrient or fat intake and provides tailored strategies for behavioral counseling. The validity of the STC is currently being examined in the WISEWOMAN Study,⁵¹ with comparisons to longer, more traditional nutritional assessments.

Secondary measure.

The Summary of Diabetes Self-Care Activities (SDSCA).²⁵ This scale includes 4 dietary items that address the most commonly targeted dietary recommendations—eg, low fat, high fiber, and increased fruit and vegetable intake. The questions correlate reasonably well with much longer reference standard instruments. The SDSCA has been widely used in various formats (paper and computer administered) and found to be sensitive to change in intervention studies with diabetes patients, but has not been validated in nondiabetic study samples. The questions seem applicable to nondiabetic adults, but we recommend the STC items because they lead directly to intervention planning.

DISCUSSION

Our goal was to derive a set of practical, yet valid, items that could evaluate patient behavior change in the 4 priority areas across the diverse P4H projects. This same set of items should be appropriate for most primary care-based research and practices in the United States. Given our mandate, the failure of traditional review procedures to identify many appropriate measures, and the frequent lack of information on key criteria (eg, sensitivity to intervention; cultural appropriateness), we were forced to rely to a moderate extent upon informal networks of leading researchers and organizations, and upon subjective estimates of which items best fulfilled our criteria.

Consistent use of the proposed measures would greatly improve the science of health promotion among primary care patients. It is difficult to compare results of studies from different interventions and populations if each study uses measures of differing or unknown reliability, validity, and sensitivity to change—even if results are converted to standard units such as effect size. We propose the items in Figure 1 ▶ and Table 2, as a relatively succinct set of items for use in primary care research and practical clinical trials.^52,53 These recommendations should also be useful to clinicians and health plans for identification of health behavior change needs and quality improvement efforts.

Additional research is needed on these measures. Some items are very new, and more data are needed on their continued performance. Several items recommended have not been fully tested in terms of traditional psychometric standards. There is, however, a compelling need for a practical set of items appropriate for assessing multiple health behaviors in primary care. We posit that the recommended items are a better solution to this need than recommending nothing, or a much longer, and less clinically useful set of measures that would be much less likely to be used. These recommendations should be periodically reevaluated and revised as new information becomes available. This report is limited to assessments of these 4 health behaviors and does not address other important self-management goals that may be set between patients and their health care team.⁵⁴ We restricted our focus to self-report measures and did not review direct observation or mechanical instrumentation methods.⁵⁵

We recommend 3 specific areas of future research. First, these proposed items should be compared directly to “gold standard” assessments. Second, more studies are needed of their sensitivity to change; acceptability in a spectrum of family medicine, internal medicine, and pediatric practices; and results under different conditions of administration, in particular computer-automated assessment. Finally, research using modern approaches, such as item response theory, is needed to determine whether it is possible to further streamline these assessments.