Validity and Reliability of a Dietary Stages of Change Measure Among Economically Disadvantaged African-American Adolescents

Abstract

Purpose

To examine the construct validity and 2-month test-retest reliability of a staging measure for assessing readiness to consume five or more daily servings of fruits and vegetables among economically disadvantaged African-American adolescents.

Design

Longitudinal survey.

Setting

Youth services agencies serving low-income communities in New York, New Jersey, and Pennsylvania.

Subjects

African-American adolescents (N = 390) aged 11 to 14 years.

Measures

Self-report measures of stage of change and fruit and vegetable consumption.

Analysis

Correlation analysis and analysis of covariance were used to examine, cross-sectionally, whether fruit and vegetable consumption differed by stage of change (validity assessment). Correlation analysis and Cohen's kappa (κ) were used to assess the degree of association and level of agreement between stages longitudinally (test-retest reliability assessment).

Results

Fruit and vegetable consumption was moderately correlated with stage of change (r ≥ .54, p = .000). Consistent with the transtheoretical model, youths’ consumption increased from earlier to later stages (p = .000). Forty-six percent of youths were in the same stage at both measurements. The correlation between stages was .26, and Cohen's κ was .20 (p = .000).

Conclusions

Despite preliminary evidence of construct validity, the low test-retest reliability found suggests that further research and testing are needed to improve the stability of the measure.

PURPOSE

The transtheoretical model (TTM) is one of the most promising models to emerge from research on the process of health behavior change.^1–3 The central organizing construct is stages of change, the temporal and motivational readiness to take action to modify a behavior as represented by five stages: precontemplation (not intending to take action within the foreseeable future, defined as the next 6 months); contemplation (intending to take action within the next 6 months); preparation (intending to take action within the immediate future, defined as the next month); action (recent change, defined as change lasting for less than 6 months), and maintenance (sustained change, defined as change lasting for 6 months or longer). Health behavior change is conceptualized as a process involving progression through these stages. Although stage transitions are often considered linear, it has been suggested that a spiral best describes movement through the stages because of the dynamic nature of change.² For example, recycling from preparation to an earlier stage is just as likely as progressing to the next stage.²

The assessment of stage of readiness is useful for describing health behavior distributions, information that can aid in tailoring intervention planning and serve as the basis for future comparisons.⁴ Stage defines if change is likely and if so, when it is likely to occur, information that practitioners can use to inform individuals of where they are in the change process prior to intervention.⁵ TTM research has shown that at each stage, individuals uniquely employ processes of change (i.e., covert and overt activities for progressing to later stages), and they differ in their endorsements of the pros (advantages) and cons (costs) of change and their self-efficacy for modifying a behavior.^6–8 The stages can therefore be used to design interventions aimed at targeting intermediate outcomes and optimizing processes of change that have the greatest salience at the different stages. The stages are also an important indicator of intervention effectiveness, as evidenced by more favorable stage transitions occurring among individuals exposed to an intervention relative to those who are not.^9–11

Divergent approaches have been used to assess stage of change, and there are variations among approaches of the same type. A common approach is the use of a multi-item measure and algorithm. The first item queries whether the respondent is at criterion (i.e., enacting the target behavior). The respondent is then presented with items for assessing either his or her intention to progress to action (if not at criterion) or the duration of the behavior (if at criterion).^12–14 Stage of readiness is then assigned using the algorithm. Variants of this approach include the use of an objective behavioral measure in lieu of the first item or, alternatively, to confirm reports of the behavior as determined by the first item.^15–19 Measures of the same behavior can differ based on the criterion for action, e.g., regular exercise versus 30 minutes of moderate activity on most days of the week, as well as the criterion for progressing to action, e.g., whether one intends to consume more fruits and vegetables versus five or more servings a day.^20–23 Another variant is to present the respondent with a single item for assessing whether he or she engages in a behavior (e.g., in the past month, have you had five or more drinks in a row?) that includes response options representing each stage (e.g., “Yes, and I do not intend to stop drinking five or more drinks in a row” [precontemplation]).^24–26 Continuous measures consisting of multiple items for probing the respondent's thoughts and feelings about a behavior and whether he or she intends to change it also have been developed.^27–29 Scores computed based on responses to items representing the different stages can then be compared with established staging profiles.

The divergent approaches for measuring stage of change underscore the need for studies examining the psychometric properties of a particular measure, ideally in various populations. Decisions to use a particular measure should be informed by data on the validity and reliability of the tool. In the absence of these data, there is the potential for incorrect classification, mismatched treatment, or the misestimation of program effects in behavior change interventions. Despite published examinations of the validity of some measures of stage of change, few studies have examined the reliability of various staging approaches. Test-retest reliability is particularly difficult to assess because naturalistic stage transitions (i.e., those that occur in the absence of intervention) are to be expected. There currently are no guidelines on the extent of naturalistic transitions that are acceptable for examining stages longitudinally and for the development and evaluation of stage-tailored behavior change interventions.³⁰

The purpose of this study was to examine the construct validity and 2-month test-retest reliability of a staging measure and algorithm for assessing readiness to consume five or more daily servings of fruits and vegetables among economically disadvantaged African-American adolescents. Despite their lower than recommended intakes,^31,32 measures are lacking for assessing readiness to consume fruits and vegetables among economically disadvantaged African-American adolescents. Information on the validity and reliability of the tool examined can aid in the assessment of its utility for the assignment of stages and for the development and evaluation of stage-tailored interventions to increase fruit and vegetable consumption in this population.

METHODS

Design and Sample

The present investigation was a secondary analysis of data obtained from participants in the control condition of a dietary intervention study.³³ Selection criteria were African-American ethnic-racial heritage and aged 11 to 14 years. Participants were recruited through 27 youth services agencies serving low-income communities in New York, New Jersey, and Pennsylvania. Following institutional review board approval, eligible youths at participating sites provided informed written assent, and informed written consent was obtained from a parent or guardian.

The intervention was evaluated in a quasi-experimental design. Youth services agency sites were assigned to study conditions (intervention and control) based on the size of their youth population. This study examined data provided by youths in the control condition who completed an outcome battery at pretest (T1) and at posttest (T2), 2 months after T1 (N = 390). The battery included items for assessing youths’ age, sex, and ethnicity, and measures of stage of change,²³ TTM constructs other than stage (α = .77–.91),³⁴ and social desirability trait.³⁵ The sample of 390 youths had a mean age of 12.58 (SD = .97) years, and was 65% female and 9% African-American Hispanic. Youths’ mean intake of 2.59 (SD = 1.49) daily servings of fruits and vegetables at T1 did not differ from their mean intake of 2.46 (SD = 1.39) servings at T2.

Procedure

Trained research staff administered the battery in a paper-and-pencil format to groups of up to 10 youths each. After youths were oriented to the study, instructions for completing each section of the measure were read aloud to the group. As youths completed the battery, assistance was provided as needed. The assistance was focused on clarifying the meaning of questionnaire items; no direction was given regarding the selection of a particular answer. The staff reviewed completed questionnaires and followed up with youths with missing or incorrectly coded responses. For completing the battery at both time points, youths received a $20 honorarium.

Measures

Stage of Change

Youths’ stage of change was assessed using a modified version of the 3-item staging measure and algorithm developed by research scientists at the Cancer Prevention Research Center.²³ Items in the original measure are shown in Table 1. Because of concerns that youths might have difficulty interpreting response options for the second and third items, the investigators simplified them to a yes/no format and split the item for assessing behavioral intentions into two questions (i.e., whether the respondent intended to start eating five or more daily servings of fruits and vegetables in the next 6 months and the next month, respectively). Changes made to the items and the algorithm for assigning stages are also shown in Table 1. Because items in the staging measure required youths to follow skip patterns, case summaries were used to examine whether youths had difficulty doing so. Fewer than 7% of respondents provided a pattern of responses indicating that they did not follow the expected skip patterns.

Table 1.

Revisions Made to Items in the Staging Measure and Algorithm for Assigning Stages

Original Item (Response Options)	Revision
1. How many servings of fruits and vegetables do you usually eat each day? (zero, one, two, three, four, five, six or more). If you answered between “zero” and “four” to question 1, go to question 3. If you answered “five” or “six or more” to question 1, go to question 2.	Item unchanged.
2. Have you been eating five or more servings of fruits and vegetables a day for more than 6 months? (Less than 6 months, More than 6 months.)	Response options modified (Yes, No).
3. Do you intend to start eating five or more servings of fruits and vegetables a day in the next 6 months? (No, and I do NOT intend to in the NEXT 6 MONTHS, Yes, I intend to in the NEXT 6 MONTHS, Yes, I intend to in the NEXT 30 DAYS.)	Response options modified (Yes, No). A fourth item (below) and instructions for following the skip pattern (i.e., “If you answered ‘Yes’ to question 3, go to question 4. If you answered ‘No’ to question 3, skip question 4”) added.
4.	Do you intend to start eating five or more servings of fruits and vegetables a day in the next month? (Yes, No)

Algorithm for Assigning Stages

Precontemplation = answer to question 1 < 5 and answer to question 3 is no

Contemplation = answer to question 1 < 5, answer to question 3 is yes, and answer to question 4 is no

Preparation = answer to question 1 < 5, answer to question 3 is yes, and answer to question 4 is yes

Action = answer to question 1 ≥ 5 and answer to question 2 is no

Maintenance = answer to question 1 ≥ 5 and answer to question 2 is yes

Fruit and Vegetable Consumption

The first item in the staging measure was used to estimate youths’ fruit and vegetable consumption. This question asks, “How many servings of fruits and vegetables do you usually eat each day?” and includes response options ranging from zero to six or more servings. Although the measure is a brief, lower-cost alternative to more extensive intake assessment methods, data on its performance among adolescents are lacking. Thus, the extent to which intake estimates derived from the measure reflected youths’ true eating behavior is unknown. There is some evidence that it is a valid indicator of intake in adults, as evidenced by the moderate correlation found between item responses and servings of fruits and vegetables estimated using a 7-item food frequency questionnaire (r = .56).³⁶ This measure has been used to track changes in intake among adolescents and adults in other dietary intervention studies.^36,37

Social Desirability Trait

Social desirability trait was measured using a short form of the Marlowe-Crowne Social Desirability Scale (M-C 2(10)).³⁵ This measure assesses the tendency to respond in a manner consistent with social norms in a testing situation, and was included to control for the potential effects of social desirability response bias on youths’ self-reported stage of change and fruit and vegetable consumption. Scores on the M-C 2(10) were highly correlated with scores on the original 33-item measure (r ≥ .80), and reliability coefficients ranged from .49 to .75 in four samples of young adults.³⁵ The measure has been used to assess social desirability trait in other research with adolescents³⁸; however, it has not been validated in adolescent samples. Examination of the internal consistency reliability of the measure in the present sample revealed that it was poor (Kuder-Richardson reliability coefficient = .45). The absence of validity data and the low internal consistency reliability found suggest that the M-C 2(10) may be less than optimal for assessing social desirability trait among younger adolescents.

Analysis

The validity of the staging measure was examined cross-sectionally using data collected at T1 and T2. Partial correlations and one-way analysis of covariance with Bonferroni post hoc tests were used to examine whether shifts in stages were associated with changes in fruit and vegetable consumption and to test for intake differences by stage. Analyses were controlled for social desirability trait. Positive and moderate correlations (i.e., at or above .50 according to Cohen's conventions)³⁹ and statistically significant intake differences across stages consistent with the TTM (i.e., intake increasing from earlier to later stages) were considered evidence of validity.

Because the criterion measure of intake was among the items in the staging measure, this increased the potential for the correlations between measures to be inflated and for intake to differ between youths in preaction (i.e., precontemplation, contemplation, and preparation) and action (i.e., action and maintenance) stages. The reason for this is that the first item in the staging measure is used to rank respondents according to intake level (i.e., intake below and at or above five daily servings) for purposes of determining whether they will be classified into preaction or action stages. The particular stage is determined based on responses to items assessing intentions to progress to action (for those with intakes below five daily servings) or the duration of the behavior (for those with intakes at or above five daily servings). One would expect intake to be lower among youths in preaction as compared to action stages because assignment to these stages is determined in part by intake level, a difference that would also be reflected in the correlations between measures. However, this would not explain intake differences within preaction stages predicted by the TTM because these differences are a reflection of whether and when youths intend to start eating five or more daily servings of fruits and vegetables. According to the TTM, intake should increase from precontemplation to preparation stages because youths in precontemplation have no intention to take action to modify their behavior, whereas those in preparation are preparing to do so. Intake increases within these stages were therefore considered further evidence of validity because this would indicate that the measure was capturing changes attributable to differences in youths’ intentions to change.

Youths’ stage distribution at each time point and the congruence of stages between T1 and T2 were examined using frequency distributions and cross-tabulations. To establish test-retest reliability, the degree of association and level of agreement between stages from T1 to T2 were examined using partial correlations and Cohen's kappa (κ). Previous work has shown that the stability of stages declines over time. For example, substantial levels of agreement were found between stages for smoking cessation (κ = .72) and reducing alcohol consumption (κ = .73) measured cross-sectionally, and between stages for exercise (κ = .72) and vigorous physical activity (κ range = .72–.76) measured 2 weeks apart.^40–42 Moderate correlations and levels of agreement were found over longer periods: for example, between stages for fruit and vegetable consumption measured 7 weeks apart (r = .51 and r = .74, respectively),⁴³ stages for fruit intake measured over a 53-day interval (κ = .64), and stages for fat (κ = .43), fruit (r = .51; κ = .43), and vegetable (r = .53; κ = .45) intake measured 3 months apart.^30,44 Based on the time frame for the present study, a moderate correlation and moderate level of agreement between stages (i.e., a κ between .41 and .60 according to Landis's conventions)⁴⁵ were anticipated. A correlation and κ of this magnitude were considered evidence of a degree of test-retest reliability comparable to that found previously for dietary stages of change measures. Analyses were conducted using the Statistical Package for the Social Sciences (SPSS for Windows, Version 17.0, 2008; SPSS Inc., Chicago, Illinois).

RESULTS

Validity Analyses

Scores on the M-C 2(10) were associated with stage of change at T1 (r = .13, p = .011) but not at T2, and were unrelated to youths’ self-reported fruit and vegetable consumption. In analyses that were controlled for social desirability trait, significant correlations were found between stage of change and fruit and vegetable consumption (T1 r = .61, p = .000; T2 r = .54, p = .000). A significant effect for stage was found for fruit and vegetable consumption measured at T1, F_4,384 = 123.60, p = .000 and T2, F_4,384 = 89.59, p = .000. Post hoc analyses revealed that at both time points, intake was lower in precontemplation, contemplation, and preparation stages than in action and maintenance, and was lower in precontemplation than in preparation (Table 2).

Table 2.

Mean (SD) Servings of Fruits and Vegetables by Stage of Change at Two Consecutive Time Points^*

	Stage of Change
Time Point	PC	C	PR	A	M	Pattern of Differences
T1	1.98 (1.04)	1.97 (1.21)	2.38 (0.98)	5.50 (0.52)	5.48 (0.50)	PC, C, PR < A, M; PC < PR
T2	1.97 (1.09)	2.00 (1.01)	2.34 (1.00)	5.37 (0.51)	5.51 (0.50)	PC, C, PR < A, M; PC < PR

^*PC indicates precontemplation; C, contemplation; PR, preparation; A, action; M, maintenance; T1, time 1; and T2, time 2 (2 months after T1). Differences across stages at each time point were significant at p < 0.001 as determined by one-way analysis of covariance.

Stage of Change Distributions and Reliability Analyses

The distributions of youths across stages are shown in Figure 1. Although the sample was similarly distributed at both time points, slightly less than one-half (46%) of respondents were in the same stage at both measurements (Figure 2). Similar proportions of youths progressed to a later stage (25%) or regressed to an earlier one (29%). As shown in Table 3, there was a moderate level of congruence between stages for precontemplation (52.3%) and preparation (55.4%), with lower levels of congruence found for contemplation (22.0%) and maintenance (23.1%). Action was the least stable stage. None of the participants in action at T1 were in this stage at T2. The partial correlation between stages was .26 (p = .000), and Cohen's κ was .20 (p = .000).

Figure 1. Percentages of Youths at the Different Stages of Change at T1 and T2.

T1, time 1; T2, time 2 (2 months after time 1); PC, precontemplation; C, contemplation; PR, preparation; A, action; M, maintenance.

Figure 2. Patterns of Stability and Change in Stages Over a 2-Month Interval.

T1, time 1; T2, time 2 (2 months after T1). Regress indicates backward movement to an earlier stage from T1 to T2; stable, no change in stage from T1 to T2; and progress, forward movement to a later stage from T1 to T2. Bars represent the percentages of youths evidencing each pattern.

Table 3.

Levels of Congruence Between Stages From T1 to T2^*

	T1
T2	PC, No. (%)	C, No. (%)	PR, No. (%)	A, No. (%)	M, No. (%)	Total, No. (%)
PC	69 (52.3)	11 (26.8)	43 (25.6)	4 (40.0)	7 (17.9)	134 (34.4)
C	17 (12.9)	09 (22.0)	19 (11.3)	1 (10.0)	5 (12.8)	051 (13.1)
P	38 (28.8)	17 (41.5)	93 (55.4)	4 (40.0)	18 (46.2)	170 (34.6)
A	2 (1.5)	1 (2.4)	5 (3.0)	0 (0.0)	00 (0.0)	08 (2.1)
M	6 (4.5)	3 (7.3)	8 (4.8)	1 (10.0)	09 (23.1)	027 (6.9)
Total	132 (33.8)	41 (10.5)	168 (43.1)	10 (2.6)	39 (10.0)	390 (100)

^*T1 indicates time 1; T2, time 2 (2 months after T1); PC, precontemplation; C, contemplation; PR, preparation; A, action; and M, maintenance. Bold numbers indicate agreement between stages at T1 and T2.

DISCUSSION

Summary

This study examined the construct validity and test-retest reliability of a staging measure and algorithm for assessing readiness to consume five or more daily servings of fruits and vegetables among economically disadvantaged African-American adolescents. Findings revealed that a priori validity criteria were met, as evidenced by the positive and moderate correlations found between stage of change and fruit and vegetable consumption and observed intake differences across stages (i.e., intake increasing from precontemplation to preparation and from precontemplation, contemplation, and preparation to action and maintenance). Because the observed correlations and intake differences found between preaction and action stages may have been an artifact of the approach used to validate the staging measure, they do not provide conclusive evidence of validity. However, intake differences within preaction stages that would be expected based on variations in whether and when youths intended to increase their intake were also found, providing additional evidence of validity.

Unclear is whether intake differences found within preaction stages were an artifact of differences in the time frames for assigning stage (i.e., 1 and 6 months). Previous research has shown that African-Americans and low-income individuals tend to orient themselves with short-term consequences, not the future.^46,47 If youths were oriented to the present, this may explain why the primary difference was between those intending to change their dietary behavior within the immediate future (i.e., the next month) as compared to those who were not. The longer 6-month time frame for assigning stages other than preparation may also explain why intake did not differ between precontemplation and contemplation and action and maintenance. This time frame may be too distal for distinguishing youths who are not intending to increase their intake from those who are considering doing so, and those who have made recent dietary changes from those who have sustained the changes. Requiring further examination is the utility of alternate time frames for capturing meaningful changes in youths’ dietary intentions and behavior.

The measure may have underestimated the proportion of youths who were consuming five or more daily servings of fruits and vegetables (i.e., those in action and maintenance stages). Between 9% and 12% of youths were found to be in these stages, data that conflict with national nutrition surveillance data suggesting that the proportion of similarly aged youths who consume five or more daily servings of fruits and vegetables is between 16% and 20%.^48,49 The difference between estimates may be due to the relatively small size of the current sample as well as the measures used to assess intake. National estimates are based on 24-hour recall and food record data, whereas in this study, a single-item measure was used to estimate youths’ intake. The item did not define a serving or provide guidance on foods and beverages to consider. Possibly, youths had difficulty conceptualizing servings or omitted items that should have been counted.⁵⁰

Youths’ stage distribution at both time points was similar to the distribution observed in a national sample of middle-school students in sixth through eighth grades, i.e., 28.5% in precontemplation, 20.7% in contemplation, 27.3% in preparation, 4.6% in action, and 18.8% in maintenance.⁵¹ The distributions were also similar to the those observed among young adults and adults participating in eight of the nine community intervention projects funded by the National Cancer Institute for the 5-a-Day initiative, i.e., the largest proportions in preparation (36%–65%) and precontemplation (5%–35%) stages, with smaller proportions in contemplation (2%–11%), action (1%–3%), and maintenance (7%–24%).⁵² Noteworthy is that the measure and algorithm used in the 5-a-Day studies classified participants into action stages based on whether they were consuming five or more daily servings of fruits and vegetables; however, those consuming less than five daily servings were classified into preaction stages based on their intention to consume more fruits and vegetables. Intentions to consume more fruits and vegetables may differ from intentions to consume at least five daily servings, underscoring the importance of using the same criterion to classify respondents into preaction and action stages. Despite differences between the measures used, the similarity between the distributions across studies suggests that, as has been found in research on staging measures for exercise behavior, the stages of change for fruit and vegetable consumption are robust across classification methods.⁵³

The modest correlation and slight level of agreement found between stages indicate that the measure evidenced poor test-retest reliability over the 2-month study time frame. This finding is surprising in light of previous research demonstrating higher correlations and levels of agreement between stages for fruit and vegetable consumption measured over similar (i.e., 53-day and 7-week) and longer (i.e., 3-month) intervals in samples of adolescents and adults.^30,43,44

A number of factors may explain the lower stability found. Frequent change in beliefs regarding dietary behavior and whether one intends to change it is one factor. Research on attitudinal stability in differing age cohorts has shown that it increases with age,^54–56 with the instability observed during adolescence attributed to the general instability during this developmental period given the numerous transitions that are occurring. Possibly, youths were unreliable reporters of their behavior. The consistency of adolescent self-reports of health behaviors has also been shown to increase with age.^57,58 Thus, the lower stability found may be an artifact of the young age of the present sample. If these age-related factors were at work, this may explain the higher correlations and levels of agreement found in samples of adults.^30,44

The correlation between social desirability trait and stage of change at T1 suggests that youths who tended to respond in a socially desirable manner reported being at later stages. If youths overreported their stage of change at T1, this may explain backward shifts in stages that occurred from T1 to T2. The determination whether this was in fact the case must await further research given the questionable use of the M-C 2(10) with younger adolescents noted previously.

Possibly, the transitions were induced by changes in the home supply of fruits and vegetables. Studies of low-income adults reveal that they shop less often and purchase fewer types of fruits and vegetables than do higher-income adults,^59,60 practices that are shaped, in part, by the disbursement schedule of food stamp and other income subsidies.⁶¹ Perhaps more fruits and vegetables were available when benefits were received than at other times, a factor that might account for shifts in youths’ readiness to consume these foods.

Because data for this study were provided by youths in the control condition of a dietary intervention study, there was the possibility that stage transitions were induced by unintentional exposure to the intervention (i.e., cross-arm contamination). This was considered unlikely, however, because youths were assigned to study condition by site. Moreover, fruit and vegetable consumption did not increase in the control condition from T1 to T2, minimizing concerns that cross-arm contamination was work. There was however, the possibility that the instability found was due to testing effects. In addition to the staging measure, the outcome battery included measures of other TTM constructs hypothesized to influence fruit and vegetable consumption. The questions asked at T1 may have raised youths’ awareness of the targeted behavior, thereby influencing their answers at T2. Because the battery was completed twice over a relatively short interval, they may also have remembered the questions.

The stages were less stable for fruit and vegetable consumption than for nondietary behaviors (i.e., smoking and alcohol consumption).⁴² The extent of variability in food intake observed among adolescents poses challenges when assessing whether and when they intend to modify their intake of particular foods or food groups. Relative to discrete behaviors such as smoking and alcohol consumption, eating is a broad and complex behavior, involving many different foods to choose from and many different compositions. The variability in youths’ intake combined with the complexity of dietary behavior may explain the lower stability found.

Possibly, the lower stability was the result of unreliable measurement, as has been suggested elsewhere.^30,44 According to the TTM, stage transitions would be less likely to occur in precontemplation and maintenance because individuals at these stages have no intention of changing their behavior. However, transitions in contemplation, preparation, and action are to be expected because individuals at these stages are considering or actively engaged in behavior change. Not surprisingly, lower levels of congruence were found for contemplation and action stages. However, maintenance was among the least stable stages and preparation was the most stable stage, findings that are contrary to what would be predicted by the model. Alternatively, the unequal representation across stages and the small sample sizes in contemplation, maintenance, and action in particular may explain the lower levels of congruence found for these stages.

If the low stability found was an artifact of unreliable measurement, further field testing of the measure is needed to improve stability. Alternatively, this may signal the need to use a different approach for assigning stages in adolescent samples. It has been suggested that the algorithmic approach may not be appropriate for younger adolescents who may not yet have acquired well-developed methods of time estimation.⁶² Domel et al. used a 32-item continuous measure to assign stages measured 7 weeks apart in a sample of similarly aged youths.⁴³ Given similarities between the age groups and time frames studied, the higher correlations found by Domel et al.⁴³ may be an indication that this approach is more appropriate for this age group.

A final explanation for the observed instability was the low prevalence of consuming five or more daily servings of fruits and vegetables observed in this study. Kappa is adversely affected by inconsistencies for items with very high or very low prevalences.⁶³ Further research with adolescent samples that evidence more heterogeneous consumption patterns is needed to determine the effect of these differences on the level of agreement between stages.

Limitations

The use of a self-selected sample limits the generalizability of study findings. Participants may have differed from economically disadvantaged African-American adolescents in other youth services agency settings. The assessment of fruit and vegetable consumption using a single-item measure of unknown validity was less than optimal. Although there is evidence that brief screening measures are adequate for ranking adolescents according to intake level, notable discrepancies in mean intake have been found relative to 24-hour recall and food record data.^64,65 The use of measures of known validity to estimate youths’ fruit and vegetable consumption is suggested to further validate the tool. The use of alternate approaches that overcome problems associated with correlating stage of stage with one of its components parts (i.e., fruit and vegetable consumption) is also recommended. Unknown is whether youths considered themselves able to control their diet. This could differ, for example, at school, where they are free to select from among foods offered, and at home, where their dependence upon adults and the probable financial and accessibility issues facing parents to buy adequate fruits and vegetables may limit their choices. The use of a global measure of intake precluded assessment of the effects on youths’ stage of change of food and eating environments influencing their food choices (e.g., home, school, after-school, retail store, and restaurant settings). Findings should be interpreted with caution given the unequal distribution across stages and the extremely small sample size in the action stage. The measures for assessing stage of change and social desirability trait were not pilot tested with the target population. Possibly, revisions were needed to improve the clarity, interpretability, and relevance of the included items. Finally, factors that may have induced stage transitions, e.g., fruit and vegetable availability, taste preferences, and social support for healthful eating were not measured; thus, the potential effects of these variables on the observed transitions is unknown.

SO WHAT? Implications for Health Promotion Practitioners and Researchers.

What is already known on this topic?

Despite examinations of the validity of some measures of stage of change, few studies have examined the test-retest reliability of various staging approaches.

What does this article add?

Findings add to the limited validity and reliability data for staging measures generally, particularly among adolescents, and on the psychometric properties of the tool examined.

What are the implications for health promotion practice or research?

Practitioners planning to use the staging measure should confirm intake as determined by the first item with a previously validated instrument, and base decisions to use it for longitudinal assessments of stage on the stability found over clinically meaningful intervals. Replication studies are needed to confirm validity findings, identify factors that may contribute to unreliable measurement, and determine the best way to assess stage of change among adolescents and the degree of reliability that is to be expected.