Umyuangcaryaraq “Reflecting”: Multidimensional Assessment of Reflective Processes on the Consequences of Alcohol Use among Rural Yup’ik Alaska Native Youth

Abstract

Concerns in some settings regarding the accuracy and ethics of employing direct questions about alcohol use suggest need for alternative assessment approaches with youth. Umyuangcaryaraq is a Yup’ik Alaska Native word meaning “Reflecting.” The Reflective Processes Scale is a youth measure tapping awareness and thinking over potential negative consequences of alcohol misuse as a protective factor that includes cultural elements often shared by many other Alaska Native and American Indian cultures. A bifactor model of the scale items with three content factors provided excellent fit to observed data. Item response theory analysis suggested a binary response format as optimal. Evidence of convergent and discriminant validity is presented.

Alaska Native (AN) people have experienced devastating consequences from alcohol abuse. Alcohol related mortality is three times higher among AN people than the Alaska general population, and this burden extends to AN youth (1). This makes developing effective alcohol measurement strategies for AN youth of particular importance.

A majority of rural Alaska communities have declared themselves ‘dry,’ so possession of alcohol is illegal (2). Most are majority AN and small population (e.g. population 100–1200) with as few as 30–100 youth. For rural AN youth to acknowledge drinking on standard alcohol measures (3, 4) creates a situation of double jeopardy. First, trust in outside researchers and confidentiality is a concern given the risks posed to a youth acknowledging illegal alcohol use. Second, concerns arise regarding harmful consequences from inferences drawn about a small potentially identifiable community, and its subgroups and individuals, based on aggregate alcohol use data. Unsurprisingly, these youth report reluctance in truthfully answering direct questions about alcohol use.

The situation poses ethical challenges, given possibilities for harm when disseminating research results about small, vulnerable populations. Such reports on a community facing disturbing consequences from substance abuse can lead outsiders to stigmatize a community or region, and community members to self-stigmatize. Collaborative research, including community based participatory approaches (5, 6), requires careful consideration of unintended consequences when reporting results (7).

These accuracy and ethical concerns with direct alcohol use questions led to our work developing assessments of youth perceptions of the likelihood of experiencing specific consequences if they use alcohol. The assessment of a reflective capacity to consider potential negative consequences was based in previous work with AN adults on factors protective from alcohol (8).

Umyuangcaryaraq is a Yup’ik word meaning “reflecting.” The Reflective Processes Scale (RPS) taps a youth’s awareness and thinking over of negative consequences of alcohol use. This is one component of a broader cultural value of awareness of interconnections between people, animal, and spirit worlds, and of the resulting consequences of one’s actions, described by the Yup’ik as a part of ellangneq, or awareness–literally ‘awake.’ (8, 9). This construct emerged and scale item content was derived from qualitative life history interviews with AN adults (10) in which participants retrospectively recounted alcohol abuse consequences in rural Alaska that led to a decision to not use or abuse alcohol (11).

Reflective processes resemble the constructs of mindfulness and alcohol expectancies in the alcohol abuse prevention literature. Mindfulness, or the clear, nonjudgmental awareness of what is happening in successive moments of perception has been the topic of significant theory and measurement development effort (12–14). Alcohol expectancies, or the effects and consequences a person believes will occur as a result of drinking are learned beliefs commonly developed out of direct experience with alcohol use(15). Reflective processes are distinct from both in that they include unique cultural elements such as notions embodied within ellangneq.

Objectives of this study were to 1) assess the multidimensional structure of the RPS, 2) investigate RPS item functioning and optimal response level calibration, and 3) assess evidence for validity of RPS score interpretations.

Method

Participants

Participants were 284 AN youth age 12–18 from rural Alaska communities; 194 attended a Southeast Alaska boarding school while the remainder attended school in a predominately Yup’ik Southwest Alaska regional hub community. Mean age was 15.5 (SD=1.5), 57.7% were female, and 72% self-identified as members of the Yup’ik cultural linguistic group, followed by Inupiat (21%), Athabaskan (11%), Aleut/Alutiq (6%), and Tlingit/Haida (4%). Some identified with two or more groups.

Measures

Reflective Processes Scale

The RPS was adapted from the adult Yup’ik Protective Factors scale (8). Adaptations included age appropriate rewording and dropping items that assumed previous alcohol use community co-researchers believed current use inappropriate to include, as it was not the case for many younger adolescents. The 12-item RPS taps reflective processes involving thinking over negative consequences of alcohol use (see Appendix). RPS instructions prompt the youth to respond to each item through the following scenario: “Pretend someone would ask you to drink alcohol and you say no. How important are these reasons for you saying no.”

Validity measures

Awareness of Connectedness Scale (ACS; α = .82)

The ACS (9) is a 12-item scale measuring youth awareness of ways individual welfare is interrelated with family, community, and natural environment, and constitutes a culture-specific protective factor from suicide and substance abuse among many Indigenous people.

Communal Mastery Scale (CMS; α = .76)

The CMS is an 8-item measure composed of the Mastery–Family and Mastery–Friends subscales from the Multicultural Mastery Scale (MMS; (16)). These MMS subscales were developed with AN youth to tap culturally mediated protective problem-solving approaches involving joining with an interwoven social network.

Reasons for Life Scale (RFLS; α = .83)

The RFLS is a 13-item scale for AN adolescents adapted from the Brief Reasons for Living Inventory for Adolescents (17). The RFLS explores beliefs and experiences that contribute to a sense of meaning in life, and is hypothesized as inversely related to suicidal ideation.

Alaska Native Cultural Identification Scale (ANCIS) is an eight-item scale adapted from the Orthogonal Cultural Identification Scale (18). The ANCIS has two four-item subscales. The AN Cultural Identification subscale (ANCI; α = .77) measures identification with AN culture; the White American Cultural Identification subscale (WACI; α = .63) measures dominant culture identification.

Procedures

Items were adapted in an iterative process using cultural consultants, community co-researchers, and cognitive interviewing with focus groups. We pilot tested and removed poorly functioning items. Participants recruited through active parental consent with youth assent procedures received $15. The survey was administered in school computer labs via a secure web server based at the University of Alaska Fairbanks (UAF). Responses on all measures used a graphical continuous analog scale with a pointer in the shape of a salmon (a local cultural icon appealing to youth). Three equally spaced semantic anchors indicated “Not at all,” “Somewhat,” and “A lot.” Research assistants administered measures as part of a larger study on adaptive developmental pathways of AN youth. The UAF Institutional Review Board, the Yukon Kuskokwim AN Health Corporation Human Studies Committee, and local school and AN advisory school boards approved procedures.

Results

The analog response scale was coded into 20 equal intervals. To allow for interpretable Item Response Theory (IRT) analysis, we recoded intervals into 5 categories where 1–10=1, 11–15=2, 16–18=3, 19=4, and 20=5. This transformed the data to normalize the distribution, which was negatively skewed. Analyses used Mplus-6 (19).

Objective 1: Assessing the Internal Structure of the Reflective Processes Scale (RPS)

Analytic strategy

Bifactor analysis evaluated RPS internal structure. Bifactor models identify one general factor on which all items load, and two or more content factors. In bifactor modeling, the general factor is not allowed to correlate with content factors, while content factors are allowed to correlate with one another.

We compared two latent variable models: 1) a unidimensional model, and 2) a bifactor model with three content factors (20). A unidimensional model is not suggested by theory, but being the most parsimonious, must be rejected before a more complex model is retained. Item composition in model 2 was guided by research with the adult instrument (8). Content factors were labeled Wangnun Piyumiutenka, “Things I Want for Myself,” Ilamnun Piyumiutenka, “Things I Want for My Family,” and Ilamnun Piyumiutenka, “Things I Want for Our Way of Life” (Figure 1).

Figure 1.

Confirmatory factor analyses comparison between the unidimensional and three-content bifactor models. ^* p < .05, ^** p < .01.

Likelihood ratio tests were used to compare models. In these tests, difference in 2 times the log-likelihood between a full model and reduced model is distributed as a chi-square with degrees of freedom equal to the difference in number of free parameters. We also assessed fit of the final model using four indices, 1) chi-square to degree of freedom ratio (χ²/df) (21), 2) comparative fit index (CFI) (22), 3) Tucker-Lewis index (TLI) (23), and 4) root-mean squared error of approximation (RMSEA) (24). This literature on covariance structure modeling suggests chi-square/df ratio of less than 2, CFI and TLI of .95 or higher, and RMSEA of .05 or lower indicate acceptable fit.

Bifactor analysis results

Table 1 shows the three content factor model displayed acceptable fit indices χ²(39, N=284)=66. 81, χ²/df =1.71, GFI=.973, CFI=.984, RMSEA=.050, and fit better than the unidimentional model, Δχ²(2) =121.86, p < .01. Aside from item 6 and 7 loadings of .45 and .42 respectively, loadings on the general factor were significant and greater than .50, indicating a strong general factor (Figure 1).

Table 1.

Model Fit Indexes for the Unidimensional, and Three-Content-Factor Bifactor Models

Model	Number of items	χ2(df)	χ2/df	TLI	CFI	RMSEA
Unidimensional	12	188.67(54)	3.49	.905	.922	.094
Three-Content Bifactor	12	66.81(39)	1.71	.973	.984	.050
Three-Content Bifactor	10	31.07(22)	1.41	.988	.994	.038

Note. CFA = confirmatory factor analysis; TLI =Tucker-Lewis Index, CFI = comparative fit index; RMSEA = root mean square error of approximation

Objective 2: Evaluate Item Functioning with Alternative Response Scales

Analytic strategy

We used the Mplus bifactor model output to estimate values analogous to the two parameter Samejima graded response IRT model (25). Item parameter estimates produced in bifactor modeling are analogous to the discrimination and location parameters of the graded response model (20). The discrimination parameter a is an estimate of item slope, providing an indicator of how well each item differentiates between people with high and low values of the latent trait measured by the scale. A value below 1 is considered small, meaning the item differentiates poorly between differing trait levels, while discrimination index values above 1 mean item scores are more closely related to the trait level. The location parameter b is the intercept for each item response category. It indicates where along the continuum of the latent trait distribution the category is placed (e.g., intercept near the top of the distribution–high item difficulty, near the bottom–low item difficulty).

IRT results

Table 2 reports the discrimination or slope parameter (a) and the location parameter (b) estimates for the five-category response calibration and a two-category response calibration where response options 1 to 4=1 and 5=2. Overall, the 12 items provided satisfactory discrimination. Item 2 provided the highest discrimination index, while items 6 and 7 displayed the lowest. Location parameter values for the five-category response calibration from b₂ to b₄ generally showed endorsing a higher response option occurred at relatively low levels of reflective processes: items provided more information for individuals engaged in fewer reflective processes. However, item 6 and 7 location parameter values were larger, indicating these items were more informative at higher levels of reflective processes.

Table 2.

Item Parameters for Five- and Two-Category Calibrations

Item	Five-category calibration					Two-category calibration
	a	b1	b2	b3	b4	a	b
1	1.10	−.85	−.36	−.09	−.04	1.20	−.03
2	1.26	−.80	−.44	−.25	−.15	1.35	−.15
3	1.22	−.84	−.42	−.21	−.18	1.44	−.18
4	1.04	−.32	.16	.31	.39	1.05	.39
5	1.23	−.92	−.41	−.20	−.12	1.18	−.12
6	0.71	.−.04	.62	.93	.97	0.73	.97
7	0.71	−.66	.04	.36	.44	0.64	.44
8	1.18	−1.21	−.56	−.32	−.29	1.23	−.29
9	0.83	−.74	−.04	.14	.21	0.98	.21
10	0.95	−1.18	−.55	−.25	−.20	1.01	−.20
11	1.00	−2.03	−1.56	−1.31	−1.19	1.02	−1.19
12	0.91	−1.08	−.37	−.11	−.05	1.03	−.05

Note. a = an item slope or discrimination parameter; b = item intercept or location parameter

Item information functions are plotted in Figure 2. These and the location parameters show item 3 produced the highest curve, indicating it provided the highest amount of information, especially at below average to slightly above average reflective levels. The near flat curves of items 6 and 7 indicate they provided the least information across most levels of reflective processes. At low to moderate reflective levels, items 2, 5, and 8 supplied moderate to high information while items 1, 4, 9, 10, and 12 provided low to moderate information. Item 11 supplied low to moderate information at low reflective levels.

Figure 2.

Item information functions from the five-category calibration for the general construct (G).

Option response and option probability curves indicated significant overlap between response categories 1 through 4, suggesting they provided redundant information. In addition, the separations between these option curves were not even approximately equal, meaning these response options did not function as equal appearing interval scales. This led us to explore collapsing response categories 1 through 4 into a two-response option (i.e., 1–4 and 5). Table 2 reports the discrimination parameters (a) of the two- and five-category calibrations are very close. Except for item 1, the location parameters (b) in the two-category calibration are identical to the b₄ values in the five-category calibration. This indicates a simpler binary response format results in essentially no information loss.

Final bifactor analyses

As items 6 and 7 had the lowest bifactor loadings and provided limited information according to IRT analyses, they were dropped from the final bifactor analysis. Table 1 shows this final 10-item three-content bifactor model using 5-category calibration resulted in very good fit indices χ²(22, N=284)=31.07, χ²/df =1.41, TLI=.988, CFI=.994, RMSEA=.038, and the improvement of fit was significant, Δχ²(17) =35.74, p < .01. Descriptive statistics for the final 10-item RPS were M=36.15, SD=10.14, α = .84 for the 5-category and M=5.56, SD=3.10, α = .85 for the 2-category calibration.

While this analysis suggests content factors are accounting for unique variance orthogonal to the general factor, the point of the bifactor analysis was to account for the variance unique to these content areas and then estimate a general factor of reflective processes. All items loaded on this general factor significantly and above .60. However, though a majority of items loaded significantly on their respective content factor, some items did not load significantly.

Objective 3: Convergent and Discriminant Evidence for Validity of RPS Score Interpretations

We evaluated evidence for convergent validity of RPS score inferences by examining associations of the 10-item RPS using 2-category calibration with ACS, CMS, RFLS, and ANCI scores and discriminant validity with WACI scores. The RPS correlated in the expected direction with the ACS (r=.44, p<.01), CMS (r=.43, p<.01), and RFLS (r=.54, p<.01), and significantly but at lower magnitude with the ANCI (r=.16, p<.01), providing support for convergent validity. Lack of association between RPS and WACI scores (r=.02, p=.72) provided evidence for discriminant validity.

Discussion

These findings suggest the RPS provides a promising measure of reflective processes about alcohol that AN youth engage in when thinking about reasons not to drink. The RPS displayed associations with other protective factors identified in previous research with this population (26), providing initial validity evidence The RPS can be understood as measuring culturally mediated protective processes similar but not identical to mindfulness and alcohol expectancies.

A bifactor model (27) in which items loaded on one of three content factors, and also loaded on one general factor uncorrelated with the content factors, provided the best fit to the data. The RPS taps reflective processes regarding negative consequences of alcohol use along three dimensions of concern: (1) personal consequences of alcohol abuse, (2) impact upon the youth’s family, and (3) effect on the way of life the youth aspires to lead. Together these attitudes and beliefs constitute a more general factor of protection from alcohol. The bifactor analysis, along with evaluation of item functioning, suggested 10 optimally functioning items for retention and that a binary response format can provide simpler responding with effectively no information loss.

In contrast to second-order confirmatory factor analysis models that assume factors subsumed under a general factor, a bifactor model assumes non-hierarchical organization of general and content factors (20), making possible measurement of unique item variance accounted for by general and content factors (27). While previous reflective processes research with adults suggested multidimensionality (8), this bifactor analysis did not find strong evidence for multidimensional structure among youth. Instead, all items loaded strongly on a general factor of reflective process, while some content factor items loaded weakly. In contrast to adults, for youth the general factor is more meaningful; emphasis shifts to understanding reflective processes as a single factor. The content factors as a secondary structure instead provide evidence of equifinality in the origins of reflective processes. The content factors are orthogonal to the general factor, mapping the multiple potential sources for scores on the RPS. For example, one high score on the RPS might derive from considerations of self, whereas another might derive from concern for family.

These data also suggest the RPS is composed of items at easy item difficulty levels. These results indicate the meaningful variation in responding on the slider response format was between maximum and all other responses. Future users of the RPS can use the slider with 5-category coding if preferred, or can use a simpler binary response option to yield essentially the same level of information.

Consistent with research and theory guiding scale construction (8, 26), the RPS displayed convergent validity through association with other protective factors measures. These included awareness of connectedness–awareness of the interdependencies with family, community, and the natural environment; communal mastery–problem solving approaches involving joining with an interwoven social network; and reasons for life–beliefs and experiences, often based in AN culture, that contribute to a sense of meaning in life. The RPS displayed moderate association with AN cultural identification and was uncorrelated with the dominant culture identification. We are not asserting a simple relationship exists between cultural identification and protection from alcohol. This is unlikely for numerous reasons, not the least of which includes how cultural identification and alcohol use norms may derive from different primary socialization norms (28). These findings instead suggest the RPS includes elements aligned with worldviews, attitudes, and beliefs associated with AN cultures, but not the majority culture, and by extension, the RPS is at least in part representative of culturally mediated and culture specific protective mechanisms (29).

The concept of culturally mediated reflective processes holds promise to extend understanding of cultural variation in mindfulness. Mindfulness has a complex multivariate structure inversely related to alcohol use and negative consequences from use (12). Future RPS research can explore for similar relationships.

Reflective processes also have promising ties to alcohol expectancies research, which studies the consequences a person believes will result from drinking (15). Alcohol expectancy theory has generated several widely used measures (30) and a body of research based in a cognitive neuroscience model showing both positive and negative expectancies explains significant proportions of variance in drinking behavior (31, 32). Expectancies are stronger predictors of drinking behavior than demographic or other background variables. Longitudinally they predict onset of alcohol use and problem drinking behavior in adolescents, with evidence for a causal relationship (15). Recent research indicates a critical shift occurs in expectancies during early adolescence that increases risk, and that this shift is mediated in complex ways. Cognitive developmental maturation in concept formation and sensation seeking traits, along with values and social exposure regarding alcohol, affect positive and negative alcohol expectancies differently (33). Interventions that challenge positive alcohol expectancies are effective in decreasing college student drinking (34), but at present, limited research explores the role of negative expectancies or culture specific elements. The RPS can further such lines of inquiry, and is currently in use testing the efficacy of a multilevel community preventative intervention (29).

Future research is needed to examine RPS associations with mindfulness, alcohol expectancies, and other alcohol risk and protective factors, and to directly test its relation to alcohol use. The RPS also has potential to advance understanding of the relationship of cultural identification to alcohol involvement (35) and culturally mediated protective factors in other AN, American Indian, and ethnic minority groups. The RPS provides a promising assessment approach addressing several ethical concerns for AN youth and other small, vulnerable populations. In addressing these concerns, the RPS may also provide a more accurate assessment of risk than measures using direct questions about alcohol use.

Appendix

Reflective Processes Scale

1. You want to stay away from being like those who drink too much.

2. You do not want to see people get hurt.

3. You do not want to lose control of yourself.

4. You would feel embarrassed to have drinking in your family.

5. You see that drinking leads to violence.

   6^*. Others tell you that you are special (inqun).

   7^*. You feel responsible for other people in your family.

6. You want to be safe.

7. You want to be the kind of person that your parents want you to be.

8. You want to be a good role model.

9. When you have children you want your kids to have a good life.

10. You think about what will happen because of what you do.

Note. ^*Items 6 and 7 were dropped for the final 10-item set