Psychological inflexibility in university students: the european portuguese version of the acceptance and action questionnaire—university students

Abstract

Psychological inflexibility is defined as a transdiagnostic mechanism characterized by rigid efforts to control psychological reactions to painful, undesirable, unpleasant internal experiences. The Acceptance and Action Questionnaire-University Students (AAQ-US) was designed to assess students’ academic context-related psychological inflexibility. This study adapted the AAQ-US to Portuguese and examined its factor structure, validity, and reliability in a college student sample (sample 1: N = 262). Exploratory factor analysis and parallel analysis suggested a two-factor structure. A confirmatory factor analysis was conducted in a second sample to cross-validate the AAQ-US factor structure (sample 2: N = 260). One higher-order factor (psychological inflexibility) with two lower-order factors (cognitive fusion and experiential avoidance) revealed a very good fit to the data. The AAQ-US total and dimensions showed good reliability, convergent and incremental validity. Overall, the Portuguese version of the AAQ-US is a reliable and valid instrument for assessing context-specific psychological inflexibility in university students.

University students face a wide range of challenges during college (Pascoe et al., 2020). They must adapt to a new and demanding faculty environment, accomplish several different tasks, achieve academic success, move away from their family and friends, and deal with various hardships and stressors. This developmental period may, therefore, be characterized by psychological difficulties leading to suffering. Saleh et al. (2017), in a French sample comprising 483 university students, found that 72.9% presented psychological distress, 86.3% reported anxiety symptoms, and 79.3% suffered from depressive symptoms. More recently, Gao et al. (2020) found that anxiety was the most prevalent problem among Chinese college students (particularly in women). Nonetheless, these authors also found that depression was a growing phenomenon among male students (Gao et al., 2020). Another frequently reported mental health issue in college students is test anxiety (e.g., Gerwing et al., 2015). According to Reynolds and Kamphaus (2015), test anxiety can be described as encompassing worry and fear students experience in testing/assessment circumstances. Given these mental health issues in this population, psychological inflexibility has been pointed out as a relevant focus in interventions aiming to improve university students’ mental health (Levin et al., 2019).

Psychological inflexibility is a transdiagnostic process described as “a pattern in which behavior is excessively controlled by one’s thoughts, feeling and other internal experiences, or to avoid these experiences, at the expense of more effective and meaningful actions” (Levin et al., 2014, p. 2). This process encompasses “an inability to effectively modify behavior in response to an immediate stressor or changing environmental demands” (Gilbert et al., 2019, p. 88). It relates to the propensity to escape, avoid, control, or decrease painful internal experiences and behave inconsistently with valued life directions (Tavakoli et al., 2019). Although in the short term, psychological inflexibility may provide a sense of relief, being a slightly non-threatening way of regulating emotions (Hayes et al., 1996; Kashdan et al., 2006), in the medium or long term, it becomes a maladaptive process, acting as a risk factor for psychological difficulties (Kashdan & Rottenberg, 2010) and reduced quality of life (Lilly & Allen, 2015).

Among college students, higher levels of psychological inflexibility have been correlated with higher stress, worry, generalized anxiety, and somatization (Tavakoli et al., 2019). Furthermore, psychological inflexibility was found to mediate the relationship between college students’ sleep problems and depression (Peltz et al., 2020), as well as between depression, anxiety and stress symptoms, and procrastination (Eisenbeck et al., 2019). This psychological process seems to be a relevant target for psychological interventions aiming at improving college students’ mental health.

The Acceptance and Action Questionnaire (AAQ-II; Bond et al., 2011) is an extensively used psychological inflexibility self-report instrument, both in clinical and research settings (e.g., Gloster et al., 2017; Pennato et al., 2013). Several other context-specific measures, derived from the AAQ-II, were developed to target the specificities of different settings/populations. Work-related psychological inflexibility (Bond et al., 2012), psychological inflexibility in teachers (Hinds et al., 2015), and in university students (Levin et al., 2019) are some of the examples of these settings/populations. Additionally, measuring psychological inflexibility in clinical populations has also been promoted through the development of AAQ-II-specific versions tailored for people facing several health conditions: chronic pain (McCracken et al., 2004), smoking (Gifford et al., 2004), diabetes (Gregg et al., 2007), tinnitus (Westin et al., 2008), epilepsy (Lundgren et al., 2008), weight-related problems (Lillis & Hayes, 2008), pain (Wicksell et al., 2010), substance abuse (Luoma et al., 2011), body image (Callaghan et al., 2015), and infertility (Galhardo et al., 2020). In general, these context-specific self-report instruments have shown to be beneficial, acknowledging more comprehensive and detailed manifestations of psychological inflexibility. They have proved to effectively measure psychological inflexibility as a change-related process in several Acceptance and Commitment Therapy (ACT) studies.

As previously mentioned, one of the psychological inflexibility domain-specific measures was the Acceptance and Action Questionnaire - University Students (AAQ-US; Levin et al., 2019). The AAQ-US authors provide the rationale for developing this questionnaire, stating that the nonspecific AAQ may lack sensitivity to reveal ACT treatment therapeutic gains in university students. Besides the AAQ-US original version (Levin et al., 2019), a Turkish (Kuru et al., 2021) and a Spanish version (Barbosa-Güiza, 2020) are also available, revealing, similarly to the original version, a single factor structure, and good psychometric properties. The current study aimed to adapt the AAQ-US to the Portuguese language and examine its factor structure, reliability, and validity in a sample of Portuguese university students. To cross-validate the AAQ-US factor structure, a second sample was recruited. A validated Portuguese version of AAQ-US allows its use in counseling/clinical and research settings, contributing to a better understanding of the role of psychological inflexibility in difficulties university students may experience and identify those at risk of developing psychological problems. Furthermore, the AAQ-US may also be a useful measure to be used in efficacy studies of intervention programs aiming at promoting well-being and academic adjustment in university students. Based on the AAQ-US original version study and other languages’ versions of this measure, it is hypothesized that the Portuguese version of the AAQ-US will present a single-factor structure and positive correlations with measures of general psychological inflexibility, as well as measures of psychopathological symptoms, particularly with test anxiety.

Method

Participants

This study was conducted in two distinct convenience samples collected from several higher education institutions in Portugal. Inclusion criteria were as follows: (a) age (18–65 years old), (b) currently being a student at any Portuguese university, (c) being able to understand the Portuguese language, and (d) agreeing to participate in the survey and give informed consent. Sample 1 encompassed 262 students, 225 females (85.9%) and 37 males (14.1%), aged between 18 and 48 years old (M = 21.75; SD = 3.17). The majority of participants were single (n = 259; 98.9%), followed by living with a partner (n = 2; 5.4%), and divorced (n = 1; 0.8%). Concerning years of education, participants presented a mean of 13.95 years (SD = 1.45). One hundred and eighty-three participants were undergraduate students (69.8%), 78 (29.8%) were attending a master’s degree, and one participant (0.4%) was a Ph.D. student. The more represented scientific areas were Psychology (n = 68; 26%), Medicine (n = 44; 16.8%), Biology (n = 11; 4.2%) and Human Resources Management (n = 10; 3.8%).

Sample 2 was used to cross-validate the questionnaire structure. This sample comprised 260 students (203 females, 78.1% and 57 males, 21.9%) aged 18 to 57 years old (M = 21.20; SD = 5.83), presenting a mean of 12.86 (SD = 0.91) years of education. Participants were mainly single (n = 250; 96.2%), followed by divorced (n = 6; 2.3%) married or living with a partner (n = 2; 0.8%), and two participants preferred not to answer (n = 2; 0.8%). Two hundred and thirty-five participants were undergraduate students (90.4%), 24 (9.2%) were pursuing a master’s degree, and one participant (0,4%) was a Ph.D. student. The more frequent scientific areas were Psychology (n = 97; 37.3%), Medicine (n = 65; 25%), and Management (n = 15; 5.8%).

Instruments

Acceptance and Action Questionnaire- University Students (AAQ-US; Levin et al., 2019). The AAQ-US was developed based on other established self-report instruments for the assessment of psychological inflexibility (e.g., Bond et al., 2011). Items were generated to address six content categories: “1) difficult internal experiences as obstacles to effective/valued actions in school, 2) perfectionism and perseveration with school work (particularly as it relates to avoiding unwanted inner experiences or fused responses to thoughts about school), 3) avoidance, suppression, and other experientially avoidant behaviors in response to inner experiences related to school, 4) fusion with difficult thoughts related to school, 5) being on autopilot (not mindful) or difficulty attending in relation to school, and 6) a sense of purpose and values in relation to school” (Levin et al., 2019, p. 201). The final 12-items AAQ-US version revealed a single-factor solution, accounting for 51.38% of the variance, and showed good internal consistency (Cronbach alpha = 0.91) (Levin et al., 2019). According to Levin et al. (2019), validity studies demonstrated good face, convergent and divergent validities, and incremental validity in the prediction of mental health outcomes and academic outcomes while controlling for the effects of the AAQ-II. In the current study, the AAQ-US was translated from English to Portuguese. As a first step, an English native speaker, fluent in Portuguese and working as an English teacher in a language school, translated the original AAQ-US items to Portuguese. Then, the research team performed a back-translation to English and examined each item’s content correspondence (Erkut, 2010). Minor inconsistencies between these two versions were acknowledged, and minimal adjustments were completed for the items to match the original version. Content validity prior to the pilot study was established by the research team [four doctoral-level researchers within the area of clinical psychology (contextual cognitive behavioral therapies) and health sciences, one master student, and one doctoral student in clinical psychology]. In a subsequent pilot study, the Portuguese version was completed by 15 undergraduate students who were invited to comment on the instructions and the items’ intelligibility and comprehensibility. These students did not report difficulties or inconsistencies. The described procedures followed Hambleton et al. (2005) and the International Test Commission (2017) recommendations.

Acceptance and Action Questionnaire II (AAQ-II; Bond et al., 2011; Portuguese version by Pinto-Gouveia et al., 2012). The AAQ-II is a 7-item self-report questionnaire designed to assess psychological inflexibility (e.g., “It seems like most people are handling their lives better than I am”). Items are answered using a 7-point Likert scale ranging from never true (1) to always true (7) regarding the extent to which each statement is true to respondents. Higher scores reveal higher psychological inflexibility. A Cronbach’s alpha mean of 0.84 (different samples) was reported for the AAQ-II original version (Bond et al., 2011). Although the discriminant validity of the AAQ-II is questionable, given it may capture psychological distress more than being a measure of experiential avoidance (Tyndall et al., 2019; Rochefort et al., 2018; Wolgast, 2014), the AAQ-II was chosen to explore convergent validity, according to the AAQ-US original version study. The Portuguese version of the AAQ-II showed a Cronbach’s alpha of 0.90 (Pinto-Gouveia et al., 2012). In the present study, the AAQ-II revealed a Cronbach’s alpha of 0.95.

Depression, Anxiety and Stress Scales − 21 (DASS-21; Lovibond & Lovibond, 1995; Portuguese version by Pais-Ribeiro et al., 2004). The DASS-21 comprise three subscales (7 items each) assessing depression symptoms (e.g., “I felt that I had nothing to look forward to”), anxiety symptoms (e.g., “I felt I was close to panic”), and stress symptoms (e.g., “I tended to over-react to situations”). The 21 items are rated on a 4-point scale, varying from did not apply to me at all (0) to applied to me very much, or most of the time (3). In the Portuguese version’s validation study, Cronbach’s alphas of 0.85, 0.74, and 0.81 were found for the depression, anxiety, and stress scales, respectively (Pais-Ribeiro et al., 2004). In this study, a Cronbach alpha value of 0.91 was found for the depression scale, 0.88 for the anxiety scale, and 92 for the stress scale (sample 1).

Reactions to Tests questionnaire (RTT; Sarason, 1984; Portuguese version by Baptista et al., 1989). The RTT assesses anxiety in test/exam situations in university students. The RTT original version encompasses 40 items distributed by four subscales (10 items each): worry (e.g., Before taking a test, I worry about failure”), tension (e.g., “I feel distressed and uneasy before tests”), test irrelevant thinking (e.g., During tests, I think about recent past events”) and bodily symptoms (“My heart beats faster when the test begins”). The RTT Portuguese version comprises 34 items and the same subscales: worry (8 items), tension (9 items), test irrelevant thinking (9 items), and bodily symptoms (8 items). Each item is rated on a 4-point scale ranging from not at all typical of me (1) to very typical of me (4). Higher scores are indicative of higher test anxiety levels. A Cronbach alfa value of 0.78 was found in the original version (Sarason, 1984). In this study, a Cronbach alfa of 0.96 was found for the RTT.

Procedures

The study was approved by the Ethical Board of the (blind for review) (reference CE-P02-20). Permission to translate and use the AAQ-US original version was obtained from the authors. The access link to the protocol study was disseminated through courses’ coordinators, students’ associations, and social media (snowball sampling). The (Portuguese higher education institution - blind for review) courses’ coordinators were solicited to disseminate the study to their students by email or by making the study link available at the Moodle platform. Students’ associations from several universities were contacted and asked to also disseminate the study on their website or social media. The study was also advertised through a social media post. The study aims, procedures, voluntary nature of participation, and confidentiality of the data were provided to participants. Informed consent was requested from all participants. Those who volunteered to participate were asked whether they would be willing to answer one of the self-report measures (the AAQ-US) 4-weeks after the first assessment. Participants who consented to complete the AAQ-US a second time provided their email addresses and were contacted 4-weeks later (a link was sent for participation). An email address was made available for queries or additional information. All participants gave their informed consent. Data collection took place during March and April 2020 (sample 1) and during April and May 2021 (sample 2). The period for data collection was set during March and May to control for potential bias related to mid-term and final exams. Sample 1 and Sample 2 sizes were determined based on the recommendations of Singh et al. (2016), suggesting that any N > 200 offers adequate statistical power for data analysis.

Data analyses

The JASP software package version 0.16.4 was used (JASP Team, 2022) was used for data analyses. Uni- and multivariate normality of the data were analyzed. Skewness and kurtosis values of the items were not suggestive of severe violations to a normal distribution (Sk < |3| and Ku < |10|; Kline, 2005). The exploratory factor analysis of the AAQ-US, psychometric properties of the items, and the association with other constructs were examined in sample (1) The AAQ-US structure was tested through Exploratory Factor Analysis, using a Principal Axis Factoring (PAF) analysis with a direct oblimin rotation. A parallel analysis was calculated to define the number of factors to be retained. Items’ mean, standard deviation, and item-total correlations were calculated. Cronbach alpha and Composite Reliability (CR; Peterson & Kim, 2013) were used to examine the AAQ-US reliability. Cronbach’s alphas or CR above 0.70 denote good reliability (Field, 2013). Independent samples t-tests were analyzed to examine differences between men and women and Cohen’s d measured effect sizes. According to Cohen et al. (2003), effect sizes between 0.20 and 0.49 are considered small, between 0.50 and 0.79 medium, and above 0.80, large. Pearson moment correlations between the AAQ-US and other measures addressing similar and related constructs were estimated. Test-retest reliability was calculated in a sub-sample of 60 participants from sample 1 through Pearson correlation analysis. Pearson correlation coefficients between 0.10 and 0.39 are considered weak, between 0.40 and 0.69 moderate, and above 0.70 strong (Dancey & Reidy, 2017). A Confirmatory Factor Analysis (CFA), using the diagonal weighted least squares (DWLS) robust estimation technique, was tested in sample (2) Mahalanobis distance (MD²) results were analyzed to inspect the existence of outliers. Three models were tested in sample 2: A single-factor model (model 1), a two-factor model (model 2), and one higher-order factor with two lower-order factors model (model 3). Model 1 was unidimensional, as reported in the literature, considering the AAQ-US original study and other language versions of this questionnaire. Model 2 was defined based on the EFA and parallel analysis results found in the current study sample1. Model 3, specifying one higher-order factor with two lower-order factors, was explored given the two factors represent interdependent dimensions of psychological inflexibility, and it might be useful to also compute a total score, capturing a global sense of psychological inflexibility. To inspect the model adequacy, the relative chi-square statistic (< 5 suggests a good fit to empirical data) and the following goodness-of-fit indices were considered: The Comparative Fit Index (CFI), the Tucker and Lewis Index (TLI), and the Standardized Root Mean Square Residual (SRMR). The CFI and the TLI indicate a good fit to the data when values range between 0.90 and 0.95, and the SRMR indicates an acceptable fit when values are < 0.08 (Marôco, 2010). The Root Mean Square Error of Approximation (RMSEA), with values lower than 0.10 being considered good, with a 90% confidence interval (Kline, 2015), was also considered. Factor loadings and squared multiple correlations were used to analyze items’ local adjustment. Factor loadings higher than 0.40 and squared multiple correlations higher than 0.25 are acceptable (Tabachnick & Fidell, 2007).

Results

The Portuguese version AAQ-US items’ analyses, exploratory factor analysis, parallel analysis, reliability, test-retest reliability, associations with other variables, and gender differences were computed in sample 1. Confirmatory factor analysis and composite reliability were computed in sample 2.

Preliminary data analyses

AAQ-US items skewness values ranged from − 0.03 to 1.12, and kurtosis values varied from − 1.23 to 0.42. Kaiser-Meyer-Olkin test (KMO = 0.93) and Bartlett’s sphericity test (χ²₍₆₆₎ = 2132.96; p < .001) results confirmed the adequacy of the data to conduct an EFA.

Items analysis

Table 1 displays means, standard deviations, item-total correlations, and Cronbach alpha if item deleted for the AAQ-US items found in sample 1.

Table 1.

AAQ-US 12 items Mean (M), standard deviation (SD), corrected item-total correlations and Cronbach α if item deleted (sample 1: N = 262)

Items	M	SD	Corrected item-total r	Cronbach α if item deleted
1. I put off schoolwork when I feel bad	3.50	1.86	0.64	0.93
2. It seems like I’m just “going through the motions” at school	2.90	1.82	0.70	0.93
3. I struggle with my thoughts about school	3.47	1.98	0.69	0.93
4. I find myself avoiding going to classes when I feel anxious or depressed	2.70	2.01	0.68	0.93
5. When I think an assignment is too hard or confusing, I give up	2.32	1.56	0.67	0.93
6. It’s hard for me to focus on what my professors are saying in classes	3.42	1.73	0.69	0.93
7. I get so worried about upcoming exams that I feel paralyzed and can’t study	2.51	1.78	0.72	0.93
8. Worries get in the way of my success at school	3.53	1.93	0.79	0.93
9. My thoughts and feelings get in the way of studying	4.01	1.89	0.78	0.93
10. I don’t get anything out of a class when I’m having negative thoughts	3.63	1.94	0.75	0.93
11. I often believe that I’m not smart enough to be in college or in this major	3.09	2.08	0.72	0.93
12. I get so caught up in my worries during tests that I have trouble focusing on the test itself	2.89	1.93	0.75	0.93
Total score	37.97	17.22	-	0.94

Item-total correlations varied between 0.64 (item 1) and 0.79 (item 8). Cronbach coefficient alpha would not improve with the exclusion of any item.

AAQ-US exploratory factor analysis

The AAQ-US EFA conducted in sample 1 showed a two-factor solution, explaining 61.7% of the total variance, with factor 1 (cognitive fusion) explaining 37.9% and factor 2 (experiential avoidance) explaining 23.8% of the variance. Parallel analysis results indicated two factors with eigenvalues exceeding the corresponding values for a randomly generated data matrix of the same size were found (12 variables x 262 participants). Table 2 presents factor loadings and communalities of the AAQ-US items.

Table 2.

AAQ-US EFA factor loadings and communalities (h2) (sample 1: N = 262)

Items	F1	F2	h2
1. I put off schoolwork when I feel bad	0.09	0.66	0.52
2. It seems like I’m just “going through the motions” at school	0.05	0.90	0.73
3. I struggle with my thoughts about school	0.39	0.38	0.51
4. I find myself avoiding going to classes when I feel anxious or depressed	0.13	0.66	0.57
5. When I think an assignment is too hard or confusing, I give up	0.01	0.78	0.62
6. It’s hard for me to focus on what my professors are saying in classes	0.56	0.20	0.51
7. I get so worried about upcoming exams that I feel paralyzed and can’t study	0.73	0.06	0.60
8. Worries get in the way of my success at school	0.95	0.10	0.78
9. My thoughts and feelings get in the way of studying	0.80	0.05	0.70
10. I don’t get anything out of a class when I’m having negative thoughts	0.74	0.07	0.63
11. I often believe that I’m not smart enough to be in college or in this major	0.60	0.19	0.56
12. I get so caught up in my worries during tests that I have trouble focusing on the test itself	0.88	0.07	0.69

Factor loadings ranged from 0.39 (item 3 – “I struggle with my thoughts about school”) to 0.95 (item 8 - “Worries get in the way of my success at school”). All items presented communalities above 0.51. Item 3 equally loaded in both factors. Nevertheless, when considering psychometric criteria such as item-total correlations and Cronbach alpha if item deleted, there were no arguments for removing this item. The two factors showed a correlation of 0.77.

Confirmatory factor analysis

To confirm the AAQ-US factor structure, three models were tested using sample 2. Model 1 was a unidimensional model (like the AAQ-US original version), model 2 a two-factor model (based on current study EFA and Parallel analysis results), and model 3 a one higher-order factor model with two lower-order factors (considering a global sense of psychological inflexibility). CFA results regarding fit indices are displayed in Table 3.

Table 3.

Fit indices for the three tested models

Model	χ2(df)	CFI	TFI	GFI	RMSEA	SRMR
Model 1	117,25(54) p < .001	0.98	0.98	0.98	0.07 [0.05 − 0.08]	0.08
Model 2	66,30(53) p = .104	1.00	1.00	0.99	0.03 [0.00 − 0.05]	0.06
Model 3	66,30(52) p = .088	1.00	1.00	0.99	0.03 [0.00 − 0.05]	0.06

The models showed a very good fit to the data, with models 2 and 3 showing slightly better results than model 1. The AAQ-US items’ factor loadings are presented in Fig. 1. Factor loadings ranged between 0.59 (item 1) and 0.84 (item 9).

Fig. 1.

One higher-order factor model with two lower-order factors (cognitive fusion and experiential avoidance)

The AAQ-US reliability was further confirmed by calculating CR, showing a value of 0.93 for the total score and of 0.91 and 0.83 for factors 1 and 2, respectively.

Item reliability analysis

The AAQ-US total revealed good internal consistency, presenting a Cronbach alpha of 0.94. Factor 1 and Factor 2 presented Cronbach alpha values of 0.92 and 0.87, respectively.

Test-retest reliability

Test-retest reliability considering a 4-week period was computed in a subsample of 60 participants from sample 1. Pearson correlations were calculated, and strong correlations were found (r = .83; p < .001) for the AAQ-US total score, (r = .81; p < .001) for the cognitive fusion subscale, and (r = .77; p < .001) for the experiential avoidance subscale.

Associations with other variables

Convergent validity was explored by computing correlation coefficients between the AAQ-US total score and subscales and other measures tapping other constructs (psychological inflexibility, depression, anxiety and stress, test anxiety). Partial correlations controlling for the effect of psychological inflexibility (measured by the AAQ-II) were used to address incremental validity. Results are displayed in Table 4.

Table 4.

Zero-order correlations between the AAQ-US total score and subscales and the AAQ-II, the DASS-21 Depression, Anxiety and Stress, and the Reactions to Tests questionnaire; partial correlations between the AAQ-US total score when controlling for the effect of the AAQ-II (N = 262)

	AAQ-US	AAQ-US-CF	AAQ-US-EA	Partial correlations
AAQ-II	0.71**	0.71**	0.62**
DASS-21 Depression	0.64**	0.59**	0.61**	0.30**
DASS-21 Anxiety	0.48**	0.52**	0.37**	0.18*
DASS-21 Stress	0.52**	0.55**	0.43**	0.20*
RTT	0.66**	0.69**	0.53**	0.44**

Note. AAQ-US = Acceptance and Action Questionnaire – University Students; AAQ-US-CF = AAQ-US Cognitive Fusion subscale; AAQ-US-EA = AAQ-US Experiential Avoidance subscale; AAQ-II = Acceptance and Action Questionnaire-II; DASS-21 Depression = DASS-21 Depression scale; DASS-21 Anxiety = DASS-21 Anxiety scale; DASS-21 Stress = DASS-21 Stress scale; AAQ-US = Acceptance and Action Questionnaire-University Students; RTT = Reactions to Tests questionnaire; **p < .001, *p < .050

Zero-order correlation results showed moderate to strong correlations. The highest value was found between the AAQ-US score and the AAQ-II (r = .71, p < .001), whereas the lower value was found between the AAQ-US and the DASS-21 Anxiety scale (r = .48, p < .001). Partial correlations results were still significant, although weaker than the zero-order correlations.

When exploring the relationships between the AAQ-US scores and sociodemographic characteristics such as age and years of education, significant negative correlations were found regarding age (r = − .16; p = .010) and years of education (r = − .21; p < .001).

Gender differences

The AAQ-US mean scores were compared between men and women, and no significant differences were found: t₍₂₆₀₎ = 0.52, p = .607.

Discussion

Psychological inflexibility has been described as a relevant transdiagnostic process linked to the onset and maintenance of an extensive variety of psychological problems (Levin et al., 2014; Masuda et al., 2014. Consequently, a spreading of numerous context-related measures of psychological inflexibility has been developed, with Ong et al. (2019) suggesting that context-specific validated measures of psychological inflexibility may capture more precise manifestations of this mechanism. The current study aimed to translate the AAQ-US to the Portuguese language and explore its factor structure (across two university students’ samples) and psychometric characteristics.

Exploratory factor analysis of the Portuguese version of the AAQ-US, conducted in sample 1, indicated a two-factor solution, and a parallel analysis also pointed to the retention of two factors. These two factors items’ content analysis indicated that factor one encompassed six items associated with cognitive fusion, and factor two comprised six items related to experiential avoidance. These two factors showed a strong correlation which corroborates the theoretical model of psychological inflexibility (Levin et al., 2014). Cognitive fusion may be conceptualized as a process in which one gets entangled with difficult thoughts (sometimes judgmental and self-descriptive), which impacts emotional and behavioral regulation (Hayes et al., 2011). The AAQ-US addresses this process through items such as “My thoughts and feelings get in the way of studying” or “I often believe that I’m not smart enough to be in college or in this major.” By perceiving one’s thoughts as literal truths, students may not only experience distress, but they can also develop a reluctance to stay in contact with these thoughts or other internal events by trying to avoid them or the circumstances that prompt them, which corresponds to experiential avoidance (Hayes et al., 1996). The experiential avoidance process is illustrated in the AAQ-US by items such as “I find myself avoiding going to classes when I feel anxious or depressed” or “When I think an assignment is too hard or confusing, I give up.” These two processes (cognitive fusion and experiential avoidance) are frequently interconnected and seem to influence each other. Being unwilling to remain in contact with painful private experiences can paradoxically intensify their manifestation (Wenzlaff & Wegner, 2000) and therefore strengthen potential occurrences of cognitive fusion (Berghoff et al., 2018). Moreover, avoiding situations (e.g., going to classes, giving up academic assignments) that may trigger painful thoughts and feelings, limits the chances of adjusting these painful private events throughout the direct experience (e.g., exposure). Although principal component analyses revealed a single-factor solution in the AAQ-US original study (Levin et al., 2019) and in the Spanish (Colombia) AAQ-US version (Barbosa-Güiza, 2020), these studies did not include a confirmatory factor analysis.

The Portuguese AAQ-US confirmatory factor analysis, conducted in sample 2, suggested that a two-factor model or one higher-order factor with two lower-order factors solution were more suitable than a single-factor structure. The one higher-order factor with two lower-order factors structure was chosen as preferable to represent the theoretical model where psychological inflexibility integrates both cognitive fusion and experiential avoidance as relevant interdependent constructs but also as part of a global sense of psychological inflexibility. Levin et al. (2019) defined six content categories based on the ACT hexaflex model, having generated four to 11 items for these school-related categories, but scale refinement was based on psychometric criteria, and there is no mention to which category belongs each of the 12 final items. The current study findings are not in line with the AAQ-US Turkish version that pointed to a single-factor structure (Kuru et al., 2021). Although some context-specific AAQ variants have shown a single factor structure [e.g., the Acceptance and Action Questionnaire for Weight-Related Difficulties (Lillis & Hayes, 2008); the Body Image Psychological Inflexibility Scale (Callaghan et al., 2015), the Psychological Inflexibility Scale – Infertility (Galhardo et al., 2020)], there were also domain-specific psychological inflexibility measures denoting a two-factor structure. Similarly to what was found in the present study, the Psychological Inflexibility in Pain Scale (Wicksell et al., 2010) revealed an avoidance of pain factor and a cognitive fusion with pain factor. Also, the Acceptance and Action Questionnaire – Substance Abuse (Luoma et al., 2011) showed a defused acceptance factor and a values commitment factor.

Regarding reliability, the item-total correlations further confirmed the adequacy of the items. Additionally, the AAQ-US as a global measure of psychological inflexibility, as well as the cognitive fusion and the experiential avoidance dimensions presented good internal consistencies, similar to that found for the original (Levin et al., 2019), the Spanish (Barbosa-Güiza, 2020) and the Turkish (Kuru et al., 2021) versions. Test-retest reliability analysis in a subsample of 60 students suggested that the AAQ-US total score and subscales Portuguese version revealed temporal stability. Although the test-retest reliability has not been examined in the original version (Levin et al., 2019) or the Spanish (Barbosa-Güiza, 2020), the current results were in line with the ones found in the Turkish version (Kuru et al., 2021), regarding the AAQ-US total score.

Overall, correlation results between the AAQ-US total score and cognitive fusion and experiential avoidance subscales, and measures of general psychological inflexibility, psychopathological symptoms, and test anxiety revealed these measures were positively associated. Higher levels of university-context psychological inflexibility, as measured by the AAQ-US Portuguese version, were related to higher levels of general psychological inflexibility, higher depression, anxiety, and stress symptoms, and higher anxiety in tests. A similar pattern was also found for the cognitive fusion and experiential avoidance subscales. As the Turkish AAQ-US (Kuru et al., 2021) mentioned, these findings suggest that university-context psychological inflexibility (AAQ-US) is related but is not redundant regarding the other considered variables. When controlled for the general psychological inflexibility (AAQ-II) effect, partial correlations results showed that the AAQ-US was still significantly associated with psychopathological symptoms and test anxiety, corroborating the AAQ-US incremental validity.

Several limitations should be acknowledged when examining the Portuguese version of the AAQ-US results. The participants’ recruitment process (online survey) encompasses some limitations, such as sampling bias, self-selection concerns, or under-representation of the population (Wright, 2005). Future research should further examine the AAQ-US model measurement and structural invariance in larger and size-equivalent samples of both genders. Furthermore, data collection took place during the Covid-19 pandemic, and it is unclear whether this condition may have influenced the results. Students were confronted with new challenges, online classes and assessments, being socially isolated, and facing difficulties in coping with new academic demands. Therefore, future studies should replicate these findings in a different timeline.

Future research may also study the AAQ-US sensitivity to therapeutic change in ACT interventions targeting university students. Previous studies have found that ACT (e.g., Katajavuori et al., 2021; Viskovich & Pakenham, 2020) has shown to be effective for college students experiencing psychological difficulties. Moreover, context-specific psychological inflexibility has been pointed sensitive to therapeutic approaches, and specific measures that allow assessing these change mechanisms are useful (Schumacher et al., 2019). Additionally, from a counseling point of view, the AAQ-US items may be used qualitatively, aiming for a more in-depth assessment of students who seek psychological support.

Despite the limitations mentioned, the Portuguese version of the AAQ-US seems to be a valid and reliable measure of students’ psychological inflexibility. It may be a unique and valuable contribution to clinical/counseling work with university students.

Authors’ contribution

All authors contributed to the study conception and design. Material preparation, data collection and analyses were performed by Ana Galhardo, Maria Neto, Bárbara Monteiro, and Ilda Massano-Cardoso. The first draft of the manuscript was written by Ana Galhardo, Cláudia Ferreira and Marina Cunha, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

No funding was received for conducting this study.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Conflict of interest disclosure

The authors have no relevant financial or non-financial interests to disclose.

Ethics approval statement

Approval was obtained from the Ethics Committee of the Instituto Superior Miguel Torga (CE-P02-20). The procedures used in this study adhere to the tenets of the Declaration of Helsinki.

Consent to participate

Informed consent was obtained from all individual participants included in the study.