A multi-model LLM framework for AI-assisted psychological scale development: initial psychometric evaluation of the Emotion Regulation Scale (ERS)

Abstract

Background

Emotion regulation is a key construct in mental health, yet existing measures differ in conceptual coverage, with some focusing on specific strategies or regulatory difficulties rather than broader cognitive, behavioral, and contextual regulation processes. This study aimed to develop an Emotion Regulation Scale (ERS) grounded in Gross’s Process Model, using large language models (LLMs) as assistive tools within an established psychometric framework.

Method

This cross-sectional study included 743 adults recruited through convenience sampling. Participants were randomly divided into two non-overlapping subsamples, one used for exploratory factor analysis and the other for confirmatory factor analysis. Multiple LLMs and AI-based tools assisted with item generation, refinement and bias screening, translation, and post-EFA construct interpretation, with human oversight throughout development. An initial pool of 25 items was generated and refined to produce the final 17-item ERS.

Results

The ERS revealed a preliminary four-factor structure, with EFA explaining 50.85% of the variance. CFA provided preliminary support for a post hoc refined 17-item model (χ²/df = 1.95, CFI = 0.934, TLI = 0.918, IFI = 0.935, RMSEA = 0.050). Reliability was acceptable for early-stage scale development (α = 0.68–0.74; ω = 0.68–0.75). Convergent validity showed positive correlations between adaptive strategies and cognitive reappraisal (r = 0.522–0.608) and between maladaptive cognitions and suppression (r = 0.400). Concurrent validity showed associations with depression (r = 0.432), anxiety (r = 0.462), well-being (r = -0.330), and quality of life (QOL1 = -0.357; QOL2 = -0.295), while adaptive strategies correlated with depression (r = -0.184 to -0.239), anxiety (r = -0.141 to -0.211), well-being (r = 0.150 to 0.203), and quality of life (r = 0.211 to 0.375). Known-groups validity indicated significant differences across depression and anxiety levels (MCR: F(4, 738) = 39.32; 64.49, p < 0.001).

Conclusion

The findings provide preliminary psychometric evidence for the 17-item ERS and suggest that it may complement existing measures by capturing broader cognitive, behavioral, and contextual emotion regulation processes. The study illustrates the potential for LLM-assisted scale development within established psychometric procedures and human oversight. Further research using diverse samples is needed to evaluate the scale’s psychometric properties, generalizability, and broader utility.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s40359-026-05759-w.

Introduction

Emotion regulation is an important psychological construct with well-established associations with mental health, well-being, and quality of life, as evidenced by meta-analytic findings [1, 2, 3, 4]. Emotion regulation refers to how individuals actively influence their internal emotional states as well as the way these emotions are outwardly expressed [5]. The process model of emotion regulation suggests that individuals utilize two broad categories of strategies [6, 7]: antecedent-focused strategies, such as situation selection, situation modification, and cognitive reappraisal, and response-focused strategies, most notably emotional suppression. These regulatory approaches are fundamental to sustaining psychological balance and overall well-being, and they are generally classified as either adaptive or maladaptive. In contrast, emotion dysregulation defined as the maladaptive, excessive, or ineffective use of otherwise adaptive emotion regulation processes has been consistently associated with poorer mental well-being and increased vulnerability to a range of psychological disorders [8].

Emotion regulation has been assessed using several established instruments with different conceptual emphases. The Emotion Regulation Questionnaire (ERQ) assesses cognitive reappraisal and expressive suppression [9]. However, its relatively focused scope provides limited coverage of other regulatory processes, such as rumination, avoidance, and behavioral or social approaches. The Difficulties in Emotion Regulation Scale (DERS) assesses difficulties in emotion regulation across six domains [10]. However, it primarily focuses on regulatory deficits rather than specific adaptive and maladaptive regulation strategies. Similarly, the Cognitive Emotion Regulation Questionnaire (CERQ) provides a multidimensional assessment of cognitive regulation strategies [11]. However, its primarily cognitive focus offers less coverage of behavioral, situational, and environmental or social processes. Taken together, these instruments provide valuable and complementary approaches to assessing emotion regulation, but none was specifically designed to integrate cognitive, behavioral, and contextual or environmental and social regulation processes within a single theoretically grounded framework. The ERQ focuses on two specific regulatory strategies, the DERS emphasizes difficulties and deficits in emotion regulation, and the CERQ primarily assesses cognitive regulation strategies. Thus, although these measures provide important coverage of specific aspects of emotion regulation, their differing conceptual emphases leave a gap in the assessment of emotion regulation as a broader, multidimensional process encompassing cognitive, behavioral, and contextual domains.

Within Health Psychology, assessment serves as a fundamental component for the systematic examination of psychological factors, health behaviors, and individuals’ reactions to disease and healthcare experiences [12]. Health Psychology, since emerging as an independent discipline, has placed strong emphasis on the development and systematic examination of psychological and behavioral interventions aimed at enhancing health outcomes. These include evidence-based approaches such as Cognitive Behavioral Therapy [CBT], Mindfulness-Based Interventions [MBIs], Dialectical Behavior Therapy [DBT], Emotion Regulation Therapy [ERT], and Positive Psychology Interventions [PPIs], which collectively aim to reduce psychological distress and enhance overall well-being. These interventions often target psychological outcomes, for instance anxiety, quality of life, emotional regulation, and depression, aiming to improve individuals’ overall mental and physical well-being (e.g., [13–14]). The success and scientific rigor of such interventions are grounded in the use of reliable and valid measurement tools that reflect the complex psychological and social components of health and illness [15, 16]. Accordingly, accurate measurement is essential for the systematic assessment of psychological constructs and for advancing psychological research and practice [17]. Nevertheless, developing such measures demands considerable resources, including careful planning, extended time, material support, and financial costs. In particular, traditional methods for instance manually generating items and conducting psychometric evaluation after data collection require considerable time and financial resources, and potential issues may only become evident at later stages of large-scale data gathering [17, 18]. These methodological and practical limitations have driven growing interest in technological solutions, particularly artificial intelligence (AI), to enhance research efficiency and scalability.

A CSIRO report [19] indicates that approximately 98% of scientific disciplines have adopted AI-based approaches, reflecting its increasing integration into research workflows. This trend is further supported by a growing body of recent literature (e.g. [20, 21, 22, 23]), . Recent advances in large language models (LLMs), including systems such as Claude, ChatGPT, and Gemini, have introduced new possibilities for enhancing survey development processes [24]. LLMs are computational language models designed to interpret human input and generate responses that closely resemble natural human language [25]. These models are able to handle multiple linguistic tasks, including text translation, content summarization, answering queries, and generating coherent text [26]. They are trained on large-scale text corpora drawn from diverse sources, enabling them to capture linguistic structures and semantic connections across words and phrases, and to generate coherent, context-aware responses [25, 27]. These developments have also prompted researchers to examine alternative approaches for supporting the construction of psychological measures, including methods informed by artificial intelligence and language modeling techniques (e.g. [28, 29, 30]), .

According to Kuru [18], LLMs offer potential solutions to methodological limitations across different stages of the process. In the item generation phase, conventional methodologies predominantly rely on manual literature reviews and the judgments of experts, which may inadvertently neglect intricate linguistic nuances or emergent conceptual themes. Conversely, approaches grounded in natural language processing [NLP] enable the structured analysis of large-scale textual datasets drawn from sources for example scholarly works, reports, and archival materials. Through this process, recurrent terms, conceptual linkages, and nuanced linguistic structures can be identified. Such approaches may contribute to the development of a more comprehensive and representative pool of items while reducing subjectivity and improving the coverage of key dimensions of the construct [31, 32, 33]. Recent empirical studies provide emerging evidence for the use of LLMs in psychological item development. For example, Terry et al. [34] found that LLM-generated items reproduced the factor structure of an established grit measure and demonstrated satisfactory psychometric properties. Similarly, Zhang et al. [35] reported that LLM-generated situational judgment items showed comparable internal consistency and convergent validity to expert-developed items. Recent research has also demonstrated that LLMs can support multiple stages of item development, including item generation, content review, linguistic evaluation, bias assessment, and item refinement, while incorporating human feedback to ensure item quality [36]. Other work has demonstrated the potential of LLMs to support both the development and refinement of psychological instruments, while emphasizing the continued importance of human judgment and psychometric validation [37].

A recent evidence synthesis by Shubina et al. [38] highlighted both the potential and the limitations of ChatGPT in psychology. The review identified several concerns, including ethical issues, potential misuse, inaccuracies in generated information and references, limited contextual understanding, and reduced emotional sensitivity. It also emphasized the importance of preventing over-reliance on such systems and developing clear ethical guidelines and policies for their use. Accordingly, the authors called for further research to examine the effectiveness, ethical use, and long-term impact of these technologies in psychological research and practice.

Cultural context is an important consideration in the assessment of emotion regulation because the ways individuals experience, express, and regulate emotions may be shaped by cultural norms, social expectations, and interpersonal contexts. Although several established emotion regulation measures are widely used across populations, their applicability across different cultural contexts warrants careful consideration, particularly regarding the linguistic and cultural adaptation of their items to the target population [39]. A cross-cultural study of the ERQ-8 across 29 countries found that emotion regulation strategies did not readily converge across cultures, highlighting the importance of examining measurement equivalence across cultural settings [40]. This consideration is particularly relevant in Saudi Arabia, where ongoing social and cultural changes may influence patterns of emotional expression, interpersonal interaction, and the use of cognitive and behavioral regulation strategies. Recent advances in artificial intelligence have demonstrated the potential of large language models (LLMs) to support the generation and adaptation of psychological items while maintaining attention to psychometric quality, construct representation, and factorial structure [41, 42, 43]. However, existing research has primarily examined the use of individual LLMs and the adaptation of pre-existing psychological instruments, with comparatively less attention to the development of entirely new scales for complex psychological constructs [41]. Moreover, the potential contribution of multi-model LLM frameworks, in which outputs from multiple models can be compared and refined, remains insufficiently examined in psychological scale development. To address these gaps, the present study employs a multi-model AI-assisted framework to develop and preliminarily evaluate an Emotion Regulation Scale grounded in Gross’s Process Model, examining its psychometric properties, including factor structure, reliability, and convergent and concurrent validity, among a community sample of Saudi adults.

Method

This study adopted a quantitative cross-sectional research design. The study sample comprised Saudi participants recruited using a convenience sampling approach. This approach was selected to facilitate access to a broad range of adult participants and enable efficient recruitment through online channels. Data were collected using a structured electronic questionnaire distributed online via social media platforms, providing a practical means of reaching eligible participants across different demographic backgrounds. Data collection was conducted between January and March 2026. The sample size was determined using a participant-to-item ratio of 3–10 participants per item [44]. The total sample of 743 participants yielded approximately 29.7 participants per item, while the EFA and CFA subsamples yielded approximately 14.8 and 14.9 participants per item, respectively, exceeding the recommended minimum ratio. The survey page provided information about the study’s purpose and procedures, emphasizing that participation was voluntary and that participants could withdraw at any time without penalty. It also assured that all data would be kept confidential, securely stored, not shared with third parties, and analyzed in a manner that prevents individual identification, with use restricted to research purposes only. Informed consent was obtained electronically on the first page of the survey, where only those who selected “agree” were allowed to proceed, while selecting “do not agree” terminated the survey. The study was carried out in full compliance with the ethical standards outlined in the Declaration of Helsinki, and all procedures involving human participants adhered strictly to established ethical guidelines.

Participants

The study included a total of 743 adult participants, comprising 298 males (40.1%) and 445 females (59.9%). For the purpose of ensuring rigorous validation, the sample was randomly split into two independent groups. The first group (n = 370) was utilized to perform Exploratory Factor Analysis (EFA), while the second group (n = 373) was designated for Confirmatory Factor Analysis (CFA). The sociodemographic characteristics of the full sample, as well as those of the two subsamples employed in the exploratory and confirmatory analyses, are summarized in Table 1.

Table 1.

Demographic characteristics of the total sample and subsamples (EFA and CFA)

Demographic Characteristics		EFA Sample		CFA Sample		Overall Sample		Pearson χ² (df)	P-value
		n	%	n	%	n	%
Age	Mean ± SD	33.1 ± 10.7		32.9 ± 10.8		33.0 ± 10.7
	18 to 29 years	160	43.2%	165	44.2%	325	43.7%	0.592 (3)	0.898
	30 to 39 years	104	28.1%	104	27.9%	208	28.0%
	40 to 49 years	81	21.9%	75	20.1%	156	21.0%
	50 years & older	25	6.8%	29	7.8%	54	7.3%
Education	Less than High School	3	0.8%	2	0.5%	5	0.7%	3.362 (3)	0.339
	High School	80	21.6%	68	18.2%	148	19.9%
	Undergraduate Degree	206	55.7%	232	62.2%	438	59.0%
	Graduate Degree	81	21.9%	71	19.0%	152	20.5%

The participants’ ages spanned from 18 years to over 50 years. The overall mean age was 33.0 years (SD = 10.7), indicating a relatively young adult sample. The sample was skewed toward younger adults, with 43.7% aged 18–29, followed by 30–39 (28.0%) and 40–49 (21.0%), while those 50 + were the smallest group (7.3%). Educational levels were high, with 59.0% holding undergraduate degrees, 20.5% graduate degrees, 19.9% high school diplomas, and 0.7% below high school.

A critical aspect of the methodological design was the comparability of the two randomly split subsamples. As shown in the data, the EFA and CFA groups were remarkably similar across all demographic variables. The mean ages (33.1 vs. 32.9 years) and the proportional distributions across age categories,, and education levels, showed only minimal, non-systematic variation. This demographic equivalence confirms the success of the random allocation procedure, ensuring that any differences in the factor structures derived from the two subsamples are attributable to the validation process itself rather than to confounding sample characteristics. This strengthens the methodological rigor and the validity of the subsequent analytical findings.

Measures

Emotion Regulation Scale (ERS)

The Emotion Regulation Scale (ERS) is a 17-item self-report instrument developed to assess multiple cognitive, behavioral, and engagement-based emotion-regulation strategies. The scale comprises four dimensions: Maladaptive Cognitions & Rumination (MCR; 6 items), Cognitive-Behavioral Avoidance (CBA; 5 items), Positive Cognitive Reappraisal (PCR; 3 items), and Environmental & Social Engagement (ESE; 3 items). The items assess a range of emotion-regulation processes, including maladaptive cognitive patterns, avoidance and attentional redirection, positive cognitive reframing, and proactive environmental and social engagement.

All items are rated on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). No items are reverse-scored in the final ERS scoring. The ERS is interpreted at the subscale level, with each dimension representing a distinct emotion-regulation strategy.

• Maladaptive cognitions & rumination (MCR). This dimension assesses maladaptive cognitive processes characterized by persistent rumination on distress, negative cognitive bias, self-devaluation, and suppression of positive affect. Higher MCR scores indicate greater endorsement of maladaptive cognitions and rumination. This dimension includes Items 1–6.

• Cognitive-behavioral avoidance (CBA). This dimension assesses behavioral and attentional strategies used to manage emotional distress, including modifying or avoiding situations expected to elicit distress and redirecting attention away from emotional triggers. Higher CBA scores indicate greater use of cognitive-behavioral avoidance and attentional-redirection strategies. This dimension includes Items 7–11.

• Positive cognitive reappraisal (PCR). This dimension assesses cognitive strategies used to reinterpret difficult situations and modify their emotional impact, including de-catastrophizing, adopting a growth-oriented perspective, and viewing difficulties as temporary. Higher PCR scores indicate greater use of positive cognitive reappraisal. This dimension includes Items 12–14.

• Environmental & social engagement (ESE). This dimension assesses proactive strategies involving modification of the environment and engagement in positive social situations to regulate emotional states. Higher ESE scores indicate greater use of environmental and social engagement strategies. This dimension includes Items 15–17.

Subscale scores are calculated separately for each of the four dimensions. Given the multidimensional structure of the ERS and the distinct conceptual meaning of its four correlated dimensions, an overall total score is not recommended at this preliminary stage of scale development. The results of both exploratory and confirmatory factor analyses supported a four-factor structure, demonstrating adequate psychometric properties. The finalized version of the scale is presented in Appendix 1.

Procedure for developing the emotion regulation scale

Drawing on the methodology proposed by Keane and McNaughton [45], which comprises five stages: Construct Setup, Item Generation, Refinement and Bias Checks, Validation and Resolution, and Finalisation. These five overarching stages comprised multiple development steps, detailed documentation of the multi-model AI-assisted workflow used in the development of the Emotion Regulation Scale (ERS) is provided in Appendix 2. we used GPT-5 to generate a self-report emotion regulation scale grounded in Gross’s Process Model. In the first stage, GPT-5 was provided with a structured conceptualization of emotion regulation, defined as the set of processes through which individuals shape the emotions they have, the timing of these emotions, and the manner in which they are experienced and expressed. The theoretical framework adopted was Gross’s Process Model, with its five dimensions specified: situation selection, situation modification, attentional deployment, cognitive change, and response modulation. A distinction was also made between strategies aimed at shaping emotions before they fully develop and strategies used to manage emotional reactions once they occur, and brief illustrative examples were provided to clarify each dimension and support the generation of accurate and representative items. Following this, item generation guidelines were set, specifying a Likert format and requesting 5–6 concise (10–20 words), first-person items per dimension, focused on observable behaviors, free of jargon and double-barrelled phrasing, and balanced between positively and negatively worded statements. In the second stage (Item Generation), an initial item pool was generated by OpenAI GPT-5, with five items produced for each dimension. In the third stage, several steps were undertaken. First, reverse-coding was applied by designating one negatively worded item per dimension; OpenAI GPT-5 was used to identify such items, which were then reviewed and refined by the authors. Second, redundancy and ambiguity screening was conducted using Google Gemini to identify overlap, reduce ambiguity, and ensure that items reflected specific, observable behaviors, while also checking for linguistic and cultural bias and suggesting more neutral phrasing. Third, grammar and clarity were improved using a combination of AI tools and Grammarly, with Gemini reviewing the revised items for readability and coherence. Fourth, the items were translated into Arabic independently by OpenAI GPT-5 and Claude 3.5; following the completion of both translations, the author and an independent translator reviewed and compared the versions to finalize the Arabic text. The translated items were reviewed by three associate professors of psychology at King Abdulaziz University, all specializing in psychometrics and research methodology. The experts evaluated each item using predefined criteria covering accuracy, clarity, conceptual equivalence, methodological rigor, and inclusivity. Expert comments and suggested revisions were subsequently reviewed by the research team. Items identified as requiring improvement on any of these criteria were revised where appropriate, and the resulting item set was finalized by the research team. Finally, randomization was performed by shuffling the items into a random order to reduce sequence bias. In the fourth stage, cross-model validation, construct verification, and ambiguity resolution procedures were conducted using two AI models (Google Gemini and NotebookLM). Both models were provided with the results of the Exploratory Factor Analysis, including the item loadings for each factor, along with the definition of emotion regulation and Gross’s Process Model with its five dimensions and the distinction between antecedent-focused and response-focused strategies. Each model was asked to review the items within each factor, identify common themes, assign descriptive labels, and provide conceptual and operational definitions aligned with the theoretical framework. The outputs from the two models were then compared to examine the consistency of factor interpretations and to identify any discrepancies in classification. Based on this comparison, construct verification was achieved by retaining factor interpretations that were theoretically justified and aligned with Gross’s model, while weaker or unsupported associations were excluded. Ambiguity resolution was addressed by removing items with primary factor loadings below 0.40 in the EFA (items 6, 7, 10, and 25) and refining factor labels based on the resulting factor structure. The finalized factor structure, labels, and definitions were subsequently reviewed by the authors. Although LLMs were used as an AI-assisted tool during the item-generation and refinement stages, all theoretical interpretations, methodological judgments, and final decisions regarding item revision, retention, and deletion were made by the research team. The overall procedure, including construct setup, item generation, refinement and bias checks, validation and resolution, and finalization, is summarized in Fig. 1.

Fig. 1.

Workflow for the AI-assisted development of the emotion regulation scale

The generalized anxiety disorder scale (GAD-7) [46]

The GAD-7 is a self-report measure consisting of seven items used to evaluate the severity of anxiety symptoms experienced during the previous four weeks. Responses are recorded on a 4-point Likert scale ranging from (never = 0) to (nearly every day = 3). The instrument captures a range of symptoms, including feelings of nervousness or tension, excessive worry, difficulty relaxing, irritability, and fears that something adverse may occur. Total scores range from 0 to 21, with higher scores reflecting greater anxiety severity. In the present study, the scale demonstrated strong internal consistency, with Cronbach’s alpha and McDonald’s omega both equal to 0.90.

The patient health questionnaire (PHQ-9) [47]

The PHQ-9 is a nine-item self-report measure used to evaluate depressive symptoms in accordance with established diagnostic criteria over the preceding two weeks. Responses are rated on a 4-point Likert scale ranging from (never = 0) to (nearly every day = 3). Total scores can range from 0 to 27, with higher scores indicating more severe depressive symptomatology. In the present study, the instrument demonstrated high internal consistency, with both Cronbach’s alpha and McDonald’s omega reported at 0.90.

The world health organization-five well-being index (WHO-5) [48]

The WHO-5 Well-Being Index comprises five items that assess individuals’ subjective well-being over the previous two weeks, including statements such as feeling cheerful and in good spirits. Each item is rated on a 6-point scale ranging from (at no time = 0) to (all of the time = 5), with higher scores reflecting better well-being. The total raw score ranges from 0 to 25 and is converted into a percentage score (0–100) by multiplying by four. Scores below 50 (or raw scores below 13) are indicative of reduced well-being and may signal the need for further psychological evaluation, such as screening for depressive disorders. In the current study, the scale demonstrated excellent internal consistency, with Cronbach’s alpha and McDonald’s omega both reaching 0.91.

Quality of life scale (QOL) [49, 50]

Subjective quality of life was assessed using two items adapted from the abbreviated version of the World Health Organization Quality of Life Scale, which was translated and validated by Almohammadi et al. [50]. The first item evaluates overall quality of life (“How would you rate your quality of life?”), while the second examines satisfaction with general health (“How satisfied are you with your health?”). Responses are provided on a 5-point Likert scale ranging from (very poor/completely dissatisfied = 1) to (very good/completely satisfied = 5). In the present study, the measure demonstrated acceptable internal consistency, with a Cronbach’s alpha of 0.72.

Translation and cross-cultural adaptation procedures

All questionnaires were administered to participants in Arabic. The GAD-7, PHQ-9, and WHO-5 were translated and culturally adapted from English into Arabic following the guidelines of Beaton et al. [51]. The process included forward translation, synthesis, back-translation, and expert committee review to ensure conceptual and cultural equivalence. The QOL items were adapted from the previously translated and validated Arabic version of the WHOQOL-BREF reported by Almohammadi et al. [50].

Statistical analysis

Data analysis followed a staged, sequential procedure aimed at developing and validating the Emotional Regulation Scale (ERS). Preliminary screening, descriptive analyses, and Exploratory Factor Analysis (EFA) were conducted using IBM SPSS Statistics (Version 27), whereas Confirmatory Factor Analysis (CFA) was performed with IBM Amos (Version 26). In addition, the R-based Parallel Analysis Engine [52] was used to support decisions regarding factor retention. EFA was applied to the first subsample using Principal Axis Factoring with Varimax rotation.

The suitability of the 25-item set for factor analysis was evaluated through the Kaiser–Meyer–Olkin (KMO) index and Bartlett’s Test of Sphericity. The number of retained factors was determined based on the results of parallel analysis and examination of the scree plot, both of which supported a four-factor structure. Item selection criteria included primary factor loadings of at least 0.40 and the absence of substantial cross-loadings, defined as loadings of ≥ 0.30 on more than one factor. Items exhibiting substantial cross-loadings were considered for removal based on both statistical and theoretical considerations. To assess the robustness of the EFA solution to the choice of rotation method, a sensitivity analysis was conducted by repeating the EFA using Principal Axis Factoring with Promax oblique rotation. The sensitivity analysis was conducted for both the initial 25-item solution and the final 21-item solution to evaluate the stability of the item-loading patterns and factor assignments when correlations among factors were permitted.

CFA was subsequently conducted on the second subsample to verify the factor structure identified in the EFA. The model parameters were estimated using the Maximum Likelihood method. Model adequacy was assessed using multiple fit indices, including the chi-square to degrees of freedom ratio (χ²/df), Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Incremental Fit Index (IFI), Goodness-of-Fit Index (GFI), and Root Mean Square Error of Approximation (RMSEA). Model refinement was carried out post hoc by excluding items with factor loadings below 0.40 and by examining modification indices to introduce theoretically justifiable correlations between error terms.

Internal consistency of the ERS dimensions was evaluated using both Cronbach’s alpha and McDonald’s omega coefficients. Convergent validity was examined through Pearson correlation coefficients between ERS dimensions and ERQ subscales, reflecting the extent of association among related constructs. Concurrent validity was assessed by correlating ERS dimensions with measures of depression, anxiety, psychological well-being, and quality of life, to ensure differentiation from conceptually distinct variables. To assess the reliability and construct validity of the measurement model, Composite Reliability (CR) and Average Variance Extracted (AVE) were calculated to evaluate internal consistency and convergent validity, respectively. Discriminant validity was assessed using the Heterotrait–Monotrait (HTMT) ratio of correlations. Following Henseler et al. [53], HTMT values below 0.85 were considered indicative of adequate discriminant validity under the conservative criterion. Known-groups validity was tested using one-way analysis of variance (ANOVA) to compare ERS scores across five depression severity groups based on the PHQ-9 (none/minimal, mild, moderate, moderately severe, and severe) and four anxiety severity groups based on the GAD-7 (minimal, mild, moderate, and severe). Where significant differences were observed, Tukey’s HSD post hoc tests were conducted to identify specific group contrasts.

Results

Exploratory factor analysis (EFA)

To investigate the latent factor structure of the initial 25-item Emotion Regulation Scale (ERS), an Exploratory Factor Analysis (EFA) was performed. Principal Axis Factoring (PAF) with Varimax rotation was employed using a randomly selected subsample (n = 370). Varimax rotation was selected to facilitate a parsimonious and readily interpretable exploratory solution with clear item–factor assignments, rather than to imply theoretical independence among the emotion-regulation dimensions. As Varimax is an orthogonal rotation method, correlations among the rotated factors are not estimated in the EFA solution; potential associations among the factors were subsequently examined within the correlated-factor CFA model. Before conducting the extraction, the appropriateness of the dataset for factor analysis was evaluated. The Kaiser-Meyer-Olkin (KMO) index of sampling adequacy reached 0.837, reflecting meritorious adequacy [54]. In addition, Bartlett’s Test of Sphericity yielded a statistically significant result (χ²(300) = 2566.86, p < 0.001), indicating that the correlations among items were sufficient to justify factor analysis.

The determination of the number of factors to retain relied on parallel analysis and examination of the scree plot (see Fig. 2). Parallel analysis, which involves comparing eigenvalues derived from the observed data with those obtained from randomly generated datasets, served as a rigorous retention criterion. Factors were retained when their observed eigenvalues exceeded the corresponding 95th percentile eigenvalues from the parallel analysis. The findings showed that the first four factors had eigenvalues surpassing their respective threshold values, whereas the fifth factor did not meet this criterion. This result indicated that only four factors explained variance beyond what would be expected by chance. This interpretation was further supported by the scree plot, which displayed a distinct point of inflection following the fourth component. Accordingly, a four-factor solution was retained for subsequent extraction and rotation.

Fig. 2.

Scree plot of the EFA model

The initial EFA identified four items that did not meet the specified primary factor-loading criterion of ≥ 0.40. Specifically, Items 6, 7, 10, and 25 had maximum factor loadings of 0.247, 0.314, 0.283, and 0.340, respectively, with corresponding communalities of 0.078, 0.207, 0.193, and 0.140. These items were therefore removed, and the EFA was rerun with the remaining 21 items. The resulting four-factor solution obtained through Principal Axis Factoring with Varimax rotation is presented in Table 2. This solution accounted for a total of 50.85% of the variance.Factor labels were assigned based on the shared conceptual content of the items within each empirically derived factor and their alignment with Gross’s Process Model, supported by the cross-model review using Gemini and NotebookLM described in the scale-development procedure; final factor labels and interpretations were determined by the research team.The first factor, which accounted for 22.85% of the variance (Eigenvalue = 4.80), was defined by six items. The highest loadings on this factor were observed for items describing a persistent focus on negative emotions (0.669), a tendency to interpret difficult situations as confirmations of personal failure (0.639), and a reluctance to propose modifications even when potentially helpful (0.558). This factor thus appears to represent a cognitive-emotional pattern centered on rumination and negative self-appraisal.

Table 2.

Rotated factor matrix for the 21-item EFA solution (n = 370)

Items	Factor 1	Factor 2	Factor 3	Factor 4
18.  I focus my attention on negative emotions for a long time.	0.669
14.  In difficult situations, I tell myself that the situation confirms my failure.	0.639
22.  I refrain from suggesting modifications, even if I think they would be helpful.	0.558
24.  I view setbacks as indicators of my self-worth.	0.533
3.    I find myself constantly thinking about upsetting events.	0.462
20.  I hide my positive feelings (like joy or delight).	0.46
12.  I replace in-person meetings with phone communication to reduce anxiety.		0.599
2.    I change current plans to reduce my stress level.		0.592
13.  When I am stressed, I shift my focus to tasks that do not provoke my emotions.		0.537
11.  I change my route to avoid meeting specific people.		0.493
1.    I avoid going to places where I expect to feel upset.		0.472
23.  I consciously observe pleasant details to redirect my attention.		0.445
8.    I focus on a specific task to avoid thinking about my feelings.		0.437
9.    I try to view difficult events as opportunities for growth and learning.			0.635
4.    I remind myself that the situation is not as bad as it seems.			0.531
19.  I interpret problems as temporary obstacles, not permanent ones.			0.523
5.    I try not to show my emotions, even when I feel them strongly.			0.499
16.  I deliberately attend events that I believe will improve my mood.				0.733
17.  I rearrange my surroundings to feel more comfortable.				0.604
21.  I seek out social situations that I expect will make me feel joyful.				0.477
15.  I openly express my negative feelings (like sadness or anger).				0.462
Eigenvalues	4.80	3.34	1.31	1.23
% of Variance explained	22.85	15.88	6.26	5.86
Cumulative %	22.85	38.73	44.99	50.85

The second factor explained an additional 15.88% of the variance (Eigenvalue = 3.34) and was comprised of seven items. This dimension was characterized by strategies involving the substitution of direct social interaction with mediated communication to manage anxiety (0.599), the alteration of plans to reduce stress (0.592), and the strategic redirection of attention towards neutral tasks when experiencing tension (0.537). Collectively, these items suggest a proactive, avoidance-oriented regulatory strategy aimed at minimizing exposure to anticipated stressors.

Factor 3 explained 6.26% of the variance (Eigenvalue = 1.31) and was marked by four items with primary loadings emphasizing a cognitive reappraisal approach. This includes reframing challenging events as opportunities for development (0.635), consciously moderating one’s perception of a situation’s severity (0.531), and viewing problems as transient rather than permanent (0.523). This factor appears to reflect a constructive cognitive style aimed at altering the emotional impact of events through positive reinterpretation.

Finally, the fourth factor explained 5.86% of the total variance (Eigenvalue = 1.23) and comprised four items. It was defined by behaviors such as deliberately seeking out mood-enhancing events (0.733), actively modifying one’s physical environment to increase comfort (0.604), and pursuing social situations anticipated to foster positive affect (0.477). This factor appears to reflect a behavioral approach strategy focused on actively modifying one’s context to regulate emotional experience. The retained items demonstrated primary factor loadings above the specified 0.40 threshold. Cross-loadings were also examined alongside primary factor loadings and theoretical interpretability during item evaluation. Given the exploratory nature of the analysis, these factor interpretations should be regarded as preliminary empirical dimensions requiring confirmation through subsequent psychometric evaluation.

Sensitivity analysis using oblique rotation

To assess the robustness of the EFA findings to the choice of rotation method, the four-factor EFA was re-estimated using Principal Axis Factoring with Promax oblique rotation. In the initial 25-item analysis, the Promax solution was generally consistent with the Varimax solution, although several items showed weak or ambiguous loading patterns. Specifically, the four items removed in the original analysis (ERS6, ERS7, ERS10, and ERS25) also had primary loadings below 0.40 under Promax. The final 21-item solution showed greater stability across rotation methods. All 21 retained items had the same primary factor assignments under Promax as under Varimax. Promax pattern coefficients ranged from 0.398 to 0.790, with ERS23 showing a marginal primary loading of 0.398 but remaining assigned to the same factor as in the Varimax solution. Inter-factor correlations in the Promax solution ranged from − 0.249 to 0.572. The oblique-rotation sensitivity analysis indicated that the factor structure of the final four-factor solution was stable across rotation methods. Detailed results of the sensitivity analysis are presented in Appendix 3.

Confirmatory factor analysis (CFA)

To evaluate the four-factor structure obtained from the exploratory analysis, a Confirmatory Factor Analysis (CFA) was conducted using an independent validation sample (n = 373). The hypothesized model specified the 21 retained items to load onto their four respective correlated latent factors, as defined by the EFA. The model was estimated using maximum likelihood estimation in AMOS software (Appendix 4).

The initial model demonstrated a poor fit to the data. Although the chi-square to degrees of freedom ratio (χ²/df = 593.018/184 = 3.22) fell within the acceptable threshold (i.e., less than 5.0), and the Root Mean Square Error of Approximation (RMSEA = 0.077) satisfied the commonly accepted criterion for reasonable fit (< 0.08), several incremental fit indices did not reach recommended levels. Specifically, the Comparative Fit Index (CFI = 0.786), Tucker-Lewis Index (TLI = 0.756), and Incremental Fit Index (IFI = 0.789) were all below the suggested cutoff of 0.90. The Goodness-of-Fit Index (GFI = 0.869) was considered marginally acceptable. Furthermore, the statistically significant chi-square value (p < 0.001), despite its sensitivity to large sample sizes, suggested a degree of misfit between the proposed model and the observed data.

To improve the model’s fit to the data, a post hoc refinement was undertaken. Items with standardized factor loadings below the recommended threshold of 0.40 were considered for removal. Furthermore, modification indices were consulted to identify theoretically justifiable error covariances between items within the same latent factor, which can account for shared method variance or specific content overlap. Based on these criteria, four items (5, 11, 12, and 15) were sequentially removed from the model. A complete item-level trajectory from the initial 25-item pool through EFA and CFA to the final 17-item scale is provided in Appendix 5. This process resulted in a 17-item model for the present study (Fig. 3).

Fig. 3.

Post hoc refined CFA model (17-item)

The revised 17-item model showed acceptable-to-good fit to the data. All standardized factor loadings for the 17 items were statistically significant (p < 0.001), indicating significant relationships between the items and their intended latent constructs. All factor loadings were at or above the 0.40 threshold. Specifically, items for Factor 1 loaded between 0.400 and 0.790; for Factor 2 between 0.400 and 0.685; for Factor 3 between 0.581 and 0.722; and for Factor 4 between 0.613 and 0.741.

The fit indices all met or exceeded standard benchmarks for good model acceptance: χ²(109) = 212.02, p < 0.001; χ²/df = 1.95; CFI = 0.934; TLI = 0.918; IFI = 0.935; GFI = 0.940; RMSEA = 0.050. The substantial improvement in model fit achieved by removing four low-loading items is detailed in the comparison presented in Table 3.

Table 3.

Comparison of model fit indices for initial and improved CFA models

Fit Index	Acceptable Threshold	Initial 21-item Model	Post Hoc Refined 17-item Model
χ²/df	< 3.0 (Good) < 5.0 (Acceptable)	3.223	1.945
CFI	≥ 0.90 (Acceptable) ≥ 0.95 (Excellent)	0.786	0.934
TLI	≥ 0.90 (Acceptable) ≥ 0.95 (Excellent)	0.756	0.918
IFI	≥ 0.90 (Acceptable) ≥ 0.95 (Excellent)	0.789	0.935
GFI	≥ 0.90 (Acceptable) ≥ 0.95 (Excellent)	0.869	0.94
RMSEA	≤ 0.08 (Acceptable) ≤ 0.06 (Good)	0.077	0.05

Note. CFI  Comparative Fit Index, TLI  Tucker-Lewis Index, IFI  Incremental Fit Index, GFI  Goodness-of-Fit Index, RMSEA  Root Mean Square Error of Approximation

This final four-factor, 17-item solution provides preliminary evidence of an interpretable measurement structure. The model demonstrated acceptable-to-good global fit, with all standardized factor loadings statistically significant and at or above 0.40. These findings provide preliminary evidence relevant to the structural validity of the ERS; however, the final item configuration requires independent confirmation in a separate sample. Based on the final model, the Emotion Regulation Scale (ERS) comprises four empirically derived dimensions (Table 4).

Table 4.

reliability and convergent validity coefficients for ERS dimensions

Dimensions	No. of items	Cronbach’s Alpha	McDonald’s Omega	CR	AVE
Maladaptive Cognitions & Rumination (MCR)	6	0.73	0.73	0.71	0.30
Cognitive-Behavioral Avoidance (CBA)	5	0.71	0.70	0.64	0.27
Positive Cognitive Reappraisal (PCR)	3	0.68	0.68	0.66	0.40
Environmental & Social Engagement (ESE)	3	0.74	0.75	0.70	0.44

Reliability

The internal consistency of the four ERS dimensions was evaluated using Cronbach’s alpha and McDonald’s omega.

The four dimensions demonstrated varying levels of internal consistency. Maladaptive Cognitions & Rumination (MCR) demonstrated modest but potentially acceptable reliability coefficients, particularly given the preliminary stage of scale development and the relatively short subscale. Cognitive-Behavioral Avoidance (CBA) showed comparable reliability, whereas Positive Cognitive Reappraisal (PCR) demonstrated somewhat lower reliability. Environmental & Social Engagement (ESE) showed the highest reliability among the four dimensions (Table 4). Overall, the reliability estimates were generally within an acceptable range for research instruments composed of relatively few items, particularly given the preliminary stage of scale development. PCR demonstrated relatively modest reliability, which may be attributable, among other factors, to the limited number of items included in the subscale, as it consists of only three items [55, 56].

Convergent validity

Construct validity was evaluated by examining evidence of both convergent and discriminant validity using established psychometric criteria. Convergent validity was assessed primarily through Composite Reliability (CR) and Average Variance Extracted (AVE). CR reflects the internal consistency of the indicators within each construct, whereas AVE represents the proportion of variance captured by a construct relative to measurement error. As shown in Table 4, CR values ranged from 0.635 to 0.705, indicating modest internal consistency, with all values exceeding the 0.60 criterion [57]. All AVE values ranged from 0.266 to 0.436 and were below the conventional threshold of 0.50, indicating limited evidence of convergent validity based on AVE alone. Although Fornell and Larcker [57] noted that convergent validity may still be considered when CR exceeds 0.60, the evidence for convergent validity based on CR and AVE should be interpreted cautiously. Similarly, Vo Thai et al. [58] emphasized that AVE values below 0.50 may be retained when the corresponding CR is acceptable. Thus, the findings provide only partial and preliminary support for convergent validity, and the low AVE values warrant cautious interpretation.

Convergent validity was further examined through the pattern of associations between the four ERS dimensions and the two theoretically established regulatory strategies assessed by the Emotion Regulation Questionnaire (ERQ): Cognitive Reappraisal and Expressive Suppression.

A theoretically consistent pattern of associations emerged from the correlation matrix. Positive Cognitive Reappraisal (PCR) demonstrated a positive association with ERQ Cognitive Reappraisal (r = 0.608, p < 0.01). This finding provides supportive evidence that the PCR dimension is consistent with an adaptive reappraisal-based regulatory style represented in the ERQ framework.

Cognitive-Behavioral Avoidance (CBA) was positively associated with ERQ Cognitive Reappraisal (r = 0.588, p < 0.01), indicating that higher CBA scores were associated with greater endorsement of cognitive reappraisal in this sample.

Environmental & Social Engagement (ESE) also correlated positively with ERQ Cognitive Reappraisal (r = 0.522, p < 0.01), indicating that individuals who actively shape supportive environments and seek positive social contexts also tend to endorse cognitive reframing strategies.

In contrast, Maladaptive Cognitions & Rumination (MCR) showed a small but significant negative correlation with ERQ Cognitive Reappraisal (r = -0.152, p < 0.01). Although the magnitude of this association was small, the inverse relationship is broadly consistent with the hypothesized distinction between maladaptive cognitive tendencies and constructive cognitive reframing.

Turning to ERQ Expressive Suppression, a different pattern of associations emerged. MCR demonstrated a moderate positive association with Suppression (r = 0.400, p < 0.01), indicating that higher levels of maladaptive cognitions and rumination were associated with greater expressive suppression in this sample. The associations between Suppression and the remaining ERS dimensions were comparatively weak. CBA showed a small positive correlation (r = 0.180, p < 0.01), suggesting limited overlap between avoidance-based strategies and expressive inhibition. PCR exhibited only a weak positive association (r = 0.110, p < 0.01), whereas ESE was not significantly related to Suppression (r = 0.071). Overall, these patterns are broadly consistent with the hypothesized relationships between the ERS dimensions and established emotion-regulation strategies, while providing preliminary supportive evidence for convergent validity. The correlations are presented in Appendix 6 (Convergent Validity).

Concurrent validity

Concurrent validity was evaluated by examining the associations between ERS dimensions and indices of psychological distress (PHQ-9; GAD-7), subjective well-being (WHO-5 Well-Being Index), and two global indicators of quality of life (QOL1: overall quality of life; QOL2: satisfaction with health). The objective was not to demonstrate an absence of correlation, but rather to establish that the ERS dimensions are related to but not reducible to distress and well-being constructs.

MCR displayed moderate positive correlations with depressive symptoms (r = 0.432, p < 0.01) and anxiety symptoms (r = 0.462, p < 0.01). These coefficients are of sufficient magnitude to indicate moderate association, yet they remain well below thresholds that would imply construct redundancy. MCR was moderately negatively correlated with well-being (r = − 0.330, p < 0.01) and both quality-of-life indicators (QOL1: r = − 0.357; QOL2: r = − 0.295; p < 0.01). These findings indicate that MCR is associated with lower well-being and quality of life, while its associations with depression and anxiety suggest that it represents a specific aspect of emotion regulation rather than being synonymous with depression or anxiety.

The CBA dimension demonstrated near-zero associations with depression (r = 0.032) and anxiety (r = 0.033), neither reaching statistical significance. This relative independence from symptom severity suggests that avoidance-based regulatory strategies are not necessarily indicative of psychopathology in this sample. However, CBA showed small but significant positive correlations with both quality-of-life indicators (QOL1: r = 0.199; QOL2: r = 0.113; p < 0.01), while its association with well-being (WHO-5) was negligible. This pattern suggests that certain forms of anticipatory avoidance or attentional redirection may, in some contexts, coexist with perceived quality of life without showing a clear association with broader subjective well-being.

The results indicated that PCR exhibited small to moderate negative correlations with depression (r = -0.239, p < 0.01) and anxiety (r = -0.211, p < 0.01), alongside positive associations with well-being (r = 0.203, p < 0.01) and both quality-of-life indicators (QOL1: r = 0.375; QOL2: r = 0.258; p < 0.01). Finally, ESE showed small negative associations with depression (r = -0.184, p < 0.01) and anxiety (r = -0.141, p < 0.01), and positive correlations with well-being (r = 0.150, p < 0.01), overall quality of life (r = 0.309, p < 0.01), and health satisfaction (r = 0.211, p < 0.01). The correlations are presented in Appendix 7 (Concurrent Validity).

Discriminant validity was further assessed using the heterotrait–monotrait (HTMT) ratio of correlations. Following Henseler et al. [53], HTMT values below 0.85 provide evidence of adequate discriminant validity under the conservative criterion. As presented in Appendix 8, the HTMT values ranged from 0.228 to 0.759 across the four ERS dimensions, with MCR–CBA = 0.228, MCR–PCR = 0.382, MCR–ESE = 0.321, CBA–PCR = 0.680, CBA–ESE = 0.709, and PCR–ESE = 0.759. All values were below the 0.85 threshold, providing evidence that the four dimensions were sufficiently distinct from one another. This finding is particularly relevant given that HTMT has been shown to provide a more sensitive assessment of discriminant validity than the Fornell–Larcker criterion, particularly in detecting discriminant validity concerns that may not be identified using the Fornell–Larcker criterion [53].

Known-groups validity

Known-groups validity was examined by testing whether ERS dimensions differentiate between groups defined by clinically meaningful levels of depressive symptoms (PHQ-9), and anxiety symptoms (GAD-7).

Depression severity groups

A gradient emerged for Maladaptive Cognitions & Rumination (MCR), with mean scores increasing steadily from the none/minimal depression group (M = 2.68) to the severe group (M = 4.43), with a highly significant group effect (F(4, 738) = 39.322, p < 0.001, η² = 0.17). Post hoc comparisons showed that the none/minimal depression group (M = 2.68, SD = 1.11) reported significantly lower levels of maladaptive cognitions and rumination than all other groups. The mild and moderate depression groups did not differ significantly from each other, whereas the moderately severe and severe groups formed higher-scoring clusters. The severe depression group (M = 4.43, SD = 1.30) reported the highest levels of maladaptive cognitions and rumination, significantly exceeding those of the moderately severe group.

The results for Cognitive-Behavioral Avoidance (CBA) revealed a significant group effect (F(4, 738) = 3.06, p = 0.016, η² = 0.01). Unlike the linear trend observed for rumination, avoidance displayed a U-shaped pattern. The lowest avoidance scores were observed in the moderate (M = 4.84, SD = 1.08) and moderately severe (M = 4.67, SD = 1.26) groups, with no significant difference between them. However, the severe depression group reported the highest level of avoidance (M = 5.33, SD = 1.06), significantly exceeding the moderate and moderately severe groups, while the mild and none/minimal groups fell in the middle and did not differ significantly from any other group.

Turning to the adaptive regulation strategies, significant group differences were found for both Positive Cognitive Reappraisal (PCR) (F(4, 738) = 11.13, p < 0.001, η² = 0.05) and Environmental & Social Engagement (ESE) (F(4, 738) = 6.75, p < 0.001, η² = 0.03). For PCR, the none/minimal depression group (M = 5.55, SD = 1.26) reported significantly higher use of reappraisal than all other groups, while the mild group did not differ from any of the depressed groups. The moderate, moderately severe, and severe groups all reported similarly low levels of reappraisal.

A similar trend was observed for ESE. The none/minimal group reported the highest level of environmental and social engagement (M = 5.78, SD = 1.34), which was significantly greater than that of the moderate, moderately severe, and severe groups. The mild depression group again occupied an intermediate position, not differing significantly from either the none/minimal group or the more severe depression groups. The differences in ERS dimension scores across depression severity groups (PHQ-9 classification) are presented in Appendix 9.

Anxiety severity groups

For MCR, the gradient was more pronounced than in the depression analysis, with scores increasing from the minimal anxiety group (M = 2.54) to the severe anxiety group (M = 4.34), yielding a highly significant group effect (F(3, 739) = 64.486, p < 0.001, η² = 0.20). The magnitude of this effect suggests that maladaptive rumination is strongly intertwined with anxiety severity, consistent with cognitive models emphasizing repetitive negative thinking as a transdiagnostic process. Post hoc comparisons showed that the minimal anxiety group (M = 2.54, SD = 1.11) reported significantly lower levels of maladaptive cognitions and rumination than all other groups. The mild and moderate anxiety groups formed a homogeneous intermediate cluster, with means of 3.38 and 3.67, respectively. The severe anxiety group (M = 4.34, SD = 1.26) stood apart, reporting significantly higher levels of maladaptive cognitions and rumination than the mild and moderate groups.

The analysis for CBA yielded a statistically significant group effect (F(3, 739) = 2.74, p = 0.043, η² = 0.01); however, post hoc comparisons showed that pairwise differences were largely non-significant. The only significant difference emerged between the mild anxiety group (M = 4.88, SD = 1.17) and the severe anxiety group (M = 5.29, SD = 1.15), with the latter reporting significantly higher levels of avoidance. The minimal and moderate groups fell in between and did not differ significantly from any other group.

Regarding PCR, the results showed significant group differences (F(3, 739) = 11.95, p < 0.001, η² = 0.04), with the minimal anxiety group reporting the highest use of reappraisal (M = 5.58, SD = 1.27), significantly exceeding all other groups. The mild, moderate, and severe anxiety groups, however, were statistically indistinguishable from one another, reporting similar reappraisal scores. A similar pattern emerged for the ESE dimension (F(3, 739) = 7.56, p < 0.001, η² = 0.03). The minimal anxiety group again reported the highest level of engagement (M = 5.85, SD = 1.31), significantly higher than all other groups. The mild, moderate, and severe groups formed a homogeneous cluster, with means ranging from 5.22 to 5.38. Differences in ERS dimension scores across anxiety severity groups (GAD-7 classification) are presented in Appendix 10.

Discussion

The present study developed a new Emotion Regulation Scale (ERS) grounded in Gross’s process model through a hybrid human–AI workflow that combined multiple LLMs with conventional psychometric procedures. The findings provided preliminary evidence supporting a coherent four-factor structure and the overall measurement properties of the 17-item ERS. The scale demonstrated acceptable model fit and varying levels of internal consistency across its dimensions. Associations with depression, anxiety, well-being, and quality of life provided preliminary evidence of concurrent and nomological validity, while the known-groups analyses provided additional validity evidence. Overall, these findings suggest that the ERS may serve as a useful preliminary measure of emotion regulation, while further research is needed to establish its psychometric properties across independent and diverse samples. The findings provide preliminary and post hoc refined evidence for a four-factor, 17-item measurement structure. This provisional structure requires independent validation to establish its stability and replicability. The findings also suggest that AI-assisted item generation, refinement, translation, and construct evaluation may be incorporated into established scale-development workflows while retaining the central roles of theoretical grounding, expert judgment, and empirical validation. The four-factor solution is broadly consistent with Gross’s process model, although it should be interpreted as both theoretically informed and empirically emergent rather than as a direct one-to-one operationalization of all five regulatory stages in the model. Gross’s framework proposes that emotion regulation may occur at multiple points in the emotion-generative sequence, including situation selection, situation modification, attentional deployment, cognitive change, and response modulation [59, 6]. However, in self-report scale development, these stages are not always psychometrically separable because they may overlap conceptually, co-occur in naturalistic behavior, or be experienced by respondents as blended regulatory tendencies rather than discrete processes. The emergence of four factors in the present study may therefore reflect a parsimonious empirical clustering of broader regulatory domains, in which adaptive and maladaptive processes organized around similar cognitive or behavioral features grouped together across items. This interpretation is consistent with the idea that emotion regulation is multidimensional and that different strategies may be more usefully represented as related but nonidentical constructs rather than as a fixed set of isolated stages [59, 6, 8].

The four-factor structure identified in the present study represents an empirical organization of emotion-regulation strategies rather than a direct replication of Gross’s process model. Gross’s model [59] provided the theoretical framework for the initial item pool, whereas the four dimensions emerged empirically through the exploratory and confirmatory factor analyses. Thus, the correspondence with Gross’s processes is conceptual rather than one-to-one, with some dimensions reflecting multiple regulatory processes. The MCR dimension includes negative cognitive appraisals, such as interpreting challenges as evidence of failure, which can be interpreted in relation to Gross’s [59] concept of Cognitive Change as modifying cognitive appraisals or evaluations. Its ruminative content is also relevant to Attentional Deployment, particularly the sustained allocation of attention to distressing thoughts and emotions. MCR also includes the suppression of positive affect, which is relevant to Gross’s concept of Response Modulation, defined as directly influencing physiological, experiential, or behavioral responding. Thus, MCR appears to integrate related cognitive, attentional, and response-regulation tendencies rather than representing a one-to-one operationalization of a single process in Gross’s model. This integrative pattern may help explain why MCR emerged as a distinct factor in the present study, capturing related cognitive, attentional, and response-regulation tendencies within a broader maladaptive dimension. The CBA dimension draws on two of Gross’s regulatory processes. First, Situation Selection, which refers to “approaching or avoiding certain people, places, or objects in order to regulate emotions,” is reflected in the avoidance component. Second, Attentional Deployment, which Gross describes as strategies for changing attentional focus, including “distraction” and “concentration,” is reflected in the cognitive-distraction component. Thus, CBA represents an antecedent-focused regulation strategy aimed at minimizing exposure to emotional triggers through both behavioral avoidance and attentional redirection. The PCR dimension is consistent with Gross’s concept of Cognitive Change, specifically the process of reappraisal. Gross described reappraisal as “one form of cognitive change that has received particular attention.” This involves cognitively transforming the situation so as to alter its emotional impact. Thus, PCR is theoretically grounded in the reappraisal component of Cognitive Change and reflects positive reframing and the modification of subjective appraisals. The ESE dimension aligns with Gross’s concept of Situation Modification, which he defines as active efforts to directly modify the situation so as to alter its emotional impact. ESE extends this concept by emphasizing social engagement as a key contextual modification, reflecting an antecedent-focused strategy that actively shapes the environment to support positive emotional experiences.

The structure observed here also aligns with prior emotion regulation research showing that narrower two-factor representations, such as the ERQ, capture only part of the construct. The ERQ was originally developed to assess habitual cognitive reappraisal and expressive suppression and has been highly influential in emotion regulation research because of its brevity and strong theoretical grounding [9]. Nevertheless, subsequent work has emphasized that emotion regulation extends beyond these two strategies and that broader process-based assessments may better capture individual differences in how people manage emotions across situations [6, 60]. The present four-factor solution is therefore not necessarily a contradiction of the ERQ literature but may be viewed as an extension of it. It suggests that when item content is intentionally designed to reflect a fuller range of regulatory processes, the resulting empirical structure may be more differentiated than the classic reappraisal-suppression model.

The validity findings provide additional evidence relevant to the construct validity of the ERS. Adaptive dimensions were positively associated with cognitive reappraisal, which is consistent with the extensive literature linking reappraisal to more favorable emotional, interpersonal, and psychological outcomes [9, 8]. In contrast, maladaptive cognition was positively associated with expressive suppression, a finding consistent with previous research indicating that suppression is more strongly linked to emotional costs, diminished well-being, and less effective social functioning compared to antecedent-focused strategies such as cognitive reappraisal [9, 8]. These associations suggest that the ERS may capture theoretically meaningful distinctions among regulatory tendencies rather than merely reflecting a generic tendency to endorse socially desirable responses.

The adaptive dimensions demonstrated a positive association with cognitive reappraisal, which is consistent with the substantial literature linking reappraisal to more favorable emotional, interpersonal, and psychological outcomes [9, 61, 8]. Conversely, maladaptive cognition demonstrated a positive relationship with expressive suppression, a trend that is congruent with previous investigations indicating that suppression has been associated with lower well-being and less favorable social functioning compared to antecedent-focused strategies such as reappraisal [9, 8]. These correlations suggest that the ERS may be sensitive to theoretically significant distinctions among varying regulatory tendencies rather than merely reflecting a broad inclination to endorse socially acceptable responses.

The correlations with depression and anxiety are also consistent with this pattern. Meta-analytic evidence indicates that maladaptive emotion regulation strategies are reliably associated with internalizing psychopathology, whereas more adaptive strategies tend to show weaker or inverse associations [62]. The present findings are consistent with that pattern: maladaptive regulation was positively related to depression and anxiety, whereas adaptive regulation was inversely related to these symptoms. This pattern is consistent with the expected relationships for a measure grounded in emotion regulation theory. The associations with well-being and quality of life provide additional evidence relevant to the interpretation that the ERS captures clinically relevant individual differences. Emotion regulation is not only linked to symptom burden but also to positive functioning, and measures that relate in predictable directions to both distress and well-being may reflect the intended latent construct [6, 62].

Known-groups validity further provides evidence relevant to the scale’s utility. Significant differences across depression and anxiety levels indicate that the ERS may discriminate between groups that are expected to differ in emotional functioning. This finding is relevant because it suggests that the scale is not only statistically coherent but may also be sensitive to meaningful variation in a criterion-relevant way. In psychometric terms, such evidence is potentially valuable for a new instrument because it indicates that the scale may be capable of detecting variation in a criterion-relevant way. Taken together, the convergent, concurrent, and known-groups findings provide evidence relevant to the interpretation of the ERS as a measure of emotion regulation, while the convergent validity evidence should be interpreted cautiously.

A major contribution of the present study is methodological: it shows that AI-assisted item generation and refinement can be integrated into a rigorous psychometric workflow while retaining core measurement standards. The study goes beyond earlier discussions of AI in assessment by using multiple AI systems for different stages of development, including construct operationalization, item generation, reverse-item identification, redundancy screening, ambiguity reduction, bias detection, readability assessment, linguistic refinement, independent translation, post-EFA construct verification, factor interpretation, and factor labeling. This breadth of use is important because most prior work has focused primarily on AI as a drafting aid or as a source of candidate items, whereas the present study treats AI as one component within a controlled and multi-step measurement-development pipeline [37, 63]. The use of multiple models—GPT-5, Gemini, Claude 3.5, and NotebookLM—may have increased methodological rigor through cross-model triangulation, because different systems can surface different linguistic formulations, conceptual concerns, and potential ambiguities. At the same time, the study does not suggest that model plurality is intrinsically superior; rather, it demonstrates a cautious strategy for reducing single-model dependence and preserving human oversight.

This hybrid approach is consistent with recent literature emphasizing that generative AI may support psychometric development, but only when embedded within established principles of construct definition, content validity, and empirical evaluation. Prior studies have shown that LLMs can assist with scale development, item drafting, and refinement, but they have also warned that AI outputs may reflect training-data biases, overgeneralization, or superficial semantic similarity that is not sufficient for valid measurement [64,63]. The present study extends this literature in two ways. First, AI was not limited to item generation but was also used for translation and construct verification, which broadens the practical scope of AI support in measurement development. Second, the study demonstrates that a multi-model workflow can be aligned with established psychometric procedures, encompassing factor-analytic investigations at both the exploratory and confirmatory stages, reliability assessment, and several approaches to examining validity. In this sense, the study provides a concrete example of human-AI collaboration in psychometrics rather than a claim about automation. Human review remained essential at each stage, and the final interpretation of the scale depended on psychometric expertise rather than on model output alone.

The study has several strengths. The scale was theory-driven from the outset, which improves conceptual coherence and reduces the likelihood of producing a purely algorithmically generated item pool with weak construct anchoring. The use of randomly split samples for EFA and CFA provides a basis for examining the structural stability of the proposed model while reducing the risk of overfitting within a single analysis. Multiple forms of validity were examined rather than relying solely on internal consistency, which is especially important in early scale development. The inclusion of cross-model construct verification and expert review also provides additional safeguards that the item content and factor labels were not based on a single AI source or a single human interpretation. Finally, the study’s bilingual and translation-related steps are particularly valuable for future cross-cultural work, because they suggest a workflow that can be adapted to multilingual scale development while preserving psychometric discipline.

Conclusions and implications

Gross’s process model informed the theoretical framework and initial item-development process, whereas the four-factor structure identified in the present study was empirically derived from the data and should not be interpreted as a direct replication of Gross’s theoretical stages. The findings supported a preliminary, coherent four-factor structure, acceptable internal consistency, and evidence relevant to construct validity, and the scale differentiated individuals across levels of depression and anxiety. More broadly, the study demonstrates that AI-assisted item generation, refinement, translation, and construct evaluation can be incorporated into established psychometric procedures to support the development of preliminary measures of psychological constructs. The principal contribution is therefore not that AI replaces traditional scale development, but that carefully supervised AI can serve as a useful adjunct within rigorous psychometric research, expanding methodological possibilities while leaving theoretical judgment and validation in human hands. The ERS may complement existing emotion regulation measures by providing broader coverage of cognitive, behavioral, and contextual regulatory tendencies within a single theoretically grounded framework. As a preliminary measure, it may offer a useful tool for future research examining emotion regulation across diverse populations and psychological outcomes. However, further independent external validation, longitudinal investigations, and cross-cultural replication are needed to establish the stability, broader applicability, and generalizability of the proposed structure, as well as its incremental and comparative utility. Although the 17-item model demonstrated acceptable fit, it was refined post hoc using the CFA sample. Therefore, the present findings should be regarded as preliminary, and the stability of the proposed item configuration requires validation in an independent sample. Accordingly, the present findings should be interpreted as preliminary support for the feasibility of integrating LLMs into psychometric instrument development, rather than as evidence that AI-assisted scale development or the proposed ERS structure has been fully established.

Limitations and suggestions for further studies

Several limitations need to be considered. The convenience sample limits generalizability, and the cross-sectional design prevents conclusions about temporal stability, predictive validity, or sensitivity to change. Although the CFA was conducted on a randomly selected holdout subsample, it was drawn from the same overall convenience sample and therefore represents internal cross-validation rather than independent external validation. The initial item pool was also relatively limited, with five items generated for each of the five dimensions. Although the items underwent subsequent screening, refinement, and expert review, a larger initial pool could have provided greater flexibility for item selection and refinement. The EFA employed an orthogonal Varimax rotation to facilitate an interpretable initial factor structure, although the emotion-regulation dimensions may be theoretically and empirically correlated. Future studies should examine the stability of the identified four-factor structure using oblique rotation methods, such as Promax or Oblimin. The evidence for convergent validity should also be interpreted cautiously, as AVE values were below the commonly used 0.50 criterion across the four subscales. These findings suggest that convergent validity remains preliminary and warrants further evaluation in independent samples. In addition, the study did not include a head-to-head comparison with a traditionally developed scale, limiting the extent to which the incremental benefit of the AI-assisted workflow can be evaluated. Item Response Theory (IRT) analyses were also not conducted, which limits the assessment of item-level functioning. Although multiple AI systems were used to reduce bias and improve wording, AI-generated outputs may still be influenced by model variability, prompt framing, language artifacts, and potential biases. The reproducibility of AI-generated items may therefore vary across models, model versions, and generation conditions. Finally, broader cross-cultural validation and measurement invariance testing are needed before the ERS can be assumed to function equivalently across linguistic, cultural, and clinical groups.

Future iterations of the ERS should include a larger initial pool of items for each dimension. IRT analyses should also be applied to further examine item-level functioning. Longitudinal studies should assess test–retest reliability, temporal stability, and sensitivity to change, while future studies should also evaluate measurement invariance across demographic and cultural groups. Head-to-head comparisons between AI-assisted and conventionally developed item pools would be particularly informative for determining whether AI-assisted workflows improve efficiency, content coverage, readability, or psychometric performance. Further research should also examine the reproducibility of AI-assisted scale development across different models, prompts, and model versions and incorporate systematic procedures to identify and address potential biases. The practical utility and psychometric performance of the ERS should also be examined across diverse populations and contexts. More broadly, future AI-assisted psychometric research may benefit from multi-model workflows in which different systems are assigned specialized tasks, provided that such workflows remain transparent, theory-based, and subject to expert review. A limitation of the present study is that the generated items were not systematically compared with items from existing international emotion regulation measures to establish their originality. Future research should conduct systematic comparisons with established international measures to further evaluate item originality and distinctiveness.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Conceptualization, Iman Almohammadi; Data curation, Iman Almohammadi; Formal analysis, Iman Almohammadi; Investigation, Iman Almohammadi and Malak Alzahrani; Methodology, Iman Almohammadi; Writing original draft–review & editing, Iman Almohammadi and Malak Alzahrani.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data availability

Data availability statementThe data that support the findings of this study are available from the corresponding author, upon reasonable request.

Declarations

Ethics approval and consent to participate

Ethical approval for this study was granted by the Scientific Research Ethics Committee at the Faculty of Arts and Humanities, King Abdulaziz University, Jeddah, on 17 December 2025 (Approval No. REC-FAH-KAU-2026-049). Before participation, participants were provided with information about the study’s purpose and procedures and were informed that participation was voluntary and that they could withdraw at any time without penalty. Electronic informed consent was obtained on the first page of the survey; only participants who selected “agree” were able to proceed. All data were collected anonymously, kept confidential, securely stored, and used solely for research purposes. The study was conducted in accordance with the ethical principles of the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.