Development and validation of a scale for measuring irrational behavior in undergraduate students’ online discourse

Abstract

Introduction

This study aims to develop a scientifically valid and effective scale for measuring irrational behavior in online discourse among undergraduate students, thereby providing a scientific basis for understanding online public opinion in universities.

Methods

The study adopted a mixed-methods design. In the qualitative phase, semi-structured interviews were conducted, and 153 valid open-ended questionnaires were collected. The responses were coded by 5 independent coders, generating an initial item pool spanning four theoretical dimensions, which were then refined by a cross-disciplinary expert panel. In the subsequent quantitative phase, data from 500 students were used for item analysis and exploratory factor analysis (EFA). Confirmatory factor analysis (CFA) and reliability and validity assessments were then performed on a separate sample of 476 students.

Results

The final scale comprises 28 items across four dimensions: cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance. The scale demonstrated acceptable construct validity (χ²/df = 2.094; CFI = 0.934, TLI = 0.928, SRMR = 0.057, RMSEA = 0.058), and an internal consistency coefficient of 0.95. Application of the scale revealed significant differences in irrational behavior among university students of different gender, internet usage duration, and family background, whereas no significant grade-related difference was observed.

Conclusions

This newly developed scale provides preliminary psychometric support for assessing irrational behavior in undergraduate students’ online discourse. Analysis using this scale may help identify at-risk subgroups in university contexts.

Clinical trial number

Not applicable.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s40359-026-05678-w.

Introduction

With advances in internet technology, the methods and forms of public opinion dissemination have undergone significant transformation, giving rise to online public opinion [1]. This shift has fundamentally transformed public participation, enabling unprecedented connectivity, information access, and participatory democracy. However, these same technological developments also pose challenges to the quality and rationality of online communication. The speed, anonymity, and algorithm-driven nature of digital platforms have created an environment in which emotional appeals often outweigh factual evidence, and cognitive biases are systematically amplified. As a result, irrationality tends to become a social norm, and public opinion itself often exhibits inherent irrationality [2, 3].

Psychologists have predominantly explained individual-level irrationality through dual-process theory [4]. This theory posits that human cognition operates via two distinct, interacting systems: System 1 produces fast, automatic, effortless, emotionally charged judgments, while System 2 engages in slow, deliberate, effortful, and logical reasoning. From this perspective, irrationality is not a fundamental flaw in human reasoning but rather the default outcome when System 1 processes dominate decision-making and System 2 fails to activate or exert sufficient regulatory control. This mechanism is particularly pronounced in online environments, where information overload and rapid content scrolling systematically favor System 1 processing and undermine deliberate rational evaluation [5]. Within this framework, individual emotions play a pivotal driving role in the propagation of online public sentiment [6–8], manifesting in behaviors such as psychological resonance, blind spectatorship, and limited cognition [9]. Given the pervasive influence of these irrational cognitive mechanisms, an effective indicator system is essential for assessing and managing online public-opinion crises [10].

Undergraduate students constitute a uniquely important demographic in digital ecosystems. As digital natives who have grown up immersed in online environments, undergraduate students generally exhibit heightened sensitivity to online information and substantial enthusiasm for participatory engagement. Yet they simultaneously face practical limitations such as limited life experience, underdeveloped discernment capabilities, and uneven media literacy. Within complex and diverse online information environments, they remain susceptible to emotional mobilization and cognitive biases, potentially triggering irrational expressions and behaviors [11]. These issues not only threaten the integrity of cyberspace and social stability but also pose severe challenges to the effectiveness of governance within universities. Consequently, scientifically identifying, effectively measuring, and thoroughly analyzing irrational behavior in university students’ online discourse and its influencing factors [12] has become a critical issue that higher education institutions must address.

University students’ online public opinion is the aggregate of their emotions, attitudes, and opinions toward focal issues in social environments and campus life within cyberspace [13]. Research indicates that irrational elements within online discourse, coupled with the anonymity and relative freedom of cyberspace, predispose university students towards irrational online behavior [14, 15]. The anonymity of the internet fosters the mistaken belief that virtual networks conceal one’s real-world identities, age, gender, and occupation. The perception that cyberspace is a channel to shed constraints and achieve stress relief enables individuals to express opinions they would not dare voice in reality. Some even utilize internet platforms for irrational emotional venting [16]. Research indicates that over 70% of university students engage in provocative or inflammatory online discourse at least once a week, deliberately instigating multi-party conflicts and verbal altercations [17]. Some scholars attribute this to algorithmic recommendations which create ‘information silos’ and reinforce the primacy effect, thereby diminishing rational cognition, stimulating irrational communication, and even inducing group polarization [18, 19]. Concurrently, researchers contend that excessive focus on group consensus can lead to irrational collective decisions, resulting in groupthink phenomena [20]. Alternatively, dismissing mainstream opinions, choosing silence, and concealing genuine thoughts can also result in collective disorientation. This manifests as “the widespread loss of rationality among individuals within online group interactions, subsequently leading to collective phenomena such as blind conformity, extremism, foolishness, or madness” [21].

Based on the existing research regarding university students’ online irrational behavior, we formally propose the following definition of the core construct under investigation: Irrational Behavior in University Students’ Online Discourse refers to a set of psychological tendencies and behavioral manifestations exhibited by university students during their participation in online public discourse. These are characterized by a deviation from factual logic and rational deliberation due to the dominance of cognitive biases or intense emotions, which may potentially or actually disrupt constructive communication, disseminate unverified claims, or harm the online discursive environment. To delineate the boundaries of the construct under investigation and distinguish it from adjacent constructs frequently discussed in the literature, we clarify the following distinctions: Problematic internet use refers to excessive internet engagement that impairs daily functioning [22], whereas the construct under investigation focuses specifically on the quality and rationality of discourse participation, not merely the amount of usage. Online aggression and cyberbullying entail intentional, targeted harmful acts toward specific individuals [23], while the construct under investigation is a broader one that may include such aggressive acts but also encompasses non-aggressive yet irrational communicative behaviors, and some harmful outcomes of the irrational behavior may occur without conscious intent. Disinformation sharing represents one specific behavioral manifestation of the construct under investigation, rather than a synonymous construct, though both may share similar cognitive-emotional underpinnings [24].

While academic research on irrational online behaviors among university students is extensive, encompassing analyses of influencing factors and underlying psychological processes [18, 25], few scholars have systematically developed targeted, psychometrically validated measurement tools to quantify these behaviors, particularly in the context of online discourse. This critical methodological gap has significantly impeded progress in this field. Although overt behaviors are the most observable and measurable outcomes, the underlying psychological mechanisms that drive these behaviors are equally central to understanding their progression. Accordingly, the present study adopts an integrated cognitive-processing framework encompassing cognition, emotion, and behavior to develop a comprehensive measure that captures the full spectrum of irrational behavior in online discourse.

Specifically, our theoretical framework is grounded in three interrelated psychological processes. First, at the cognitive level, individuals exhibit biased information processing, including selective attention to negative information and negative interpretation of ambiguous stimuli [26]. Second, at the emotional level, self-relevant or identity-threatening cues are preferentially detected, leading to emotional reasoning that overrides rational judgment [27]. Critically, biased cognition and emotional arousal form a self-reinforcing cycle: selective attention to negative information triggers intense emotional responses, which in turn narrow attentional focus and reinforce pre-existing cognitive biases. Beyond cognition and emotion, the ability to reflect on and control one’s own cognitive processes is equally critical. This includes awareness that the internet, where cognitive biases and emotional reactions may be amplified, has characteristics distinct from those of the physical world [28]. Individuals also require critical thinking skills to evaluate information quality, particularly when exposed to media content produced with specific intentions [29]. These are higher-order metacognitive processes and self-regulatory abilities [30]. From a dual-process theory perspective, biased cognition and heightened emotional sensitivity represent overactivation of intuitive System 1 processing, while impaired self-regulation reflects deficient deployment of analytical System 2 functions. These internal psychological processes ultimately manifest as overt irrational acts. Accordingly, our scale development process incorporates four interrelated dimensions that capture this complete causal chain: cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance.

Recent studies have highlighted that the development of instruments measuring irrational beliefs typically involves multi-stage procedures including qualitative item generation, expert validation, and factorial testing in large samples [31]. Following the same procedure, the present study aimed to develop and validate a self-assessment scale measuring the psychological states, emotional experiences, and behavioral tendencies associated with irrational behavior in online discourse among undergraduate students. The resulting scale will enable researchers and educators to monitor the subjective psychological states of university students engaging in online discourse and identify individuals at elevated risk of exhibiting irrational behavior in online discourse. Based on the theoretical framework outlined above, we propose the following three hypotheses: The construct of irrational behavior in online discourse is multidimensional, comprising cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance; The developed scale will demonstrate satisfactory psychometric properties; The scale will exhibit sufficient sensitivity to detect differences across demographic subgroups.

Methods

Design

This study employed a mixed-methods design following a four-stage scale development framework integrating qualitative and quantitative approaches. First, semi-structured interviews explored the concrete manifestations and psychological characteristics of university students’ irrational behavior in online discourse. Second, items’ content was coded, and preliminary dimensions were constructed. Third, a quantitative cross-sectional survey was administered, followed by item analysis, exploratory factor analysis, confirmatory factor analysis, and psychometric testing to screen items, optimize the factor structure, and verify the scale’s reliability and validity. Fourth, an online survey was conducted using the scale among university students.

Semi-structured interviews and open-ended questionnaire survey

This study first employed qualitative research methods, conducting semi-structured interviews with five university counsellors and 157 undergraduate students. Participants were also invited to complete an open-ended questionnaire, primarily focusing on the following topics: online discourse engagement patterns, emotional responses to online content, engagement in problematic online behaviors, information literacy practices, misinformation experiences, opinion expression strategies, and reactions to irrational online discourse. Specific probing questions (Supplementary Table S1) were developed for each domain to elicit detailed participant responses.

After several simplistic responses were excluded, a total of 153 valid open-ended questionnaires were collected, establishing a textual database of irrational behavior in online discourse among undergraduate students, which served as the basis for item development.

Expert-based item development

Preliminary analysis of open-ended questionnaire responses identified 169 psychological expressions associated with irrational behavior in online discourse. To code these expressions, the research team developed a coding framework organized around four core theoretical dimensions: cognition, emotion, self-regulation, and behavior. Each dimension contained three mutually exclusive sub-dimensions with clear operational definitions and representative examples drawn from the textual database. The four dimensions and their sub-dimensions were defined as follows:

The cognitive dimension assesses distorted information processing, understanding, and judgment in online discourse: (1) Information identification ability. Inability to distinguish false, exaggerated, or biased content from facts (e.g., “I cannot accurately identify false information or exaggerated claims in online discourse”); (2) Information filtering preferences. Tendency to selectively accept belief-consistent information and reject contradictory evidence without critical evaluation (e.g., “I believe extreme viewpoints in online discourse are reasonable”); (3) Information interpretation bias. Prioritizing emotional intensity or narrative coherence over factual accuracy (e.g., “I disregard factual accuracy in online discourse, focusing solely on the intensity of arguments”).

The emotional dimension examines dysregulated emotional responses in online interactions: (1) Emotional volatility. Rapid, intense reactions to controversial topics or opposing views (e.g., “Controversial topics in online discourse frequently provoke strong emotional reactions in me”); (2) Emotional contagion. Susceptibility to the collective mood of online communities or the tone of individual comments (e.g., “When my views are challenged in online discourse, I experience feelings of frustration”); (3) Emotion-driven response tendency. Prioritizing emotional expression over rational argumentation (e.g., “I exhibit intense emotional responses to topics in online discourse that involve personal interests”).

The self-regulation dimension evaluates impaired self-control and rational thinking capacity in online discourse: (1) Impulse control. Difficulty restraining immediate emotional reactions and impulsive comments before posting (e.g., “I am easily swayed by online discourse to alter my stance”); (2) Rational deliberation ability. Lack of patience for careful analysis and reflection on contentious topics (e.g., “I lack sufficient patience and reflection regarding contentious topics in online discourse”); (3) Resistance to social influence. Tendency to uncritically accept views from online opinion leaders and influencers (e.g., “I readily accept the views of online opinion leaders and prominent influencers”).

The behavioral dimension describes actual problematic online conduct patterns: (1) Quality of speech production. Use of inappropriate, offensive, or factually incorrect language (e.g., “I have made erroneous decisions in online discourse due to a lack of discernment”); (2) Information dissemination practices. Sharing unverified, misleading, or false information without fact-checking (e.g., “I have disseminated unverified information within online discourse”); (3) Norms of interactive behaviors. Engagement in hostile disputes, personal attacks, or other norm-violating interactions (e.g., “I have engaged in prolonged disputes with others over differing viewpoints within online discourse”).

Content coding was performed independently by five trained coders: two PhD-level psychologists, one Master of Psychology, and two Masters of Journalism and Communication. For each expression, coders first identified all relevant dimensional and sub-dimensional elements, then assigned it to the primary dimension that best captured its core meaning; expressions that could not be clearly mapped to any dimension were flagged for potential removal. Inter-coder reliability was assessed using Fleiss’ kappa, which yielded a value of 0.92. All coding discrepancies were resolved through consensus discussion among all coders, with reference to the established framework.

After consensus coding, the expressions were refined into a standardized preliminary item pool through three sequential stages: (1) Removal of misaligned expressions that addressed unrelated topics or lacked clear relevance to irrational behavior in online discourse; (2) Elimination of redundant items (semantically identical, highly similar, or overly specific to individual experiences); (3) Standardization of the remaining expressions into formal self-report items.

Following the initial coding and item consolidation stage that yielded 68 preliminary items, we convened an expert content validation panel comprising 24 members (Journalism/Communication: 3 undergraduates, 3 master’s students, 2 doctoral students, 1 associate professor, and 1 professor; Psychology: 2 undergraduates, 7 master’s students, 4 doctoral students, and 1 professor). The panel evaluation was conducted through discussions focused on two tasks: (1) eliminating items that lacked theoretical interpretability for their assigned dimensions; (2) refining ambiguous or potentially misleading wording. Items were retained if more than 80% of panel members endorsed their dimensional appropriateness. Finally, 31 items were retained. With respect to item phrasing, we adopted differentiated temporal framing across dimensions. Items for the cognitive bias, emotional sensitivity, and self-regulation deficiency dimensions were formulated in the present tense to capture stable, trait-like patterns of information processing and emotional response. In contrast, items for the behavioral deviance dimension used past-tense formulations describing specific prior behavioral occurrences. This wording strategy was designed primarily to mitigate social desirability bias: framing concrete behaviors as past factual events, rather than current self-evaluative traits, was expected to reduce respondents’ defensiveness and promote more honest self-reporting. The scale adopted a 5-point Likert response format (1 = strongly disagree, 5 = strongly agree), in which higher total scores indicate a greater tendency toward irrational behavior in online discourse.

Participants

An online version of the questionnaire was distributed to seven universities across Shaanxi, Hunan, and Hainan provinces, China, via the Chinese online survey platform Wenjuanxing. This study adopted a convenience sampling approach. Participants were recruited from multiple schools in each cooperative university. Freshmen, sophomores, and juniors were recruited with balanced sample distribution across grades. Given that the majority of recruited institutions are for teacher education, the sample naturally contains a higher proportion of female students. Online questionnaires were distributed from April to May 2024. A total of 1067 preliminary responses were collected. After screening, 91 questionnaires were excluded due to identical responses across all items or excessively short completion times, resulting in 976 valid responses (valid response rate: 91.47%). After randomly splitting the valid data into two halves, sample 1 (N = 500) was used for item analysis and exploratory factor analysis, while sample 2 (N = 476) was used for confirmatory factor analysis. Participants’ demographic characteristics are presented in Table 1. The final version of the Irrational Online Behavior Scale was administered to a separate cohort of undergraduate students from May to June 2024. A total of 1047 valid responses were collected for group difference analyses. All participants were informed that completing the questionnaire was an optional after-class activity associated with a course, which counted toward the “general daily performance” component of the course grade; although no alternative assignment was offered to non-participants, students could still obtain full credit for this component through other regular class activities, such as attendance and active participation. The complete recruitment procedure is provided in Supplementary Figure S1.

Table 1.

Demographic characteristics of participants

Class	Number of people	Percentage (%)
Gender:
male	186	19.1
female	790	80.9
Grade:
First year	352	36.1
Second year	325	33.3
Third year	299	30.6
Daily internet usage time:
< 1 h	38	3.9
1–4 h	435	44.6
5–8 h	396	40.6
> 8 h	107	11.0
Father’s educational level:
Junior high school and below	536	54.9
High school and vocational school	286	29.3
College diploma	140	14.3
Postgraduate or above	14	1.4
Mother’s educational level:
Junior high school and below	612	62.7
High school and vocational school	257	26.3
College diploma	96	9.8
Postgraduate or above	11	1.1
Whether it is a single-parent family:
yes	113	11.6
no	863	88.4

Note. This table presents the demographic characteristics of the scale-development sample

Data analysis

Item analysis

In sample 1, an independent-samples t-test was performed for each item between the high (top 27%) and low groups (bottom 27%). Any item for which no statistically significant difference (t < 3) was found between the two groups was removed[32]. The correlation between each item and the total scale score was calculated. The criterion for retention was a correlation coefficient above 0.30[33], with the additional provision that removing any item below this threshold should not increase the scale’s Cronbach’s α by more than 0.02[34]. Subsequently, an item difficulty analysis was performed, with item difficulty operationalized as the ratio of each item’s mean score to the maximum possible score of 5.

Exploratory factor analysis

The data from sample 1 were analyzed using SPSS 26.0 for exploratory factor analysis (EFA). To assess the factorability of the data in sample 1, the Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test of sphericity were examined prior to conducting an EFA. For multicollinearity, we inspected the correlation matrix. For multivariate outliers, we computed Mahalanobis distance for each case using the chi-square critical value at p < 0.001. EFA was performed with principal-axis factoring as the extraction method, followed by promax oblique rotation to facilitate interpretation of the factor structure. Parallel analysis and scree plot inspection were jointly applied to determine the optimal number of factors to retain. Parallel analysis (conducted by using psych package version 2.6.1 in R) was performed using principal-axis factoring extraction based on 100 randomly generated permuted datasets; a factor was retained if its eigenvalue from the actual sample exceeded the corresponding mean eigenvalue from the simulated dataset[35]. Items were deleted based on the following criteria: (1) items with factor loadings below 0.40; (2) items exhibiting loadings greater than 0.40 on multiple factors.

Confirmatory factor analysis

The data from sample 2 were analyzed using the R package lavaan (version 0.6–21) for confirmatory factor analysis (CFA) and tests of reliability and validity. Prior to the main analysis, multicollinearity and multivariate outliers were assessed. A CFA was conducted employing robust maximum likelihood (MLR) estimation. Model fit was evaluated using the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). Generally, CFI and TLI values above 0.90 indicate an acceptable fit, while RMSEA values below 0.05 and SRMR values below 0.08 suggest a satisfactory model fit[36]. Pairwise correlations were computed to preliminarily examine the construct distinctiveness of the four dimensions. Internal consistency was assessed using composite reliability (CR). Convergent validity was evaluated via the average variance extracted (AVE), with an ideal threshold of ≥ 0.50 as recommended by Fornell and Larcker (1981)[37]. Discriminant validity was tested using the Heterotrait-Monotrait (HTMT) ratio of correlations, applying a conservative cutoff value of < 0.85 as suggested by Henseler et al. (2015) [38].

Group difference analysis

To investigate the influence of participants’ sex, grade, internet use duration, and family background on their scale scores, an additional dataset was collected. Sex was coded as female and male; grade referred to first-, second-, and third-year undergraduates from four-year universities; daily internet use duration was categorized into four groups: less than 1 h, 1–4 h, 5–8 h, and more than 8 h; family background was divided into single-parent families and two‑parent (non‑single‑parent) families. Prior to formal between-group comparisons, measurement invariance across sex was tested as a prerequisite to confirm the cross-group equivalence of the scale structure and ensure the validity of subsequent mean comparisons. A hierarchical sequence of measurement invariance models (configural, metric, scalar, and strict invariance) was estimated using robust maximum likelihood (MLR) estimation, and changes in CFI (ΔCFI ≥ − 0.01) and RMSEA (ΔRMSEA ≤ 0.015) were adopted as criterion for judging invariance retention at each hierarchical level [39]. Between-group comparisons of the scale were analyzed using SPSS 26.0. Descriptive statistics are presented as means and standard deviations. Statistical comparisons were conducted using independent-samples t-tests or ANOVA. For independent-samples t-tests, standard test results were reported when variances were homogeneous, and Welch-adjusted t-tests were applied when the equal-variance assumption was violated. For post hoc pairwise comparisons following ANOVA, Tukey’s test was adopted under homogeneous variance conditions, while the Games-Howell test was used when variances were heterogeneous. Given that the comparisons across the four dimensions and four grouping variables produced 16 p‑values, we used the Benjamini–Hochberg FDR procedure to correct these p‑values, and adjusted p‑values < 0.05 were considered statistically significant. For each of the four dimensions, 95% confidence intervals for mean-score (calculated as subscale scores divided by the number of items within each subscale) differences are reported for statistically significant pairwise group comparisons.

Results

Item analysis

In Sample 1 (n = 500), apart from one item, all other items demonstrated significant differences between high- and low-scoring groups, yielding t-values ranging from 8.59 to 33.84 (p < 0.001), confirming good item discrimination. Correlations between each item score and the total score ranged from 0.59 to 0.92 (p < 0.001), warranting retention of all items. Item difficulty indicators ranged from 0.39 to 0.54, indicating moderate difficulty across all items. Detailed results are presented in Supplementary Table S2.

Construct validity

After removing the item that failed to pass the item analysis, the KMO value for sample 1 was 0.98, and the Bartlett’s test of sphericity was statistically significant (₍₃₇₈₎ = 18836.34, p < 0.001), confirming the scale’s suitability for factor analysis. Inter-item correlations ranged from 0.20 to 0.87, indicating acceptable levels of collinearity. A total of 64 multivariate outliers (12.8% of the sample) were detected using Mahalanobis distance. Parallel analysis suggested retaining four factors (Fig. 1). Two items were deleted because they had low loadings or high loadings in more than one factor (Supplementary Table S2). Finally, 28 items were retained, and the results of the EFA for the final 28 items are presented in Table 2. A supplementary robustness check, which excluded all detected outliers, reproduced an identical four-factor structure, confirming the stability of the factor solution.

Fig. 1.

Parallel analysis and scree plot for determining the optimal number of factors in exploratory factor analysis (EFA). The analysis was performed on 28 items using principal-axis factoring extraction. Filled markers represent the eigenvalues derived from the actual study sample; hollow markers represent the average eigenvalues calculated from 100 randomly generated parallel datasets. The horizontal dashed line indicates an eigenvalue of 1. Parallel analysis results supported the retention of four factors for the scale.

Table 2.

EFA results and factor loadings for the 28-item irrational online discourse behavior scale

Item	Cognitive bias	Emotional sensitivity	Self-regulation deficiency	Behavioral deviance
1. I often change my opinion on certain events due to online public opinion.	0.61	0.29	0.24	-0.39
2. I think the extreme views in the online public opinion are reasonable.	0.71	0.07	0.01	0.07
3. I believe that aggressive remarks in online public opinion are normal and should not be avoided.	0.74	-0.04	0.04	0.16
4. I don’t care about the factual truth in online discussions, but only the intensity of the arguments.	0.74	-0.13	0.00	0.28
5. I believe all the opinions of online opinion leaders are correct.	0.75	-0.07	0.05	0.12
6. Controversial topics in online discourse often stir my emotions.	0.25	0.77	-0.09	-0.11
7. I often get into arguments with others over differing opinions in online discussions.	0.39	0.47	-0.16	0.21
8. I feel frustrated when my views are challenged in online discussions.	0.06	0.90	-0.16	0.02
9. I feel fear or unease about certain remarks in online public opinion.	-0.04	0.71	0.15	0.05
10. I have strong emotional reactions to topics involving personal interests in online public opinion.	-0.13	0.72	0.20	0.07
11. I tend to get emotionally charged when online discussions touch on collective interests.	-0.20	0.73	0.21	0.06
12. I find it hard to control my emotional outbursts in online discussions.	0.12	0.03	0.52	0.23
13. I am easily swayed by online public opinion and may change my stance.	0.09	0.02	0.73	0.06
14. I lack sufficient judgment on certain topics in online public opinion.	-0.01	0.04	0.88	-0.05
15. I often get caught up in conflicting emotions and confusion due to online public opinion.	0.02	0.05	0.67	0.21
16. I lack sufficient patience and critical thinking when dealing with controversial topics in online discussions.	0.06	-0.04	0.92	-0.11
17. I find it hard to stay calm and rational when dealing with online public opinion.	0.00	0.07	0.68	0.17
18. I lack the ability to cope with negative remarks in online public opinion.	0.07	0.11	0.50	0.23
19. I once made offensive remarks in online public discourse.	-0.06	0.21	-0.12	0.86
20. I once spread unverified information in online discussions.	0.13	-0.01	-0.02	0.82
21. I once engaged in malicious online hype or rumor-mongering in public discourse.	0.09	-0.03	-0.06	0.93
22. I once engaged in personal attacks or insults against others in online public discourse.	0.11	-0.12	-0.01	0.91
23. I once made irrational remarks in online discussions due to emotional outbursts.	0.02	-0.10	0.04	0.93
24. I once got into a heated online argument with someone over differing opinions.	-0.06	0.17	-0.06	0.85
25. I once changed my original intention in online discussions due to others’ influence.	-0.04	0.01	0.28	0.69
26. I once made a wrong decision in online discussions due to poor judgment.	-0.03	0.07	0.06	0.78
27. I once made extreme remarks in online discussions about matters affecting my personal interests.	-0.03	-0.01	0.06	0.90
28. I once made extreme remarks in online discussions about public affairs related to university administration and social development.	0.02	-0.03	0.02	0.88
Eigenvalues	15.97	2.30	1.18	0.91
Contribution rate (%)	57.03	8.22	4.20	3.25

Note. Factor loadings are from exploratory factor analysis (EFA). Primary factor loadings (≥ 0.40) are shown in bold

For the CFA sample (Sample 2), inter-item correlations ranged from 0.14 to 0.82, with no evidence of multicollinearity. Mahalanobis distance identified 49 multivariate outliers (10.29% of the sample). Outliers were retained rather than excluded, as robust maximum likelihood (MLR) estimation is robust to multivariate outliers. Factor loadings for the CFA model are shown in the supplementary Figure S2. The ratio of the chi-square to the degrees of freedom (χ²/df) was 2.094, indicating a reasonable fit. The CFI = 0.934 and the TLI = 0.928 both exceeded the 0.90 threshold for acceptable fit. The SRMR = 0.057 was well below the 0.08 cutoff, indicating a good fit. The RMSEA = 0.058 fell within the “acceptable” or “reasonable” range (0.05 to 0.08). Thus, CFA indicated that all fit indices met psychometric standards (Table 3). Since the CFI and TLI values did not reach the more stringent threshold of 0.95, modification indices (MI) were further examined (Supplementary Table S6). Three pairs of items had MI values exceeding 50; however, none of the suggested modifications were supported by strong theoretical rationale. Therefore, no post hoc modifications were made to the original model.

Table 3.

Model fit indices for the 28-item four-dimensional scale

χ2	df	χ2/df	CFI	TLI	SRMR	RMSEA
720.317	344	2.094	0.934	0.928	0.057	0.058

Note. These fit indices are from the confirmatory factor analysis (CFA) conducted using robust maximum likelihood estimation (MLR)

Pairwise correlation analyses between the cognitive, emotional, self-regulation, and behavioral dimensions of the scale yielded correlation coefficients ranging from 0.56 to 0.67 (Table 4). These moderate correlations across all dimensions suggested that a bifactor model might also explain the data well. We therefore tested a bifactor model with one general common factor and four independent specific factors. The results showed that the bifactor model was not superior to the original four-factor model (Supplementary Table S7).

Table 4.

Correlation matrix of the four dimensions of the 28-item scale

	Cognitive bias	Emotional sensitivity	Self-regulation deficiency
Cognitive bias	-
Emotional sensitivity	0.56**	-
Self-regulation deficiency	0.59**	0.67**	-
Behavioral deviance	0.61**	0.56**	0.66**

Note: Values are Pearson product-moment correlation coefficients. ** p < 0.01

The internal consistency coefficients for the four dimensions ranged from 0.79 to 0.96, with the total scale achieving a coefficient of 0.95. The composite reliability (CR) values for all four dimensions exceeded 0.70. The average variance extracted (AVE) values were 0.47 for cognitive bias, 0.39 for emotional sensitivity, 0.51 for self-regulation deficiency, and 0.69 for behavioral deviance (Table 5). The AVE values of the first two dimensions were below the ideal threshold of 0.50. All pairwise Heterotrait-Monotrait (HTMT) values fell between 0.54 and 0.76, which were below the conservative cutoff of 0.85 (Table 6).

Table 5.

Convergent validity indicators of the 28-item scale

	Cognitive bias	Emotional sensitivity	Self-regulation deficiency	Behavioral deviance
AVE	0.47	0.39	0.51	0.69
CR	0.81	0.79	0.88	0.96

Note. AVE = average variance extracted; CR = composite reliability

Table 6.

HTMT matrix of the four dimensions of the 28-item scale

	Cognitive bias	Emotional sensitivity	Self-regulation deficiency
Cognitive bias	—
Emotional sensitivity	0.61	—
Self-regulation deficiency	0.74	0.76	—
Behavioral deviance	0.69	0.54	0.76

Note. Heterotrait-Monotrait Ratio of Correlations. This matrix evaluates the discriminant validity between latent variables

Irrational behavior in online discourse among university students

The participants’ overall mean score on the scale was 2.31. The mean scores for the four dimensions were as follows: cognitive bias, 2.20; emotional sensitivity, 2.67; self-regulation deficiency, 2.33; and behavioral deviance, 2.03 (Table 7). Strict measurement invariance across gender was verified prior to group comparisons (see Supplementary Table S8), which confirmed the statistical validity of conducting between-gender mean-score comparisons. Male students exhibited greater cognitive bias (95% CI [0.102, 0.392]) and higher levels of behavioral deviance (95% CI [0.239, 0.556]) than female students. No significant differences were found among the three grades groups. For daily internet usage duration, post hoc pairwise comparisons following one-way ANOVA showed that students in the < 1 h group and the > 8 h group scored significantly higher on cognitive bias than those in the 1–4 h group, with 95% confidence intervals for the mean differences of [0.037, 0.724] and [0.073, 0.522], respectively. For behavioral deviance, students in the < 1 h group scored significantly higher than those in both the 1–4 h and 5–8 h groups, with 95% CIs of [0.147, 0.904] and [0.156, 0.921], respectively. Undergraduate students raised in single-parent families exhibited significantly higher levels of cognitive bias in online discourse (95% CI [0.069, 0.439]) compared to their non-single-parent counterparts (Table 8). Given the relatively small sample sizes of the male group, single-parent family group, and < 1 h daily internet usage group, post hoc statistical power analyses were conducted. Results indicated that the sex and internet usage group comparisons had adequate statistical power (≥ 0.80), while the single-parent family group comparison had insufficient power (Supplementary Table S9).

Table 7.

Means and standard deviations of irrational online behavior in online discourse among undergraduate students

Dimension	Mean	Standard deviation
Cognitive bias	2.20	0.96
Emotional sensitivity	2.67	0.91
Self-regulation deficiency	2.33	0.97
Behavioral deviance	2.03	1.05

Table 8.

Group differences in irrational online behavior in online discourse among undergraduate students

	Cognitive bias	Emotional sensitivity	Self-regulation deficiency	Behavioral deviance
Gender:
Males (n=207)	2.39 ± 0.95	2.62 ± 0.91	2.44 ± 0.98	2.34 ± 1.03
Female (n = 840)	2.14 ± 0.94	2.50 ± 0.91	2.29 ± 0.96	1.94 ± 1.04
t	3.346	1.579	1.969	4.918
p	< 0.001	0.115	0.049	< 0.001
pFDR	0.003	0.117	0.099	< 0.001
Cohen’s d	0.260	0.123	0.153	0.382
Grade:
First year (n = 404)	2.20 ± 0.98	2.54 ± 0.93	2.32 ± 1.01	2.08 ± 1.07
Second year (n = 323)	2.19 ± 0.95	2.53 ± 0.91	2.35 ± 0.97	2.04 ± 1.07
Third year (n = 320)	2.20 ± 0.91	2.51 ± 0.87	2.30 ± 0.90	1.93 ± 0.99
F	0.014	0.097	0.273	1.813
p	0.986	0.907	0.761	0.164
pFDR	0.986	0.968	0.870	0.202
Partial	< 0.001	< 0.001	< 0.001	0.003
Daily internet usage duration:
< 1 h (n = 57)	2.48 ± 1.07	2.56 ± 1.08	2.51 ± 1.11	2.49 ± 1.13
1–4 h (n = 450)	2.10 ± 0.89	2.47 ± 0.87	2.25 ± 0.94	1.96 ± 1.02
5–8 h (n = 380)	2.18 ± 0.95	2.53 ± 0.92	2.31 ± 0.98	1.95 ± 1.05
> 8 h (n = 160)	2.40 ± 1.01	2.66 ± 0.91	2.49 ± 0.95	2.19 ± 1.04
F	5.697	1.835	3.203	6.222
p	< 0.001	0.139	0.023	< 0.001
pFDR	0.003	0.186	0.060	0.003
Partial	0.016	0.005	0.009	0.018
Post hoc	1, 4 > 2	-	-	1 > 2, 3
Is it single-parent family:
YES (n = 115)	2.42 ± 1.01	2.70 ± 0.95	2.47 ± 0.95	2.17 ± 1.05
No (n = 932)	2.17 ± 0.94	2.50 ± 0.90	2.31 ± 0.97	2.00 ± 1.05
t	2.700	2.218	1.690	1.606
p	0.007	0.027	0.091	0.109
pFDR	0.023	0.062	0.162	0.167
Cohen’s d	0.267	0.219	0.167	0.159

Note. Independent-samples t-tests were used for gender and family structure comparisons. One-way ANOVA was used for grade and daily internet usage comparisons. Benjamini–Hochberg FDR procedure was used to correct p-values. Cohen’s d and partial η² are reported as effect size measures. Standard independent-samples t-tests were applied for all two-group comparisons, as the equal-variance assumption was satisfied across all analyses. For ANOVA models with statistically significant omnibus effects, post hoc pairwise comparisons were performed using Tukey’s test, given that homogeneity of variance was confirmed for all significant ANOVA results

Discussion

Adopting a combined qualitative and quantitative approach from a psychological perspective, this study developed a scale for irrational behavior in undergraduate students’ online discourse. The findings offer preliminary evidence for the scale’s internal structure, reliability, and discriminant validity. Exploratory factor analysis (EFA) identified four core dimensions: cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance, which aligned closely with the theoretical framework. Confirmatory factor analysis (CFA) yielded acceptable model fit, supporting the scale’s structural validity. Its application among university students revealed significant variations in irrational behaviors linked to gender, daily internet usage, and family background, but not grade. Consequently, the hypothesis—which predicted that the scale would be sufficiently sensitive to detect differences across all demographic subgroups—is only partially supported.

The development of the scale

Although the overall factor structure is clear, the self-regulation deficiency dimension is less conceptually clear than the other three. This dimension encompasses emotional control, critical thinking, and resistance to social influence, reflecting higher-order cognitive processes—specifically metacognition—that monitor and coordinate lower-level cognitive operations. Recent research highlights metacognition as a critical regulator of online behavior, involving monitoring one’s own cognitive–emotional states and suppressing impulsive responses [30, 40]. Despite theoretical overlap with metacognition, we adopted the term self-regulation to avoid conceptual confusion with the cognitive bias dimension. Studies further link self-control to resistance to social media persuasion [41], which renders this term more appropriate for the current dimension. Thus, the use of self-regulation aligns more closely with the content encompassed by this dimension. Nevertheless, this label does not fully capture the breadth of the construct, and caution is warranted when interpreting this dimension solely based on its name.

Notably, the behavioral deviance dimension comprises more items and yields higher factor loadings compared with the other three dimensions. This design aligns with the core conceptual positioning of the scale: irrational behavior in online discourse serves as the primary outcome construct, whereas the cognitive bias, emotional sensitivity, and self-regulation deficiency dimensions capture the underlying psychological predispositions that precede and drive such behavioral manifestations. During item development, we intentionally developed a larger item pool for the behavioral dimension. Accordingly, the behavioral subscale has a relatively narrower construct scope and higher internal item homogeneity than the other three dimensions. For practical application, dimension-specific scores are recommended as the primary reference for targeted assessment and intervention design. When a total scale score is required, it is advisable to compute it as the unweighted average of the four dimension mean scores, rather than summing all individual items directly. This approach mitigates weighting bias caused by the uneven number of items across dimensions.

Strong intercorrelations among the four factors were observed. This finding is theoretically expected, as the dimensions are conceptually interdependent components. High inter-factor correlations are common in multidimensional psychological scales measuring related facets of a single overarching phenomenon, and do not necessarily invalidate the distinctiveness of individual dimensions [42]. To formally disentangle shared variance from dimension-specific variance, we tested a bifactor model that explicitly partitions item variance into a general irrationality factor and four dimension-specific factors [43]. However, this model did not yield a substantive improvement in model fit compared with the original four-factor correlated structure. Collectively, these results indicated that while the four dimensions share a common underlying core of irrationality, each also captures unique, theoretically meaningful variance that supports retaining the original four-factor structure.

Regarding convergent validity, average variance extracted (AVE) values for cognitive bias (0.47), emotional sensitivity (0.39), fell below the conventional threshold of 0.50 proposed by Fornell and Larcker (1981) [37], which indicates that these latent constructs explain less than half the variance of their respective items. Cognitive biases and emotional sensitivity, by their very nature, operate at an introspective level to which individuals may lack full conscious access, introducing substantial unique variance unrelated to the underlying construct [44]. Accordingly, the results for the two dimensions should be interpreted with caution. In contrast, the self-regulation dimension, which focuses on explicit and observable behavioral control, and the behavioral dimension, which comprises directly measurable indicators, are more likely to yield higher AVE estimates because their items tap into concrete, readily reportable content, thus reducing random measurement error.

The application of the scale

The results revealed significant gender disparities in students’ irrational online discourse behavior. Specifically, male students exhibited greater cognitive bias and higher levels of behavioral deviance than female students when participating in online discourse. Despite the relatively small sample size of the male subgroup, post hoc power analysis and sensitivity analysis confirmed that it was sufficient to detect the magnitude of gender differences observed in the present study (Supplementary Tables S9 and S10). From a biological perspective, some research suggests that gonadal hormones may contribute to a greater propensity for risk-taking behaviors in males, which could manifest in more confrontational or less restrained online expression [45]. Concurrently, sociocultural explanations emphasize the role of gender socialization: individuals socialized toward traditional masculine norms often display lower impulse control [46], and research by Shoenberger and Rocheleau (2017) further demonstrates that women have stronger self-control capabilities than men [47]. These existing studies have predominantly focused on emotional and impulsivity-related pathways of gender differences. In contrast, the present study found that gender differences were mainly manifested in the cognitive dimension, rather than in emotional and self-regulation dimensions, indicating that deviant online behaviors among males are more directly driven by cognitive factors. This finding aligns with recent empirical evidence: compared with women, men are more likely to adopt irrational cognitive strategies and exhibit aggressive responses during online interactions, with self-control influencing behavior through a mediating pathway while cognition exerts a direct effect [48]. Another study also reported that males had higher levels of cyber-aggression and hostile attribution bias than females, reflecting gender divergence in cognitive processing and deviant behavioral tendencies in online contexts, with self-control playing a moderating role [49]. Thus, cognitive factors appear to be the primary driver of the observed gender differences in irrational online discourse behavior.

No significant differences were found among three groups of different grades. Theoretically, first-year students, who are newly enrolled and adapting to a new environment while establishing social connections, tend to be more enthusiastic about participating in various social activities, including online discourse [50]. In this process, they often exhibit herd mentality and conformity to maintain positive interpersonal relationships, making them susceptible to the influence of online discourse. In contrast, second- and third-year students, having passed the initial novelty phase of transitioning from secondary school to university, have adapted to campus life. With increasing age and knowledge, their ability to discern online information and regulate their emotions has gradually strengthened [51]. Although these theoretical considerations would predict significant grade-related differences, the absence of such findings in the present study suggests that there is no simple linear relationship between irrational online discourse behavior and grade level during the undergraduate years.

Analysis of daily internet usage among university students revealed that cognitive bias was most pronounced in the following order: under 1 h > over 8 h > 5–8 h > 1–4 h. Behavioral deviance was highest among those online for < 1 h, followed by > 8 h, 1–4 h, and 5–8 h. However, this finding should be interpreted with caution, as the < 1 h group represents a small subset of the sample. Woods and Scott (2016) contend that adolescents addicted to online social media use the internet more frequently, leading to poorer sleep quality and heightened anxiety and depression [52]; such negative emotional states and sleep disturbances may further result in diminished rational thinking during online discourse. Our results generally support the trend that irrationality levels increase with usage duration: over 8 h > 5–8 h > 1–4 h. Nevertheless, the unexpectedly high irrationality observed in the under-1-hour group calls for further investigation. To explore this unexpected finding, we dichotomized the sample into two groups: <1 h and ≥ 1 h. Chi-square tests revealed that the < 1 h group contained a significantly higher proportion of males (Supplementary Table S12). Consistent with our findings, male students scored higher than female students on the Irrational Behavior in Online Discourse Scale. This distinct demographic profile may therefore partially account for the unexpectedly elevated levels of irrationality in the < 1 h group. Additionally, it is plausible that university students with very limited daily internet exposure (under 1 h) may lack sufficient experience in identifying negative or misleading information online, which could impair their ability to resist the influence of such information in online discourse.

Analysis of whether university students’ family backgrounds were single-parent households revealed that those raised in single-parent families exhibited significantly higher levels of cognitive bias in online discourse compared to their non-single-parent counterparts. This result, however, is not fully consistent with previous literature, which has predominantly emphasized the negative emotional consequences of single-parent family structures. Prior research has shown that parental divorce exerts a range of adverse effects on adolescent mental health, including persistent negative affect, lowered self-esteem, and reduced adaptive and cognitive functioning relative to children raised in intact families [53–55]. Single-parent families tend to have relative disadvantages in emotional support, economic conditions, and educational resources [56], and family disruptions and adverse life events may foster psychological traits such as heightened sensitivity, low self-esteem during development [57]. Given this context, our finding—which observed an effect on cognition but not on emotion, self-regulation, or behavioral deviance—warrants further interpretation. One plausible explanation is that cognitive biases may precede or predispose subsequent emotional disturbances and behavioral manifestations. Although between-group differences in behavioral deviance were not detected in the current sample, the observed cognitive disparity may signal an early precursor to more pronounced risks of irrationality over time, meriting attention in both theoretical research and practical university intervention settings.

Our results showed that the cognitive bias dimension exhibited significant group differences across three of the four demographic factors examined, and was more sensitive to individual variations than the behavioral deviance dimension. This finding highlights that cognitive processes are the core determinant of irrational online behavior. This conclusion aligns with recent theoretical and empirical advances. Studies applying the Job Demands-Resources model have demonstrated that irrational beliefs, as key personal cognitive factors, interact with psychological demands to influence various behavioral outcomes such as innovative and proactive behavior [58]. Additional empirical evidence also confirms that irrational beliefs shape psychological functioning and behavioral responses in different contexts, forming distinct psychological profiles associated with specific behavioral patterns [59]. Taken together, these studies and our findings collectively support the critical role of cognitive biases in driving irrational behaviors.

Limitations

This study has several limitations. First, the participants were recruited from only three provinces in China, which may not fully represent the diverse cultural, economic, and educational backgrounds of undergraduate students. Second, the cross-sectional design limits the ability to capture the dynamic evolution of irrational behavior in online discourse over time. Third, the reliance on self-reported data introduces potential social desirability bias, whereby participants might underreport their irrational behavior. Fourth, the sample exhibits a significant gender imbalance, which may reduce the scale’s measurement accuracy for male participants. Finally, AVE values for cognitive bias and emotional sensitivity fell below the ideal standard of 0.50, suggesting that the explanatory power of these dimensions for their respective measurement items needs further strengthening through item revision in future research.

Conclusions

This study developed and initially validated the Irrational Behavior in Online Discourse Scale for undergraduate students. The scale’s four-dimensional structure—comprising cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance—received preliminary empirical support, with acceptable overall internal consistency and initial evidence of construct and discriminant validity. Its application among university students revealed significant variations in irrationality levels across different genders, daily internet usage durations, and family backgrounds, thereby contributing to the understanding of irrational online behavior in university settings.

Supplementary information

Below is the link to the electronic supplementary material.

Author contributions

ZL: Data curation, investigation, formal analysis, conceptualization, methodology, and writing the paper. CZ: Data collection and review. JL: Data collection, data curation, and analysis. BY: Supervision, validation, and review.

Funding

This study was supported by the following grants: “Research on Risk Prevention and Guidance Mechanism of Group Polarization in University Network Public Opinion” (24JDSZ3098), a Project for University Counselors in Humanities and Social Science Research of the Ministry of Education; and “Research on Online and Offline Interactive Evolution Mechanism of University Network Public Opinion” (22YBA260), a General Project of Hunan Provincial Philosophy and Social Science.

Data availability

Data will be shared by the corresponding author upon reasonable request.

Declarations

Ethics Statement

This study was conducted in accordance with the principles of the Declaration of Helsinki. The research protocol was reviewed and approved by the Ethics Committee of Hunan First Normal University (Approval Number: 202301005; Approval Date: January 10, 2024). All other participating universities recognized and accepted this central IRB approval. All research procedures were conducted in strict accordance with the approved protocol at every participating institution. All participants provided informed consent.

Competing interests

The authors declare no competing interests.