Using Biclustering to Detect Cheating in Real Time on Mixed-Format Tests

Abstract

We evaluated a real-time biclustering method for detecting cheating on mixed-format assessments that included dichotomous, polytomous, and multi-part items. Biclustering jointly groups examinees and items by identifying subgroups of test takers who exhibit similar response patterns on specific subsets of items. This method’s flexibility and minimal assumptions about examinee behavior make it computationally efficient and highly adaptable. To further finetune accuracy and reduce false positives in real-time detection, enhanced statistical significance tests were incorporated into the illustrated algorithms. Two simulation studies were conducted to assess detection across varying testing conditions. In the first study, the method effectively detected cheating on tests composed entirely of either dichotomous or non-dichotomous items. In the second study, we examined tests with varying mixed item formats and again observed strong detection performance. In both studies, detection performance was examined at each timestamp in real time and evaluated under three varying conditions: proportion of cheaters, cheating group size, and proportion of compromised items. Across conditions, the method demonstrated strong computational efficiency, underscoring its suitability for real-time applications. Overall, these results highlight the adaptability, versatility, and effectiveness of biclustering in detecting cheating in real time while maintaining low false-positive rates.

The widespread adoption of computerized testing in educational assessments and high-stakes examinations has been driven by its flexibility and efficiency (Bennett, 2015; Tippins, 2009). This trend was significantly accelerated during the COVID-19 pandemic as educational institutions and certification bodies were compelled to shift toward remote testing solutions. Although online testing environments have facilitated the continuation of assessments, they have also raised concerns regarding the maintenance of test integrity. In particular, remote and unsupervised settings increase the potential for cheating, exacerbated by the ease with which the internet enables the dissemination and sharing of exam content through unauthorized channels (Newton & Essex, 2024).

To address these issues, many institutions rely on human proctors who remotely monitor examinees via webcams to detect and prevent cheating behaviors in real time (Muckle et al., 2022; Selwyn et al., 2023). Proctors can intervene during the test by issuing warnings or taking action such as pausing or terminating the exam to prevent further dishonest behavior (Topuz et al., 2022). Although effective, this approach is resource-intensive, often requiring many proctors to monitor all examinees simultaneously. Moreover, human subjectivity introduces inconsistencies in how cheating is identified and addressed (Belzak et al., 2024), which has led to increasing interest in automated real-time cheating detection systems for the potential they offer to provide greater consistency and objectivity.

However, many traditional cheating detection methods, such as response similarity indices (Angoff, 1974; Wollack & Maynes, 2016) and item response theory-based approaches (Drasgow et al., 1987; Van der Linden & Sotaridona, 2006), are unsuitable for real-time application. These methods typically require complete response data or are computationally intensive, restricting their use during live testing. Consequently, there has been limited research on real-time cheating detection using traditional methods with a few notable exceptions. Han and Kang (2023) proposed a multivariate sequential monitoring procedure that focuses on non-effortful behavior (e.g., random guessing, careless responding), while Lu et al. (2024) introduced a sequential Bayesian changepoint detection method targeting both cheating and rapid guessing. However, despite their ability to function in real time, these approaches fail to distinguish cheating from other aberrant responses. Becker and Meng (2022) developed a matrix-based similarity index to detect collusion in real time, but this method relies solely on response data without incorporating response times.

To address these challenges, several machine learning-based approaches have been proposed. For example, Meng and Ma (2023) used response scores, time data, and item difficulty to detect cheating in real time, whereas Tiong and Lee (2021) leveraged both response accuracy and response time as well as internet protocol data for real-time detection. While promising, these methods rely on supervised learning that require accurately labeled training data to identify cheaters. Such data are not always available, making this approach less feasible in many contexts. Another common approach involves the use of biometric data, such as facial recognition (Fayyoumi & Zarrad, 2014; Ozdamli et al., 2022), eye-tracking (Bawarith et al., 2017), and mouse movement (H. Li et al., 2021). While these techniques can be effective, they are often impractical in many testing environments due to high implementation costs, computational complexity, and concerns over privacy (Coghlan et al., 2021; Langenfeld, 2020).

To address these limitations, we propose the use of biclustering, a method previously applied in post-test analysis to detect collusion (Lee & Vispoel, 2025), and adapt it for real-time detection. Specifically, we extend this approach to mixed-format tests, incorporating statistical significance tests for relative clusters to reduce false positives (i.e., flagging non-cheaters as cheaters). By offering an effective and computationally efficient solution, this method can provide a robust framework for real-time cheating detection in a wide range of testing environments.

Background

Biclustering

Biclustering, was first introduced by Hartigan (1972) as a powerful unsupervised learning method designed to simultaneously group both rows and columns within a dataset by forming biclusters—submatrices that exhibit consistent patterns. Unlike traditional clustering techniques that classify data along a single dimension (either rows or columns), biclustering jointly analyzes both dimensions, allowing it to uncover localized structures that may not be apparent through standard clustering approaches. This method has been widely applied in bioinformatics, recommendation systems, and text mining where identifying localized patterns is crucial (Castanho et al., 2024).

Given an input matrix A = (X, Y) in which X = { $x_1,\dots ,x_N$ } represents rows and Y = { $y_1,\dots ,y_M$ } represents columns, a bicluster B = (I, J) is defined as a submatrix of size $n\times m$ , where I = ( $i_1,\dots ,i_n)\subset X$ and J = ( $j_1,\dots ,j_m)\subset Y$ . Each bicluster captures a subset of rows and columns that demonstrate a particular form of coherence, such as similar values or trends.

In educational assessment and cheating detection, biclustering serves as a powerful analytical tool for identifying subgroups of examinees (I) who exhibit similar response behaviors on specific items (J). In cases of test collusion, cheaters share unauthorized test information, leading to highly correlated response patterns that form examinee clusters (rows). Since cheating typically occurs on a subset of items rather than the entire test, biclustering also can simultaneously identify compromised item clusters (columns). By jointly clustering both examinees and items, biclustering enables the detection of localized collusion patterns that are particularly effective for identifying suspicious response behaviors. Like other machine learning methods, biclustering makes fewer assumptions about examinee behavior and provides greater flexibility and computational efficiency in identifying patterns of cheating.

Several biclustering algorithms have been developed for various applications (Padilha & Campello, 2017; Xie et al., 2019), but QUBIC (QUalitative BIClustering; G. Li et al., 2009) can be especially effective for detecting patterns of cheating. One of its key advantages is its ability to form overlapping clusters that are particularly useful in testing situations where multiple cheating groups may have access to some of the same compromised items. By accommodating this overlap, QUBIC facilitates the identification of distinct yet interconnected cheating groups. Furthermore, it does not require every data point to be assigned to a cluster but instead specifically identifies subsets of data points that exhibit clear cheating behaviors. This targeted approach not only enhances the interpretability of results but also reduces the need for post hoc identification of cheating clusters, thereby improving computational efficiency (Lee & Vispoel, 2025).

The QUBIC algorithm follows a structured process to identify biclusters within a dataset. The QUBIC algorithm begins by transforming the original data matrix to enhance patterns of detection, followed by a graph-based clustering process to identify biclusters. The process typically begins with a qualitative transformation of the original data matrix in which each cell is assigned an integer based on its comparative ranking within the corresponding row. This transformation enhances the algorithm’s ability to detect key patterns by focusing on relative relationships rather than absolute values. However, in this study, this step was omitted, because the input dataset already consisted of integer values, making additional transformation unnecessary. The construction of the input dataset for cheating detection is discussed in the next section.

Once the dataset is prepared, the algorithm constructs a weighted graph in which examinees are represented as vertices, and edges are formed between pairs of examinees. The weight of each edge reflects the number of items for which both examinees share the same non-zero integer values to identify response patterns indicative of potential collusion. After the weighted graph is established, biclusters are identified through an iterative expansion process. The algorithm begins by selecting the heaviest unused edge as an initial seed bicluster that serves as the foundation for further expansion. Additional examinees are incrementally added based on their similarity to the existing bicluster so that detected groups maintain strong internal coherence. The expansion process continues until the consistency level threshold is met to ensure that the final bicluster retains a minimum proportion of identical non-zero integers within each column relative to the total number of rows in the sub-matrix.

Real-Time Cheating Detection Algorithm

The real-time cheating detection algorithm in this study monitors examinees’ responses at fixed 10-min intervals, though this interval can be adjusted depending on the testing context. This approach enables timely detection of potential cheating to allow test administrators to intervene before the exam concludes. Since examinees complete items at different speeds, the number of completed items within each interval varies. Figure 1 outlines the key steps in this process.

Figure 1.

Flowchart illustrating the real-time cheating detection algorithm via biclustering.

Step 1: Constructing the Input Matrix

The first step is constructing an input matrix that helps detect suspicious patterns in examinee responses. Cheating behaviors often involve high accuracy or selecting the same distractor choice as other examinees on a given item, particularly when responses are submitted unusually quickly (Angoff, 1974; Belov, 2011; Sinharay, 2020). By incorporating both high accuracy and shared distractor choices, this method reduces the risk of falsely identifying fast responses due to low motivation or time constraints as instances of cheating (Lee & Vispoel, 2025). To capture these patterns, the matrix is built using three key pieces of information: response accuracy, response time, and distractor selection. At each 10-min interval, an input matrix is generated with the responses made up to this timestamp. The input matrix is structured with examinees as rows and test items as columns.

A response choice or score is included in the matrix only if its response time is less than half the median response time for that item across all examinees who answered it (see Figure 2). For dichotomous items completed within the response time threshold, the matrix records the examinee’s raw response choice (e.g., options 1–4). Note that item correctness (i.e., scoring as 0 or 1) is not considered at this stage but is applied later in Step 3 of the biclustering procedure. For polytomous items, the matrix records the assigned score reflecting partial or full credit. For multi-part items, the matrix preserves the entire sequence of values, combining raw choices for dichotomous sub-items and assigned scores for polytomous sub-items. If a response takes longer than this threshold or if the item has not yet been attempted, the matrix assigns a value of zero. Since the QUBIC algorithm ignores zeros when detecting patterns, this structure ensures that the analysis focuses specifically on rapid responses that are more likely to indicate potential cheating while reducing the influence of normal variations in response speed.

Figure 2.

Input matrix construction.

Note. RT _i  = an examinee’s response time for item i; median RT _i  = the median response time for item i across all examinees who responded to it.

Step 2: Applying the QUBIC Algorithm

The QUBIC algorithm is applied to detect biclusters—groups of examinees and items that exhibit similar response patterns. Key parameters can be adjusted to optimally reduce either false-positive or false-negative error rates. For example, the consistency level set at 1 in this study ensured that the examinees in a bicluster have identical values in each column. We also set a minimum of four columns per bicluster to focus on larger clusters of similar responses to facilitate more conservative detection.

Step 3: Reducing False Positives

To minimize false positives, three filters are applied. First, at least 80% of the responses in a bicluster must be correct, as cheating is more likely to involve correct answers. Second, a p-value threshold of .05 is used to remove biclusters likely to have occurred by chance. Third, for biclusters that exceed the p-value threshold, an additional ability-level consistency check is made to identify examinees who are more likely to be true cheaters. To achieve this, we categorize examinees based on their total score up to the current timestamp into 10 ability levels. We then calculate the proportion of examinees within each ability level who exhibit the same response pattern across the items in the bicluster. If this proportion is less than .1 and at least two examinees within the bicluster meet this condition, it confirms that the observed response pattern is not simply a result of ability-level differences but instead suggests potential collusion or answer sharing. (The detailed development and implementation of the p-value and ability-level filters are further discussed in the next section).

Step 4: Iterative Detection

The process described above is repeated every 10 min, flagging examinees who are part of statistically significant biclusters while applying the ability-level consistency check to identify potential cheaters within statistically insignificant biclusters. Our method does not require repeated identification across multiple timestamps to classify an examinee as a potential cheater. As long as examinees surpass the p-value threshold or satisfy the ability-level consistency check at any given evaluation point, they are flagged as a potential cheaters, independent of whether they are detected in subsequent iterations. ¹ The algorithm continues this detection process until the test concludes or all examinees complete the items. The R code used for applying our method to real-time cheating detection is available in a public repository. ²

Statistical and Ability-Based Criteria for Identifying Cheaters

Minimizing false positives is critical in real-time cheating detection. To ensure statistical rigor, a p-value is calculated for each bicluster to determine whether its occurrence could be due to chance alone. The first step involves calculating the probability of observing the same response pattern for each item in the bicluster based on the assumption that responses are independent across items to simplify the calculation. For each column (representing an item) in the bicluster, the probability that the examinees provide the same response is calculated. Unlike conventional biclustering approaches (Henriques & Madeira, 2018) and the approach used in Lee and Vispoel (2025) that base calculations on input data for biclustering, probabilities are derived here from the original response data to preserve the natural response distributions. This is crucial because the transformed matrix, filtered by the response time, can lead to artificially low p-values that cause many biclusters to pass the p-value threshold and thereby increase false-positive rates. By using the original response data, we ensure that the probability reflects the actual likelihood of selecting the same answer irrespective of response time. To account for fast response times, we adjust the probability of each suspected cheating response by multiplying it by .5, which still yields higher p-values than those derived from the transformed data.

The probabilities for each item are multiplied to obtain the overall probability of observing the same response pattern across all items in the bicluster. For example, if a bicluster is generated as shown in Table 1, we calculate the probabilities for each item in the bicluster using the original response data. Suppose the probabilities are as follows: p(item 5 = 1) = .4, p(item 11 = 3) = .3, p(item 13 = 4) = .2, p(item 18 = 20,201) = .1. The probability that an examinee provides the same response across these items is p =p(item 5 = 1) × p(item 11 = 3) × p(item 13 = 4) × p(item 18 = 20,201) × 0.5⁴ = .00015.

Table 1.

Example of a Bicluster.

Examinee	Item 5	Item 11	Item 13	Item 18
ID 156	1	3	4	20,201
ID 519	1	3	4	20,201
ID 824	1	3	4	20,201

Finally, the p-value is calculated using a binomial distribution: $\sum _{k=n}^N\left(\begin{array}{c}N\hfill \\ \multicolumn{1}{c}{k}\end{array}\right)p^k(1-p{)}^{N-k}$ This step estimates the probability of observing the same number of rows (examinees) in the bicluster purely by chance. For the case in Table 1, where there are 1,000 examinees, the p-value is computed as: $\sum _{k=3}^{1,000}\left(\begin{array}{c}1,000\hfill \\ \multicolumn{1}{c}{3}\end{array}\right)(0.00015{)}^k(1-0.00015{)}^{1,000-k}$ . If the resulting p-value is below the significance threshold of .05, the bicluster is considered statistically significant, meaning that the observed pattern is unlikely to have occurred by random chance, and the examinees in the bicluster are flagged as potential cheaters.

Based on this logic, while a statistically insignificant bicluster is unlikely to represent a group of cheaters as a whole, it does not guarantee that all examinees within such a bicluster are innocent. The QUBIC algorithm identifies biclusters by first selecting a pair of examinees with the highest correlation in response patterns. It then iteratively expands the bicluster by adding examinees whose response patterns closely align with this initial pair. During this process, a bicluster may include both actual cheaters and normal respondents, meaning that the statistical insignificance of the entire group does not preclude the presence of individual cheaters.

To address this issue, we implement an additional step to identify potential cheaters who might be included in statistically non-significant biclusters. Specifically, we introduce an ability-based filtering mechanism that differentiates between response patterns attributable to an examinee’s ability level and those indicative of potential misconduct. Examinees with similar ability levels are naturally more likely to exhibit similar response patterns that may inadvertently lead to their grouping within the same bicluster. To account for this, we categorize examinees into 10 quantiles based on their cumulative sum scores up to the current timepoint.

For each bicluster that does not meet the p-value threshold, we examine how common the response patterns are within the same ability group. Specifically, we calculate the proportion of examinees in the same ability category who have identical response patterns across the items in the bicluster. If this proportion is less than .1 for at least two examinees in the bicluster, their response patterns are considered highly unlikely to be due to ability alone. In such cases, these examinees are flagged as potential cheaters.

Although the bicluster shown in Table 1 was previously used to illustrate a statistically significant case, let us now consider it hypothetically as a statistically insignificant bicluster (p > .05). Rather than discarding all examinees in this bicluster, we examine the likelihood of their response patterns within their respective ability groups. Examinee ID 156 falls into the fourth decile of the cumulative score distribution, where only 2% (0.02) of examinees display the same response pattern. Examinee ID 824 is in the sixth decile, where the matching proportion is 6% (.06). These illustrative values are summarized in Table 2. Since both proportions fall below the .1 threshold, these two examinees are still flagged as potential cheaters.

Table 2.

Example of Ability-Based Filtering for an Insignificant Bicluster.

Examinee	Cumulative score at the current timestamp	Ability group	Matching proportion in ability group
ID 156	9	Fourth decile	.02
ID 519	13	Sixth decile	.15
ID 824	15	Seventh decile	.06

Note. Examinees are grouped into deciles based on their cumulative score up to the current time point. The matching proportion represents the percentage of examinees in the same ability group who exhibit the identical response pattern found in the bicluster shown in Table 1.

Purpose of this Investigation

Our primary purpose in this investigation was to evaluate the performance of the proposed real-time cheating detection methods across various testing conditions, particularly in the context of mixed-format assessments that include dichotomous, polytomous, and multi-part items. To achieve this, we divided the study into two phases. In the first phase, we focused on validating the proposed method by evaluating its performance when the compromised items were exclusively dichotomous or non-dichotomous. Following this initial validation, the second and main phase of the study was to assess the method’s performance under more complex conditions in which cheating involved all item types across a variety of relevant assessment conditions.

Methods

Data Generation

When simulating response and response time data, we sought to replicate real-world testing conditions as closely as possible. Hierarchical Item Response Theory models (H-IRT; Van der Linden, 2007) are often employed in simulation studies, because they provide a framework for jointly modeling item responses and response times. H-IRT models can handle both dichotomous and polytomous items, but we aimed to apply the cheating detection method in a more realistic scenario—a mixed-format test that includes dichotomous, polytomous, and multi-part items. This mixed-format structure reflects many real-world tests but poses challenges for traditional models that typically analyze each item format separately. In addition, real-world data often deviate from idealized model assumptions due to various forms of noise, such as weakly motivated examinees who engage in rapid guessing or test anxiety causing irregular response patterns. These factors can distort response patterns and add complexity to the data, making accurate modeling more difficult. To address this challenge and generate data that more closely reflects real testing environments, we incorporated real test data into our simulation.

Specifically, we used fixed-form test data from the 2019 computer-based Trends in International Mathematics and Science Study (eTIMSS) as the baseline for normal, non-cheating responses. TIMSS is assumed to be free from significant cheating for several reasons. First, it is conducted under strict standardized conditions with trained administrators overseeing the exam in a controlled environment that minimizes opportunities for dishonest behavior (Martin et al., 2020). Second, students are randomly given different test booklets to reduce the likelihood of copying answers. Finally, and perhaps most importantly, TIMSS is a low-stakes assessment, meaning that the results do not affect students’ grades or future opportunities. This low-stakes nature of testing decreases students’ motivation to cheat, as there is little personal gain from doing so.

For this study, we used the eighth-grade mathematics and science data from booklet 3 of the 2019 eTIMSS. A total of 1,000 examinees were sampled from the United States, Canada, and England. The eTIMSS assessment includes 54 items across both subjects. One item did not have a recorded response time for any student, so it was excluded from the analysis. Among the remaining 53 items, 13 were multi-part items, that consisted of a series of related questions within a single item. Three items were polytomous (having multiple scoring levels), and 37 were dichotomous with correct/incorrect answers.

The data included complete response times (i.e., there were no missing values). Each student was given 45 min for each subject, summing to 90 min in total, and the response time data were recorded in seconds for each item. However, the response data themselves contained missing values—317 examinees had at least one missing response–resulting in a total of 1,668 missing data points. This inclusion of missing responses adds a layer of realism to the dataset, as actual testing environments often have incomplete data due to factors like skipped questions or time constraints.

We treated the eTIMSS data as normal, non-cheating responses and simulated cheating behavior by manipulating the responses and response times of randomly selected examinees and items. The manipulation was applied in two ways: (a) Response time manipulation: The original response time for compromised items was divided in half to simulate the effect of prior knowledge of the item while maintaining individual differences in response speed (Gorney & Wollack, 2022; Meijer & Sotaridona, 2006). (b) Response manipulation: If the original response was incorrect, it was changed to the correct answer to reflect cheating. In cases of incorrect answer sharing, all members of the cheating group were assigned the same incorrect response for specific items to mimic coordinated cheating behavior.

The simulation conditions, including the proportion of cheaters, the number of compromised items, and the size of each cheating group, are detailed in the next section. To ensure the stability and accuracy of the evaluation, each condition was replicated 100 times. All data generation and analyses were performed using R version 4.4.0 (R Core Team, 2024).

Simulation Study 1: Cheating Detection for Dichotomous and Non-Dichotomous Items

In this simulation, we focused on evaluating the biclustering method’s ability to detect cheating when the compromised items were either exclusively dichotomous or non-dichotomous. In the eTIMSS data, there were only three polytomous items. To ensure a more balanced comparison, we grouped the polytomous items with 13 multi-part items, collectively referring to them as “non-dichotomous items.” We designed this study to validate the method’s effectiveness in detecting cheating separately within each item format.

To simulate cheating responses, we assumed that cheating occurred in groups of examinees who shared the same information on a subset of test items. Multiple cheating groups were simulated within each test, and the size of these groups varied across three conditions (5, 10, and 15 examinees). In addition, we varied the overall proportion of cheaters in the test population, examining proportions of 5%, 25%, and 50%. In all scenarios, we assumed that approximately 25% of the total items (13 out of 53 items) were compromised within each cheating group.

For the dichotomous-only condition, 13 items were randomly selected from the pool of 37 dichotomous items for each cheating group. For the non-dichotomous-only condition, 10 multi-part items were selected from the pool of 13, and all 3 polytomous items were treated as compromised items. We further assumed that approximately 90% (12 out of 13) of the items involved correct answer sharing, while the remaining roughly 10% (1 out of 13) involved incorrect answer sharing. To account for potential errors, such as misremembered information or simple mistakes during correct answer sharing, the accuracy for these items was modeled using a Bernoulli distribution with a probability of .9 (Sinharay, 2017). In cases of incorrect answer sharing, the accuracy was set to 0. In total, Simulation Study 1 addressed the biclustering method’s performance under 18 study conditions (2 item formats × 3 group sizes × 3 cheater proportions).

Simulation Study 2: Cheating Detection in Mixed-Format Tests

After establishing that the biclustering method could detect cheating effectively across dichotomous and non-dichotomous items in Simulation Study 1, our goal in Simulation Study 2 was to evaluate the method’s performance in a more complex, mixed-format test environment. Cheating was simulated across a combination of dichotomous, polytomous, and multi-part items. As in Study 1, we examined the method’s real-time cheating detection performance under varying proportions of cheaters (.05, .25, and .5) and group sizes (5, 10, and 15). In addition, we introduced a new variable: the proportion of compromised items, which was set at 25%, 50%, and 75%. To maintain consistency, the proportion of compromised items for each format (dichotomous, polytomous, and multi-part) was held constant relative to their distribution in the total set of items (70% dichotomous, 5% polytomous, and 25% multi-part). As in the previous study, we also simulated incorrect answer sharing for 10% of the compromised items. For instance, when the proportion of compromised items was set to 25%, each cheating group had nine dichotomous, one polytomous, and three multi-part items, with one item involving incorrect answer sharing. At 50%, there were 19 dichotomous, 2 polytomous, and 7 multi-part items, with 3 items involving incorrect answer sharing. At 75%, the selection included 28 dichotomous, 3 polytomous, and 10 multi-part items, with four items involving incorrect answer sharing. Overall, Simulation Study 2 included 27 study conditions (3 group sizes × 3 cheater proportions × 3 compromised item proportions).

Evaluation Criteria

To evaluate the performance of our cheating detection model’s ability to correctly classify cheaters and non-cheaters, we used several key metrics: False-negative rate, false-positive rate, precision, F1 score, and balanced accuracy. The false-negative rate measures the proportion of actual cheaters that are incorrectly classified as non-cheaters, while false-positive rate indicates the proportion of non-cheaters that are incorrectly identified as cheaters. Precision is defined as the ratio of correctly identified cheaters to the total number of flagged examinees. The F1 score is the harmonic mean of precision and true-positive rate (1 minus false-negative rate), which balances the model’s ability to avoid false positives while detecting cheaters. Balanced accuracy, calculated as the average of the true-positive rate and true-negative rate, ensures equal consideration of both cheaters and non-cheaters to provide a fairer evaluation of model performance than standard accuracy, which can be skewed by the majority class (non-cheaters).

Results

Simulation Study 1: Cheating Detection for Dichotomous and Non-Dichotomous Items

In Simulation Study 1, the algorithm’s performance was evaluated across 100 replications per research condition, yielding a small average standard error of 0.03 that revealed a high level of consistency in the evaluation metrics. To evaluate how early cheaters were detected, we report the number of compromised items a cheater engaged with by the time they are first flagged. On average, cheaters were detected after engaging with 10.7 compromised items out of a total of 53 items. This metric quantifies the delay between the onset of cheating and its detection.

Figure 3 is presented to illustrate the dynamic nature of the detection process, demonstrating how the algorithm continuously identifies potential cheaters at each timestamp. For simplicity, we focus on the scenario where the cheating group consists of 10 examinees, as the graphs for group sizes of 5 and 15 exhibited similar patterns to those in Figure 3. As detection time advances, the evaluation metrics evolve, confirming the effectiveness of the real-time cheating detection method. The false-positive rate remains low, with a mean of 0.03 across all time points (range: 0.01–0.04). Meanwhile, the false-negative rate steadily decreases, reflecting the increasing number of cheaters being accurately identified as time progresses (range: 0.36–0.99, M = 0.58). Although precision shows some variability due to its dependence on the proportion of true cheaters among flagged examinees, it remains mostly strong across time and study conditions (range: 0.40–1.00, M = 0.74). The mean balanced accuracy increased from 0.51 at the first timestamp to 0.70 at the final timestamp, while the F1 score improved from 0.05 to 0.49 over the same period, indicating enhanced detection performance as more items were included throughout the exam.

Figure 3.

Simulation Study 1: Real-time changes in cheating detection performance across study conditions.

After evaluating how the biclustering method detects cheaters at each time point, we then assessed its overall detection performance at the final timestamp that accounts for the responses of all examinees. To examine the statistical significance of key factors, an Aligned Rank Transform (ART) analysis was conducted (Table 3). This analysis reveales that all three main effects—compromised item type, cheater proportion, and group size—haved statistically significant impacts on most evaluation metrics (p < .05). However, group size does not significantly affect false-positive rate (p = .19). In addition, interaction effects are significant in most cases, except for the three-way interaction (compromised item type × cheater proportion × group size) on false-positive rate (p = .75) and F1 score (p = .06). Overall, these results suggest that the study conditions strongly influence detection metrics, whereas false-positive rate is less affected by group size and the three-way interaction.

Table 3.

Simulation Study 1: Aligned Rank Transform ANOVA Test.

Effect	False negative	False positive	Precision	F1	Balanced accuracy
Compromised item type	1,160.18 (0.00)	3,316.89 (0.00)	2,957.88 (0.00)	1,060.46 (0.00)	1,422.19 (0.00)
Cheater prop	3,461.84 (0.00)	3,021.15 (0.00)	6,975.81 (0.00)	2,006.42 (0.00)	3,135.78 (0.00)
Group size	68.12 (0.00)	1.65 (0.19)	11.66 (0.00)	37.11 (0.00)	65.39 (0.00)
Compromised item type × cheater prop	810.62 (0.00)	728.98 (0.00)	195.82 (0.00)	930.60 (0.00)	712.14 (0.00)
Compromised item type × group size	41.74 (0.00)	3.56 (0.03)	28.74 (0.00)	46.51 (0.00)	41.94 (0.00)
Cheater prop × group size	7.40 (0.00)	3.20 (0.01)	15.99 (0.00)	2.99 (0.02)	6.83 (0.00)
Compromised item type × cheater prop × group size	3.56 (0.01)	0.47 (0.75)	3.73 (0.01)	2.23 (0.06)	3.33 (0.01)

Note. Cheater Prop represents the proportion of true cheaters among examinees. The first value in each cell represents the F-statistic, and the value in parentheses represents the corresponding p-value.

Table 4 includes the cheating detection results at the final timestamp. When cheating occurs only on non-dichotomous items, the model has a lower false-negative rate (M = 0.54 vs. 0.62) and achieves higher precision (M = 0.78 vs. 0.70), F1 scores (M = 0.52 vs. 0.46), and balanced accuracy (M = 0.72 vs. 0.67) compared to the scenario where only dichotomous items involve cheating. This indicates that biclustering performs better when detecting cheating on polytomous and multi-part items than on dichotomous items in this simulation. Such improvement may be attributed to the lower likelihood of response consistency occurring by chance in non-dichotomous items. In addition, since the number of non-dichotomous items is smaller than dichotomous items in our data, there is a greater overlap in the cheating patterns among groups, making detection easier.

Table 4.

Simulation Study 1: Cheating Detection Results at the Final Timestamp.

Item type		Evaluation indices
Cheater prop	Group size	False negative	False positive	Precision	F1	Balanced accuracy
Dichotomous
0.05	5	0.574	0.038	0.371	0.396	0.694
	10	0.520	0.037	0.402	0.437	0.721
	15	0.504	0.038	0.381	0.430	0.729
0.25	5	0.642	0.035	0.772	0.488	0.661
	10	0.611	0.035	0.786	0.520	0.677
	15	0.597	0.033	0.806	0.537	0.685
0.50	5	0.728	0.027	0.910	0.418	0.623
	10	0.716	0.027	0.913	0.434	0.629
	15	0.697	0.026	0.919	0.455	0.638
Non-dichotomous
0.05	5	0.376	0.036	0.475	0.539	0.794
	10	0.364	0.037	0.476	0.544	0.800
	15	0.350	0.037	0.451	0.532	0.806
0.25	5	0.505	0.019	0.897	0.637	0.738
	10	0.507	0.019	0.897	0.636	0.737
	15	0.514	0.018	0.900	0.630	0.734
0.50	5	0.765	0.009	0.964	0.378	0.613
	10	0.762	0.009	0.962	0.381	0.614
	15	0.761	0.009	0.963	0.383	0.615

Note. Cheater Prop represents the proportion of true cheaters among examinees.

As the size of cheating groups increases, all evaluation metrics generally improve with a few exceptions, while the false-positive rate remains consistently low, as reflected in the difference due to group size ranging from 0.000 to 0.002 under identical conditions. This outcome is expected, because larger cheating groups make the cheating pattern more distinct to facilitate easier detection, while the p-value threshold and the ability-based filtering effectively control false-positive rate.

In addition, as but proportion of total cheaters increases, precision improves (M(difference) = 0.26), but false-negative rate also rises (M(difference) = 0.15), leading to a decline in balanced accuracy (M(difference) = −0.07). The F1 score initially increases as the proportion of total cheaters rises from 5% to 25% (M(difference) = 0.10) due to improved precision but then declines (M(difference) = −0.17) when the proportion further increases to 50%. This fluctuation occurs because false-negative rate continues to rise while precision improves at different rates, affecting the harmonic mean for both metrics. This trend suggests that the detection thresholds and parameters used in this study, which remain constant across all conditions, may be too conservative when the number of cheaters is high. Consequently, at a 50% cheater proportion in which most flagged examinees are indeed cheaters, a substantial number of true cheaters are undetected, leading to reduced overall detection performance, as reflected in the lower balanced accuracy and F1 score.

Finally, the algorithm demonstrates strong computational efficiency, requiring an average of only 0.54 s per cheating detection per timestamp using 8 cores and 8 GB of RAM that is sufficiently fast to support real-time detection during testing.

Simulation Study 2: Mixed Format Test

Following the confirmation of our method’s effectiveness for both dichotomous and non-dichotomous items in Simulation Study 1, we extended the evaluation to cases where the compromised items comprise a mix of both types. Across 100 replications per condition, the evaluation metrics display an average standard error of 0.02, again indicating a high degree of consistency in performance. To gauge how early cheaters are identified, we report the number of compromised items a cheater engaged with before being flagged. On average, cheaters are detected after 19.9 compromised items out of a total of 53 items.

Figure 4 illustrates how the algorithm consistently detects cheaters over time, focusing on the scenario with a group size of 10 examinees for simplicity. The general trend remains consistent with the results from Simulation Study 1. False-positive rate stays near zero across all time points, demonstrating that the p-value threshold and the ability-based filtering effectively prevent false positives (range: 0.00–0.04, M = 0.02). Precision, while showing some minor fluctuations due to its sensitivity to the ratio of flagged true cheaters, remains high throughout (range: 0.34–0.99, M = 0.80). Meanwhile, false-negative rate decreases (M = 0.92–0.43) and both balanced accuracy (M = 0.54–0.77) and F1 scores (M = 0.15–0.61) increase over time, reflecting enhanced overall detection performance as more test items are processed.

Figure 4.

Simulation Study 2: real-time changes in cheating detection performance across study conditions.

After confirming that the biclustering method performs effectively across individual timestamps, we proceeded to evaluate its detection performance at the final timestamp, that includes the responses of all examinees. To determine the significance of the study conditions on the evaluation criteria at the final timestamp, we again performed an Aligned Rank Transform (ART) analysis (Table 5). This analysis confirms that cheater proportion, group size, and compromised item proportion significantly influence most detection metrics (p < .05), while group size has no significant effect on false-positive rate ( p = .17). Interaction effects are also largely significant, except for the three-way interaction on false-positive rate (p = .36). Overall, these results suggest that the study conditions strongly influence detection metrics, whereas false-positive rate remains stable and largely unaffected by variations in group size and the three-way interaction.

Table 5.

Simulation Study 2: Aligned Rank Transform ANOVA Test.

Effect	False negative	False positive	Precision	F1	Balanced accuracy
Cheater prop	8,483.20 (0.00)	7,526.76 (0.00)	10,592.04 (0.00)	3,848.66 (0.00)	7,696.06 (0.00)
Group size	272.27 (0.00)	15.06 (0.00)	71.02 (0.00)	228.75 (0.00)	277.22 (0.00)
Compromised item prop	6,592.44 (0.00)	1,367.33 (0.00)	4,124.75 (0.00)	6,419.99 (0.00)	6,954.68 (0.00)
Cheater prop × group size	9.89 (0.00)	1.58 (0.17)	38.85 (0.00)	13.66 (0.00)	9.11 (0.00)
Cheater prop × Compromised item prop	1,143.12 (0.00)	623.34 (0.00)	247.73 (0.00)	640.90 (0.00)	1,021.63 (0.00)
Group size × Compromised item prop	53.01 (0.00)	7.23 (0.00)	63.17 (0.00)	87.39 (0.00)	55.62 (0.00)
Cheater prop × group size × Compromised item prop	2.33 (0.02)	1.10 (0.36)	7.29 (0.00)	1.99 (0.04)	2.47 (0.01)

Note. Cheater Prop represents the proportion of true cheaters among examinees, while compromised item Prop indicates the proportion of true compromised items among all items. The first value in each cell represents the F-statistic, and the value in parentheses represents the corresponding p-value.

Table 6 includes the final timestamp results of the compromised detection model. As the proportion of cheating items increases, all evaluation metrics improve: false-negative rate decreases (M(difference) = −0.15), false-positive rate declines slightly (M(difference) = −0.01), precision rises (M(difference) = 0.05), F1 scores increase (M(difference) = 0.11), and balanced accuracy increases (M(difference) = 0.08). This trend suggests that a higher proportion of compromised items makes cheating behaviors more distinguishable, thereby enhancing detection performance.

Table 6.

Simulation Study 2: Cheating Detection Results at the Final Timestamp.

Compromised item prop		Evaluation indices
Cheater prop	Group size	False negative	False positive	Precision	F1	Balanced accuracy
0.25
0.05	5	0.556	0.037	0.382	0.410	0.703
	10	0.486	0.036	0.427	0.465	0.739
	15	0.462	0.036	0.409	0.464	0.751
0.25	5	0.635	0.033	0.788	0.499	0.666
	10	0.588	0.031	0.815	0.547	0.690
	15	0.557	0.029	0.839	0.579	0.707
0.50	5	0.719	0.025	0.917	0.430	0.628
	10	0.687	0.024	0.929	0.468	0.645
	15	0.663	0.023	0.935	0.495	0.657
0.5
0.05	5	0.237	0.039	0.505	0.607	0.862
	10	0.201	0.039	0.519	0.628	0.88
	15	0.181	0.039	0.500	0.620	0.89
0.25	5	0.391	0.023	0.897	0.725	0.793
	10	0.367	0.024	0.898	0.742	0.804
	15	0.357	0.023	0.905	0.751	0.810
0.50	5	0.603	0.011	0.972	0.564	0.693
	10	0.586	0.011	0.974	0.581	0.702
	15	0.580	0.011	0.975	0.587	0.705
0.75
0.05	5	0.085	0.039	0.551	0.687	0.938
	10	0.072	0.040	0.553	0.693	0.944
	15	0.068	0.040	0.526	0.672	0.946
0.25	5	0.253	0.018	0.933	0.829	0.865
	10	0.241	0.018	0.933	0.837	0.870
	15	0.249	0.018	0.935	0.833	0.867
0.50	5	0.593	0.003	0.992	0.577	0.702
	10	0.592	0.003	0.992	0.578	0.702
	15	0.583	0.003	0.992	0.587	0.707

Note. Cheater prop represents the proportion of true cheaters among examinees, while Compromised item prop indicates the proportion of true compromised items among all items.

In addition, increasing the group size results in slight improvements in detection, as larger groups make cheating patterns more apparent and easier to identify. False-negative rate decreases (M(difference) = −0.02), while false-positive rate and precision remain unchanged (M(difference) = 0.00). Consequently, overall performance—measured by F1 scores (M(difference) = 0.01) and balanced accuracy (M(difference) = 0.01)—shows marginal gains with increasing group size.

As the proportion of cheaters increases, false-negative rate rises significantly (M(difference) = 0.18), while precision also improves (M(difference) = 0.24). This pattern is likely due to the fixed thresholds and parameters that become more conservative as the number of cheaters grows. False-positive rate shows a slight decline (M(difference) = −0.01). Although precision improves, the model fails to detect a substantial number of cheaters, while false positives remain nearly constant, leading to a decrease in balanced accuracy (M(difference) = −0.14). F1 scores exhibit a non-monotonic trend, increasing when the proportion of cheaters rises from 0.05 to 0.25 (M(difference) = 0.12) but subsequently declining as the proportion increases from 0.25 to 0.50 (M(difference) = −0.16). This fluctuation occurs because, in the transition from 5% to 25% cheaters, false-negative rate increases moderately (M(difference) = 0.14), whereas precision improves substantially (M(difference) = 0.40). However, when the proportion of cheaters further increases from 25% to 50%, false-negative rate increases more sharply (M(difference) = 0.22), while precision shows only a slight improvement (M(difference) = 0.08), ultimately lowering F1 scores.

Across all conditions, false-positive rate remains consistently low (range: 0.00–0.04, M = 0.03), reinforcing the method’s reliability. In addition, the algorithm demonstrates strong computational efficiency, requiring an average of only 0.56 seconds per cheating detection per timestamp.

Discussion

Overview of Findings

In this study, we evaluated the effectiveness of a real-time biclustering method for detecting cheating across different testing conditions and item formats. The method was designed to address the complexities of mixed-format assessments, including dichotomous, polytomous, and multi-part items, by incorporating enhanced statistical significance tests to reduce false positives. Within two simulation studies, we evaluated the method’s performance at various timestamps, demonstrating its effectiveness at different stages of the exam. Due to its ability to handle small sample sizes and its computational efficiency, the biclustering approach proved to be a viable option for real-time cheating detection.

Cheating Detection for Dichotomous and Non-Dichotomous Items

Following the our initial evaluations at different time points, we assessed the method’s overall cheating detection performance at the final timestamp incorporating various study conditions to evaluate its stability and accuracy across a range of scenarios. In Simulation Study 1, we confirmed that the biclustering method successfully detected cheating in real time for both dichotomous and non-dichotomous items. Our results further showed that the biclustering approach effectively identified suspicious response patterns, such as quick responses and similar correct and incorrect answers, regardless of item format. Thus, item format—whether dichotomous or non-dichotomous—did not hinder its detection capabilities. Sinharay (2015) developed an IRT-based method for mixed-format tests that adapts person-fit statistics but requires additional modeling and computational overhead while incorporating posterior predictive model checks that further increase complexity and computing time. In contrast, biclustering directly analyzes observed response patterns without estimating item parameters, reducing computational demands and making it more suitable for real-time detection. This adaptability is crucial for contemporary assessments that increasingly feature diverse and complex item types such as technology-enhanced items and multi-part questions. In this context, the biclustering method provides a robust solution for maintaining test integrity across a wide variety of modern assessment formats.

Effects of Study Conditions

In Simulation Study 1, we focused on detecting cheating on dichotomous and non-dichotomous items, while in Simulation Study 2, we examined how the detection method performs when the compromised items include a mix of both item types. Across both studies, we evaluated how different study conditions influenced cheating detection. As the proportion of cheaters increased, false-negative rate and precision also increased, while balanced accuracy and F1 scores decreased. and false-positive rate remained stable across all conditions. This pattern is likely due to the fixed thresholds and parameters that which remained constant and became too conservative as the proportion of cheaters grew. Consequently, the model missed a significant number of cheaters, leading to higher false negatives, even though false-positive rate was effectively controlled by the p-value threshold. These results underscore the importance of adjusting parameters based on the risk of cheating that can vary depending on the testing context.

The size of cheating groups also had a substantial impact on detection performance across both simulation studies. As group size increased, all evaluation metrics improved except for false-negative rate that remained relatively constant across study conditions. However, the difference between small groups (5 examinees) and large groups (15 examinees) was minimal, with an average improvement of only 0.03 in Study 1 and 0.04 in Study 2 across all evaluation metrics. These results indicate that the proposed method can detect cheating effectively even with smaller group sizes, though performance improves as group size increases, likely due to more pronounced cheating patterns when larger groups are involved.

In Simulation Study 2, we introduced proportion of compromised items as a variable and observed its effects on detection performance. The biclustering method demonstrated that it could detect cheaters in real time regardless of the proportion of compromised items. However, as also noted by Lee and Vispoel (2025), detection performance improved when the proportion of compromised items was higher. When cheaters colluded on more items, the biclustering algorithm more effectively identified examinee-by-item cheating patterns, making them easier to detect.

Overall, the findings underscore the biclustering approach’s effectiveness in detecting cheaters under various testing conditions while maintaining a low false-positive rate when using p-value thresholds. Moreover, the study demonstrates the value of combining real test data with simulated cheating behavior to create a more authentic evaluation environment. By simulating varied item formats and incorporating natural variations in response behaviors, we ensured that the simulation closely mirrored real-world conditions, thereby enhancing the relevance and realism of our findings.

Application of Real-Time Biclustering Cheating Detection Approach

The real-time biclustering-based cheating detection method is highly adaptable, making it suitable for both high-stakes and low-stakes testing environments. Its adaptability is particularly evident in its ability to adjust the p-value threshold used to flag potential cheaters. In low-stakes exams, where the consequences of being falsely flagged are minimal, a more liberal threshold can be applied. This broader identification of potential cheaters allows for real-time interventions such as issuing warnings during the test. By providing these warnings, examinees can reconsider their actions without facing serious consequences, thus preserving the exam’s integrity while minimizing harm to them. Such immediate interventions could serve as deterrents and help promote honest behavior during the test.

Across our studies, we targeted a false-positive rate of 0.05. However, for research settings that require a more conservative detection criterion, this method can be adjusted by flagging only examinees who pass the ability-level filtering within biclusters that also meet the p-value threshold. Under this stricter approach, false-positive rate is controlled below 0.01, though at the cost of an increased false-negative rate in most cases (see Supplemental Materials for details). Consequently, for high-stakes exams in which the ramifications of cheating are severe, a more conservative p-value threshold should be used to minimize false positives. In this context, the system can flag suspicious behavior without directly intervening during the test to allow proctors or administrators to monitor flagged individuals more closely. The flagged examinees can be observed during the exam, with further evidence collected before any definitive actions are taken. In addition, the system’s outputs can complement other post-exam cheating detection methods to provide supporting evidence for a comprehensive review of the results. This indirect approach ensures that any accusations of cheating are well-supported by multiple layers of evidence, thereby protecting the fairness and credibility of the high-stakes assessment.

The biclustering method evaluated here is primarily designed for fixed-form tests in which all examinees respond to the same set of items. However, it is also highly applicable to multistage testing environments. The system can offer preventive measures by dynamically adapting the test content for examinees flagged as potential cheaters. For instance, if a cheater is flagged early on, subsequent test items can be replaced with new ones to reduce the chance of further item compromise. In scenarios where multiple examinees are detected as part of a coordinated cheating effort, the system can assign them distinct sets of items in subsequent stages. This strategy disrupts their ability to collaborate, ensuring that they receive different content to minimize the effectiveness of coordinated cheating.

Limitations and Future Directions

While the biclustering approach to real-time cheating detection offers significant advantages, it also has some limitations that require further attention. One key limitation observed in the simulation studies is the relatively high false-negative rates (M = 0.52 across the two simulation studies). Although adjusting the p-value threshold and other biclustering parameters could potentially reduce these rates, maintaining a low false-positive rate—which is crucial in real-time detection—made high false negatives unavoidable. However, the false-negative rate reported in this study is comparable to or lower than rates from previous research that focused on cheating in post-exam real data, while yielding similar or lower false-positive rates. For instance, Lee and Vispoel (2025) found that biclustering methods resulted in false-negative rates of 0.67 to 0.70 when the false-positive rate was maintained at 0.01. In a study by Man et al. (2019), the norm conformity index (Tatsuoka & Tatsuoka, 1983) exhibited false-negative rates between 0.83 and 0.87 when the false-positive rate was 0.08. In addition, self-organizing mapping in the same study showed false-negative rates of 0.48 to 0.54, though this was accompanied by a higher false-positive rate of 0.27.

Another challenge lies in the method’s difficulty in detecting individual cheaters. The method excels at identifying patterns across groups, meaning that if a cheater acts independently and does not follow the same patterns as others, their actions may go undetected. To address these issues, future research should explore the collaborative use of multiple cheating detection methods rather than relying solely on the biclustering approach. By integrating other methods – such as statistical approaches that focus on response time anomalies or machine learning models that analyze individual-level behaviors—examiners can achieve a more comprehensive and effective detection system capable of identifying both group-based and individual cheaters (Man et al., 2019; Marianti et al., 2014; Qian et al., 2016).

Another challenge involves manipulation of behaviors by cheaters such as deliberately altering their response times to avoid detection. In these cases incorporating additional behavioral variables, such as answer changes, item revisits, or eye-tracking data could provide a more nuanced view of test-taking behavior (Liao et al., 2021; Man & Harring, 2023; Sinharay & Johnson, 2017). These extra layers of data could help counteract attempts to game the system by manipulating timing patterns.

In contexts where item banks are reused or similar items are employed across multiple test sessions, there is also the risk of cheaters exploiting prior knowledge of test content. To mitigate this, future implementations of the biclustering approach could incorporate historical data from previous test sessions. By adding rows (representing examinees) from past sessions to the current session’s matrix, the model can identify patterns that persist across different exam administrations to detect similarities among examinees who may have shared compromised test content across sessions.

Lastly, for cases where cheaters are successfully identified, the fast computation time of the biclustering method allows it to be used as a feature in supervised learning models. This integration could further enhance the system’s ability to detect cheating in real time by using the biclustering results as part of a broader feature set for machine learning models. This hybrid approach, combining unsupervised and supervised methods, could refine cheating detection models and improve their accuracy in real-time settings. Future research should focus on these collaborative approaches in integrating different detection techniques, incorporating behavioral data, and expanding the application to a variety of assessment contexts to ensure a robust, fair, and practical detection of cheating behaviors.