Frontier Language Models and Optical Character Recognition Preprocessing Against Invisible Text Injection in AI Peer Review

Abstract

This study assesses the integration of frontier large language models including tools such as optical character recognition (OCR) into the scientific peer review process to detect invisible text injection in an article under review.

Introduction

As journals explore integrating large language models (LLMs) into the peer review process, the integrity of artificial intelligence (AI)–generated evaluations becomes a pressing concern. Choi et al¹ demonstrated that invisible text injection (ITI) of adversarial instructions imperceptible to human readers (eg, white text) but readable by LLMs can inflate scores during AI-assisted peer review. Whether current frontier models remain vulnerable and whether document preprocessing defenses such as optical character recognition (OCR) can neutralize ITI have not been tested.

Methods

In this diagnostic study using a published clinical trial article, we created 6 variants: 1 unaltered control and 5 with a single injected systematic flaw each.² Each article variant was rendered as a portable document format (PDF) under 4 ITI conditions using the exact Choi et al¹ payload twice per page (eMethods in Supplement 1). Article text was extracted via 2 paths: direct PDF text extraction (preserves invisible text) and an open-source OCR vision model that only processes visible text (olmOCR-2-7B-1025).³ This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis–Large Language Model (TRIPOD-LLM) reporting guideline.⁴

Six LLMs served as AI reviewers: 3 Choi-utilized models (Claude 3 Haiku, GPT-4o-mini, and Gemini 2.0 Flash Lite) and 3 early-2026 frontier models (Claude Sonnet 4.5, GPT-5.2, and Gemini 3 Pro).^5,6,7 Each combination of article variant, ITI condition, extraction path, and LLM was evaluated 10 times (2880 total reviews). The AI review prompt was adapted directly from Choi et al¹ (eMethods in Supplement 1). The primary outcome was mean quality score on a Likert scale (1 = very poor, 5 = excellent). We analyzed scores using a linear mixed-effects model with ITI, extraction path, and model generation as fixed effects; flaw type as a covariate; and LLM as a random intercept to account for nonindependence of observations within models. Effect sizes were estimated using Cohen d for the ITI effect within each generation × extraction path cell (eMethods in Supplement 1). Statistical significance was set at P < .01. P values were 2 sided. All analyses used Python, version 3.13 with scipy 1.15.1, statsmodels 0.14.4, and pandas 2.3.2. This study used only publicly available, deidentified data and did not involve interaction or intervention with individuals. Therefore, it does not meet the definition of human subjects research as defined by 45 CFR §46.102. All analyses were performed on February 24, 2026.

Results

On direct text extraction, ITI inflated quality scores for Choi-utilized models by 0.82 points (mean, 4.19 vs 3.37 points; Cohen d = 1.11; 95% CI, 0.93-1.28; P < .001) (Table). In contrast, frontier models showed a negative ITI effect on direct extraction (−0.15 points; mean, 2.23 vs 2.38 points; d = −0.39; 95% CI, −0.56 to −0.22; P < .001). With OCR-based extraction, the ITI effect was eliminated for both model generations. The 3-way interaction between ITI presence, model generation, and extraction path was statistically significant in the mixed-effects model (coefficient, −1.00; P < .001). Among individual models, Claude 3 Haiku was the most susceptible, with scores increasing from 4.00 to 4.86, whereas Gemini 3 Pro was the most resistant, with scores decreasing from 2.17 to 1.99 (Figure). In the no-ITI condition, frontier models showed greater separation across manuscript variants than Choi-utilized models (mean score range, 2.15-2.55 vs 3.33-3.43).

Table. Mean Quality Scores by Model Generation, Flaw, and Condition^a.

Model	No ITI, mean (SD)	Direct, mean (SD)			All ITI, OCR, mean (SD)
		White on white	Zero-point font	Near-white color
Choi-utilized models	3.37 (0.48)	4.19 (0.82)	4.20 (0.81)	4.16 (0.83)	3.34 (0.48)
Control	3.43 (0.50)	4.23 (0.73)	4.27 (0.69)	4.20 (0.71)	3.33 (0.47)
Statistical	3.33 (0.48)	4.07 (0.78)	4.03 (0.81)	4.03 (0.81)	3.33 (0.47)
Methodological	3.38 (0.49)	4.30 (0.88)	4.30 (0.92)	4.17 (0.91)	3.34 (0.48)
Reporting	3.37 (0.49)	4.23 (0.94)	4.17 (0.91)	4.23 (0.94)	3.33 (0.47)
Conclusion overreach	3.33 (0.48)	4.17 (0.91)	4.20 (0.92)	4.20 (0.92)	3.33 (0.47)
Transparency	3.37 (0.49)	4.17 (0.70)	4.23 (0.63)	4.13 (0.73)	3.38 (0.49)
Frontier models	2.38 (0.49)	2.25 (0.46)	2.22 (0.48)	2.23 (0.46)	2.29 (0.46)
Control	2.53 (0.50)	2.43 (0.50)	2.50 (0.51)	2.40 (0.50)	2.42 (0.50)
Statistical	2.15 (0.36)	2.13 (0.35)	2.10 (0.40)	2.13 (0.35)	2.11 (0.32)
Methodological	2.28 (0.45)	2.17 (0.38)	2.03 (0.32)	2.07 (0.25)	2.21 (0.41)
Reporting	2.47 (0.50)	2.10 (0.31)	2.20 (0.41)	2.23 (0.50)	2.33 (0.50)
Conclusion overreach	2.37 (0.49)	2.20 (0.55)	2.13 (0.43)	2.27 (0.52)	2.27 (0.47)
Transparency	2.48 (0.50)	2.47 (0.51)	2.33 (0.61)	2.30 (0.53)	2.38 (0.49)

Abbreviations: ITI, invisible text injection; OCR, optical character recognition.

^a Scores represent the mean (SD) of quality ratings on a 1 to 5 scale (1 = very poor, 5 = excellent) across all article variants and repetitions (2880 reviews). The “No ITI” column includes both extraction paths. ITI conditions pool all 3 injection methods (white on white, zero-point font, and near-white color).

Figure. Mean Quality Scores, Invisible Text Injection (ITI) Effect Sizes, and Per-Model Score Shifts.

A, Bar graph showing mean quality scores (1-5 scale [1 = very poor, 5 = excellent]; error bars indicate SD) for each of 6 large language models under 3 conditions: no ITI, ITI with direct PDF text extraction, and ITI with optical character recognition (OCR)–based extraction. A dotted line separates Choi-utilized models (left) from frontier models (right). There was zero variance for some of the bars. B, Dot plot showing Cohen d (95% CI) for the ITI effect (ITI present vs absent) by model generation and extraction path. C, Dot plot showing per-model score shift between no-ITI baseline (circles) and ITI with direct extraction (squares); numeric differences are annotated.

Discussion

Our findings reveal 2 complementary defenses against ITI in AI-assisted peer review. Frontier LLMs not only resisted ITI but actively penalized articles containing it, a striking reversal from the score inflation seen with Choi-utilized models and evidence of improved adversarial robustness across all 3 model families.^5,6,7 OCR preprocessing eliminated the ITI effect entirely regardless of model generation, offering a model-agnostic safeguard that journals could implement today. Limitations of this study include use of a single base article and language (English), 1 ITI payload phrase, and 1 OCR engine; generalizability to diverse manuscripts, languages, and evolving attack strategies warrants further study. As journals consider adopting AI-augmented review, both the threat of adversarial manipulation and the most effective countermeasures demand attention. ITI is but one example of what is likely to be an ongoing and dynamic challenge to the integrity of scholarly publishing. The vulnerability documented by Choi et al¹ appears addressable today through model advancement and OCR preprocessing, but continued vigilance will be essential.

Supplement 1.

eMethods

Supplement 2.

Data Sharing Statement