Skip to main content
NCBI home page
Royal Society of Chemistry logoLink to Royal Society of Chemistry
. 2026 Oct 13. Online ahead of print. doi: 10.1039/d6dd00425c

RXNMapper-v2: updated validation and evaluation on expanded chemical space

Yves Grandjean a, Sacha Raffaud b, Annie M Westerlund c, Philippe Schwaller b, Samuel Genheden c, Jean-Louis Reymond a,✉
PMCID: PMC13678721  PMID: 42899099

Abstract

Atom-to-atom mapping (AAM) assigns reactant atoms to product atoms and is fundamental for reaction classification, template extraction, and synthesis planning. RXNMapper derives AAM from the attention weights of a transformer model trained by masked language modeling on unmapped reaction SMILES. However, its original model selection relied on USPTO-50K, a validation set later shown to contain erroneous mappings. Here we revisit RXNMapper using a curated validation set of 4′533 reactions from the dataset Pistachio. We retrained ALBERT-based RXNMapper models on USPTO-full and Pistachio and selected the best head, layer, and checkpoint on this new validation set. To improve the comparison of AAM algorithms, we developed an evaluation framework that removes ambiguous reference reactions and compares predictions based on reaction graph isomorphism rather than direct string matching, avoiding false negatives from inconsistent formats and symmetry-equivalent mappings. Across the golden dataset, Pistachio, and an internal AstraZeneca medicinal chemistry benchmark, the updated models outperform the original RXNMapper and LocalMapper, while rearrangements remain persistent challenges.

Atom-to-atom mapping (AAM) assigns reactant atoms to product atoms and is fundamental for reaction classification, template extraction, and synthesis planning.graphic file with name d6dd00425c-ga.webp

Introduction

Atom-to-atom mapping (AAM), namely the assignment of reactant atoms to their corresponding product atoms, is a basic operation in cheminformatics as it describes how atoms are redistributed during a reaction.1 Accuracy of atom mapping has implications on downstream tasks such as reaction search,1 reaction classification2,3 reaction template extraction for computer-assisted synthesis planning,4–13 and several template-free applications including reaction-center tagging,14–19 reaction tokenization,20 and reaction representation learning for property prediction.21–24

Early AAM methods relied on structural comparison, optimization procedures, or expert-defined transformation rules to assign atom correspondences.1,25–40 More recently, neural-network approaches based on transformers41–43 and graph models have enabled data-driven atom mapping, such as RXNMapper,44 Graphormer,45 LocalMapper,46 and AMNet.47 Among these, RXNMapper is particularly attractive because it is trained by masked language modelling (MLM) on unmapped reaction SMILES, making it simple to train and fast to apply.44 However, its original model selection relied on a mapped validation set derived from USPTO-50K, which was later shown to contain erroneous mappings, while LocalMapper was subsequently reported to outperform RXNMapper.46

Here we revisit the RXNMapper approach using a curated validation set designed to cover the chemical space of Pistachio. We first establish an evaluation framework for atom mapping that removes ambiguous reference reactions and compares predictions based on reaction graph isomorphism rather than direct string matching. We then retrain ALBERT-based RXNMapper models on USPTO-full (2.8 M reactions, RXNMapper-v2) and Pistachio (15.06 M reactions, RXNMapper-P) and improve model selection at the level of head, layer, and checkpoint using the curated validation set from Pistachio (4′533 reactions). The original RXNMapper, the two updated models, and LocalMapper are compared on three benchmarks: the golden dataset (1′758 reactions),46,48 a Pistachio test set (16′248 reactions), and an internal AstraZeneca medicinal chemistry dataset (108′611 reactions). We find that reliable model selection has a larger effect on performance than retraining on a new reaction corpus, and that the updated RXNMapper models outperform both the original RXNMapper and LocalMapper across the datasets examined here. To support reproducibility, we provide the training scripts, model-selection workflow, updated checkpoints, and evaluation framework in dedicated repositories.

Methods

Datasets

We used four reaction datasets in this study. USPTO-full, composed of 2′847′312 reactions extracted from the United States Patent and Trademark Office by Lowe,49,50 contains 2.8 M unmapped reaction SMILES and was used to train the original RXNMapper and the updated RXNMapper-v2 models. USPTO-50K, introduced by Schneider et al.,3 consists of 50 K reactions sampled from USPTO-full and atom-mapped and classified with NameRXN (v2.1.84). The golden dataset, introduced by Lin et al.48 and later filtered by Chen et al.46 contains 1′758 reactions and was used as a benchmark for atom-mapping performance. Pistachio,51 the proprietary reaction dataset from NextMove, contains 15′065′981 mapped and classified reactions in the 2025Q2 release. To obtain a benchmark covering broad chemical space, we sampled up to 10 reactions per reaction class randomly across the 2′460 available classes and performed a random stratified 20/80 split of the resulting set by reaction class into validation (4′533 reactions) and test (16′248 reactions) subsets. Finally, an internal AstraZeneca medicinal chemistry dataset containing 108′611 mapped (by NextMove) reactions recorded between January 2024 and June 2025 with yields above 20% was used as a third benchmark representing a narrower chemical space.

Models, training and inference

We compared four atom-mapping models: the original RXNMapper,44 LocalMapper,46 and two updated ALBERT-based RXNMapper models retrained in this work. The original RXNMapper was trained by Masked Language Modeling (MLM) on USPTO-full and used as published. LocalMapper was used from the author's public implementation. The two updated RXNMapper models, RXNMapper-v2 and RXNMapper-P, were trained with a new MLM training pipeline on the unlabeled (i.e. not atom-mapped) USPTO-full (2′847′312 reactions) and Pistachio (15′065′981 reactions), respectively, using the same ALBERT architecture as the original model and training each model for 1.5 M steps. For RXNMapper-v2, the original vocabulary was used, whereas a new one was built for RXNMapper-P. Vocabularies were used to tokenize the reactions before running MLM. For each model, the final atom-mapping configuration was selected by choosing the best head, layer, checkpoint combination on the Pistachio validation set. All heads of the three last layers (indices 9, 10, 11) were used to atom-map the validation set reactions across all 150 model training checkpoints. Our evaluation pipeline was used to find the head/layer/checkpoint that led to the best performance on the validation set.

Standardization and evaluation of atom-mapped reactions

Before evaluation, atom-map indices were canonicalized using a deterministic ordering procedure derived from the original RXNMapper workflow. Ground-truth reactions were then screened for formatting inconsistencies and excluded when they contained duplicated product-side atom-map indices, product atoms without reactant-side counterparts, invalid or non-canonicalizable structures, unsanitizable molecules, or unmapped product atoms. Excluded reactions were flagged to allow later inspection.

For evaluation, each atom-mapped reaction was converted into a labeled reaction graph. Atom correspondences between reactant and product sides were encoded through shared atom-map indices, and graph equivalence between ground-truth and predicted reactions was assessed by labeled graph isomorphism. This comparison was implemented with the VF2 isomorphism algorithm as available in NetworkX.52,53 Graph-based comparison treats symmetry-related but chemically equivalent atom mappings as identical.

TMAPs

TMAP54 was used to visualize the structure of the reaction error sets. Reactions were represented with RXNFP fingerprints55 derived from reaction SMILES, and nearest-neighbor relationships were computed using cosine distance. The resulting maps were used to examine how incorrectly predicted reactions are distributed in reaction space and whether distinct regions of model agreement or failure could be identified.

Results and discussion

Validation set design

In the benchmarking study used for LocalMapper (a graph neural network trained via human-in-the-loop currently state-of-the-art on USPTO-50 K),46 the authors reported errors in the USPTO-50 K dataset,3 including in the validation and test sets, and removed part of these automatically, leaving 969 of the original 1000 validation reactions. Because USPTO-50 K is widely used for benchmarking atom-mapping models, we examined reactions where AAM from RXNMapper and LocalMapper differed, and discovered around 100 additional reactions which were clearly unreliable as atom-mapping references. We observed typographical errors in reactions (Fig. 1a), errors in the ground-truth atom-maps (Fig. 1b), and cases in which multiple atom mappings could reasonably be assigned (Fig. 1c). We also found reactions whose recorded transformation appeared chemically questionable. Taken together, these issues make USPTO-50 K poorly suited as a validation or test set for the present study.

Fig. 1. Examples of NameRXN 2.1.84 atom-mapping errors or ambiguities in USPTO-50 K. (a) Typographical and reagent-description errors in a Mitsunobu reaction (Ph3 instead of the full reagent set PPh3/DEAD). (b) Incorrect atom mapping in an unbalanced reaction (the product O-atom should correspond to a water molecule or the acetone O-atom). (c) A reaction for which several atom mappings are chemically plausible, which can lead to false negatives during evaluation (the product amide O-atom might correspond to any of the three O-atoms in reagents and reactants).

Fig. 1

Open in a new tab

We therefore used Pistachio (2025Q2) as the reference dataset for model selection. Pistachio covers broader chemical space and provides NameRXN-based reaction classes in a three-level x.y.z scheme, where x denotes the superclass and y.z the reaction class. To obtain representative validation and test sets, we sampled up to 10 reactions per reaction class and applied a stratified split by reaction class, assigning up to 8 reactions per class to the test set and up to 2 to the validation set. This yielded 16′248 test reactions and 4′533 validation reactions. The Pistachio validation set was then used to select the best model configuration in the remainder of this work.

Training RXNMapper models

Because the original RXNMapper publication released model checkpoints but not the corresponding training scripts, we implemented a new training pipeline reproducing the original ALBERT setup. Two models were trained by unsupervised learning on unmapped reaction SMILES: one on USPTO-full (RXNMapper-v2) and one on the full Pistachio dataset (RXNMapper-P) (Fig. 2a). The Pistachio-trained model was included to test whether exposure to a broader chemical space improves atom-mapping performance under the same unsupervised training scheme.

Fig. 2. Overview of RXNMapper training and model selection. (a) ALBERT pretraining by masked language modeling on large corpora of unmapped reaction SMILES. (b) Selection of the best head/layer/checkpoint combination by performance comparison on the validation set.

Fig. 2

Open in a new tab

After pretraining, the atom-mapping head was selected empirically. In the RXNMapper framework, atom mapping is derived from the attention pattern of a single head, which is used to associate each product atom with the corresponding reactant atom. We therefore evaluated head, layer, and checkpoint combinations on the Pistachio validation set and retained the best-performing configuration as the final model (Fig. 2b).

Preprocessing and evaluation of atom-mapping predictions

Atom-mapped reactions are not represented consistently across datasets, which can introduce false negatives during evaluation. To address this, we developed a preprocessing and evaluation pipeline extending the approach of Schwaller et al.44 After atom-map canonicalization, ground-truth reactions were excluded when (1) two or more product atoms shared the same mapping number, (2) a product atom had no mapped counterpart on the reactant side, or (3) a product atom was left unmapped (Fig. 3a). These cases were removed because they make the reference mapping ambiguous and unsuitable for benchmarking.

Fig. 3. Preprocessing and evaluation framework. (a) Atom-mapped reaction formats excluded during preprocessing: (i) duplicated product-side atom-map indices, (ii) product-side atom-map indices without reactant-side counterparts, and (iii) unmapped product atoms. (b) Examples of highly symmetrical reaction pairs that require symmetry-aware evaluation. (c) Graph-based comparison procedure in which atom-mapped reactions are converted into labeled reaction graphs and tested for isomorphism. (d) Difference in measured performance between string matching and reaction graph comparison for the original RXNMapper model on the Pistachio test set (13′862 reactions).

Fig. 3

Open in a new tab

We then replaced string-based comparison with graph-based evaluation. For each ground-truth/predicted pair, the mapped reactions were converted into reaction graphs and compared by labeled graph isomorphism (Fig. 3c, see Theory section in the SI). Under this representation, two mappings were considered equivalent when they described the same underlying transformation, including symmetry-related alternatives (Fig. 3b). This avoids penalizing chemically correct predictions that differ only by an alternative but equivalent atom numbering.

On the Pistachio test set, the measured accuracy of the original RXNMapper differed by 4.92 percentage points depending on whether evaluation was based on string matching or reaction graph comparison (Fig. 3d). This gap arises from false negatives produced by the string-based procedure, especially for highly symmetrical reactions (Fig. 3b). These results show that robust atom-mapping evaluation requires both filtering of ambiguous references as well as symmetry-aware comparison of predicted and reference mappings. Reactions removed from the evaluation are not necessarily wrong, but their atom-mapping format should be updated to remove all ambiguity and allow a clear evaluation.

Performance of the updated model on different chemical spaces

We next compared the two updated RXNMapper models with the original RXNMapper and with LocalMapper. The models were benchmarked on three datasets representing complementary reaction spaces: the golden dataset as an open benchmark, the Pistachio test set as a broad chemical space benchmark, and the AstraZeneca medicinal chemistry set as a narrower pharmaceutical benchmark. In all cases, ground-truth and predicted reactions were preprocessed before evaluation. This left 1′758 reactions in the golden dataset, 13′862 in the Pistachio test set, and 103′286 in the AstraZeneca set.

On the preprocessed golden dataset, RXNMapper reached 86.42% accuracy, RXNMapper-v2 89.12%, RXNMapper-P 88.23%, and LocalMapper 89.02% (Table 1). On the preprocessed Pistachio test set, RXNMapper reached 85.10%, RXNMapper-v2 87.25%, RXNMapper-P 87.96%, and LocalMapper 80.59%. On the preprocessed AstraZeneca medicinal chemistry set, the same ranking was observed, with accuracies of 96.79% for RXNMapper, 97.34% for RXNMapper-v2, 97.35% for RXNMapper-P, and 94.13% for LocalMapper.

Table 1. Evaluation by reaction graph comparison of the different atom-mapping models on the preprocessed golden dataset (1′758 reactions), Pistachio test set (13′862 reactions), and internal AstraZeneca medicinal chemistry set (103′286 reactions).

Model Golden dataset [%] Pistachio test set [%] AZ med chem set [%]
RXNMapper 86.42 85.10 96.79
RXNMapper-v2 89.12 87.25 97.34
RXNMapper-P 88.23 87.96 97.35
LocalMapper 89.02 80.59 94.13
Open in a new tab

Across the three benchmarks, the original RXNMapper consistently gives the lowest accuracy among the ALBERT-based models. The largest gain is observed when the original USPTO-trained model is retrained and reselected under the updated workflow, while additional pretraining on Pistachio provides only a smaller benefit. On Pistachio, RXNMapper-v2 improves by 2.15 percentage points over the original model, whereas RXNMapper-P adds a further 0.71 points. On the AstraZeneca set, RXNMapper-v2 and RXNMapper-P are nearly identical (97.34% and 97.35%), indicating that a representative validation set has a larger effect than retraining on a broader reaction corpus.

The absolute accuracy values also show a clear benchmark effect. Performance is lowest on Pistachio, where even the best model remains below 88%, and highest on the AstraZeneca medicinal chemistry set, where all models exceed 94%. This is consistent with Pistachio covering a broader and more difficult reaction space, whereas the AstraZeneca benchmark reflects a narrower medicinal chemistry domain.

LocalMapper performs competitively on the golden dataset but underperforms on Pistachio and AstraZeneca relative to the ALBERT-based models. This trend may reflect differences in the training domain, as LocalMapper was developed on USPTO-centered data, whereas Pistachio spans a broader chemical space.

Reaction class performance

We used the NameRXN superclass annotation of the Pistachio test set to examine whether model performance differs across broad reaction classes (TMAP of the test set colored by NameRXN superclass, Fig. S1). Table 2 reports the number of mistakes per superclass for each model. Across the 12 superclasses, LocalMapper gives the highest error count overall. Among the ALBERT-based models, RXNMapper-P gives the lowest number of errors in 9 of the 12 superclasses, including the frequent superclasses 4 (heterocycle formation) and 9 (functional group interconversion, FGI), whereas RXNMapper-v2 gives the lowest counts in superclasses 3 (C–C bond formation), 5 (protections), and 7 (reductions). Relative to superclass size, superclass 12 shows 5/5 reactions in the error set, but this category contains only five miscellaneous examples. Among the larger classes, the highest error-set proportions are observed for superclass 4 (heterocycle formation) and superclass 8 (oxidations), both at 36.7%, followed by superclass 10 (functional group addition, 33.6%), superclass 9 (functional group interconversion, 29.6%), and superclass 3 (C–C bond formation, 27.8%). In absolute terms, superclasses 4 (heterocycle formation), 9 (FGI), and 3 (C–C bond formation) contribute the largest numbers of reactions to the error set, with 916, 829, and 758 reactions, respectively (Table 2).

Table 2. Number of mistakes per superclass for each model on the Pistachio test set (13′862 reactions). The error-set column gives the number of reactions mistaken by at least one model and the corresponding percentage within that superclass.

Class description Super class RXNMapper RXNMapper -v2 RXNMapper -P LocalMapper Error set [# (%)] Full test set
Heteroatom alkylation/arylation 1 187 173 166 274 386 (19.6) 1′974
Acylation related processes 2 185 176 152 198 298 (23.5) 1′269
C–C bond formation 3 449 360 371 576 758 (27.8) 2′727
Heterocycle formation 4 486 425 394 657 916 (36.7) 2′494
Protections 5 11 2 3 28 28 (23.3) 120
Deprotections 6 29 17 12 83 107 (14.4) 741
Reductions 7 39 37 45 68 102 (15.0) 679
Oxidations 8 127 112 96 153 221 (36.7) 603
Functional group interconversion 9 478 399 375 558 829 (29.6) 2′804
Functional group addition 10 58 39 37 65 93 (33.6) 277
Separation processes 11 13 22 14 26 42 (24.9) 169
Miscellaneous 12 4 5 4 5 5 (100) 5
Total 2′066 1′767 1′669 2′691 3′785 (27.3) 13′862
Open in a new tab

To further analyze model behavior, we examined where models make mistakes and how errors overlap across them. The error set comprises all reactions that have been wrongly predicted by at least one of the models. Table 3 summarizes the 3′785 reactions in the Pistachio error set. Exactly one model is wrong for 1′640 reactions, exactly two models for 706 reactions, exactly three models for 615 reactions, and all four models for 824 reactions. Among single-model errors, LocalMapper accounts for 1′097 reactions, compared with 277 for the original RXNMapper, 139 for RXNMapper-v2, and 127 for RXNMapper-P.

Table 3. Number of mistaken models on the Pistachio test set. Number of times each model belongs to a given error-overlap class.

Mistaken models ALBERT orig. ALBERT repr. ALBERT pistachio LocalMapper Error set
1 284 139 127 1′090 1′640
2 433 291 279 409 706
3 525 513 439 368 615
4 824 824 824 824 824
Total 2′066 1′767 1′669 2′691 3′785
Open in a new tab

The largest single-model error group consists of reactions missed only by LocalMapper (1′097 reactions), mainly from superclasses 4, 3, and 9. Among two-model errors, the most frequent pair is the original RXNMapper together with LocalMapper (206 reactions), again enriched in superclasses 4, 9, and 3. In the three-model error class, LocalMapper is the only correct model for 247 reactions, whereas RXNMapper-P is the only correct model for 176 reactions. The 824 reactions missed by all four models present over-represented superclasses 3, 4, and 9. Representative examples from these groups are shown in Fig. 4 and 5, where the difference between ground truth and mistaken predictions are illustrated (picked from the RXNFP TMAP of the error set, Fig. S2).

Fig. 4. Examples of the 3′785 reactions from the Pistachio test set for which at least one model gives an incorrect prediction. (a) Reactions missed only by LocalMapper.56 (b) Reactions missed only by the original RXNMapper and LocalMapper.57,58.

Fig. 4

Open in a new tab

Fig. 5. Examples of the 3′785 reactions from the Pistachio test set for which at least one model gives an incorrect prediction. (a) Reaction correctly mapped only by LocalMapper.59 (b) Reaction correctly mapped only by RXNMapper-P. (c) Reactions missed by all models.60,61.

Fig. 5

Open in a new tab

At the superclass level, the models differ mainly in the overall error rate rather than in a clear reaction-type specialization. The two updated RXNMapper models consistently make fewer mistakes than the original RXNMapper and LocalMapper, and RXNMapper-P gives the lowest total error count. The most informative superclasses are 3, 4, and 9 because they combine high frequency with high absolute numbers of errors, whereas superclasses 4 and 8 are the most error-prone relative to class size.

Table 3, and Fig. 4 and 5 show that the largest room for improvement lies in the reactions missed by all models. These 824 cases are less likely to reflect model-specific weaknesses and more likely to correspond to systematic challenges, such as rearrangements, migrations, multi-center transformations, or residual ambiguity in the reference mapping. At the same time, the TMAP does not reveal a clearly separated region of reaction space that is uniquely difficult for one model and easy for another. This suggests that, at the level of NameRXN superclasses, the main difference between models is robustness rather than specialization.

Table 1 demonstrates an increase in atom-mapping performance on average. However, no reaction class drastically improved from always being predicted wrong to only correct predictions. We hypothesize that this comes from the stochastic nature of the training and the lack of direct feedback during the unsupervised MLM training.

Together, Table 2, and Fig. 4 and 5 indicate that future improvements should focus on reaction classes that are both common and error prone. In the present dataset, these are primarily superclasses 3, 4, and 9.

Mechanistically challenging atom-mapping cases

To move beyond superclass-level trends, we examined representative reaction types that are known or expected to be difficult for atom-mapping algorithms. Fig. 6 shows selected examples from the Pistachio test set together with the alternative mappings predicted by the different models. We focused on rearrangements, for which the atom correspondence is often less intuitive, and on Mitsunobu ester synthesis, which is frequently associated with mapping errors in esterification chemistry.

Fig. 6. Representative atom-mapping errors on selected difficult reaction types from the Pistachio test set. (a–c) Rearrangement reactions, including Cope62 (a), oxy-Cope (b), and Robinson annulation63 (c) examples. (d) Mitsunobu ester synthesis.64 For each case, the reference mapping and alternative model predictions are shown, highlighting typical failure modes such as missing skeletal rearrangements or incorrect oxygen assignment.

Fig. 6

Open in a new tab

Fig. 6a–c shows representative rearrangements, including Cope, oxy-Cope, and Robinson annulation reactions. Across these examples, the dominant error is failure to capture the underlying skeletal reorganization. Instead, incorrect predictions tend to preserve local structural similarity and assign atom maps as if a substituent had migrated without rearrangement or as if the product double bond arose from the most similar alkene fragment in the reactants. This behavior is particularly apparent for Cope-type transformations, for which most examples are missed by several or all models. Oxy-Cope rearrangements appear somewhat better handled by the updated ALBERT-based models, but the same general failure mode remains. In Robinson annulation, typical errors assign the enone double bond by local similarity rather than by the actual annulation pattern.

Fig. 6d shows a representative Mitsunobu ester synthesis. In this case, the reproduced and Pistachio-trained RXNMapper models recover the reference mapping, whereas the original RXNMapper and LocalMapper assign the ester oxygen incorrectly. The error corresponds to treating the hydroxyl group of the carboxylic acid as the leaving group, a failure mode consistent with other esterification-like transformations in which multiple mechanistic possibilities can compete.

These examples show that the main remaining failures are not random but mechanistically structured. Rearrangements remain difficult because correct atom mapping requires recognition of a global reorganization of the carbon framework rather than simple local similarity matching. Mitsunobu ester synthesis highlights a second source of difficulty, namely competing oxygen assignments in transformations related to ester formation. In both cases, the errors often involve only a small number of exchanged atom labels, yet they correspond to a chemically distinct interpretation of the reaction. This suggests that future improvements will likely require training or evaluation strategies that better account for mechanistic context, especially for rearrangements and esterification-like processes.

Conclusion

We revisited the RXNMapper approach by addressing two limitations of the original workflow: unreliable validation data and evaluation by direct string matching. Using a curated validation set covering Pistachio chemical space, we retrained ALBERT-based RXNMapper models and optimized head, layer, and checkpoint selection under a common framework. We also developed a preprocessing and graph-based evaluation pipeline that filters ambiguous reference reactions and compares atom mappings at the level of reaction graph equivalence, thereby avoiding false negatives caused by inconsistent formatting or symmetry-equivalent mappings.

Across the golden dataset, the broad Pistachio benchmark, and an internal AstraZeneca medicinal chemistry set, the updated RXNMapper models consistently outperformed the original RXNMapper and LocalMapper. The results show that careful model selection on a representative validation set can improve atom-mapping performance more than retraining on a new reaction corpus alone. At the same time, performance remained strongly dependent on the benchmark, with the broad Pistachio reaction space proving substantially more challenging than the narrower medicinal chemistry set.

Superclass-level analysis and selected case studies further showed that current atom-mapping errors are not uniformly distributed. Frequent and error-prone classes such as C–C bond formation, heterocycle-forming reactions, and functional group interconversions contribute a large fraction of the remaining failures, while rearrangements and Mitsunobu ester synthesis illustrate persistent mechanistic challenges. These cases suggest that future gains will require models and evaluation schemes that better capture mechanistic context rather than local structural similarity alone.

Overall, this work provides both improved RXNMapper models and a more rigorous evaluation framework for atom mapping. We expect that the released training scripts, model-selection workflow, and benchmarking tools will support more reproducible development of atom-mapping algorithms and more reliable evaluation on chemically relevant reaction spaces.

Author contributions

YG designed and realized the project with SR and wrote the paper. AMW provided technical guidance. SG, PS and JLR designed and supervised the project. JLR wrote the paper. All authors reviewed and approved the final manuscript.

Conflicts of interest

The authors declare that they have no competing interests.

Supplementary Material

DD-OLF-D6DD00425C-s001

Acknowledgments

YG would like to thank Marcos E. González Laffitte for the useful discussion. This work was supported financially by the Swiss National Science Foundation, Grant no. 200020_207976. Part of the calculations were performed on UBELIX (http://www.id.unibe.ch/hpc), the HPC cluster at the University of Bern. The rest of calculations were performed on SCP, AstraZeneca's HPC cluster. SR acknowledges the support from the AiChemist project via MSCA Doctoral Network and SERI.

Data availability

The LocalMapper model and inference code are publicly available from the authors' GitHub repository https://github.com/snu-micc/LocalMapper. The updated atom-mapping and training scripts used in this work to retrain ALBERT-based RXNMapper models, the workflow used to select the best head/layer/checkpoint combination, and the evaluation framework for preprocessing and reaction-graph-based comparison of atom-mapping predictions are available at https://github.com/yvsgrndjn/RXNMapper_v2 and https://github.com/yvsgrndjn/atommap_eval. The updated RXNMapper-v2 model checkpoints are available on the GitHub and on HuggingFace (https://huggingface.co/yvsgrndjn/alberta-uspto-2800k). USPTO-full is publicly available and was used here in the version extracted from the United States Patent and Trademark Office reaction corpus by Lowe.49,50 USPTO-50 K and the golden dataset are available from their original sources.3,48 Pistachio is a proprietary dataset from NextMove Software and is therefore not publicly redistributed. The AstraZeneca medicinal chemistry benchmark used in this study is proprietary and cannot be shared publicly. The updated TMAP code is available on GitHub: https://github.com/afloresep/TMAP2.

Supplementary information (SI): Description of reaction graph isomorphism check, overview of Pistachio reaction dataset as TMAPs (Fig. S1 and S2), and examples of incorrect atom mapping predictions (Fig. S3). See DOI: https://doi.org/10.1039/d6dd00425c.

References

  1. Moock T. E., Nourse J. G., Grier D., Hounshell W. D., The Implementation of Atom-Atom Mapping and Related Features in the Reaction Access System (REACCS), in Chemical Structures, ed. W. A., Warr, Springer, Berlin, Heidelberg, 1988, pp. 303–313. 10.1007/978-3-642-73975-0_33 [DOI] [Google Scholar]
  2. Varnek A. Fourches D. Hoonakker F. Solov’ev V. P. Substructural Fragments: An Universal Language to Encode Reactions, Molecular and Supramolecular Structures. J. Comput.-Aided Mol. Des. 2005;19(9):693–703. doi: 10.1007/s10822-005-9008-0. [DOI] [PubMed] [Google Scholar]
  3. Schneider N. Stiefl N. Landrum G. A. What's What: The (Nearly) Definitive Guide to Reaction Role Assignment. J. Chem. Inf. Model. 2016;56(12):2336–2346. doi: 10.1021/acs.jcim.6b00564. [DOI] [PubMed] [Google Scholar]
  4. Chen S. Jung Y. A Generalized-Template-Based Graph Neural Network for Accurate Organic Reactivity Prediction. Nat. Mach. Intell. 2022;4(9):772–780. doi: 10.1038/s42256-022-00526-z. [DOI] [Google Scholar]
  5. Coley C. W. Green W. H. Jensen K. F. RDChiral: An RDKit Wrapper for Handling Stereochemistry in Retrosynthetic Template Extraction and Application. J. Chem. Inf. Model. 2019;59(6):2529–2537. doi: 10.1021/acs.jcim.9b00286. [DOI] [PubMed] [Google Scholar]
  6. Szymkuć S. Gajewska E. P. Klucznik T. Molga K. Dittwald P. Startek M. Bajczyk M. Grzybowski B. A. Computer-Assisted Synthetic Planning: The End of the Beginning. Angew. Chem., Int. Ed. 2016;55(20):5904–5937. doi: 10.1002/anie.201506101. [DOI] [PubMed] [Google Scholar]
  7. Segler M. H. S. Waller M. P. Neural-Symbolic Machine Learning for Retrosynthesis and Reaction Prediction. Chem. – Eur. J. 2017;23(25):5966–5971. doi: 10.1002/chem.201605499. [DOI] [PubMed] [Google Scholar]
  8. Genheden S. Thakkar A. Chadimová V. Reymond J.-L. Engkvist O. Bjerrum E. AiZynthFinder: A Fast, Robust and Flexible Open-Source Software for Retrosynthetic Planning. J. Cheminf. 2020;12(1):70. doi: 10.1186/s13321-020-00472-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Saigiridharan L. Hassen A. K. Lai H. Torren-Peraire P. Engkvist O. Genheden S. AiZynthFinder 4.0: Developments Based on Learnings from 3 Years of Industrial Application. J. Cheminf. 2024;16(1):57. doi: 10.1186/s13321-024-00860-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Dai H., Li C., Coley C. W., Dai B. and Song L., Retrosynthesis Prediction with Conditional Graph Logic Network, arXiv, 2020, preprint, arXiv:2001.01408, 10.48550/arXiv.2001.01408 [DOI]
  11. Fortunato M. E. Coley C. W. Barnes B. C. Jensen K. F. Data Augmentation and Pretraining for Template-Based Retrosynthetic Prediction in Computer-Aided Synthesis Planning. J. Chem. Inf. Model. 2020;60(7):3398–3407. doi: 10.1021/acs.jcim.0c00403. [DOI] [PubMed] [Google Scholar]
  12. Coley C. W. Barzilay R. Jaakkola T. S. Green W. H. Jensen K. F. Prediction of Organic Reaction Outcomes Using Machine Learning. ACS Cent. Sci. 2017;3(5):434–443. doi: 10.1021/acscentsci.7b00064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Wei J. N. Duvenaud D. Aspuru-Guzik A. Neural Networks for the Prediction of Organic Chemistry Reactions. ACS Cent. Sci. 2016;2(10):725–732. doi: 10.1021/acscentsci.6b00219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Westerlund A. M. Saigiridharan L. Genheden S. Human-Guided Synthesis Planning via Prompting. Chem. Sci. 2025;16(32):14655–14667. doi: 10.1039/D5SC00927H. https://dx.doi.org/10.1039/d5sc00927h [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Grandjean Y. Kreutter D. Reymond J.-L. Data Augmentation in a Triple Transformer Loop Retrosynthesis Model. Digit. Discov. 2026;5(2):653–661. doi: 10.1039/D5DD00465A. https://dx.doi.org/10.1039/d5dd00465a [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Thakkar A. Vaucher A. C. Byekwaso A. Schwaller P. Toniato A. Laino T. Unbiasing Retrosynthesis Language Models with Disconnection Prompts. ACS Cent. Sci. 2023;9(7):1488–1498. doi: 10.1021/acscentsci.3c00372. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Kreutter D. Reymond J.-L. Multistep Retrosynthesis Combining a Disconnection Aware Triple Transformer Loop with a Route Penalty Score Guided Tree Search. Chem. Sci. 2023;14(36):9959–9969. doi: 10.1039/D3SC01604H. https://dx.doi.org/10.1039/d3sc01604h [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kreutter D. Reymond J.-L. Chemoenzymatic Multistep Retrosynthesis with Transformer Loops. Chem. Sci. 2024;15(43):18031–18047. doi: 10.1039/D4SC02408G. https://dx.doi.org/10.1039/d4sc02408g [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Westerlund A. M., Saigiridharan L., Genheden S., Constrained Synthesis Planning with Disconnection-Aware Transformer and Multi-Objective Search, ChemRxiv, May 24, 2024, preprint, 10.26434/chemrxiv-2024-c77p4 [DOI]
  20. Ucak U. V. Ashyrmamatov I. Lee J. Improving the Quality of Chemical Language Model Outcomes with Atom-in-SMILES Tokenization. J. Cheminf. 2023;15(1):55. doi: 10.1186/s13321-023-00725-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Sulpizio G., Gerhaher C., Heid E., Jorner K., Bridging CGR Representations and Language Models for Reaction Property Prediction, March 16, 2026, 10.26434/chemrxiv.15000926/v1 [DOI] [Google Scholar]
  22. Nugmanov R. I. Mukhametgaleev R. N. Akhmetshin T. Gimadiev T. R. Afonina V. A. Madzhidov T. I. Varnek A. CGRtools: Python Library for Molecule, Reaction, and Condensed Graph of Reaction Processing. J. Chem. Inf. Model. 2019;59(6):2516–2521. doi: 10.1021/acs.jcim.9b00102. [DOI] [PubMed] [Google Scholar]
  23. Lin A. I. Madzhidov T. I. Klimchuk O. Nugmanov R. I. Antipin I. S. Varnek A. Automatized Assessment of Protective Group Reactivity: A Step Toward Big Reaction Data Analysis. J. Chem. Inf. Model. 2016;56(11):2140–2148. doi: 10.1021/acs.jcim.6b00319. [DOI] [PubMed] [Google Scholar]
  24. Bort W. Baskin I. I. Gimadiev T. Mukanov A. Nugmanov R. Sidorov P. Marcou G. Horvath D. Klimchuk O. Madzhidov T. Varnek A. Discovery of Novel Chemical Reactions by Deep Generative Recurrent Neural Network. Sci. Rep. 2021;11(1):3178. doi: 10.1038/s41598-021-81889-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Koda S., Saito S., General and Scalable Atom-to-Atom Mapping via Weisfeiler–Lehman-like Approximate Graph Matching, ChemRxiv, November 25, 2025, preprint, 10.26434/chemrxiv-2025-hthwn [DOI]
  26. NameRXN, 2026, https://www.nextmovesoftware.com/namerxn.html, accessed 2026-03-23
  27. Latendresse M. Malerich J. P. Travers M. Karp P. D. Accurate Atom-Mapping Computation for Biochemical Reactions. J. Chem. Inf. Model. 2012;52(11):2970–2982. doi: 10.1021/ci3002217. [DOI] [PubMed] [Google Scholar]
  28. Lynch M. F. Willett P. The Automatic Detection of Chemical Reaction Sites. J. Chem. Inf. Comput. Sci. 1978;18(3):154–159. doi: 10.1021/ci60015a009. [DOI] [Google Scholar]
  29. Funatsu K. Endo T. Kotera N. Sasaki S.-I. Automatic Recognition of Reaction Site in Organic Chemical Reactions. Tetrahedron Comput. Methodol. 1988;1(1):53–69. doi: 10.1016/0898-5529(88)90008-5. [DOI] [Google Scholar]
  30. AutoMapper User's Guide - Documentation, https://docs.chemaxon.com/latest/automapper_user-guide.html, accessed 2026-03-26
  31. Akutsu T. Efficient Extraction of Mapping Rules of Atoms from Enzymatic Reaction Data. J. Comput. Biol. 2004;11(2–3):449–462. doi: 10.1089/1066527041410337. [DOI] [PubMed] [Google Scholar]
  32. McGregor J. J. Willett P. Use of a Maximum Common Subgraph Algorithm in the Automatic Identification of Ostensible Bond Changes Occurring in Chemical Reactions. J. Chem. Inf. Comput. Sci. 1981;21(3):137–140. doi: 10.1021/ci00031a005. [DOI] [Google Scholar]
  33. Wang T. Zhou J. EMCSS: A New Method for Maximal Common Substructure Search. J. Chem. Inf. Comput. Sci. 1997;37(5):828–834. doi: 10.1021/ci9601675. [DOI] [Google Scholar]
  34. Körner R. Apostolakis J. Automatic Determination of Reaction Mappings and Reaction Center Information. 1. The Imaginary Transition State Energy Approach. J. Chem. Inf. Model. 2008;48(6):1181–1189. doi: 10.1021/ci7004324. [DOI] [PubMed] [Google Scholar]
  35. Apostolakis J. Sacher O. Körner R. Gasteiger J. Automatic Determination of Reaction Mappings and Reaction Center Information. 2. Validation on a Biochemical Reaction Database. J. Chem. Inf. Model. 2008;48(6):1190–1198. doi: 10.1021/ci700433d. [DOI] [PubMed] [Google Scholar]
  36. Rahman S. A. Torrance G. Baldacci L. Martínez Cuesta S. Fenninger F. Gopal N. Choudhary S. May J. W. Holliday G. L. Steinbeck C. Thornton J. M. Reaction Decoder Tool (RDT): Extracting Features from Chemical Reactions. Bioinformatics. 2016;32(13):2065–2066. doi: 10.1093/bioinformatics/btw096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. First E. L. Gounaris C. E. Floudas C. A. Stereochemically Consistent Reaction Mapping and Identification of Multiple Reaction Mechanisms through Integer Linear Optimization. J. Chem. Inf. Model. 2012;52(1):84–92. doi: 10.1021/ci200351b. [DOI] [PubMed] [Google Scholar]
  38. Jochum C. Gasteiger J. Ugi I. The Principle of Minimum Chemical Distance (PMCD) Angew Chem. Int. Ed. Engl. 1980;19(7):495–505. doi: 10.1002/anie.198004953. [DOI] [Google Scholar]
  39. Jaworski W. Szymkuć S. Mikulak-Klucznik B. Piecuch K. Klucznik T. Kaźmierowski M. Rydzewski J. Gambin A. Grzybowski B. A. Automatic Mapping of Atoms across Both Simple and Complex Chemical Reactions. Nat. Commun. 2019;10(1):1434. doi: 10.1038/s41467-019-09440-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Indigo Toolkit, https://lifescience.opensource.epam.com/indigo/index.html, accessed 2026-03-26
  41. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser Ł. ukasz, Polosukhin I., Attention Is All You Need, in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2017, vol. 30 [Google Scholar]
  42. Devlin J., Chang M.-W., Lee K. and Toutanova K., BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding, arXiv, 2019, preprint, arXiv:1810.04805, 10.48550/arXiv.1810.04805 [DOI]
  43. Lan Z., Chen M., Goodman S., Gimpel K., Sharma P. and Soricut R., ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations, arXiv, 2020, preprint, arXiv:1909.11942, 10.48550/arXiv.1909.11942 [DOI]
  44. Schwaller P. Hoover B. Reymond J.-L. Strobelt H. Laino T. Extraction of Organic Chemistry Grammar from Unsupervised Learning of Chemical Reactions. Sci. Adv. 2021;7(15):eabe4166. doi: 10.1126/sciadv.abe4166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Nugmanov R. Dyubankova N. Gedich A. Wegner J. K. Bidirectional Graphormer for Reactivity Understanding: Neural Network Trained to Reaction Atom-to-Atom Mapping Task. J. Chem. Inf. Model. 2022;62(14):3307–3315. doi: 10.1021/acs.jcim.2c00344. [DOI] [PubMed] [Google Scholar]
  46. Chen S. An S. Babazade R. Jung Y. Precise Atom-to-Atom Mapping for Organic Reactions via Human-in-the-Loop Machine Learning. Nat. Commun. 2024;15(1):2250. doi: 10.1038/s41467-024-46364-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Astero M. Rousu J. Learning Symmetry-Aware Atom Mapping in Chemical Reactions through Deep Graph Matching. J. Cheminf. 2024;16(1):46. doi: 10.1186/s13321-024-00841-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Lin A. Dyubankova N. Madzhidov T. I. Nugmanov R. I. Verhoeven J. Gimadiev T. R. Afonina V. A. Ibragimova Z. Rakhimbekova A. Sidorov P. Gedich A. Suleymanov R. Mukhametgaleev R. Wegner J. Ceulemans H. Varnek A. Atom-to-Atom Mapping: A Benchmarking Study of Popular Mapping Algorithms and Consensus Strategies. Mol. Inform. 2022;41(4):2100138. doi: 10.1002/minf.202100138. [DOI] [PubMed] [Google Scholar]
  49. Lowe D., Chemical Reactions from US Patents (1976-Sep2016), 2017, 10.6084/m9.figshare.5104873.v1 [DOI] [Google Scholar]
  50. Lowe D. M., Extraction of Chemical Structures and Reactions from the Literature, 2012 [Google Scholar]
  51. NextMove S., Pistachio, 2026, https://www.nextmovesoftware.com/pistachio, accessed 2026-03-23
  52. Cordella L. P. Foggia P. Sansone C. Vento M. A (Sub)Graph Isomorphism Algorithm for Matching Large Graphs. IEEE Trans. Pattern Anal. Mach. Intell. 2004;26(10):1367–1372. doi: 10.1109/TPAMI.2004.75. [DOI] [PubMed] [Google Scholar]
  53. Hagberg A. A., Schult D. A., Swart P. J., Exploring Network Structure, Dynamics, and Function Using NetworkX, Pasadena, California, 2008, pp. 11–15, 10.25080/tcwv9851 [DOI] [Google Scholar]
  54. Probst D. Reymond J.-L. Visualization of Very Large High-Dimensional Data Sets as Minimum Spanning Trees. J. Cheminf. 2020;12(1):12. doi: 10.1186/s13321-020-0416-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Schwaller P. Probst D. Vaucher A. C. Nair V. H. Kreutter D. Laino T. Reymond J.-L. Mapping the Space of Chemical Reactions Using Attention-Based Neural Networks. Nat. Mach. Intell. 2021;3(2):144–152. doi: 10.1038/s42256-020-00284-w. [DOI] [Google Scholar]
  56. Lombaert S. D., Composés destinés à être utilisés dans le traitement de troubles neurologiques. WO2023107722A1, June 15, 2023, https://patents.google.com/patent/WO2023107722A1/fr, accessed 2026-05-12
  57. Boulet S. L., Johansson A. M., Smith S. M., Pharmaceutical Compounds. WO2005092835A1, October 6, 2005, https://patents.google.com/patent/WO2005092835A1/en?oq=WO2005092835A1, accessed 2026-05-12
  58. Matsumoto M., Watanabe N., Process for Preparing 4-Oxo-4,5,6,7-Tetrahydrobenzofuran Derivative. EP0101003B2, January 8, 1992, https://patents.google.com/patent/EP0101003B2/en?oq=EP0101003B2, accessed 2026-05-12
  59. Crew A. P., Werner D. S., Tavares P. A. R., UNSATURATED mTOR INHIBITORS. WO2007087395A2, August 2, 2007, https://patents.google.com/patent/WO2007087395A2/en?oq=WO2007087395A2, accessed 2026-05-12
  60. Berthel S. J., Brinkman J. A., Hayden S., Haynes N.-E., Kester R. F., Mcdermott L. A., Qian Y., Sarabu R., Scott N. R., Tilley J. W., Pyrrolidinone Glucokinase Activators. EP2274297B1, May 9, 2012, https://patents.google.com/patent/EP2274297B1/en?oq=EP2274297B1, accessed 2026-05-12
  61. Mathews C. J., Robinson L., Jeanmart S. A. M., Novel Herbicides. US20120065066A1, March 15, 2012, https://patents.google.com/patent/US20120065066A1/en?oq=US20120065066A1, accessed 2026-05-12
  62. GRENNING A. J., Methods and Compositions for Terpenoid Synthesis. WO2018053322A1, March 22, 2018, https://patents.google.com/patent/WO2018053322A1/en?oq=WO2018053322A1, accessed 2026-05-12
  63. Krause J., Coated Parts and Use Thereof. WO2011161047A1, December 29, 2011, https://patents.google.com/patent/WO2011161047A1/en, accessed 2026-05-12
  64. Hagiya K., Sugimura T., Bis(2-Alkoxyethyl) Azodicarboxylate Ester Compound and Intermediate for Production of the Same. EP2112136B1, May 23, 2012, https://patents.google.com/patent/EP2112136B1/en?oq=EP2112136B1, accessed 2026-05-12

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

DD-OLF-D6DD00425C-s001
DD-OLF-D6DD00425C-s001.pdf (855.8KB, pdf)

Data Availability Statement

The LocalMapper model and inference code are publicly available from the authors' GitHub repository https://github.com/snu-micc/LocalMapper. The updated atom-mapping and training scripts used in this work to retrain ALBERT-based RXNMapper models, the workflow used to select the best head/layer/checkpoint combination, and the evaluation framework for preprocessing and reaction-graph-based comparison of atom-mapping predictions are available at https://github.com/yvsgrndjn/RXNMapper_v2 and https://github.com/yvsgrndjn/atommap_eval. The updated RXNMapper-v2 model checkpoints are available on the GitHub and on HuggingFace (https://huggingface.co/yvsgrndjn/alberta-uspto-2800k). USPTO-full is publicly available and was used here in the version extracted from the United States Patent and Trademark Office reaction corpus by Lowe.49,50 USPTO-50 K and the golden dataset are available from their original sources.3,48 Pistachio is a proprietary dataset from NextMove Software and is therefore not publicly redistributed. The AstraZeneca medicinal chemistry benchmark used in this study is proprietary and cannot be shared publicly. The updated TMAP code is available on GitHub: https://github.com/afloresep/TMAP2.

Supplementary information (SI): Description of reaction graph isomorphism check, overview of Pistachio reaction dataset as TMAPs (Fig. S1 and S2), and examples of incorrect atom mapping predictions (Fig. S3). See DOI: https://doi.org/10.1039/d6dd00425c.

Articles from Digital Discovery are provided here courtesy of Royal Society of Chemistry

RESOURCES