PIMENTO: a primer inference toolkit to facilitate large-scale calling of amplicon sequence variants

Abstract

The identification of amplicon sequence variants from DNA metabarcoding data is a common method for revealing the taxonomic makeup of environmental samples, and for allowing comparative studies between similar datasets. A significant hurdle to the large-scale calling of amplicon sequence variants from publicly available nucleotide datasets is the heterogeneous presence of primer sequences in reads, the removal of which is a necessary pre-processing step for this form of analysis. Furthermore, as the details of the experimental primers are rarely captured in the metadata associated with the sequence records, there is a need for a method that can automatically infer the presence and identity of primers in sequencing data. In this work, we introduce the PrIMER infereNce TOolkit (PIMENTO), a Python package that uses a dual-strategy approach for identifying primers that are present in sequencing reads to enable their removal, and therefore facilitate amplicon sequence variant calling at scale.

Introduction

DNA metabarcoding is a molecular technique that targets a diagnostic marker gene and obtains genetic sequences that can be assigned to a taxonomy. This method is typically applied to genetic material present in environmental samples termed environmental DNA (eDNA), which is increasingly being applied to elucidate the organisms present in an ecosystem [1, 2]. Depending on the taxonomic group of interest in a metabarcoding experiment, there are different marker genes that are typically chosen for amplification and DNA sequencing. For example, the mitochondrial cytochrome C oxidase subunit I (COI) gene is commonly used to resolve eDNA from macro-organisms like insects, while the ribosomal RNA (rRNA) gene is a typical marker used for microbial diversity resolution, including bacteria and archaea, but even some eukaryotes like fish, fungi, and plankton. The use of metabarcoding experiments to reveal the taxonomic composition of samples is popular due to cost-effectiveness, standard protocols, and ease of use.

One common approach for the analysis of metabarcoding sequence data is the clustering of reads into high sequence identity groups, termed operational taxonomic units (OTUs). These OTUs can then be taxonomically annotated by comparing to marker-gene-specific databases (Table 1), a process termed closed-reference OTU picking. Closed-reference OTUs therefore help describe taxonomic diversity within a sample, and can be compared across samples when used with a consistent reference database. However, the OTU clustering can easily hide diversity at the lowest taxonomic ranks like species, including unknown organisms that might be of interest. Another approach for the analysis of metabarcoding data, which is growing in popularity, is the calling of amplicon sequence variants (ASVs). ASVs have various benefits compared to OTUs, including more precise resolution down to single-nucleotide differences, and more reliable cross-study comparisons [10]; however, comparisons can only be made between experiments amplifying the same marker and utilizing a common informatics pipeline.

Table 1.

Reference databases commonly used in metabarcoding analysis, separated by marker gene.

Reference database	Marker genes
SILVA [3]	rRNA—SSU and LSU
PR2 [4]	rRNA—SSU (Protist focused)
Greengenes [5]	rRNA—SSU (16S only)
UNITE [6]	rRNA—ITS1 and ITS2
MIDORI [7]	Mitochondrial markers (COI, 12S)
MitoFish [8]	Mitochondrial markers (COI, 12S)
BOLD [9]	COI

Like many bioinformatics methods that handle sequencing reads, ASV calling necessitates a degree of quality control (QC) on the reads [11]. In addition to common read QC steps like quality filtering, the most widely used tools for generating ASVs, including DADA2 [12] and deblur [13], have an additional strict requirement: the removal of synthetic nucleotides, particularly those coming from amplification primers. Synthetic nucleotides, especially ambiguous bases of primers, are likely to result in high numbers of chimeric ASVs being called, which are made up of 2 or more abundant “parent” ASVs. As chimeric ASVs are typically filtered out in downstream analyses, it is possible to lose the majority of the data to this filtering step if primer sequences are included in the initial ASV calling, making precise primer trimming necessary for a robust ASV analysis.

Performing primer removal on an individual study basis is usually trivial: the chosen primer sequences are normally known to the scientist conducting the experiment, which are passed to tools like Cutadapt [14] for primer removal. However, in larger-scale meta-analyses, where multiple studies may be considered in a single analysis, or where publicly available data from heterogeneous sources is being analysed, it can be more complicated: individual studies might use multiple different primer pairs, and primers may or may not have been removed from sequence reads [15]. Also, within sequence records in databases like the European Nucleotide Archive (ENA) [16], the experimental metadata attached to a sequence record often does not contain information about used primers [15]. Such variation in metadata availability is problematic for analysis services such as MGnify [17], where automation of analysis pipelines is necessary. In this case, applying the ASV analysis to vast collections of diverse data that are deposited in sequence archives such as ENA, from large studies (like Tara Oceans [18] and the Earth Microbiome Project [19]) to smaller but more specific studies, can prove more challenging.

Consequently, there is a need for a method to perform automated inference of primer sequences for the purpose of large-scale ASV calling. To this end, we introduce a tool called PIMENTO that implements a dual-strategy method for addressing this important need: it first searches a set of amplicon reads for a predefined list of standard primers that are commonly used in metabarcoding experiments, and if a match is not found a de novo prediction for the presence of primers not in the list is performed.

Method

PIMENTO is a Python command-line tool for performing primer inference on sequencing read files. The source code is available on GitHub [20], and is packaged and released on both PyPI and Bioconda for ease of distribution. PIMENTO’s dual primer-search strategy (Fig. 1) is as follows:

Figure 1.

Schematic workflow of the PIMENTO tool. PIMENTO employs a dual primer-search strategy, with the standard library primer search method used first. If primers are found, they can be trimmed from the input reads using a tool like Cutadapt [14]. If no primers are matched, the general primer search detects if potential primers are present, then the primer cutoff prediction method is used to determine a consensus sequence by inferring a cutoff point for the primer to be trimmed.

• Standard primer search: fuzzy regular expression search queries against a curated library of standard primer sequences.

• Primer cutoff prediction: identification of the primer cutoff point based on analysis of patterns of base conservation at the beginning (and end, for single-end libraries) of reads.

Performance benchmarking for PIMENTO’s different functions shows high scalability in its implementation (S1, S2, S3). Below, we explain these 2 strategies in detail.

Standard primer search strategy

This first strategy utilizes a manually curated library of standard primers for metabarcoding of the rRNA gene. Typically, specific regions of the rRNA gene are targeted for amplification, including 16S rRNA for both bacteria and archaea, and 18S rRNA and the internal transcribed spacer (ITS1 and ITS2) for eukarya. This library was curated from a survey of the literature, sequencing platform websites, and other resources. The library currently contains 110 primers (names and sequences), of which 55 are forward strand direction (5′ to 3′) and 55 are reverse strand direction (3′ to 5′).

The strategy takes every primer sequence as a query, and performs a search against the first 50 bases of each read, the region where untrimmed primers are expected to reside. The search is fuzzy, allowing for at most 2 mismatches, to model for sequencing errors, and uses regular expressions, to model for ambiguous bases in some primers. The proportion of reads in which a primer is found is recorded, and any primers with a read proportion lower than a default threshold of 0.60 are disregarded. Once all primers have been searched against the reads, the primer pair with the highest proportion is identified. The default parameters for this step are optimized for the identification of a single pair of primers, but the user can modify this threshold to find multiple sets. However, changing this threshold does not assert the primer pairing, and as the threshold is lowered, the users should take more care in verifying the results.

Corresponding PIMENTO command:  pimento std <args>

Primer cutoff prediction strategy

The second strategy is based on the analysis of base-conservation patterns at the start of reads to predict the “cutoff point” of the primer, i.e., where the primer sequence ends and the biological sequence begins. Specifically, it is expected that bases in the “primer space” will have high levels of conservation if the primer has not been removed, as metabarcoding primers are chosen for sequence conservation by design. Complementarily, bases in the immediate “post-primer space” are typically less conserved, as the natural sequence diversity of the organisms in the sample starts revealing itself. However, identifying the cutoff point between the “primer space” and “post-primer space” is complex, as the bases immediately after a primer can still lie in the conserved regions of amplicons like the rRNA gene. Another difficulty is caused by ambiguous bases near the end of a primer sequence, as the patterns of conservation can look similar to a true cutoff point. However, application of the method has demonstrated that the heuristic does select cutoff points that are effective in trimming reads of existing primer sequences.

This approach has one optional but recommended step, and 3 required steps, all of which use a common concept of “base-conservation,” which we define here. First, a window starting at the ith base of the sequence and of size n bases is defined, the value of which will vary depending on the step of the method. Second, at each index j of this window, the proportion p of the most commonly occurring base between A/C/T/G is computed. Finally, the mean base proportion of all the indices of the window is calculated. This mean value will represent the base-conservation BC starting at base i of the window of size n (Equation 1).

Equation 1: The base-conservation BC of the window starting at base i and of size n is defined as the mean proportion p of the most commonly occurring base at each index j.

Optional preliminary step

Before running the primer cutoff prediction method, it is optional but recommended to use one of PIMENTO’s utilities to flag as true or false whether there are primers present in the reads. Using base-conservation patterns, this utility splits the first 100 bases of reads into 10 different base-conservation windows of size 10 bases. By comparing the base-conservation of the first 2 base-pair windows, to the other 8 windows, the tool will declare a primer as existing if the bases in those initial 2 windows are more conserved, based on the sample-specific “average” conservation across all windows.

Corresponding PIMENTO command: pimento are_there_primers <args>

• Step 1. The first step of the primer cutoff prediction method slides a base-conservation window of size 5 across the first 25 indices. The sequence of base-conservation values computed by sliding this window represents a “base-conservation vector” (BCV) for the first 25 bases of the reads.

Corresponding PIMENTO command: pimento generate_bcv <args>

• Step 2. In the second step, the negative inflection points of this BCV are computed. Every identified negative inflection point, i.e., a point at which the base conservation drops, is considered a potential primer cutoff point that warrants further investigation in the next step.

Corresponding PIMENTO command: pimento find_cutoffs <args>

• Step 3. In the third and final step, the “best” cutoff point is chosen out of the original set of inflection points. This decision is based on the difference between base-conservation windows before and after a point—the inflection point that has the largest difference is greedily chosen as the primer cutoff point. As a final output of this process, a consensus sequence is generated based on the previously selected cutoff point. This consensus sequence can then be used downstream to remove the primer from the reads with a tool like Cutadapt.

Corresponding PIMENTO command: pimento choose_primer_cutoffs <args>

For ease of use, it is possible to run the 3 steps of the primer cutoff inference strategy in just one PIMENTO command: pimento auto <args>

Results and discussion

To assess the primer inference capabilities of PIMENTO, 855 paired-end, marine microbiome, amplicon sequencing runs were randomly selected and retrieved from the ENA, comprising what we will call the marine survey dataset (MSD) hereafter. The MSD was built by first querying the ENA Browser API with the following query:

tax_eq(408172) AND library_strategy=“AMPLICON” AND library_source=“METAGENOMIC” AND library_layout=“PAIRED”

From the result of this query, 855 different studies were randomly chosen, and a random paired-end sequencing run was selected from each study. Thus, the MSD is composed of the 855 pairs of FASTQ files of these selected runs. Both the PIMENTO standard primer search and primer cutoff prediction strategies were then run on the MSD, generating primer inference results for all runs.

As popular choices of primers in metabarcoding experiments depend on the biome being studied, 2 additional, but smaller biome-specific datasets were built and tested the same way to assess PIMENTO’s consistency across different sampling environments. Specifically, a soil survey dataset (SSD) containing 79 random runs was built from ENA, swapping the “tax_eq” value for “410658.” Finally, a human gut survey dataset (HGSD) containing 78 runs was built from ENA, swapping the “tax_eq” value for “408170.”

To assess PIMENTO’s performance on long-read data, a benchmark survey was undertaken on studies comprising Oxford Nanopore Technologies (ONT) long reads, which also demonstrated PIMENTO’s applicability on ONT data (see S4 in supplementary data).

Primer inference outcomes

The standard primer search strategy and the optional preliminary step of the primer cutoff prediction strategy were applied to all runs of the MSD. The results are summarized in Fig. 2. For both forward and reverse FASTQ files, flags were assigned depending on the outcome of each strategy.

Figure 2.

Bar plot showing the distribution of primer inference outcomes for the 855 runs of the MSD. The outcomes are classified into high and low confidence depending on the likely accuracy of the outcome category, with 80.9% of the outcomes considered high confidence.

• “S”—if a standard primer was found in a direction.

• “s”—if no standard primer was found in a direction.

• “G”—if a primer was predicted by the cutoff prediction in a direction.

• “g”—if no primer was predicted by the cutoff prediction in a direction.

For example, if the presence of primers was found to be likely in both directions by step 1 of the primer cutoff prediction strategy, and standard primers were found in both directions, the flag “SSGG” was assigned. If only standard primers were found, but the pipeline could not detect the presence of a primer the flag would be “SSgg.”

We assessed the different combinations of flags as high and low confidence of being accurate about the presence of primers based on the following interpretations. As the standard primer search strategy is more reliable, outcomes where standard primers were found on both strands were considered high confidence. Cases where neither strategies found primers (i.e., ssgg) were also considered high-confidence outcomes as both methods indicated a negative presence of primers. All remaining outcomes are considered to be low confidence, as they mostly include cases with negative standard primer presence in combinations that are harder to interpret.

As can be seen in Fig. 2, of the 855 runs, 692 were assessed to be high-confidence outcomes (80.9%). This number includes 447 runs which were unanimously found to have primers by both methods (i.e., SSGG), while 201 runs (23%) have no primers (i.e., ssgg), indicating that the primers have been removed. Such a low proportion of the runs being confidently primer-free illustrates the problem and shows the current necessity for a tool like PIMENTO.

Low-confidence outcomes make up 18.9% of the runs (162 out of 855), with almost half of those (80 out of 162) being from runs where the presence of primers was deemed likely in both directions, but with no matches to standard primers (i.e., ssGG). There are multiple interpretations possible for these cases: the first one—and probably the most likely—is that primers do indeed exist but are not in the standard primer library. For this scenario, the cutoff prediction method will predict an inferred primer to be trimmed, which the method does with high accuracy (see examples in the “Consistency across biomes” section). Another possibility is that some runs will be false positives, i.e., they have been flagged for primer inference even though there are no primers, which is not implausible due to the use of the heuristic method. Strategies for assessing potential false positives through primer validation are discussed in the “Retrieval of automatically inferred primers from literature” section.

Standard primer inferences

The results in this section pertain to runs with outcomes that were positive for standard primers in both directions (i.e., every SS outcome), which total 491 runs. In Fig. 3, the distributions of primer presence proportions, in both forward and reverse directions, can be seen. These proportions show that some runs had matching primers as low as 0.61, which still require trimming despite being lower value outliers. Figure 3 also shows the fraction of reads the matched primers were found in, with the first quartiles computing to 0.98 and 0.97, for the forward and reverse directions, respectively. These high values show that for the majority of runs, matched standard primers were present in the vast majority of reads, which, after trimming, would lead to cleaner reads for ASV calling purposes.

Figure 3.

Distribution of the primer presence proportions in the MSD for matched standard primers for both 5′ to 3′ (F) and 3′ to 5′ (R) directions. The vast majority of the runs had primer proportions in the reads as high as 0.97, with a minority of outliers reaching a proportion as low as 0.61.

Figure 4 plots the counts for different amplicon regions that were identified by proxy of their standard primer identity. The most common regions are 16S hypervariable regions, which is expected as the most common form of amplicon sequencing. However, the plot also shows the breadth of regions covered by the standard primer library, with occurrences of 18S, ITS, and even Archaea-specific 16S. Figure 5 shows the actual matched primers rather than the regions they amplify. The most common primers tend to be for 16S amplification, but there is a wide range of primers being represented in the MSD.

Figure 4.

Distribution of identified amplified regions in the MSD. The most common amplified regions were 16S, but matches to other regions like 18S and ITS were also present.

Figure 5.

Distribution of identified standard primers in the MSD for both 5′ to 3′ (F) and 3′ to 5′ (R) directions. As the most common amplified region was 16S (Fig. 4), the most common standard primers were also for 16S.

Consistency across biomes

The SSD and HGSD datasets were analysed by PIMENTO to assess the consistency of performance across different biomes. In Figs 6 and 7, the distribution of outcomes of the standard primer search strategy and the optional preliminary step of the primer cutoff prediction strategy for the SSD and HGSD can be seen, respectively, using the same interpretation of high- and low-confidence outcomes as used in the “Primer inference outcomes” section.

Figure 6.

Bar plot showing the distribution of primer inference outcomes for the 79 runs of the SSD. The outcomes are classified into high and low confidence as before, with 81.0% of the outcomes considered high confidence.

Figure 7.

Bar plot showing the distribution of primer inference outcomes for the 78 runs of the HGSD. The outcomes are classified into high and low confidence as before, with 67.9% of the outcomes considered high confidence.

Of the 79 soil runs, 64 were assessed to be high-confidence outcomes (81.0%) and 15 were low-confidence outcomes (18.9%). These results are very similar to those of the marine dataset, showing consistency across these 2 biomes. Meanwhile, 53 of the 78 human gut runs were high-confidence outcomes (67.9%) and 25 were low-confidence outcomes (32%). While these results are still mostly made up of high-confidence outcomes, the low-confidence outcomes are approximately twice as common in the human gut dataset compared to both the marine and soil datasets. A deeper investigation of these low-confidence cases confirmed they are runs where the primers were already removed, but with highly conserved bases at the beginning of reads, leading to false positive primer predictions. This highlights the need to validate the primer predictions where possible. Some strategies for validating primers are discussed in the “Primer validation of automatically inferred primers” section.

Primer cutoff prediction inferences

The primer cutoff prediction inference method was developed as a secondary strategy for cases where primers are not present in the standard primer library bundled with PIMENTO. While it is less reliable than the standard primer method due to its heuristic nature, it is applicable to any type of metabarcoding study, and to any type of sequenced amplicon, from rRNA to COI. When a primer is inferred in this way, the end-product is a consensus sequence from the beginning of reads to some chosen optimal cutoff point, and will therefore naturally fit the data it was generated from.

To establish the method’s accuracy in predicting real primers, 84 runs from the SSD and HGSD in which standard primers were identified in both orientations were collected. The primer cutoff prediction method was used on these 84 runs, generating a total of 168 automatically inferred primers. These inferred primers were then aligned pairwise to their respective standard primers using the Needleman–Wunsch [21] global alignment tool.

If the primer cutoff prediction method is reliable, we would expect the inferred primers to align well to their standard primer counterparts. Figure 8 shows the distribution of 2 pairwise alignment metrics: standard primer-based alignment percent identity, which is calculated by dividing the number of matches by the length of the standard primer, and full alignment percent identity, which is calculated by dividing the number of matches by the length of the alignment, including gaps. For the former, the median percent identity is 82.8% for forward primers and 85.0% for reverse primers, while for the latter, the percent identity is lower at 69.5% for forward primers and 70.8% for reverse primers. The high percent identity when based on the standard primer lengths shows a high level of precision in the regeneration of real primer sequences by the primer cutoff prediction method. However, the full alignment percent identity distribution indicates the presence of gaps, and, therefore, the presence of extra and/or missing bases in the predicted primers.

Figure 8.

Distribution of 2 alignment percent identity metrics between sequences identified by the standard primer search strategy and the primer cutoff prediction method by global pairwise sequence alignment. The standard primer-based percent identity metric is calculated by dividing the number of matches by the length of the standard primer, and the full alignment-based percent identity is calculated by dividing the number of matches by the length of the alignment, including gaps. The former shows that the primer cutoff prediction method regenerates standard primers at a high level of identity, while the latter shows that the alignments do have gaps, and therefore extra and missing bases.

Looking closely at 3 specific examples in Table 2, we can make some assumptions about why the alignment-wide percent identities tend to be lower. For one of the primer pairs of SRR12095443, the alignment percent identity is very high at 95.2%, with only one mismatch. However, the mismatch is due to the predicted primer having the ambiguous base W, which stands for either A or T, which, while less precise, is not a true mismatch.

Table 2.

Comparison of sequences identified by the standard primer search strategy and the primer cutoff prediction method by global pairwise sequence alignment.

Run	Primer alignment	Alignment % id
SRR12095443		95.2%
SRR12580015		65.2%
SRR21642156		All reads: 14.2% Reads starting with primer: 95.5%

The alignment percent identity is calculated by dividing the number of matching bases by the length of the alignment, gaps included. These 3 examples show a high level of precision in regenerating standard primers by the cutoff prediction method, although gaps are present due to missing or extra bases. The final example shows the importance of quality control before running PIMENTO to remove additional artificial bases that were added for sequencing.

For one of the SRR12580015 primer pairs, the alignment percent identity is slightly lower than the median at 65.2%. There are 2 potential reasons for the lower quality of this alignment. The first is that the predicted primer is 2 bases shorter than the standard primer, creating a gap at the end. This is due to the heuristic nature of the primer cutoff prediction method resulting in a cutoff point 2 bases shorter than expected. The second, larger issue is the presence of 5 bases at the beginning of the predicted primer that do not exist in the standard primer, 4 of which are the ambiguous base H (A, C, or T), creating a gap. Given that the inferred primer is a consensus sequence generated from the input reads, these bases do actually exist in the reads, and are not spurious. One possibility is that these extra nucleotides are other synthetic bases like primer-tags, adapter sequences, flow-cell binding sequences, or multiplex barcodes, which would need to be trimmed for ASV generation, so their presence in the inferred primer is not problematic. All other bases are identical except for one true mismatch from C to G, which could be explained by the use of a slightly different primer than the standard one.

The final example, which is one of the primer pairs of SRR21642156, had a very low alignment percent identity of just 14.2%—there are only 3 identical matches, and a sizeable 7-base-pair gap due to missing bases at the end of the predicted primer. However, the predicted primer is mostly made up of ambiguous bases, all of which are in agreement with the standard primer bases, e.g., the first G being replaced with a D, which stands for either A, G, or T. Investigating the start of these reads, the reason is likely due to extra bases of variable lengths at the beginning of most reads. Out of 49,990 reads, only 14,323 start with the standard primer, 14,151 start with TG, 11,257 start with AC, 7,780 start with CTA, and 7 start with just T.

The standard primer method handles reads like these, since primers are searched for anywhere at the start of reads. However, this experiment does highlight a limitation of the primer cutoff prediction method, in that it generates primers through the consensus of reads once a cutoff point has been selected, which is adversely affected by varying starts to reads. Because the pre-primer length of these reads is variable, the base distributions of the primer are not in register, leading to ambiguous bases at every position. Nonetheless, if these extra bases are trimmed beforehand, which they would be at the QC stage of most bioinformatics pipelines, the primer cutoff prediction method would handle cases like these correctly. In fact, running the method on just the reads that start with the primer resulted in the prediction of the primer “GACTACHVGGGTATCTAATCCT,” which when aligned to the standard primer results in an alignment percent identity of 95.5% (Table 2), indicating that QC would allow for correct primer inference.

Retrieval of automatically inferred primers from literature

Multiple examples of inferred primers from the MSD were retrieved and compared to matches found in literature, which can be seen in Table 3. In DRR059938 a Large Subunit (LSU) primer pair was identified—LSUCop-D2—which has been used in marine metagenomics analysis using the D2 region of the LSU as a marker [22]. Both the forward and reverse primers map to the native sequences with exact identity and coverage, with the inferred reverse primer being slightly longer by 2 bases.

Table 3.

Comparison of 4 primer sequences that were predicted by the primer cutoff prediction method and their native counterparts, found in literature.

Run	Primer alignments	Alignment % id
DRR059938 [22]		F: 100% R: 100%
SRR6378319 [23]		R: 84.2%
SRR13172663 [24]		F: 100% R: 74.1%
SRR22098918 [25]		F: 73.9% R: 90.5%

All 4 examples show a high coverage, and exact sequence identity for said coverage, with some gaps.

In SRR6378319, a standard primer for the 16S V3 region was found that was not originally in the curated library—BV3r [23]. In this run, the forward primer was an existing standard primer for the V3 region: 341F. This is an example of an outcome where a standard primer was found in just one direction, but primers were predicted to exist in both directions (i.e., SsGG). In this case a standard primer was simply missing from the library, and has since been added to it. The predicted primer overlaps with 100% identity and coverage, although it does over extend by 3 bases.

In SRR13172663, a primer pair for marine eDNA studies [24] was found. This pair is actually a universal set for fish eDNA metabarcoding, with the forward primer matching perfectly. The reverse primer is however missing 7 bases out of 20, which is explained by the cutoff prediction strategy preferring to select shorter primers as part of its algorithm, as to avoid trimming biological bases and losing potential real diversity in the process. Finally, in run SRR22098918, a different marine eDNA primer pair was found, teleo, which is often compared to the MiFish universal set [25]. Sequence identity was again exact, with a small number of extended bases for both forward and reverse directions (6 and 1, respectively), and one ambiguous base (R) being replaced by a non-contradicting standard base (A). However, the amplicon targeted by the teleo primers is usually around 65 bp in length [26]. Due to the short size of this amplicon, the removal of 6 native bases could adversely impact the resulting ASV analysis more strongly. This impact could be mitigated by setting a maximum primer length downstream of the primer cutoff prediction method’s results.

These results show that the primer cutoff prediction strategy for inference is marker-agnostic. While the MSD was created by retrieving sequencing runs that were annotated as originating from microbial metabarcoding experiments, incorrect annotations in public databases resulted in some eDNA studies being selected, for which primers were still able to be inferred. A further aim was to identify sequences that could be considered for addition to the standard library, such as the V3 BV3r primer. It is encouraging to note that on every instance demonstrated in these results, the primers matched with exact sequence identity to the native primers, with no gaps or mismatches. Nonetheless, the inferred primers are not perfect: some are longer than the native primer, which could result in removal of biological diversity, and some are shorter, which could still impact ASV calling results as the primer is not fully removed. Also, there are likely to be false positives, as some runs will have been flagged for primer inference when unnecessary.

Primer validation of automatically inferred primers

As was discussed in “Consistency across biomes” and “Retrieval of automatically inferred primers from literature,” sections one potential source of error in PIMENTO is the false positive inference of primers existing in reads when it is not the case. To prevent such false positives, validating low-confidence automatically inferred primers is always recommended. This validation can be performed either by referring to existing literature that describes the primer, or by computational means. For example, using a tool like Infernal/cmsearch with the different Rfam covariance rRNA models as input, e.g., RF00177 for the bacterial 16S region, one could make sure that the generated primer sequence lies within coordinates that would be expected for a primer, i.e., before or after a hypervariable region, rather than within one.

Running PIMENTO on ONT long reads

To test PIMENTO on ONT long reads, five 12S marker gene studies from ENA were used (PRJEB81670 [27], PRJEB84473 [28], PRJNA1418494 [29], PRJNA1389880 [30], PRJNA1159001 [30]). Five runs were extracted per study, and a small custom 12S primer database was created by aggregating the different primers used in each study according to their respective literature. Using this primer database, the standard primer matching (pimento std) method was used on each run.

The reads of an ONT run are typically made up of one FASTQ file containing a mixture of both 5′–3′ and 3′–5′ orientation reads. On its own, PIMENTO is not expected to work on ONT data with default thresholds, as a primer is expected to be present in at least 60% of reads to be considered. Since PIMENTO does not currently support searching across all orientations in one use, it is therefore unlikely that matched primers will reach this 60% threshold.

However, it is possible to alter the threshold using the –minimum_primer_threshold parameter, which can be used to identify primers in cases like ONT long reads where they are expected at a lower prevalence. If primers are matching at a lower threshold, we recommend validating these primers by first reverse-complementing reads that are in the 3′–5′ orientation using a tool like Cutadapt [14]. Once all reads follow the 5′–3′ orientation, pimento std can be run again. This workflow, with the threshold changed to 15%, was applied to generate the results of this section, which can be found in Table ST1.

Table ST1 shows that in 4 out of 5 studies, matching primers were found using pimento std. In one study (PRJNA1159001), it was not possible to identify primers at any threshold, which could be due to prior primer trimming of reads before uploading to ENA. These results demonstrate that with appropriate adjustments to the default prevalence threshold, PIMENTO can also be used on ONT long-read sequencing data.

Conclusion

In this work, we have introduced PIMENTO, which is a set of Python scripts that implement a novel dual-strategy method for identifying sequencing primers, filling an important gap in large-scale ASV analysis of metabarcoding data. Using either a curated library of standard primers to search against, or base-conservation patterns to automatically predict good cutoff points for inferring a consensus sequence, we believe this toolkit will help enable analysis pipelines that generate ASVs at scale.

As was shown in “Primer cutoff prediction inferences” and “Retrieval of automatically inferred primers from literature” sections, the automatically inferred primers generated by the cutoff-based method have exact sequence identity to native primers, but can sometimes be either slightly shorter or longer than the correct sequence. During the course of this work, multiple primers that were not originally in the standard primer library, but were predicted as existing and then generated by the primer cutoff prediction method, were subsequently added to the standard primer library, including the BV3r primer that was discussed in the “Retrieval of automatically inferred primers from literature” section. The addition of further primers to the library when they are first encountered means that future runs using them will benefit, as they will now be detected by the standard primer strategy directly, leading to more precise primer trimming. There is therefore significant potential in the crowdsourcing of further additions to the standard primer library, e.g., through the creation of new issues on PIMENTO’s GitHub repository, naturally improving PIMENTO’s detection capabilities over time.

While automatically predicting the presence and identity of primers for large-scale analysis pipelines can still be complicated by study design, e.g., negative and positive controls, PIMENTO’s blindspot of sometimes falsely predicting non-existent primers can be overcome using validation methods as shown in the “Primer validation of automatically inferred primers” section. Ultimately, we believe PIMENTO will be a robust and reliable tool that helps enable calling of ASVs at scale for platforms like MGnify.

Additional files

PIMENTO_supplementary_23_07_2026.docx

Table ST1: Comparison of sequences identified by the standard primer search strategy and the canonical primers extracted from literature by global pairwise sequence alignment for the set of ONT studies. Matched primers are either identical or almost identical in all 4 studies (20 runs) represented in this table. A fifth study, PRJNA1159001, was also assessed, but no standard primers were found in any of the runs tested.

Availability of source code and requirements

Project name: PIMENTO

Project homepage: https://github.com/EBI-Metagenomics/PIMENTO

Operating system: Platform independent

Programming language: Python

Other requirements: Python (3.12 or newer)

License: Apache-2.0 license

List of abbreviations

ASVs: amplicon sequence variants; COI: cytochrome C oxidase subunit I; eDNA: environmental DNA; ENA: European Nucleotide Archive; HGSD: human gut survey dataset; LSU: Large Subunit; MSD: marine survey dataset; ONT: Oxford Nanopore Technologies; OTUs: operational taxonomic units; QC: quality control; rRNA: ribosomal RNA; SSD: soil survey dataset.

Author contributions

Conceptualization, data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft, writing—review & editing: C.A.

Methodology, project administration, supervision, validation, writing—review & editing: L.R.

Software, project administration, supervision, writing—review & editing: M.B.

Project administration, supervision, funding acquisition, writing—review & editing: R.D.F.

Supplementary Material

giag083_Reviewer_1_Report_Original_Submission

Reviewer 1 -- 11/4/2025

giag083_Reviewer_1_Report_Revision_1

Reviewer 1 -- 4/30/2026

giag083_Reviewer_1_Report_Revision_2

Reviewer 1 -- 7/6/2026

giag083_Reviewer_2_Report_Original_Submission

Reviewer 2 -- 2/1/2026

Funding

Funded by the European Union under the Horizon Europe Programme with Grant Agreement No. 101112823 (DTO-BioFlow) and 101094227 (Blue-Cloud 2026). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Climate, Infrastructure and Environment Executive Agency (CINEA). Neither the European Union nor the granting authority can be held responsible for them. This project is also funded by European Molecular Biology Laboratory core funds.

Data availability

The files making up the standard primer library can be found in the GitHub repository [31]. The data used to create the figures of this manuscript and its supplementary material are also available on the GitHub repository. PIMENTO is available on both PyPI and Bioconda as the “mi-pimento” package.

Competing interest

The authors declare that they have no competing interests.