GMFOLD: Subgraph matching for high-throughput DNA-aptamer secondary structure classification and machine learning interpretability

Abstract

Aptamers are oligonucleotide receptors that bind to their targets with high affinity. Here, we consider aptamers comprised of single-stranded DNA that undergo target-binding-induced conformational changes, giving rise to unique secondary and tertiary structures. Given a specific aptamer primary sequence, there are well-established computational tools (notably mfold) to predict the secondary structure via free energy minimization algorithms. While mfold generates secondary structures for individual sequences, there is a need for a high-throughput process whereby thousands of DNA structures can be predicted in real-time for use in an interactive setting, when combined with aptamer selections that generate candidate pools that are too large to be experimentally interrogated. We developed a new Python code for high-throughput aptamer secondary structure determination (GMfold). GMfold uses subgraph matching methods to group aptamer candidates by secondary structure similarities. We also improve an open-source code, SeqFold, to incorporate subgraph matching concepts. We represent each secondary structure as a lowest-energy bipartite subgraph matching of the DNA graph to itself. These new tools enable thousands of DNA sequences to be compared based on their secondary structures, using machine-learning algorithms. This process is advantageous when analyzing sequences that arise from aptamer selections via systematic evolution of ligands by exponential enrichment (SELEX). This work is a building block for future machine-learning-informed DNA-aptamer selection processes to identify aptamers with improved target affinity and selectivity and advance aptamer biosensors and therapeutics.

1. Introduction

Aptamers are rare, single-stranded nucleotide polymers (oligonucleotides, e.g., DNA, RNA) selected for high affinity binding to specific targets [1], including ions, organic molecules, peptides, proteins, and cells. Their small size, reproducible chemical synthesis, biocompatibility, low immunogenicity, and structural stability relative to proteins make aptamers advantageous in biosensing [2,3], therapeutics [4], and beyond [5,6]. Aptamers are identified from large combinatorial libraries, theoretically consisting of $4^N$ sequences (> 1 billion), where $N$ is the number of nucleotides in the sequence random variable region. The identification process, called SELEX, generates thousands of aptamer candidate sequences per selection (Fig. 1).

Fig. 1.

Flowchart for high-throughput aptamer secondary structure determination (e.g., GMfold) and machine learning to downselect from thousands of potential aptamer sequences derived from SELEX to a tractable and more relevant number of aptamer sequences for further target affinity and selectivity determination via physical experiments.

Aptamer candidates are sequenced by next-generation sequencing (NGS) and organized hierarchically by numbers of ‘hits,’ i.e., the number of times each sequence occurs in the NGS output. While SELEX + NGS is considered high-throughput, yielding thousands of candidates, from a practical standpoint, only a few dozen of the highest copy number sequences are typically advanced for experimental determination of target binding properties and suitability for application use. Thus, aptamer candidate sequence spaces are experimentally intractable, and numerous potentially useful, even exemplary candidates are left unexplored.

In solution, aptamers form secondary and tertiary structures determined by their primary sequences. When an aptamer binds its target, conformational rearrangements facilitate energetically favorable interactions with the target and surrounding solution ions. This results in global changes in aptamer secondary and tertiary structure via intramolecular rearrangements.

Some AI tools are available for aptamer sequence clustering. Aptacluster, proposed in [7], uses statistical methods to group aptamer sequences by similarity. Here, similarity is defined by the number of nucleotide edits needed to convert one sequence to another. AptaTrace, published in [8], detects motifs associated with minimum face energy (MFE) binding properties (vide infra). The work [9] uses particle display to partition a library of aptamers by affinity and then train machine learning models on these data to predict affinity. The paper [10] reviews computational tools for aptamer clustering. And, the work [11] develops a string-based method for cluster analysis. Nonetheless, existing methods do not focus on aptamer secondary structure, information that is critical to the target binding potential.

Modern machine learning methods enable sorting, comparing, and learning from massive amounts of data, with examples ranging from the analysis of text documents to high-dimensional video sequences [12,13] to large heterogeneous knowledge graphs [14]. Given their large numbers of candidate sequences, aptamer selections would benefit significantly from a high-throughput pipeline accounting for secondary structures. To enable this pipeline, secondary structures must be computed efficiently (Fig. 1). Machine learning methods can then be used to sort and categorize aptamer candidates by secondary structure and, in the future, tertiary structure, including target binding. Strategies that identify low-sequence-count aptamers with high target affinity, selectivity, or other advantageous properties that would otherwise be discarded after SELEX are of particular interest. Here, we developed an open-source pipeline for high-throughput secondary structure calculation coupled with machine learning analysis of thousands of secondary structures to advance this strategy.

SeqFold is a recent open-source code for high-throughput DNA secondary structure determination developed in [15]. We improved this code, called SeqFold 2.0, to require fewer computations and lead to secondary structures closer to those predicted by mfold, the proprietary industry standard. Limitations intrinsic to the SeqFold algorithm are addressed by an mfold alternative based on subgraph matching. We created an open-source Python implementation of this approach called GMfold. We show that GMfold can compute high-throughput MFE structures for thousands of DNA sequences in a few minutes (see Sec 4). We compare implementations of GMfold with mfold, SeqFold, and SeqFold 2.0, to illustrate the challenges in calculating aptamer secondary structures. This is not a complete list of codes with the capability to compute these structures (see e.g. NUPACK [16]).

GMfold is the first computational step in a proposed workflow (see Fig. 1). The workflow next classifies aptamer candidates by several proposed metrics for similarity among secondary structures. Evaluating the (dis)similarity between secondary structures is essential for machine-learning techniques to detect structural patterns (clustering) and predict binding properties. Based on our proposed similarity metrics, we apply several well-known machine learning methods, including dimensionality reduction for data visualization, spectral clustering for cluster identification, and topic modeling for identifying common underlying properties characterizing groups of aptamer candidate sequences. In the case of topic modeling, we adapt the classical “bag of words” method to a “bag of faces” concept in which sections of DNA sequences are viewed as having the same role as words in text documents. We apply our proposed framework to a pool of 4550 single-stranded DNA candidate sequences from several selections designed to identify aptamers that recognize the neurotransmitter and hormone norepinephrine.

Our approach and findings are organized as follows. In Section 2, we orient readers by providing details on the SELEX method, existing single-stranded DNA secondary structure determination methods, and relevant literature for subgraph matching that inspired this work. Section 3 presents the detailed mathematical definitions and free energies pertinent to the DNA secondary structure folding problem, along with the details of the algorithms. We review both the classical algorithms and new methods in Section 3 in some detail — for the benefit of readers who are not experts in these methods. We also show how the existing methods can be improved using ideas from subgraph matching and using other ideas that decrease the computational cost of this problem. Section 4 shows a number of computational examples for aptamers, comparing the different algorithms for secondary structure calculations. Section 5 develops machine learning methods for clustering aptamer candidates learned from the high-throughput GMfold algorithm.

GMfold provides an effective and efficient open-source alternative to mfold and SeqFold. By generating a large number of secondary structures, and then analyzing a collection of such structures with modern machine learning methods, researchers can identify clusters of sequences that share similarities with the highest-count sequences identified by NGS. Our approach uncovers aptamer motifs underlying target recognition and advances aptamer identification for applications-driven research.

2. Background

2.1. Aptamer chemistry and the SELEX process

Aptamers are identified by an in vitro directed evolution selection method called SELEX [17,18]. A large combinatorial library containing billions of oligonucleotide sequences is synthesized with a randomized sequence region flanked by two constant regions. The constant regions are used for polymerase chain reaction (PCR) sequence amplification and identification (NGS). The randomized region is where the target potentially binds. The length of the randomized region correlates with the theoretical diversity of oligonucleotide-folded secondary structures. Longer random regions impart greater diversity and, thus, the probability of identifying aptamer hits. However, the length of the random region must be balanced by difficulties in working with longer sequences and the fact that there are practical limits to the number of sequences that can be synthesized and sampled.

Large targets, e.g., proteins, are immobilized to a stationary phase, and the library is passed through the stationary phase. Sequences with low target affinity (most sequences) pass through; the target-functionalized stationary phase captures higher affinity sequences. The latter are then eluted via competition with the target in solution and collected. Following amplification by PCR, sequences with target affinity form a new pool that acts as the selection library for the next round. Stringency conditions are increased with each selection round to enrich the pool with sequences having increasing target affinity and selectivity (via counter-selection rounds to remove candidates with affinity for interferants). Once the intended stringency conditions are reached, NGS is performed to identify the primary sequences of all candidates in the pool of potential aptamers [19]. This step can now take advantage of high throughput sequencing for fast analysis of the primary sequences (see [20] for a review).

Selecting aptamers for small molecule targets, such as neurotransmitters, hormones, metabolites, and ions is challenging due to the limited number of target functional groups. Small targets afford fewer opportunities for noncovalent interactions with aptamers via hydrogen bonds, electrostatic interactions, $\pi$-stacking, and hydrophobic interactions. Solution-phase SELEX is used to overcome this challenge. Here, the selection library is immobilized to a stationary phase (instead of the target). A short sequence complementary to a portion of one constant region is immobilized on the stationary phase. The library is designed so that both constant regions are also complementary to each other. A target in solution is passed through the library-immobilized stationary phase. Sequences with target affinity change conformation upon target recognition and are thereby released and collected, with additional selection steps the same as above at increasing stringency. Solution-phase selections identify stem-loop aptamers, i.e., aptamers with complementary 3’- and 5’-ends that form a stem upon target binding, as exemplified by the selection screens reported herein.

The in vitro SELEX process is laborious and not guaranteed to succeed. In silico pre- and post-SELEX simulations have been developed to complement empirical selections. In any case, the prohibitively large combinatorial space of the libraries used in SELEX cannot be empirically or computationally accessed at present. Small-molecule-aptamer structures are less abundant than aptamer structures for large targets, such as proteins, limiting the possibility of modeling using previous data sources. Computational approaches have been explored but have not yet been fully leveraged for small molecule targets for ab initio oligonucleotide structure/folding predictions, modeling oligonucleotide-target interactions, and ultimately, de novo functional predictions from primary oligonucleotide sequences.

2.2. Existing methods for aptamer structure

Two approaches are mainly used for computing secondary structures of single-stranded oligonucleotides. The first computes conformations that minimize free energy [21]. The second relies on partition functions and the computation of base pair probabilities [22]. Both approaches rely on a dynamic programming principle first applied by [23]. Other less common approaches for folding single-stranded oligonucleotides into secondary structures are either based on maximum matching [24], where the goal is to determine the secondary structure with the maximal number of base pairs, or simple rules relying on basic assumptions related to the folding process [25].

Various programs exist to predict aptamer secondary structure, including mfold [26], NUPACK [16], RNA composer [27,28], web 3DNA [29–31], Mode RNA [32], among others. Mfold is currently the gold-standard modeling program. It uses free energy minimization algorithms to predict the lowest energy-folded structure and includes temperature and ion concentrations as inputs. Empirical confirmation of secondary structure predictions relies on brute-force experimentation. The mfold code is written in C/C++ and Academic and nonprofit users can use it under a free license; commercial use requires payment.

SeqFold [15] is an open-source alternative to mfold. Similar to mfold, SeqFold implements the dynamic programming approach from Zuker and Stiegler [21]. Moreover, SeqFold has the non-trivial advantage of being implemented in Python, the primary programming language for developing machine learning (ML) procedures. This makes it possible to integrate SeqFold into a workflow that inputs large numbers of single-stranded DNA sequences (aptamer candidates), generates the related MFE secondary structures, and applies various ML data analysis procedures.

Here, we show that ML methods can be powerful tools for analyzing large sets of DNA secondary structures. Moreover, ML has the potential to be effective across a range of tasks, including visualizing the configuration space of secondary structures, identifying clusters of structures with shared properties, and predicting binding activity.

The current version of SeqFold has several pitfalls, including inconsistencies between the algorithm proposed in [21] and the implemented code. We found that these inconsistencies can lead to the computation of MFE structures that are substantially different from those obtained with mfold, which we take as ground truth for this work.

2.3. Multigraphs and graph matching for DNA

We draw on the recent work of one of the authors [33–35] on subgraph matching for multiplex networks, referred to here as labeled directed multigraphs. These are defined as:

$$G=\left(V,E,L,C\right),$$

where $V$ is the node-set and $E$ is the edge set (in our case, not directed). $L$ assigns a label to each node, (in our case, the nucleotide bases $A,T,G$, or $C$). $V$ assigns a channel to each edge. We have two (or potentially three) types of channels (backbone-nucleotide and nucleotide-nucleotide).

In this context, perfect matching is a subgraph isomorphism from some template graph $G_t$ to the background graph $G_b$ (sometimes called the world graph). Subgraph isomorphism is NP-complete [36]. Subgraph matching algorithms date back to Ullman’s work [37], which uses a tree search, keeping track of a search state, and navigating the tree of possible search states, backtracking when the end of a branch is reached. Due to the enormity of the tree, computational complexity is limited as much as possible by refining the search space at each step of the search to avoid unnecessary branches. Other tree search methods include VF2 [38] and its variants (VF2 Plus [39], VF3 [40], VF2++ [41]), and for specific graphs, RI/RI-DS [42]. The techniques of constraint propagation and filtering have been shown to reduce computation time significantly [33–35,43–46]. The subgraph matching problem is generally known to have combinatorially complex solution spaces. In recent work by [33], there are examples in which SMPs are shown to have 10¹⁰⁰ or more isomorphisms.

We view the aptamer secondary structure folding problem as a bipartite incomplete graph matching problem in which nodes in the DNA graph are matched to conjugate nodes within the same structure. A secondary structure can be represented by making a second copy of the DNA primary sequence and searching for a bipartite matching of the DNA sequence to its copy. We apply the constraint that any base in one copy can only pair with a single conjugate base in the second copy. As such, this is an incomplete or inexact matching problem, with the optimal matching defined by the MFE. Existing algorithms for DNA secondary structure are constructed similarly to Ullman’s tree search and backtracking method. One goal of this work is to introduce more ideas from constraint propagation to speed up DNA secondary structure algorithms. We then show how to leverage our new code GMfold to analyze thousands of secondary structures simultaneously using modern machine learning methods.

3. Subgraph matching and free energy minimization for DNA folding

3.1. Overview

Here, we develop the full algorithm for the DNA secondary structure. We follow general ideas used in [33–35] for subgraph searches on multiplex networks. We treat each DNA strand as a linear string of nodes whose labels correspond to the primary sequence of nucleobases (adenine (A), thymine (T), guanine (G), and cytosine (C)). The graph-matching problem is solved using hierarchical filters applied to matched base-paired candidate nodes and an exhaustive tree search through the rest of the solution space. The tree search can be accelerated using ideas of structural equivalence and the node cover, as described in [33], or by ordering the search based on decreasing energy [35].

The DNA secondary structure problem is an annotated incomplete subgraph matching problem in which the graph is matched to itself through internal interactions between nodes (nucleotides) to find the optimal number and arrangement of interactions to minimize free energy.

To this end, we build our method on an elimination scheme as follows:

Step 1: The DNA strand is a linear graph with edges between adjacent nodes in the primary sequence. We want to map the strand to a copy of itself using the following rules. For graph matching, nucleotide bases (i.e., A, T, C, G) act as labels on the graph nodes. The labels correspond in their order to the primary DNA sequence.

We create a list of candidate nodes in the target graph to which every other node can match. The rules for matching are defined by canonical nucleotide conjugate base pairs. That is, for each adenine (A) in the primary sequence, the candidates for matching are all of the nodes with thymine (T) as their label. Likewise, for each guanine (G), the candidate list corresponds to all of the cytosine (C) nodes. We complete the list by identifying all the candidate nodes for (T) and (C). At the end of step one, we have many possible matched candidate nodes. We then systematically eliminate matched nodes (see below) until we have an optimal structure. An example showing Step 1 output is shown in Fig. 3-left panel.

Fig. 3.

Serotonin aptamer showing secondary structure at Step 1 of our method (left), the removal of edges ($i,j$) with $|j-i|\le 4$ (middle), and Step 2 (right). This is a matrix of all candidate pairs for the secondary structure. Our strategy is to eliminate pairings until there is at most one interaction per nucleotide with the final configuration having the lowest free energy. Step 2, filtering all candidate nodes that do not participate in at least one stack connection greatly reduces the search space prior to Step 3.

Step 2: This step is an analog of the “topology” filter in [34]. We apply a constraint that eliminates matched candidate nodes that do not connect to nodes that match the first-order connections in the template graph in the subgraph matching problem. For DNA aptamer structure, we search based on stack configurations, as shown in Fig. 2. We eliminate all matched candidate nodes that are not part of a stack, which is defined by two adjacent matched nodes. In Fig. 2, two primary/adjacent edges/connections are shown in black along the oligonucleotide backbone, and two secondary edges/connections (base pairs) are shown in red. Thus, nodes 1 and 2 are conjugate to nodes 3 and 4, respectively. In the following discussion, we refer to the secondary edges remaining after Step 2 filtering as “admissible” edges. Fig. 3-right shows the resulting secondary edges left after Step 2 was carried out for a serotonin aptamer [47]. The filtering removes a little over a factor of 2 edges from the graph, as seen in Fig. 5.

Fig. 2.

Template graph consisting of 4 nodes. We refer to such a graph as a “stack”.

Fig. 5.

${\ell }_1$ norm of the adjacency matrix (twice the number of edges) before (solid) and after (dashed) filtering for 1000 random sequences for lengths from 10 to 150, and the ratio of the two (top left). Error bars show the standard deviation.

Step 3: This final step eliminates additional matching candidate nodes to achieve a minimal free energy configuration, with at most one candidate per nucleotide. This requires an understanding of face configurations and their energies. We consider five classes of faces: hairpin loops, stacking regions, bulge loops, interior loops, and multi-branches, as shown in Fig. 4. This step can also be framed in the context of subgraph matching, in which the face classes provide a natural subgraph grouping of matched nodes to simplify the search for the lowest energy configuration. Each face has an associated scalar energy value. The total energy is the sum of all the face energies. Our goal is to find the minimal energy configuration by selecting from the remaining candidate edges between different matched nodes (nucleotide pairs). Note that the energy depends on the face configuration and the actual nucleotides comprising each face.

Fig. 4.

Face classes for calculating minimal free energy configurations. (a) Hairpin loops are faces with a single interior edge. (b) Inner loops have stacks on either end with no secondary edges connecting the nodes between each stack. (c) Bulges are similar to inner loops but with no nodes on one side of the loop. (d) Multibranches are faces with more than two interior edges. In all of the face classes, the rest of the aptamer continues out from each stack (colored green).

Step 3 consists of an overall search of substrings derived from the DNA strand. For each candidate interaction ($i,j$), we compute all possible face energies associated with any admissible face that contains ($i,j$). We start with smaller distances between $i$ and $j$ and proceed to larger and larger distances. The algorithm concludes by choosing the edge ($\mathrm{i},\mathrm{j}$) with minimal internal energy and designating it as the “last pair”, defined such that there are no faces outside of the substring. We choose all pairs in the optimal configuration to be part of the final structure.

Optimization involving the hairpin, inner loop, and bulge face energies are fairly straightforward compared to optimization for multi-branch structures. The issue is that for a multi-branch, one has to choose from all subsets of allowable edges, which is a combinatorially large search space. The details of this optimization can be found in the appendix.

We now outline the recursive process used to find the minimum free energy as originally proposed by Zuker and Stiegler [21]. This process is what mfold and SeqFold use in their respective algorithms. To do this, we need to store content in structure caches, $S_v$, and $S_w$, which are functions from pairs of indices ($\mathrm{i},\mathrm{j}$) with an energy and a “structure”. (These caches are stored as a list of lists containing objects that store an energy and an interaction configuration). The caches are defined as follows: $S_v(i,j)$ is the optimal energy and structure of the substring $[i,j]$, assuming that $(i,j)$ is an internal edge, i.e., assuming that $i$ and $j$ interact. $S_w(i,j)$ similarly is the optimal energy and structure of the substring $[i,j]$ with no additional assumptions. Note that if $i$ interacts with $j$ in the optimal configuration, then $S_v(i,j)=S_w(i,j)$.

The recursive process is defined as follows.

$$S_w(i,j)=\mathrm{m}\mathrm{i}\mathrm{n}\left\{S_w(i+1,j),S_w(i,j-1),S_v(i,j)\right\}$$ (1)

Where $S_v(i,j)$ must be computed explicitly using the values of the cache for all subsequences of $[i,j]$. The method used to fill the $S_v$ cache is by computing the minimal energy that the string $[i,j]$ can have if it ends in a face, as described in 3.4.

3.2. SeqFold 2.0

Recall the SeqFold algorithm from Section 1, as the first DNA-specific open source Python code [15]. Building on the SeqFold code, we developed an improved version, called SeqFold 2.0, that implements the following ideas:

1. Cache efficiency

   We can avoid such unnecessary recursive passages by computing the caches in a specific order. We propose two approaches that would lead to the same result. The first approach consists of computing first $S_w(i,j)$ such that $|i-j|=k$ with $\mathrm{k}=4$, then those for which $k=5,6,\dots ,n$. The second approach consists of computing for each $j=5,6,\dots ,n$, $S_w(i,j)$ for $i=j-4$, $j-5$, $j-4,\dots ,1$. Both approaches are based on the fact that to compute $S_w(i,j)$, we only need to determine the value of $S_w\left(i^{\prime },j^{\prime }\right)$ for all ($i^{\prime },j^{\prime }$) such that $i\le i^{\prime }<j^{\prime }\le j$. By following one of the two suggested orders, no unnecessary recursive passages are needed leading to a reduction of the computational cost of order $n$.

2. Unlike SeqFold, SeqFold 2.0 uses step 2, the graph-matching approach detailed in the previous section. Notably, by first performing the subgraph matching algorithm to find all embeddings of the stack graph. This then gives the collection of nucleotides that are “allowed” to bind, and the set of allowable edges between them. Notably by restricting to these nucleotides and edges, we have made the assumption that no isolated interactions form. This is accounted for in earlier algorithms [21] by setting the energy of such an interaction to a large value (in SeqFold, this was 1600 kcal/mol [15,48]). As indicated above, this search takes $4\cdot \left(\genfrac{}{}{0pt}{}{n}{2}\right)$ computations, and thus scales as $𝒪\left(n^2\right)$ (a lower scaling when compared to the original implementation of the dynamic algorithm). Now we note that by restricting edges to be in this set, we can cut down on the number of operations in the inner loop and multibranch energy calculation, and also in the calculation of $S_v$. Note that if ($i,j$) is not an edge in the valid set, then we can assume that $S_v(i,j)>S_w(i,j)$ and therefore do not have to compute it.

3. Fixes incorrect energies

   The SeqFold code has some incorrect energies. We fixed these in SeqFold 2.0. Details are discussed in Appendix A.1.

3.3. GMfold

We have developed an entirely new method, the GMfold algorithm. Our algorithm computes the minimal energy structure, using the energy function provided in [21]. GMfold has the following advantages over prior methods:

1. Subgraph matching: As in SeqFold 2.0, we perform the subgraph matching algorithm (Step 2 in the previous section) to determine all admissible interactions. This reduces the candidate set we search over, and allows us to be more flexible with the optimization approach we use.

2. Multibranch face optimization To determine the lowest energy multibranch possible containing ($i,j$), we conduct a more robust search than the one used in [21]; we compute all combinations of at most $m$ edges in the substring [$i,j$]. This gives more fine control over the types of structures that the algorithm produces and ensures the entire space is searched, at the cost of computation time. Details of this step can be found in Appendix.

3. Cache reduction: Performing the multibranch optimization step above eliminates the need for a secondary cache, meaning we only need to store structure and energy information of the pair ($i,j$) if ($i,j$) is an admissible pair, reducing the size of the cache.

4. Our code allows for different temperatures, see Appendix A.1 for details.

5. Missing items from GMfold

   Here are items not in GMfold which could be added to a future version.

   1. Our code uses a salt concentration 1 M NaCl. For other salt concentrations, the user will need to update the reference energies in the code.
   2. We include limited coaxial stacking stabilization energies. We use a naive value for all coaxial stacking, whereas the Ref. [49] has experimentally determined values dependent on the sequence. This might explain why some of our results differ from mfold when the code predicts a complicated multibranch structure (See e.g. Fig. 7).
   3. We only include admissible edges as defined in Step 2 above. Infrequently, mfold finds structures that include non-canonical pairing and neither GMfold nor SeqFold/SeqFold 2.0 will produce these structures.

Fig. 7.

Comparison between SeqFold, SeqFold 2.0, GMfold and mfold, as implemented within the UNAfold software, for an exemplary sequence. The energies are computed by inputting each folded structure into the mfold software. The sequence is 5’-GGGACGACGGGGCACATTGTGCTATTCAGTTGTTCCGCAGGAGAGTCGTCCCGCCTAGCTATTCAGTTGTTCCGCAGGAGAGTCGTCCC-3’. Structure visualization generated using ViennaRNA, specifically forgi [50].

3.4. Energy optimization and algorithmic complexity

At each step in our energy minimization algorithm, we must decide what type of face will be formed and how big it will be. To do this, we assume that in the substring [$i,j$] there will be a face ending at ($i,j$). If this face has no other internal interactions, it will be a hairpin loop, if it has one other internal interaction, it will be an inner loop, bulge loop, or stack, and if there are two or more other internal interactions, it will be a multibranch loop. In order to determine the optimal face, we in general need to look for the optimal placement of these other internal pairs. We will now discuss the systematic way to compute these structures and energies.

Assuming that $i$ interacts with $j$, we must determine what face structure will have lowest energy. We have to compute the optimal energies associated to the three types of faces: the hairpin energy, all possible inner loop/bulge/stacks, and all possible multibranches (note that none of the algorithms check all possible multibranches). For the hairpin, there is only one choice. The complexity of this calculation is sublinear time.

Next, we group the inner loop, bulge, and stack structures together, as they are defined by exactly two interior edges. We search over all admissible edges (from Step 2) in the substring ($i,j$) to determine which face is optimal. This generically has computational complexity $𝒪(\left(\genfrac{}{}{0pt}{}{|j-i|}{2}\right))=𝒪({|j-i|}^2)$, but in our SeqFold 2.0 and GMfold algorithms this will be fewer than $\left(\genfrac{}{}{0pt}{}{|j-i|}{2}\right)$ checks, as the allowable edges have been greatly reduced by the graph matching algorithm Fig. 3.

Finally, we have the multibranch optimization. We consider all combinations of admissible pairs in the string [$i,j$] which could form a multibranch structure. This is potentially a combinatorially large search space. The best upperbound one can place on this is $2^N$ where $N$ is the number of admissible pairs in the substring [$i,j$]. [21] avoid this by assuming that there exists some $k\in [i,j]$ such that the optimal multibranch structure will be given by the optimal structure on $[i+1,k]$, joined with the optimal structure on $[k,j-1]$. Using this assumption, along with the need to store not only the interaction structure between $(i,j)$ but also the overall optimal structure, their algorithm is able to find the optimal multibranch in $𝒪\left(\right|j-i\left|\right)$ steps, i.e. one for each $k\in [i,j]$. This is the algorithm that SeqFold, mfold, and SeqFold 2.0 use.

For GMfold, we wanted more control. In order to solve the combinatorially complex search space, we take in a parameter $m$, the maximum number of branches one allows in a multibranch structure. This reduces from searching through all combinations of edges between $i$ and $j$, to searching through all combinations of at most $m$ edges between $i$ and $j$. This then requires at maximum $𝒪(\left(\genfrac{}{}{0pt}{}{|j-i|}{2m}\right))=𝒪({|j-i|}^{2m})$ computations. Again, this should be greatly reduced after the graph matching algorithm in (Step 2) reduces the search space.

Combining all of these complexities, we see that the complexity of computing the optimal face for ($i,j$) is

$$\mathrm{m}\mathrm{a}\mathrm{x}\left\{𝒪(|j-i|),𝒪(|j-i{|}^2),𝒪(|j-i|)\right\}=𝒪(|j-i{|}^2)$$

for the algorithms based on the Zuker-Steigler approach, and

$$\mathrm{m}\mathrm{a}\mathrm{x}\left\{𝒪(|j-i|),𝒪(|j-i{|}^2),𝒪(|j-i{|}^{2m})\right\}=𝒪(|j-i{|}^{2m})$$

for GMfold.

This face energy optimization described above is carried out for all admissible edges ($i,j$). We do not have an estimate for the number of admissible edges so we use the naive upper bound of all possible pairs $i,j$. This gives upper bounds for the overall complexity of

$$\sum _{1\le i<j\le n}𝒪(|j-i{|}^2)=𝒪\left(n^4\right)$$

for the Zuker-Steigler algorithms, and

$$\sum _{1\le i<j\le n}𝒪(|j-i{|}^{2m})=𝒪\left(n^{2m+2}\right)$$

for the GMfold algorithm. These estimates include all possible terms, many of which may not need to be included. In our calculations, we used the parameter $m=4$. For the experimental runtime, see Fig. 6 for the comparison between SeqFold and SeqFold 2.0. Notably, the observed scaling of SeqFold is $𝒪(n^{3.696})$, where as SeqFold 2.0 exhibits $𝒪(n^{3.477})$.

Fig. 6.

Running time differences between SeqFold and SeqFold 2.0 on randomly generated single-stranded DNA sequences of various lengths. Ten sequences were computed per string length with a variance of about 0.5%, not shown. Algorithms were performed on AMD Ryzen 9 5900HX clocked to 3.80 GHz (Base clock 3.30 GHz, boosted to 3.80), NVIDIA RTX 3070 Laptop GPU, 16 GB system ram.

4. Numerical results for DNA folding

We show computational examples that illustrate advantages to our approach. We compare our proposed approach with two benchmark algorithms for predicting MFE secondary structures: SeqFold by [15] and mfold by [26]. We provide several benchmark sequences in this section and in Appendix A.3, illustrating the differences between these algorithms. The sequences are from either SeqFold’s comparison library [15], or from actual aptamers that bind to particular targets.

We start with five aptamer sequences from the SeqFold library, as seen in Table 1, originally presented by the authors of SeqFold to validate their approach against mfold. The authors of SeqFold provide seven DNA sequences; here we consider those for which SeqFold and mfold disagree. For each sequence, we compute the secondary structure using our proposed approach, SeqFold 2.0 and the two baseline methods. Next, we use mfold to compute the ground truth energies of the folded structures obtained with each method. For the first three sequences, the MFE structures computed by GMfold have the same energy as those obtained with mfold. In particular, the two approaches lead to the same MFE structures. The structures predicted for sequences S1, S2 and S3 indicate our approach based on subgraph matching can be effectively employed.

Table 1.

Results of our experiments on five single-stranded DNA sequences. The five single-stranded DNA sequences are among those used by the authors of SeqFold to validate their approach against mfold. These can be found in SeqFold GitHubrepository. The table reports the structure energies, computed with mfold, associated with the structures obtained by computing the MFE with GMfold, SeqFold 2.0, SeqFold and mfold, respectively. Lowest energies are in bold.

N°	Sequence	SeqFold (kcal/mol)	SeqFold 2.0 (kcal/mol)	GMfold (kcal/mol)	mfold (kcal/mol)
S1	5’-GGGAGGTCGTTACATCTGGGTAACACCGGTAC TGATCCGGTGACCTCCC-3’	−8.84	−8.84	−10.94	−10.94
S2	5’-GGGAGGTCGCTCCAGCTGGGAGGAGCGTTGGG GGTATATACCCCCAACACCGGTACTGATCCGGT GACCTCCC-3’	−17.17	−17.61	−23.36	−23.36
S3	5’-TAGCTCAGCTGGGAGAGCGCCTGCTTTG CACGCAGGAGGT-3’	−4.75	−4.75	−6.85	−6.85
S4	5’-GGGGGCATAGCTCAGCTGGGAGAGCGCCT GCTTTGCACGCAGGAGGTCTGCGGTTC GATCCCGCG CGCTCCCACCA-3’	−10.01	−15.11	−15.11	−15.48
S5	5’-TGTCAGAAGTTTCCAAATGGCCAGCAATC AACCCATTCCATTGGGGATACAATGGTAC AGTTTCGCATATTGTCGGTGAAAATGGTT CCATTAAACTCC-3’	−2.22	−4.70	−5.70	−9.35

If we consider the secondary structures and energies from mfold as the ground truth, GMfold might lead to suboptimal results. The experiments on S4 and S5, reported in Table 1, provide examples where our approach and mfold lead to different results. Specifically, the secondary structures of S4 and S5 computed by GMfold do not agree with the results of mfold. This is evidenced by the higher energy values associated with the structures. Nonetheless, it is worth mentioning that in all our experiments GMfold consistently outperforms SeqFold. In particular, for S1, S2, and S3, SeqFold does not compute the MFE structures found by mfold.

We now present three examples that illustrate the structural differences that can arise when comparing the algorithms. Fig. 7 considers a longer aptamer found in the SeqFold comparison catalog. SeqFold and SeqFold 2.0 vary drastically, GMfold gets a close approximation to mfold. Differences like these are often due to a difference in multi-branch energy computation between the two algorithms. Specifically, GMFold does not find an interaction between positions 34 and 42 in one of the multibranches and finds an interaction between positions 71 and 79 in the other multibranch. SeqFold 2.0 fails to find the additional branch on the external loop, and finds different lengths of the branches in the multibranch loop, again due to a slight difference in how GMfold and SeqFold 2.0 calculate multibranch energies.

Fig. 8 highlights the similarities between the GMFold and mfold algorithms in contrast to the SeqFold and SeqFold 2.0 algorithms. Both SeqFold and SeqFold 2.0 fail to predict coaxial stacked multi-branch structure that mfold predicts, whereas GMFold can identify this structure.

Fig. 8.

Comparison folding of two exemplary sequences between SeqFold, SeqFold 2.0, GMfold and mfold. The figure shows that both SeqFold and SeqFold 2.0 consistently fail to predict the junctions represented in the secondary structures produced by GMfold and mfold. We consider two sequences from Table 1: S1 (top row) with a three-way junction and S2 (bottom row) with a four-way junction. Note that Table 1 also mentions the mfold energies associated with each illustrated secondary structure. Structure visualization generated using ViennaRNA, specifically forgi.[50].

Fig. 9 depicts an aptamer for Sgc-3b, a particular membrane protein. Similar to the example in Fig. 7, GMfold finds an aptamer most similar to what mfold predicts. The only discrepancy is the initial stacking region that GMFold finds is one shorter than what mfold finds. Both SeqFold and SeqFold 2.0 predict entirely different multibranches. Using the mfold energies, SeqFold and SeqFold 2.0 show much less favorable energies.

Fig. 9.

This example may reflect either a difference in coaxial stacking energies or a difference in the multibranch energy function. Comparison folding of an exemplary sequence between SeqFold, SeqFold 2.0, GMfold and mfold as implemented within the UNAfold software. The sequence is an aptamer for Sgc-3b [51] The sequence being folded is 5’-TTTACTTATTCAATTCCCGTGGGAAGGCTATAGAGGGGCCAGTCTATGAATAAGTTT-3’. Structure visualization generated using ViennaRNA, specifically forgi. [50].

Fig. 10 depicts an aptamer for theophylline. This example illustrates noncanonical pairing, which is not found by GMfold or SeqFold 2.0, due to Step 2. The interaction (12, 32) is a $T-G$ non-canonical pairing found by mfold. This causes the $G-C$ pair right before it to be deleted by Step 2. This causes the SeqFold 2.0 and GMFold algorithm to neglect the central stacking region, as neither algorithm finds having the $G-C$, $G-C$ stack to be favorable without the rest of that stretch. This example highlights possible future improvements that include adding the possibility for noncanonical base pairings to the code. More examples comparing the secondary structures of various exemplary aptamers are shown in Appendix A.3.

Fig. 10.

Example with non-canonical base pair in one of the stacks. Comparison between SeqFold, SeqFold 2.0, GMfold and mfold as implemented within the UNAfold software. This sequence has been found to be an aptamer for Theophylline. The sequence being folded is 5’-GACGACGATTGTGGTCTATTCATAGGCGTCCGCTGAGTCGTC-3’ [52]. Structure visualization generated using ViennaRNA, specifically forgi.[50]. The first three methods do not find the non-canonical base pair.

5. Machine learning for high-throughput DNA structure clustering

In this section, we show how machine learning tools can be used to analyze a large collection of DNA strands from the SELEX process. We start with a particular target binding problem (in this case norepinephrine) and process the raw data obtained from next generation sequencing after SELEX. In contrast to prior statistical methods such as [7], our methods directly address the global geometry of the DNA secondary structure in an unsupervised way, with the goal of identifying structures that may not have a high “count”, however they are structurally close to the top count aptamers. We see analogous approaches arising in related areas of bioinformatics, for example the recent Snekmer algorithm [53] which develops a scalable pipeline for protein sequence fingerprinting based on amino acid recoding, thus linking sequences with distant sequence similarity.

5.1. Dataset and data pre-processing

In this section, we detail the process of generating raw data and the procedures we implement to clean them.

5.1.1. Raw data generation process

Solution phase systematic evolution of ligands by exponential enrichment (SELEX) was performed according to previously published protocols to select new aptamers for the small molecule neurotransmitter target, norepinephrine [47,54–59]. Two libraries were used. One library had a 48-nucleotide randomized region and the other had a 58-nucleotide randomized region. Standard desalted oligonucleotides were used for both libraries, as well as their primers. Both libraries contained identical sequences flanking the randomized regions. The five nucleotides on the 5’ and 3’ ends were complementary to one another [55]. As reported previously, this library design preferentially selects for aptamers that undergo stem closure upon target binding, a design approach advantageous for biosensing applications.

Iterative selection rounds were followed by semiquantitative PCR to determine how/when to increase the stringency of the selection conditions [55]. Selections were carried out in phosphate-buffered saline (PBS) with 2 mM MgCl₂ at pH 7.4. The PCR started at 95 °C for 2 min, followed by $N$ cycles of [95 °C for 15 s, 60 °C for 20 s, 72 °C for 30 seconds] and a final cycle of 72 °C for 3 minutes [54,55]. Each PCR run was 13 ± 2 cycles. Generally, target concentrations were decreased when the band densities of the prewash in the previous step and the first wash in the next step were similar, and the bands remained visible. Negative selection was performed with dopamine, serotonin, and epinephrine, all of which are structurally similar to norepinephrine (target).

Once the prewash and first target wash repeatedly showed no increase in band densities, the oligonucleotide samples were sent to Genewiz for amplicon-EZ next-generation sequencing (NGS). Samples from the 9th and 13th SELEX cycles were sequenced for the 48-nucleotide library. Samples from the 12th and 16th cycles were sequenced for the 58-nucleotide library. We added longer overhang NGS primers to increase the sequencing speed.

We preliminarily cleaned the NGS data in Excel by removing the primer sequences from the 3’ and 5’ ends, as they do not contribute to aptamer function, tertiary structure, or sensing performance [54]. We filtered the data to focus on the randomized regions by choosing nucleotides contained within the 5 outermost complementary nucleotides on the 5’ and 3’ ends, which form the stem. All sequences should have had lengths representing the 48 or 58 random region libraries. Nonetheless, we observed other lengths. These sequences likely arose from random insertions or PCR amplification errors. We retained variable-length sequences because they persisted in the library screening pool, indicating target recognition. We ranked all sequences by the number of reads (counts) in the NGS files. Typically, we would choose only the high-count-number sequences for empirical analysis of target recognition and sensing performance. However, this practice, while expeditious, potentially excludes high-affinity, high-specificity aptamers with low count numbers. Here, we developed new algorithms that enabled us to mine low-count sequences based on structural similarities with the highest-count sequences via high-throughput secondary structural analysis and clustering.

The data cleaning process produced four files containing single-stranded DNA sequences and the associated number of reads or “counts.” Two files contained sequences from the 48-nucleotide randomized region library: one with sequences after 9 PCR cycles and the other after 13 PCR cycles. The remaining two files contained sequences from the library that had a 58 nucleotide randomized region: one that had sequences obtained after 12 PCR cycles and the other after 16 PCR cycles.

5.1.2. Data pre-processing

We considered all sequences arising from the data generation and cleaning procedures discussed in the previous subsection. In total, we analyzed 16,631 single-stranded DNA sequences and associated counts. These sequences included the five complementary nucleotides on the 5’ and 3’ ends, which form the terminal stem. If the same sequence was reported more than once, i.e., was reported in files associated with different cycles, we remove duplicates, keeping the sequence associated with the highest count. Moreover, we removed all sequences with fewer than 20 nucleotides and those with an invalid nucleotide, e.g., sequences with a nucleotide that was not A, T, C, or G.

After applying filtering procedures, we had a dataset containing 4933 sequences. We removed additional outlier sequences based on length, keeping only those with lengths between the 5th and 95th percentiles. The final dataset contained 4450 sequences with lengths ranging from 33 to 83 nucleotides. We used GMfold to compute the secondary structures and face energies of all sequences in our processed dataset. We forced the five complementary nucleotides on the 5’ and 3’ ends to base pair. Folding all 4450 sequences with GMfold took approximately three minutes on a commercial laptop with an 11th Gen Intel(R) Core(TM) i5–1135G7 CPU running @ 2.42 GHz with 8 GB RAM.

5.2. Mathematical representation of secondary structures

We propose three vector-valued descriptors to represent the secondary structure of single-stranded DNA sequences. The first is based on the adjacency matrix of the secondary structure graph, which we call the structural matrix. The second is based on Motzkin paths, combinatorial objects used in mathematics, particularly in the study of lattice paths. The third type is inspired by the Bag of Words, which are text-mining descriptors used in natural language processing.

5.2.1. Structural matrix descriptors

The structural matrix descriptors consist of the adjacency matrix of the secondary structure graph, where we set to zero all matrix entries corresponding to the primary structural bonds between nucleotides (e.g., Fig. 11 lower left). The structural matrix descriptors provide information only on the topology of the secondary structures. Information related to the nucleotides that make up the single-stranded DNA sequence or the energies of the folded structures are not represented.

Fig. 11.

Visual representation of descriptors we propose to represent the secondary structure of single-stranded DNA sequences. The secondary structure can be represented as a graph (upper left corner). The Structural Matrix and Motzkin path descriptors convey information about the topology of the secondary structure graph. The Bag of Faces(BoF) descriptors provide information about the energetic configuration of the secondary structure.

5.2.2. Motzkin paths descriptors

A Motzkin Path is a path of length $N$ in $\mathbb{N}\times \mathbb{N}$ starting at (0, 0) and ending at ($0,N$) with each step being one of the three following types (+1, +1), (+1, 0), (+1, −1). We can record such a path as a balanced parenthetical sequence with rests, i.e., a sequence $s\in {\{(,.,)\}}^N$ that is balanced in the parentheses. We note that we can go from such a sequence to a path by the map ( ↦ (+1, +1), . ↦ (+1, 0), ) ↦ (+1, −1). The balanced condition ensures that we never cross below the 𝑥 axis. Noting that each path step moves one to the right, we can also record the sequence as $\in {\{-1,0,1\}}^N$, where the condition is that all partial sums of $s$ are non-negative. This leads to the fourth way of describing such paths, i.e., by their partial sums. A sequence $s\in {\mathbb{N}}^N$ is admissible as a partial sum sequence if $s_1=s_N=0$, and $\left|s_{i+1}-s_i\right|\le 1$ for all $i$.

These objects are well known to be in correspondence with the secondary structures of RNA sequences without pseudoknots [60–63]. Motzkin numbers are a generalization of Catalan numbers and have been studied thoroughly, with several identities known [64]. At the level of two-dimensional structures, DNA and RNA sequences are in bijection under the map $U\mapsto T$,. However RNA also exhibits pseudoknots [65], a feature not commonly present in DNA [66]. While single-stranded DNA sequences can form pseudo knots, the conditions under which they form are specialized [67]. This means we can use modified versions of older RNA methods to predict DNA structure efficiently, as described in the algorithms above.

We include a proof here. The quickest heuristic is to look at the balanced parenthetical sequence with spaces, and note that when a pair of parentheses balances, those two indices will interact in the DNA strand. Notably, assuming a lack of pseudoknots, we can draw the pairings as non-crossing paths in the plane, and thus the secondary structure of a sequence with length $N$ is a non-crossing chording of an $N$-gon. For what follows, we show that this structure is completely determined by a balanced parenthetical sequence with “.”s inserted, or equivalently, a Motzkin path.

To go from a collection of chords of an $N$–gon to a parenthetical sequence, we choose a starting point of the $N$–gon to hold constant (say the top) and move clockwise, replacing each start of a chord with ‘(‘, each position without a chord by ‘.’ and each end of a chord by ‘)’. This results in a sequence with an equal amount of ‘(‘ and ‘)’ parentheses. In terms of the DNA sequence, we start by laying the sequence along the $N$-gon in order. When we encounter a base pairing, we place open and closed parentheses on each side of the pair, and draw a chord from one nucleotide to the other. Note, we place the open parenthesis before the close parenthesis in the sequence, and we will have an equal number of both, as a pair corresponds to a chord, meaning that the parenthetical sequence will be balanced. As the sequence is balanced, two different chordings will have two different words, as they must differ by at least one chord position. One can also verify that putting the parentheses along the circle and connecting corresponding open and closed parentheses will give a non-crossing chording of the circle, showing that this mapping is, in fact, a bijection. Note, that a balanced parenthetical sequence with periods can be replaced with a ${\{1,0,-1\}}^N$ vector whose cumulative sum is non-negative via the substitution $(\to 1,.\to 0)\to -1$.

To visualize how this description gives us the secondary sequence of this DNA sequence, we place the backbone clockwise along the outside of the circle, and draw a chord between any two nucleotides interacting in the secondary structure. The non-crossing condition comes from the fact that DNA sequences tend not to form so-called pseudoknots [67]. We show an example sequence below 12.

Note that all information about the topology of the secondary structure is stored in this Motzkin path formalism. Thus, we can use the Motzkin path vectors (both $v\in {\{-1,0,1\}}^N$ and $,w_i=\sum _{j=0}^i{v}_j$) to represent the sequence secondary structures. By comparing the Motzkin path vectors, we can identify structural similarities between different folded sequences. We use the cumulative sum vector to describe the secondary structure of the sequences in our numerical experiments. Moreover, we suggest excluding dimensions where the Motzkin path vectors are identical across all data points.

This additional processing step eliminates redundant information from the Motzkin path descriptors, resulting in a more compact representation. We remove six dimensions: five related to the initial stack of five base pairs present in all the secondary structures in our dataset, and one from the fact that all Motzkin path descriptors we consider have value zero in the last dimension. The processed Motzkin path descriptors have 77 dimensions.

The Motzkin path and structural matrix descriptors only describe the secondary structure’s topology. However, the chemical properties of a folded single-stranded DNA sequence also depend on the secondary structure’s energetic configuration. Next, we propose the Bag of Faces representation to characterize the energetic configurations of secondary structures.

5.2.3. Bag of faces descriptors

The folded secondary structure of a single-stranded DNA sequence is characterized by the faces of the secondary structure graph and their associated energies. Based on this observation, we propose a descriptor that counts the occurrences of face/energy configurations in a given folded sequence. We call such a descriptor Bag of Faces (BoF).

The concept of BoF is inspired by the Bag-of-Words descriptor commonly used in natural language processing to provide vectorized representations of text. The Bag-of-Words descriptor encodes the frequency of word occurrences in a given text. In this work, we propose using faces and their corresponding energies in the secondary structures in an analogous manner to generate Bag of Faces descriptors.

In the BoF model, a secondary structure is mapped onto a vector consisting of bags, where each bag, or entry of the vector, counts the occurrences of a particular face/energy type, e.g., [stack, −1.5], [hairpin, 2.2], and so on. Fig. 11 (upper right) provides a visual representation of a BoF descriptor representing the secondary structure of an exemplar single-stranded DNA sequence. The sequences in our datasets are characterized by a total of 277 face/energy configurations.

Note that in this work, we considered bags for the faces of the secondary structure graph and related energies, but other characteristics of the secondary structures can be considered instead. Future work in this direction is warranted. We note that subgraph structures called “graphlets” were recently considered for molecular fingerprinting using a linear model and histogram structure similar to ours, but also with a hierarchical structure of the graphlets [68]. We believe that such an approach is less relevant for aptamers, which already possess well-defined substructures through the faces.

5.3. Visualizing the space of secondary structures

Data visualization assists in exploring the configuration space by transforming large amounts of secondary structure data into compact visual formats. This facilitates the identification of patterns, relationships, and outliers that might not be evident if only a few data samples are analyzed at a time. The data visualization process provides two- or three-dimensional representations of vector-valued descriptors representing sequences in our dataset. Such lower-dimensional representations can be obtained using dimensionality reduction techniques.

A key aspect of dimensionality reduction is the preservation of the intrinsic geometry of the high-dimensional data manifold, e.g., preserving pairwise distances. Recent advancements in ML have produced efficient dimensionality reduction algorithms, such as t-distributed stochastic neighbor embedding (t-SNE) [69] (used here) and principal component analysis (PCA) [70]. Conceptually, a t-SNE is constructed by minimizing the divergence between two probability distributions: one that measures pairwise similarities of the points in the high-dimensional space and another that measures pairwise similarities of the points in the lower-dimensional space. The goal is to ensure that similar points in high-dimensional space remain close in the lower-dimensional representation.

A t-SNE performs non-linear dimensionality reduction. The t-SNE projection into the lower dimensional space preserves complex, non-linear relationships contrary to a PCA projection, which is linear. The t-SNE algorithm also depends on a hyperparameter called perplexity that balances the focus between local and global data structures during dimensionality reduction. Perplexity influences how many neighbors each point considers when embedding the data into lower dimensions. Smaller perplexity values emphasize local relationships by focusing on nearby points, while larger values consider a broader neighborhood, capturing more global patterns.

Choosing the right perplexity, typically through trial and error, is crucial as it can significantly affect the visualization outcome. We set the perplexity value to 100. Note that the computation cost of t-SNE is quadratic in the number of data points because it needs to compute similarities for all pairs of data points. Thus, this method may not be suitable for analyzing extremely large datasets. We employed the t-SNE algorithm from the scikit-learn Python library [71]. In our experiments, t-SNE computations took less than 32 s to run on our dataset consisting of 4450 sequences represented by the Motzkin-path descriptors³

Fig. 13 displays the t-SNE two-dimensional representations of the Motzkin path descriptors of the sequences in our dataset, introduced in Section 5.1. Each data point corresponds to a specific sequence with a color corresponding to its count number from the SELEX-NGS output; the brighter the color, the higher the count. Fig. 13 clearly illustrates that data points tend to cluster, indicating that there are sets of sequences whose secondary structures share high similarities. Additionally, the plot indicates that higher-count structures with brighter colors tend to be located in specific clusters or regions of the visualized space. Conversely, there are regions with solely low-count structures, for example, in the lower right corner. Note that the high-count structures are not isolated, so that we can find low-count sequences with similar secondary structure as the high-count sequences.

Fig. 13.

Two-dimensional representation using t-SNE of Motzkin path descriptors for our dataset of 4450 unique single-stranded DNA sequences. Motzkin path descriptors characterize the topology of the sequences’ secondary structure graph. Each plotted point represents a unique single-stranded DNA sequence. The brighter the color, the higher the count of the sequence. Illustrated are also the secondary structures of sequences ${\tilde{\mathit{x}}}_1$, ${\tilde{\mathit{x}}}_2$, ${\tilde{\mathit{x}}}_3$ and ${\tilde{\mathit{x}}}_4$ with counts in the top 0.1 percentile. Next to each of these is plotted an exemplary secondary structure of a single-count sequence in our dataset with the same topology.

For example, consider the four sequences with the highest SELEX-NGS counts. For each one, we can identify another sequence with a similar secondary structure and a different primary sequence, but having the lowest possible count of 1. In Fig. 13, we illustrate exemplary folded sequences, provide the associated counts for each sequence, and indicate the region of the two-dimensional space where those structures are represented. Fig. 15 shows the foldings of the top count apatmer from this screen.

Fig. 15.

A comparison of the folding algorithms using the high-count norepinephrine aptamer determined by the SELEX process. An initial stem length of four nucleotide pairs was forced in the folding. Notably, GMfold gives an intermediate structure between SeqFold and mfold/UNAfold. The sequence is 5’-ACGACGGGGCACATTGTGCTGTTCATCTGTTCCGCAGGAGAGTCGT-3’. Structure visualization generated using ViennaRNA, specifically forgi [50].

5.4. Similarity search

One important task for analyzing large datasets of single-stranded DNA sequences representing aptamer candidates is to find sequences with folded structures similar to those hypothesized to have ‘good’ binding properties, e.g., those associated with high NGS counts. When investigating the similarities between secondary structures, we can consider two aspects: structural and energetic. Structural similarity pertains to the topology of folded sequences, while energetic similarity refers to the energetic configurations of the secondary structures, such as the energy associated with the faces composing the secondary structures.

Using Motzkin path (or structural matrix) descriptors, we can identify sequences with secondary structures topologically equivalent to that of a given sequence of interest. Using the BoF, we can identify sequences with secondary structures with similar energetic configurations.

Consider $𝒟:={\left\{{\mathit{x}}_i\right\}}_{i=1}^n$ dataset of single-stranded DNA sequences, $ℳ:={\left\{{\mathit{m}}_i\right\}}_{i=1}^n$ set of associated Motzkin path descriptors, and $ℱ:={\left\{{\mathit{f}}_i\right\}}_{i=1}^n$, which is a set of related BoF representations, where ${\mathit{m}}_i$ and ${\mathit{f}}_i$ are the Motzkin path and BoF representations of the sequence ${\mathit{x}}_i$, respectively. Moreover, let us consider a sequence of interest $\stackrel{\mathit{‾}}{\mathit{x}}$, e.g., a high count sequence, with Motzkin path and BoF descriptors $\overline{\mathit{m}}$ and $\overline{\mathit{f}}$, respectively. Given ${ϵ}_m,{ϵ}_f\in {\mathbb{R}}^{+}$, we define the sets ${𝒯}_{\tilde{\mathit{x}},{ϵ}_m},{ℰ}_{\tilde{\mathit{x}},{ϵ}_m}\subset 𝒟$ of sequences with secondary structures topologically ${ϵ}_m$-similar and energetically ${ϵ}_f$-similar to $\tilde{\mathit{x}}$ as follows

$${𝒯}_{\tilde{\mathit{x}},{ϵ}_m}:=\left\{{\mathit{x}}_i\in 𝒟\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}{‖{\mathit{m}}_i-\tilde{\mathit{m}}‖}_2\le {ϵ}_m\right\},$$

$${ℰ}_{\tilde{\mathit{x}},{ϵ}_f}:=\{{\mathit{x}}_i\in 𝒟\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}{‖{\mathit{f}}_i-\tilde{\mathit{f}}‖}_2\le {ϵ}_f\}$$

where $\Vert \cdot {\Vert }_2$ is the Euclidean norm and ${ϵ}_m$ and ${ϵ}_f$ are parameters quantifying structural and energetic similarities between the representations. In particular, ${𝒯}_{\tilde{\mathit{x}},0}$ and ${ℰ}_{\tilde{\mathit{x}},0}$ are the set of sequences with secondary structures topologically and energetically equivalent to that of the sequence $\tilde{\mathit{x}}$, respectively.

We perform a structural end energetic similarity search on the dataset introduced in Section 5.1, consisting of 4450 unique single-stranded DNA sequences. One of the goals of the similarity search is to find sequences with secondary structures that have the same topological and energetic configurations as those with highest-count sequences, which are then predicted (but not known) to have good binding properties. In particular, we aim to find sequences with properties similar to those sequences with counts in the top 0.1 percentile. These are sequences whose counts are among the highest 0.1%, meaning their count values exceed that of 99.9% sequences in the dataset. There are only five such sequences in our dataset.

Notably, the third-highest-count sequence (with count 48,708) has a secondary structure with the same topology as the top high-count sequence (with count 77,352). Thus, to restrict our analysis to a diverse set of topological structures, we only study the first, second, fourth, and fifth highest-count structures. We refer to the four sequences we analyze as ${\tilde{\mathit{x}}}_1$ with count $\mathrm{77,352}$, ${\tilde{\mathit{x}}}_2$ with count $\mathrm{67,049}$, ${\tilde{\mathit{x}}}_3$ with count 28,432 and ${\tilde{\mathit{x}}}_4$ with count 12,126. The secondary structures of ${\tilde{\mathit{x}}}_1$, ${\tilde{\mathit{x}}}_2,{\tilde{\mathit{x}}}_3$ and ${\tilde{\mathit{x}}}_4$ are illustrated in Fig. 13.

Performing structural similarity searches on our dataset we find that it contains 103 sequences with secondary structures sharing the same topology as ${\tilde{\mathit{x}}}_1$, 59 sequences similar to ${\tilde{\mathit{x}}}_2$, 34 sequences sim ${\tilde{\mathit{x}}}_3$, and 36 sequences similar to ${\tilde{\mathit{x}}}_4$. That is, $\left|{𝒯}_{{\tilde{\mathit{x}}}_1,0}\right|=103$, $\left|{𝒯}_{{\tilde{\mathit{x}}}_2,0}\right|=59$, $\left|{𝒯}_{{\tilde{\mathit{x}}}_3,0}\right|=34$ and $\left|{𝒯}_{{\tilde{\mathit{x}}}_4,0}\right|=36$.

The bar plots in Fig. 14 illustrate the number of different energetic profiles associated with sequences in ${\left\{{𝒯}_{{\tilde{\mathit{x}}}_i,0}\right\}}_{i=1}^4$ and the number of sequences for each of the energetic profiles. Clearly, sequences may share the same topology in their folded configuration but be associated with different energetic profiles. In our dataset, the topology of the secondary structure of ${\tilde{\mathit{x}}}_1$ can be associated with nine distinct energetic profiles, ${\tilde{\mathit{x}}}_2$ with three, ${\tilde{\mathit{x}}}_3$ with seven, and ${\tilde{\mathit{x}}}_4$ with eleven energy profiles. In the bar plots in Fig. 14, the striped bars are associated with the energetic configurations of at least one high-count sequence. In particular, Fig. 14 indicates that our dataset contains 80 sequences with secondary structures sharing the same topology and energetic configuration as ${\tilde{\mathit{x}}}_1$, 56 as ${\tilde{\mathit{x}}}_2$, 26 as ${\tilde{\mathit{x}}}_3$, and 20 as ${\tilde{\mathit{x}}}_4$. That is, $\left|{𝒯}_{{\tilde{\mathit{x}}}_1,0}\cap {ℰ}_{{\tilde{\mathit{x}}}_1,0}\right|=80,\left|{𝒯}_{{\tilde{\mathit{x}}}_2,0}\cap {ℰ}_{{\tilde{\mathit{x}}}_2,0}\right|=56$, $\left|{𝒯}_{{\tilde{\mathit{x}}}_3,0}\cap {ℰ}_{{\tilde{\mathit{x}}}_3,0}\right|=26$ and $\left|{𝒯}_{{\tilde{\mathit{x}}}_4,0}\cap {ℰ}_{{\tilde{\mathit{x}}}_4,0}\right|=20$.

Fig. 14.

The bar plots show the Bag of Faces (BoF) energetic configurations on the x-axes, and the number of sequences with each energetic configuration on the y-axes, for sequences in ${𝒯}_{{\tilde{\mathit{x}}}_1,0}$, ${𝒯}_{{\tilde{\mathit{x}}}_2,0}$, ${𝒯}_{{\tilde{\mathit{x}}}_3,0}$ and ${𝒯}_{{\tilde{\mathit{x}}}_4,0}$, ${\left\{{𝒯}_{{\tilde{\mathit{x}}}_i,0}\right\}}_{i=1}^4$ are the sets of sequences with secondary structure topologically equivalent to one of the four high-count sequences we analyze: ${\tilde{\mathit{x}}}_1$ with count 77352, ${\tilde{\mathit{x}}}_2$ with count 67049, ${\tilde{\mathit{x}}}_3$ with count 28432 and ${\tilde{\mathit{x}}}_4$ with count 12126. The heights of the bars are shown in log scale. The numbers on top of each bar are the numbers of sequences with the respective energetic configuration. The bars with stripes are associated with the energetic configurations of high-count sequences.

Recall that the bar plots in Fig. 14 relate to the first, second, fourth, and fifth highest-count sequences, which we call ${\mathit{x}}_1$, ${\mathit{x}}_2$, ${\mathit{x}}_3$, and ${\mathit{x}}_4$, respectively. The sequence with the third-highest count (48,708) has a secondary structure with the same topology as ${\mathit{x}}_1$, the highest-count sequence.

In Fig. 14a there is only one bar with stripes. That is, only one configuration (Configuration I) corresponds to a high-count sequence. This is because the third and the first highest-count sequences share the same energetic configuration.

5.5. Topic modeling and spectral clustering

We can also cluster sequences based on the energetic configurations of their secondary structures. To achieve that, we build latent topic models using BoF descriptors. Next, we use the identified topic distributions to describe and cluster sequences.

Topic modeling is a machine learning technique used to discover hidden semantic structures within a corpus of documents [72]. By analyzing word co-occurrences across documents, topic modeling identifies underlying topics, where each topic is represented as a distribution of words, and each document is a mixture of these topics. Common unsupervised ML methods, such as Latent Dirichlet Allocation (LDA) [73] and Nonnegative Matrix Factorization (NMF) [74], enable the automatic association of each text with a distribution over topics. Given a text and a number of topics, these methods quantify how relatable the given text is to each identified topic.

To perform topic modeling, we use Nonnegative Matrix Factorization (NMF), which is a linear algebraic method. Linear approaches have the advantage of being computationally efficient and scalable to massive data sets. The NMF model discovers interpretable latent components in high-dimensional unlabeled data. It analyses a set of documents described by the counts of unique words. In particular, NMF takes as input the so-called term-document matrix $X\in {\mathbb{R}}^{w\times t}$. Each of the $w\in \mathbb{N}$ rows of $X$ correspond to a unique word in the vocabulary, and each of the $t\in \mathbb{N}$ columns correspond to a text. The ($i,j$)-th entry of $X$ counts the number of occurrences of the $i$th word in the $j$th document.

Next, NMF decomposes $X\in {\mathbb{R}}^{w\times t}$ into two lower-dimensional non-negative matrices, $W\in {\mathbb{R}}^{w,k}$ and $H\in {\mathbb{R}}^{k,t}$, such that: $X\approx W\cdot H$. $W$ captures the basis vectors (e.g., topics), and $H$ encodes the coefficients or contributions of each basis vector (e.g., the importance of each topic in each document). Both $W$ and $H$ are constrained to be non-negative, which aligns with many real-world scenarios, such as text data where counts cannot be negative. The hyperparameter $k$ must be defined a priori . It determines the number of latent components (or topics) identified by the NMF algorithm. The NMF model computes $W$ and $H$ by solving the following optimization problem:

$$\underset{W,H>0}{\mathrm{m}\mathrm{i}\mathrm{n}}{\Vert X-WH\Vert }_F^2,$$

where $\Vert \cdot {\Vert }_F$ is the Frobenius norm.

Here, we consider single-stranded DNA sequences instead of text. We analyze occurrences of face-energy configurations across secondary structures instead of words. Consequently, in the matrix $X$, we give as input to NMF, that each column is associated with a sequence, and each row corresponds to a face energy configuration. The ($i,j$)-th entry of 𝑋 counts the number of occurrences of the $i$–th face-energy configuration in the $j$th document. Here, each column of the matrix $X$ is the BoF descriptor of the related sequence.

The rows of $W$ characterize each topic as a distribution of face-energy configurations, while the columns of $H$ describe the secondary structure of each sequence by quantifying how relatable it is to each one of the identified topics. In particular, the columns of the matrix $H$ associate the secondary structure of each sequence with a mixture of different topics. Hence, we can describe secondary structures with their associated topic mixture distributions. We introduce an additional concept of similarity. The distance between any two secondary structures can be measured by comparing how dissimilar their topic mixture distributions are. This means determining how different are the columns of matrix $H$, associated with the two sequences of interest. We employ the NMF algorithm from the scikit-learn Python library [71] considering 25 topics. The number of topics for topic modeling is selected by the user. In our experiments, NMF on our 4450 BoF descriptors takes less than two seconds to run^5.3.

Next, we aim to identify clusters of sequences described by similar topic mixture distributions. Spectral clustering is a well-known graph-based clustering ML technique that leverages the eigenvalues of a similarity matrix to partition data into distinct groups. It constructs a similarity graph from the input data points and then uses the eigenvectors of the graph Laplacian to identify clusters. Spectral clustering effectively captures non-linear structures in the data, making it particularly useful for complex clustering tasks in machine learning. We employ the spectral clustering algorithm from the scikit-learn Python library [71]. The number of clusters we aim to identify must be defined a priori and provided as input to the clustering algorithm. In line with our choice to consider 25 topics, we aim to identify 25 distinct clusters. In our experiments, spectral clustering takes less than three seconds to run on our dataset consisting of 4450 sequences represented by the topic mixture distributions^5.3.

Recall that the primary goal of our data analysis is to identify sequences that have similar structural and energetic configurations as high-count sequences. In what follows, we define a sequence to be high-count if the associated count is in the top 0.1 percentile. There are five such sequences in our dataset, four of which are ${\tilde{\mathit{x}}}_1$, ${\tilde{\mathit{x}}}_2{\tilde{\mathit{x}}}_3{\tilde{\mathit{x}}}_4$ which we analyzed in the previous section. The fifth sequence is the third highest count sequence, which is topologically and energetically equivalent to ${\tilde{\mathit{x}}}_1$. The bar plot in Fig. 16 illustrates the number of sequences per cluster. The bars with stripes are those associated with a cluster containing at least one high-count sequence. Only four of the twenty-five clusters contain one of the five high-count sequences because the first and third highest-count sequences share the same structural and energetical configuration and are associated with the same cluster. Cluster M contains the highest count sequence ${\tilde{\mathit{x}}}_1$ and the third highest count sequence, cluster E contains ${\tilde{\mathit{x}}}_2$, cluster W contains ${\tilde{\mathit{x}}}_3$, and cluster X contains ${\tilde{\mathit{x}}}_4$. According to our clustering results, the clusters with at least one high-count sequence count 457 elements. That is, only less than 11% of the sequences in our dataset share similarities with a high-count sequence, suggesting that further research efforts should prioritize the analysis of this subset of sequences, screening out the remaining 89%.

Fig. 16.

On the left-hand side of the figure is a t-SNE two-dimensional representation of topic distribution descriptors for our dataset of 4450 unique single-stranded DNA sequences. Topic distribution descriptors represent information related to the energetic configuration of the secondary structures of the sequences. Each plotted point represents a unique single-stranded DNA sequence. Each color is associated with a different cluster. The alphabetic characters and the dotted circles highlight the clusters containing at least one sequence with an associated count in the top 0.1 percentile. For each such cluster, we provide an exemplary low-count secondary structure. On the right-hand side is a bar plot illustrating the computed clusters and the number of sequences per cluster. Bars with stripes are associated with sequences with counts in the top 0.1 percentile. The numbers above each bar are the counts of sequences included in the respective cluster. The heights of the bars are shown in log scale.

Fig. 16 displays a t-SNE two-dimensional representation of the descriptors obtained via topic modeling. Each data point is associated with a specific color. Each color is associated with a specific cluster. Points with the same color are in the same cluster. Fig. 16 allows us to visualize the entire space of folded structures and the cluster associated with each sequence. In the figure, we also highlight the clusters that include at least one high-count sequence with the corresponding alphabetic character. For each of these clusters, we propose an example of a low-count secondary structure that shares structural and energetic similarities with the high-count sequence in the cluster. The similarities are determined by the descriptors obtained through topic modeling.

Interestingly, even if the clusters have been computed on the high-dimensional descriptors, the two-dimensional representation in a cluster tends to aggregate in specific regions of the Euclidean space. This suggests that similarity of the high-dimensional descriptors is well represented in the lower-dimensional space. Moreover, from the geometric distribution of the two-dimensional representations, it is clear that points in the same cluster may aggregate in more than one region of the Euclidean space. This suggests that the dimensionality reduction highlights additional hidden patterns between sequences within the same cluster that can potentially be exploited to segment the larger clusters further.

6. Conclusions

We developed new Python code for high-throughput processing of DNA sequence secondary structures. This code can be directly paired with the SELEX directed evolution method to categorize large numbers of aptamer candidates using modern machine learning methods. The workflow enables comparing, contrasting, and clustering aptamer candidates based on secondary structure and energetics. Namely, it allows sequences with similar structure and energetics to the few sequences with the highest amplification counts from the SELEX process to be identified.

Our code, GMfold, can be expanded to include those items mentioned in Section 3. The pipeline presented is ready to be paired with experiments to determine aptamer candidate target-binding affinity and selectivity. The machine learning methodologies presented in Section 5 can be expanded, as additional studies are carried out, to test additional clustering methods. Of note is the fact that we focused on unsupervised clustering methods. Most other statistical and ML methods for aptamers are supervised methods. It would be interesting to consider active learning approaches for semi-supervised methods in which laboratory data are incorporated with the ML and SELEX processes.

A natural next step is to incorporate aptamer 3D structures into the machine learning process. This is more challenging due to the high dependency on the accuracy of the secondary structure information provided. Programs like AMBER24 [75,76] use the mechanical force field simulation of the target molecule, usually followed by energy minimization software to do molecular docking analyses. These types of simulations are helpful in identifying potential aptamer-target binding sites, which can be experimentally investigated via biophysical techniques like crystallography, nuclear magnetic resonance spectroscopy, or cryo electron microscopy (cryo-EM). While all-atom X-ray or cryo-EM structures for some aptamers have been reported, generally, ssRNA and ssDNA present substantial challenges for biophysical structure determination due to the harsh conditions needed (e.g., forming crystals), high negative charge densities in polyphosphate backbones, labor-intensive and expensive work, and uncertainty of success.

Importantly, crystal structures cannot capture oligonucleotide dynamic solution conformations important for target recognition, as well as signal transduction in biosensing. Structural information on shorter aptamers (in the 30–50mer range) that bind small molecules has been more successfully elucidated using NMR. However, overlap and broadening on NMR signals requires a significant amount of analysis, most notably complicating tertiary structure determination. As such, more comprehensive in silico secondary, and in the future, tertiary structure predictions will be beneficial in multiple facets of the aptamer selection process. Following SELEX, it will enable more accurate structural predictions which can guide future experimentation.

Fig. 12.

A visualization of the chording procedure introduced in Section 5.2.2. The sequence structure and Motzkin path found in Fig. 11. Lower line gives the associated parenthetical sequence. Red lines connect interacting nucleotides and have corresponding parentheses in the parenthetical sequence.

Appendix

A.1. Corrections to the SeqFold energy functions

SeqFold 2.0 and GMfold, primarily rely on the functions used in SeqFold to compute energy values of the various faces, such as hairpins, bulges and inner loops. But, we make two straightforward improvements to such energy functions. Specifically, we modify the computation of the energies associated with internal loops generated by single base-pair mismatches and with junctions, which are multi-branches with no unpaired nucleotide between the branches.

The energy associated with internal loops from single base pair mismatches depends on the energy of the terminal mismatches of the loop. Terminal mismatches occur at the ends of a loop’s double-stranded regions and consist of four nucleotides: two base pairing and the other two inside the loop and not matching. We can associate a left and right terminal mismatch to each loop depending on the 5’ to 3’ direction. Fig. 17 illustrates a folded sequence with an internal loop from a base pair mismatch and highlights the left and right terminal mismatches. The right terminal mismatch, enclosed in the dashed box, consists of nucleotides CC/AG, while the left one, enclosed in the continuous line box, consists of nucleotides GC/CA. Following along [49], we can compute the free energy associated with a base pair mismatch loop with the following formula

$$\begin{array}{r}{G}_T^{\circ }(\text{single}\text{mismatch}\text{loop})=\Delta G_T^{\circ }(\text{left}\text{terminal}\text{mismatch})\\ +\Delta G_T^{\circ }\left(\text{right}\text{terminal}\text{mismatch}\right),\end{array}$$

where $\Delta G_T^{\circ }$ (left (right) terminal mismatch) is calculated using (2) from the enthalpy and entropy associated with the terminal mismatches. Tables 2 and 3 report the energies SeqFold assigns to terminal mismatches of inner loops. Table 2 is used when the terminal mismatch does not involve any nucleotide that is either the first or the last in the sequence. We refer to such mismatches as ‘internal’. Table 3 is used when the terminal mismatch includes either the first or the last nucleotide in the sequence.

Fig. 17.

Single stranded DNA secondary structure with inner loop from single base-pair mismatch. Highlighted in the boxes are the terminal mismatches of the internal loop. The notation WX/YZ indicates that W and X are consecutive nucleotides on the DNA strand and Y and Z are the corresponding consecutive nucleotides on the complementary portion of the DNA strand. That is, W pairs or mismatches with Y, and X pairs or mismatches with Z.

Unfortunately, SeqFold inaccurately computes the energy associated with single base pair mismatch loops due to an incorrect assessment of the left terminal mismatch. Specifically, according to SeqFold, the left terminal mismatch consists of the base pairs at the end of the two stacks that determine the loop. For instance, in the example in Fig. 17, SeqFold identifies GC/CG as the left terminal mismatch. We fix the issue and assess terminal mismatches correctly.

Regarding the computation of energy associated with Junctions, SeqFold associates a junction with energy ${ℰ}_m=4.6+E_{st}$, where $E_{st}$ is the sum of the energies associated to each of the branches (or stacks) of the junction. Alternatively, we associate junctions with lower energy and set ${ℰ}_m=0.6+E_{st}$. By associating junctions with lower energy, we consider these structures to be more stable than SeqFold does. We do not back our decision to modify the energy in this way with empirical laboratory experiments. We tune the constant in the junction energy computation using a heuristic approach to emulate mfold folding behavior. We note that in our experience, both SeqFold and SeqFold 2.0 constantly fail to predict junctions where both GMfold and mfold do. We provide examples in Fig. 8, which illustrates a comparison of the secondary structures of two exemplary sequences computed with SeqFold, SeqFold 2.0, GMfold and mfold. The figure shows that SeqFold and SeqFold 2.0 fail to predict the junction in both examples. After a first empirical investigation, we speculate that this limitation of the SeqFold approaches is not only due to the junction energy computation but also to how they implement the dynamic programming approach. In particular, we think SeqFold and SeqFold 2.0 do not consider junctions as possible faces. Further investigation in this direction is warranted.

A.2. Detailed energy functions

SeqFold, SeqFold 2.0, and GMfold use the same energy functions for hairpin loops, internal loops, bulges, and stacks. As in [49], we use energies for 37° c with salt concentration 1 M NaCl. Our code does allow modification of the temperature. The free energy values are calculated from enthalpy and entropy values given in a table from [49] and adjusted for the appropriate temperature as follows (Note the enthalpy increment is in kilocalories/mol and the entropy increment is given in calories/mol, hence the factor of 1000):

$$\Delta G_T^{\circ }=\Delta H-T\cdot \frac{\Delta S}{1000}$$ (2)

For ease of writing, we will write everything from now on in terms of $\Delta G^{\circ }$, but note that all of the tables store value of entropy and enthalpy.

As discussed previously, we compute the energy of the structure associated to the substring $[i,j]$, we note that this will be the sum of the face energy that has pair ($i,j$), and for each internal pair in this face, ($i^{\prime },j^{\prime }$) we add the energy of the structure associated to the substring $\left[i^{\prime },j^{\prime }\right]$. This is implemented as finding the minimum of 3 energies, corresponding to the 3 types of faces that can be present.

For hairpins, we have a look-up table (see Table 4) for all loops of length 3 and 4 which depends on the base pairs present [49]. For hairpin lengths 5 through 30, we have a reference table depending only on the length of the loop (see Fig. 21). For anything longer, we use the Jacobson–Stockmayer energy extrapolation formula [49]:

$$\Delta G_n^{\circ }=\Delta G_{n_{max}}^{\circ }+2.44\cdot R\cdot T\cdot ln\left(\frac{n}{n_{max}}\right).$$ (3)

Table 2.

Enthalpies and Entropies associated with DNA internal terminal mismatches. The reported values are taken from SeqFold GitHub repository and are valid for temperature T = 37°. Internal terminal mismatches are sets of four nucleotides WX/YZ where W is not the first nucleotide of the DNA string and Z is not the last. The energies are the same for each terminal mismatch in the reverse direction. That is, WX/YZ has the same energy as ZY/XW.

Internal Terminal mismatch	Enthalpy (kcal/mol)	Entropy (cal/mol)
AG/TT	1.0	0.9
AT/TG	−2.5	−8.3
CG/GT	−4.1	−11.7
CT/GG	−2.8	−8.0
GG/CT	3.3	10.4
GG/TT	5.8	16.3
GT/CG	−4.4	−12.3
GT/TG	4.1	9.5
TG/AT	−0.1	−1.7
TG/GT	−1.4	−6.2
TT/AG	−1.3	−5.3
AA/TG	−0.6	−2.3
AG/TA	−0.7	−2.3
CA/GG	−0.7	−2.3
CG/GA	−4.0	−13.2
GA/CG	−0.6	−1.0
GG/CA	0.5	3.2
TA/AG	0.7	0.7
TG/AA	3.0	7.4
AC/TT	0.7	0.2
AT/TC	−1.2	−6.2
CC/GT	−0.8	−4.5
CT/GC	−1.5	−6.1
GC/CT	2.3	5.4
GT/CC	5.2	13.5
TC/AT	1.2	0.7
TT/AC	1.0	0.7
AA/TC	2.3	4.6
AC/TA	5.3	14.6
CA/GC	1.9	3.7
CC/GA	0.6	−0.6
GA/CC	5.2	14.2
GC/CA	−0.7	−3.8
TA/AC	3.4	8.0
TC/AA	7.6	20.2
AA/TA	1.2	1.7
CA/GA	−0.9	−4.2
GA/CA	−2.9	−9.8
TA/AA	4.7	12.9
AC/TC	0.0	−4.4
CC/GC	−1.5	−7.2
GC/CC	3.6	8.9
TC/AC	6.1	16.4
AG/TG	−3.1	−9.5
CG/GG	−4.9	−15.3
GG/CG	−6.0	−15.8
TG/AG	1.6	3.6
AT/TT	−2.7	−10.8
CT/GT	−5.0	−15.8
GT/CT	−2.2	−8.4
TT/AT	0.2	−1.5

Here $n$ is the length of the sequence one wants to calculate, $n_{max}$ is the maximum length with an experimentally known value, $R$ is the ideal gas constant, and 2.44 is an experimentally determined constant. The terminal mismatch penalty is added to this energy value.

The formula for an inner loop is a bit more complicated. From [49], we have the following formula.

$$\Delta G_{\mathit{inner}}^{\circ }=\Delta G_n^{\circ }+\Delta G_{\mathit{asym}}^{\circ }+\Delta G_{TMM}^{\circ }$$ (4)

Where the components are $\Delta G_n^{\circ }$, the energy given the length of the inner loop (we have a table with experimental values up to length 30 and then extend using the Jacobson–Stockmayer equation). $\Delta G_{\mathit{asym}}^{\circ }=\left|n_l-n_r\right|\times 0.3\mathrm{k}\mathrm{c}\mathrm{a}\mathrm{l}{\mathrm{m}\mathrm{o}\mathrm{l}}^{-1}$ is the asymmetric penalty ($n_l$, $n_r$ are the lengths of each side of the loop). $\Delta G_{TMM}^{\circ }$ is the terminal mismatch penalty, and is the sum of the penalty from the inner end of the loop and the outer end of the loop.

Table 3.

Enthalpies and Entropies associated with DNA terminal mismatches. The reported values are taken from SeqFold GitHub repository and are valid for temperature T = 37°. The energies are the same for each terminal mismatch in the reverse direction. That is, WX/YZ has the same energy as ZY/XW.

Terminal mismatch	Enthalpy (kcal/mol)	Entropy (cal/mol)
AA/TA	−3.1	−7.8
TA/AA	−2.5	−6.3
CA/GA	−4.3	−10.7
GA/CA	−8.0	−22.5
AC/TC	−0.1	0.5
TC/AC	−0.7	−1.3
CC/GC	−2.1	−5.1
GC/CC	−3.9	−10.6
AG/TG	−1.1	−2.1
TG/AG	−1.1	−2.7
CG/GG	−3.8	−9.5
GG/CG	−0.7	−19.2
AT/TT	−2.4	−6.5
TT/AT	−3.2	−8.9
CT/GT	−6.1	−16.9
GT/CT	−7.4	−21.2
AA/TC	−1.6	−4.0
AC/TA	−1.8	−3.8
CA/GC	−2.6	−5.9
CC/GA	−2.7	−6.0
GA/CC	−5.0	−13.8
GC/CA	−3.2	−7.1
TA/AC	−2.3	−5.9
TC/AA	−2.7	−7.0
AC/TT	−0.9	−1.7
AT/TC	−2.3	−6.3
CC/GT	−3.2	−8.0
CT/GC	−3.9	−10.6
GC/CT	−4.9	−13.5
GT/CC	−3.0	−7.8
TC/AT	−2.5	−6.3
TT/AC	−0.7	−1.2
AA/TG	−1.9	−4.4
AG/TA	−2.5	−5.9
CA/GG	−3.9	−9.6
CG/GA	−6.0	−15.5
GA/CG	−4.3	−11.1
GG/CA	−4.6	−11.4
TA/AG	−2.0	−4.7
TG/AA	−2.4	−5.8
AG/TT	−3.2	−8.7
AT/TG	−3.5	−9.4
CG/GT	−3.8	−9.0
CT/GG	−6.6	−18.7
GG/CT	−5.7	−15.9
GT/CG	−5.9	−16.1
TG/AT	−3.9	−10.5
TT/AG	−3.6	−9.8

If the interior loop is instead a stack, we have a look-up table for all stacking energies, as described above.

If the inner loop is a bulge, we have a few cases. The first case is if the bulge has size 1, then we add the stacking energies on each side, then the penalty for the internal nucleotide and due to the bulge strain, if one of the closing ends is an $A$, we get an $AT$ penalty:

$$\Delta G_{\mathit{bulge}}^{\circ }=\Delta G_{\mathit{stack}}^{\circ }+\Delta G_{NN}^{\circ }+\Delta G_{AT}^{\circ }.$$

For longer bulges, we no longer have the penalty for the loop dependent on which nucleotides are contained in the bulge, and only on the length, the values for this are given by a table for lengths 1 through 30 and the Jacobson–Stockmayer formula is used to extend to loops longer than 30.

Table 4.

Enthalpies and Entropies associated with all hairpin loops of length 3 or 4. ΔH is in units of kcal/mol, and ΔS is in units of cal/mol. The reported values are taken from SeqFold GitHub repository and are valid for temperature T = 37°. literature values given in [49].

Tri/Tetra loops	ΔH	ΔS	Tri/Tetra loops	ΔH	ΔS	Tri/Tetra loops	ΔH	ΔS	Tri/Tetra loops	ΔH	ΔS
AGAAT	−1.5	0.0	AGCAT	−1.5	0.0	AGGAT	−1.5	0.0	AGTAT	−1.5	0.0
CGAAG	−2.0	0.0	CGCAG	−2.0	0.0	CGGAG	−2.0	0.0	CGTAG	−2.0	0.0
GGAAC	−2.0	0.0	GGCAC	−2.0	0.0	GGGAC	−2.0	0.0	GGTAC	−2.0	0.0
TGAAA	−1.5	0.0	TGCAA	−1.5	0.0	TGGAA	−1.5	0.0	TGTAA	−1.5	0.0
AAAAAT	0.5	0.6	AAAACT	0.7	−1.6	AAACAT	1.0	−1.6	ACTTGT	0.0	−4.2
AGAAAT	−1.1	−1.6	AGAGAT	−1.1	−1.6	AGATAT	−1.5	−1.6	AGCAAT	−1.6	−1.6
AGCGAT	−1.1	−1.6	AGCTTT	0.2	−1.6	AGGAAT	−1.1	−1.6	AGGGAT	−1.1	−1.6
AGGGGT	0.5	−0.6	AGTAAT	−1.6	−1.6	AGTGAT	−1.1	−1.6	AGTTCT	0.8	−1.6
ATTCGT	−0.2	−1.6	ATTTGT	0.0	−1.6	ATTTTT	−0.5	−1.6	CAAAAG	0.5	1.3
CAAACG	0.7	0.0	CAACAG	1.0	0.0	CAACCG	0.0	0.0	CCTTGG	0.0	−2.6
CGAAAG	−1.1	0.0	CGAGAG	−1.1	0.0	CGATAG	−1.5	0.0	CGCAAG	−1.6	0.0
CGCGAG	−1.1	0.0	CGCTTG	0.2	0.0	CGGAAG	−1.1	0.0	CGGGAG	−1.0	0.0
CGGGGG	0.5	1.0	CGTAAG	−1.6	0.0	CGTGAG	−1.1	0.0	CGTTCG	0.8	0.0
CTTCGG	−0.2	0.0	CTTTGG	0.0	0.0	CTTTTG	−0.5	0.0	GAAAAC	0.5	3.2
GAAACC	0.7	0.0	GAACAC	1.0	0.0	GCTTGC	0.0	−2.6	GGAAAC	−1.1	0.0
GGAGAC	−1.1	0.0	GGATAC	−1.6	0.0	GGCAAC	−1.6	0.0	GGCGAC	−1.1	0.0
GGCTTC	0.2	0.0	GGGAAC	−1.1	0.0	GGGGAC	−1.1	0.0	GGGGGC	0.5	1.0
GGTAAC	−1.6	0.0	GGTGAC	−1.1	0.0	GGTTCC	0.8	0.0	GTTCGC	−0.2	0.0
GTTTGC	0.0	0.0	GTTTTC	−0.5	0.0	GAAAAT	0.5	3.2	GAAACT	1.0	0.0
GAACAT	1.0	0.0	GCTTGT	0.0	−1.6	GGAAAT	−1.1	0.0	GGAGAT	−1.1	0.0
GGATAT	−1.6	0.0	GGCAAT	−1.6	0.0	GGCGAT	−1.1	0.0	GGCTTT	−0.1	0.0
GGGAAT	−1.1	0.0	GGGGAT	−1.1	0.0	GGGGGT	0.5	1.0	GGTAAT	−1.6	0.0
GGTGAT	−1.1	0.0	GTATAT	−0.5	0.0	GTTCGT	−0.4	0.0	GTTTGT	−0.4	0.0
GTTTTT	−0.5	0.0	TAAAAA	0.5	−0.3	TAAACA	0.7	−1.6	TAACAA	1.0	−1.6
TCTTGA	0.0	−4.2	TGAAAA	−1.1	−1.6	TGAGAA	−1.1	−1.6	TGATAA	−1.6	−1.6
TGCAAA	−1.6	−1.6	TGCGAA	−1.1	−1.6	TGCTTA	0.2	−1.6	TGGAAA	−1.1	−1.6
TGGGAA	−1.1	−1.6	TGGGGA	0.5	−0.6	TGTAAA	−1.6	−1.6	TGTGAA	−1.1	−1.6
TGTTCA	0.8	−1.6	TTTCGA	−0.2	−1.6	TTTTGA	0.0	−1.6	TTTTTA	−0.5	−1.6
TAAAAG	0.5	1.6	TAAACG	1.0	−1.6	TAACAG	1.0	−1.6	TCTTGG	0.0	−3.2
TGAAAG	−1.0	−1.6	TGAGAG	−1.0	−1.6	TGATAG	−1.5	−1.6	TGCAAG	−1.5	−1.6
TGCGAG	−1.0	−1.6	TGCTTG	−0.1	−1.6	TGGAAG	−1.0	−1.6	TGGGAG	−1.0	−1.6
TGGGGG	0.5	−0.6	TGTAAG	−1.5	−1.6	TGTGAG	−1.0	−1.6	TTTCGG	−0.4	−1.6
TTTTAG	−1.0	−1.6	TTTTGG	−0.4	−1.6	TTTTTG	−0.5	−1.6

For multi-branches, the face energy is given by

$$\mathrm{\Delta }{G}_{MB}^{\circ }=a+b\cdot N_{br}+c\cdot N_{\mathit{unpaired}}-d\cdot {\delta }_{\mathit{fully}\mathit{stacked}},$$

where $a$, $b$, $c$, $d$ are energy parameters defined in [49] (We use values (2.6, 0.2, 0.2, 2.0)), $N_{br}$ is the number of branches in the multibranch structure, $N_{\mathit{unpaired}}$ is the number of unpaired nucleotides in the multi-branch, and ${\delta }_{\mathit{fully}\mathit{stacked}}$ is zero if $N_{\mathit{unpaired}}\ne 0$, and one otherwise, and corresponds to the stabilization that is gained by the multibranch being fully stacked, sometimes called the coaxial stacking stabilization. Note that this particular energy function is linear with the length of the multibranch. [49] suggest that a logarithmic dependence may be more optimal, i.e. an energy function similar to the Jacomson–Stockmayer equation. If given appropriate energy values, such a function could be implemented in either SeqFold 2.0 or GMFold.

A.3. Examples with aptamers to illustrate the difference in the algorithms

As mentioned in Section 3.3, our algorithm has a few key differences from mfold not only in optimization technique but also in energy computation. Here are some examples that illustrate these differences. Fig. 18 exemplifies the difference in coaxial stacking computation, as the mfold structure seems to gain additional stability in the stacked configuration when compared to the configuration found by the other software. Figs. 19 and 20, show aptamers for Cocaine and a membrane protein of a glioma cell line SHG44 respectively. These aptamers are examples of an open multibranch structure (i.e. the “open face” has multiple branches), where it seems that methods other than mfold do not find the same stability of this type of structure. The examples provided in the main text also show that there may be subtle changes in the energy values of particular structures used in mfold’s code that we have been unable to find in the literature.

A.4. GitHub accessibility

The source code is available in a GitHub repository https://github.com/PaClimaco/GMfold. The READ ME file describes the layout of the repository. In the repository, the environment file contains the details of the python environment used to run the above experiments. The rest of the code is broken into three folders. The data folder contains in comma separated value (CSV) format the sequences from the SeqFold repository [15] and the folded data frame from the SELEX screen. The raw SELEX screen files and cleaned files are in the respective subfolders.

The notebooks folder contains the files used to generate the figures in the paper, including notebooks for comparing the algorithms, folding DNA aptamers at scale, conducting the similarity search over folded structures, and applying the topic modeling techniques laid out in the paper.

The src or source folder contains the source code for DNA folding, including the version of SeqFold used for this paper (accessed July of 2024), our modification to this algorithm, SeqFold 2.0, and our algorithm GMfold. It also includes the necessary energy functions and energy lookup tables [49,79].

Fig. 18.

The Cocaine aptamer with sequence 5’-ACAGCTGGGTGAAGTAACTTCCTAAAAGGAACAGAGGG-3’ [77]. secondary structure as determined by SeqFold, SeqFold 2.0, GMfold and mfold as implemented within the UNAfold software. Structure visualization generated using ViennaRNA, specifically forgi.[50]. This example illustrates the different ways each method addresses co-axial stacking.

Fig. 19.

Comparison folding of an exemplary sequence between SeqFold, SeqFold 2.0, GMfold and mfold as implemented within the UNAfold software. This aptamer binds to a membrane protein of the glioma cell lineSHG44. The sequence being folded is 5’-CACAGGTTCCAGGTAATACCTAAGGGTATGCTCTCGCCTATTATATGGAGCAC-3’. Structure visualization generated using ViennaRNA, specifically forgi.[50].

Fig. 20.

Comparison folding of an exemplary sequence between SeqFold, SeqFold 2.0, GMfold and mfold as implemented within the UNAfold software. The sequence is an aptamer for Botulinum neurotoxin type A [78]. The sequence being folded is 5’-TTTTATTTTATTTTATTTTAAAAGGCGAATTCAGGGGACGTAGCAATGACTGCC-3’. Structure visualization generated using ViennaRNA, specifically forgi.[50].

Fig. 21.

Loop energy table. These values are used to compute the energy of inner loops, bulge loops, and hairpin loops. The energy of larger loops is calculated using the Jacobson–Stockmayer energy extrapolation formula.[49].

Data availability

We have made the code and data available in a github repository and we include appendix A4 describing how to access it.