The Figure of Merit for CRISPR-Based Nucleic Acid-Sensing System: Improvement Strategies and Performance Comparison

Abstract

CRISPR-based nucleic acid-sensing systems have grown rapidly in the past few years. Nevertheless, an objective approach to benchmark the performances of different CRISPR sensing systems is lacking due to the heterogeneous experimental setup. Here, we developed a quantitative CRISPR sensing figure of merit (FOM) to compare different CRISPR methods and explore performance improvement strategies. The CRISPR sensing FOM is defined as the product of the limit of detection (LOD) and the associated CRISPR reaction time (T). A smaller FOM means the method can detect smaller target quantities faster. We found that there is a tradeoff between the LOD of the assay and the required reaction time. With the proposed CRISPR sensing FOM, we evaluated five strategies to improve the CRISPR-based sensing: preamplification, enzymes of higher catalytic efficiency, multiple crRNAs, digitalization, and sensitive readout systems. We benchmarked the FOM performances of 57 existing studies and found that the effectiveness of these strategies on improving the FOM is consistent with the model prediction. In particular, we found that digitalization is the most promising amplification-free method for achieving comparable FOM performances (~ 1 fM·min) as those using preamplification. The findings here would have broad implications for further optimization of the CRISPR-based sensing.

Graphical Abstract

Sensitive, accurate, and fast diagnostics of infectious diseases is crucial to optimize clinical care and guide infection control and public health interventions to limit disease spread. The development of the clustered regularly interspaced short palindromic repeats (CRISPR)-based methods have taken center stage in biotechnology since the modified CRISPR/Cas9 system was applied for gene editing in mammalian genomes ¹. Additionally, the CRISPR-Cas9 system has shown outstanding competence in nucleic acid-sensing with high specificity ^2–7. Recently, the discovery of the collateral cleavage in other Cas proteins like Cas12 ⁸, Cas13 ⁹, and Cas14 ¹⁰ made it possible to translate the sequence-specific targeting to other detectable signals, which has led to the increasing emergence of CRISPR-mediated biosensors ^{9, 11–21}. In 2017, Gootenberg et al. introduced the specific high sensitivity enzymatic reporter unlocking (SHERLOCK), which exploits Cas13a for viral RNA detection ⁹. Simultaneously a Cas-12a-based nucleic acid-sensing tool called a one-hour low-cost multipurpose highly efficient system (HOLMES) was introduced in 2018 ⁸. The potential of CRISPR-based diagnostic systems was established in the recent global pandemic where numerous CRISPR-based tests were developed for SARS-CoV-2 (emerging virus responsible for COVID-19 pneumonia) detection ^22–31.

While CRISPR-based nucleic acid-sensing systems are growing rapidly, an objective approach to benchmark and compare the performances of different systems remains challenging. Several previous studies have reviewed the performances of various CRISPR-based methods ^32–36. As a potential diagnostic tool, two of the most important performance metrics in CRISPR-based methods are the achievable limits of detections (LODs) and the required reaction times ^32–33. It is generally favorable to obtain lower LODs in shorter reaction times. Ramachandran et al. recently presented an analytical model based on Michaelis-Menten enzyme kinetics to address the question of what are the achievable limits of detection and associated CRISPR reaction times ³⁷. This study demonstrated that the reaction time is inversely proportional to the target abundance and the Cas enzyme catalytic efficiency. Nevertheless, from the whole system perspective, the achievable LOD and the associated reaction time depend not only on the Cas protein catalytic efficiency but also on other conditions such as preamplification ^8–9, reaction volumes ^38–39, target activator ^{8, 37}, and readout systems ^{35, 40}. Due to these variations, there were almost no identical setups among different reported CRISPR-based methods.

In this work, we proposed and developed a figure of merit (FOM) for CRISPR-based nucleic acid-sensing systems with the goal to quantitatively benchmark different methods and explore the performance improvement strategies. We developed a kinetic model utilizing a single-enzyme framework and then extended it to bulk (multi-enzyme) systems. The CRISPR-based nucleic acid-sensing FOM, defined as the product of the LOD and CRISPR reaction time, is analytically established by connecting the LOD and reaction time to various reaction setup properties. Using the developed FOM model, we evaluated five strategies to achieve lower LODs with shorter reaction times (i.e., lowering the FOM value). We also compared the improved efficiency of these five strategies. Finally, we benchmarked a total of 57 published works related to CRISPR-based nucleic acid-sensing with reaction and performance parameters available. We found that digital CRISPR offers the best (lowest) FOM among various strategies and represents the most promising route towards amplification-free CRISPR-detection methods.

ESTABLISHMENT OF THE CRISPR SENSING FOM

Figure 1 presents the common steps for a CRISPR-based nucleic acid-sensing system. We assume the CRISPR nucleic acid-sensing starts with N₀ copies of the targets (DNA or RNA). Normally, a preamplification step could be performed to increase the copy numbers of the targets. For RNA targets, a reverse transcription (RT) step should be performed before or simultaneously with the amplification. Afterward, the cDNA product could be directly utilized in the Cas12 assay ^{13, 41} and should be transcribed back to RNA targets in the Cas13 assay ^{9, 42}. While each different amplification method has its unique kinetics, the number of the amplified targets (N₁) can be related to the initial target quantity N₀ as, N₁ = A N₀, where A is the amplification ratio.

Figure 1.

Typical steps in CRISPR-based nucleic acid-sensing system. As an optional step, the DNA or RNA targets could be pre-amplified before the Cas reaction to increase the target quantity. Reverse transcription or transcription will be needed depending on the Cas protein property and targets (note that the illustration shows a Cas13 assay as an example). In the CRISPR reaction, the target molecules are specifically recognized and bounded to the Cas proteins and their associated crRNA (i.e., Cas proteins activation). The trans-cleavage of the reporters could be described as an enzymatic reaction where activated Cas proteins and reporters act as enzymes and substrates, respectively. The cleaved reporter results in signal development in various forms (optical or electrical), which is detected by a readout system.

After this optional amplification step, the specific binding of the nucleic acids to the non-activated Cas proteins (Cas/crRNA binary complex) would activate the Cas proteins (Cas/crRNA/target ternary complex). Upon activation, Cas12 and Cas13 indiscriminately trans-cleavage ssDNA and ssRNA reporters, respectively ⁴³. Since the trans-cleavage activity is an enzymatic reaction, the CRISPR assay can be modeled as ³⁷,

$$E+S\underset{k_{\mathrm{o}\mathrm{f}\mathrm{f}}}{\stackrel{k_{\mathrm{o}\mathrm{n}}}{\rightleftharpoons }}ES\stackrel{k_{cat}}{\to }P+E$$ (1)

where k_on, k_off, and k_cat are the forward, reverse, and catalytic rates, respectively. E represents the enzyme (activated Cas protein), S is the substrate (intact reporters), ES is the reaction intermediate (enzyme-substrate reporter complex), and P signifies the product (i.e., cleaved reporters).

To capture the speed of product formation, we started from the reaction speed of each individual activated enzyme. Studies have shown that the single enzyme reaction is a stochastic process ⁴⁴, and the reaction speed (s⁻¹) is the reciprocal of the mean waiting time $〈\tau 〉$ and can be estimated as: $1/〈\tau 〉=k_{cat}\left[S\right]/(K_M+\left[S\right])$, where $\left[S\right]$ is substrate concentration and K_M is Michaelis constant and defined as ${({k_{off}+k}_{\mathrm{c}\mathrm{a}\mathrm{t}})/k}_{\mathrm{o}\mathrm{n}}$. Assuming the total activated enzymes is limited by the number of targets N₁ (i.e., the input Cas/crRNA binary complex is more than the nucleic acid targets, with or without amplification), we can obtain the reaction speed (s⁻¹) for the CRISPR reaction as:

$$v={N_{1}k}_{cat}\frac{\left[S\right]}{K_M+\left[S\right]}$$ (2)

With a CRISPR incubation reaction time of T and reaction volume of V_r, the concentration of the cleaved product would be vT/V_r. In order to effectively detect the cleaved products, the product concentration must be larger than the readout system’s limit of detection C_min (vT/V_r>C_min). As a result, we can obtain a critical equation for the CRISPR based nucleic acid-sensing,

$$N_0\ge \frac{V_{r\mathrm{ }}{C}_{min}}{{A\mathrm{ }Tk}_{cat}\frac{\left[S\right]}{K_M+\left[S\right]}}$$ (3)

This equation means that the lowest quantity of a target concentration (i.e., LOD) that can be detected in a specific CRISPR assay is given by,

$$LOD=\frac{{\mathrm{m}\mathrm{i}\mathrm{n}(N}_0)}{V_0}=\frac{V_{r\mathrm{ }}{C}_{min}}{{V_{0\mathrm{ }}A\mathrm{ }Tk}_{cat}\frac{\left[S\right]}{K_M+\left[S\right]}}$$ (4)

where V₀ is the target sample volume in the Cas reaction. In theory, increasing the V₀ would decrease the LOD of the system. However, V₀ between 1 to 5 μL has been used in most reported Cas reactions ^{9, 11, 18}. This is because increasing the V₀ could affect the assay buffer ¹⁸. From Eq. 4, we can observe a clear tradeoff between the LOD and CRISPR reaction time (T). To benchmark different CRISPR assays, we defined a figure of merit (FOM) for CRISPR-based nucleic acid-sensing as the product of the LOD and reaction time,

$$FOM=LOD\times T=\frac{V_{r\mathrm{ }}{C}_{min}}{{V_{0\mathrm{ }}A\mathrm{ }k}_{cat}\frac{\left[S\right]}{K_M+\left[S\right]}}$$ (5)

This CRISPR-based sensing FOM could be utilized to benchmark the performance of different assays as it is related to experimental conditions such as preamplification (A), the reaction volume (V_r), readout system (C_min), and enzymatic efficiency (k_cat, K_M). A smaller FOM value means that lower quantities of the target could be detected faster. It is noteworthy that LOD and reaction time are not equally important for different application scenarios. The FOM presented here should be used as a guide if the test needs to meet certain turnaround times or LOD requirements.

FOM IMPROVEMENT STRATEGIES

Use of preamplification

Based on Eq. 5, the FOM has a reverse relation with the amplification ratio (A). This implies that utilizing amplification with higher A would decrease the FOM and improve the overall sensing performances. In fact, various preamplification methods such as polymerase chain reaction (PCR) ^{8, 27, 45}, loop-mediated isothermal amplification (LAMP) ^{5, 16, 23}, and recombinase polymerase amplification (RPA) ^{9, 19, 46} and their reverse transcriptase (RT) version ³⁵ were adopted in the CRISPR assays. For example, in the Cas13-based SHERLOCK system, RPA was used to improve the LOD of the system up to 6 orders of magnitude ⁹. In the Cas12-based HOLMES system, the LOD was improved by 7 orders of magnitude by introducing a 45 min PCR amplification to the assay ⁸. However, it is noteworthy that while preamplification could improve the FOM of the CRISPR system significantly, utilizing this additional step complicates the assay design and could increase the cost and assay time. One might be intrigued by the question of why utilizing the CRISPR-based sensing if amplification techniques such as PCR or LAMP could already be used as the testing tools. The answer to this question is that sequence-dependent recognition of target nucleic acids by CRISPR effectors could significantly enhance the specificity and minimize the false positives in the amplification process ⁴⁷.

Figure 2a shows a radar chart comparing the six performance metrics of three common preamplification strategies used in CRISPR assays. (1) One-pot reaction. While the preamplification could be performed separately before the CRISPR assay in a two-step reaction, it is preferable to combine the preamplification and the CRISPR assay in a one-pot reaction to simplify the assay setup, decrease the assay time and reduce the risk of contaminations ³⁴. To this end, the reaction temperature between the preamplification and the CRISPR assay should be compatible. In this regard, RPA is the most suitable preamplification method to couple with CRISPR assays since the reaction temperature is similar (~37 °C) ³⁵ and PCR is incompatible with CRISPR due to its required thermal cycling. LAMP is somewhere in between due to its isothermal nature and had been used in one-pot CRISPR reactions ⁴⁸. Nevertheless, the required 65 °C working temperature is less compatible with that in the CRISPR assay ⁴⁹. (2) Primer design. Both PCR and RPA require only two primers ⁵⁰. On the other hand, the LAMP requires four to six primers that bind laterally to distinct sites of the DNA target ⁵¹. Moreover, the preamplification primer design is also restricted by the PAM (Cas12-based) ^{41, 52}, and PFS (Cas13-based) ⁹ regions in the target. As a result, designing the LAMP primer is more challenging than the PCR and RPA assay. (3) Intellectual property (IP) protection. PCR is one of the first introduced amplification methods, and the foundational patents for PCR expired in March of 2005 in USA and 2006 in Europe ⁵³. Therefore, various companies could offer PCR reagents across the world ⁵⁴. The LAMP assay was patented by Eiken chemical company (EP 1020534 B) from Japan, and this patent was expired in 2019 ⁵⁵. Currently, various companies such as New England Biolabs and Thermofisher in USA and OptiGene in Europe offer the required reagents for LAMP assay ^56–57. On the other hand, RPA was introduced recently by TwistDx Limited from United Kingdom ⁵⁵. So far, only TwistDx and Alere offer the RPA reagents ⁵⁸. (4) Sensitivity. The sensitivity of a diagnostic test is defined as the number of true positives (judged by the ‘Gold Standard’) over the total number received a positive result on this test. Li et al.⁵⁹ reviewed over 50 studies and compared the sensitivity of RPA with PCR. They showed that the sensitivity of RPA is only half as the PCR. (5) Specificity. The specificity of a diagnostic test is defined as the number of true negatives (judged by the ‘Gold Standard’) over the total number received a negative result on this test. Although the sensitivity of the RPA was not comparable to PCR results, their specificity is comparable ⁵⁹. On the other hand, the complexity of primer design and the number of primers involved in LAMP reaction can lead to false positives from non-specific primer interactions ⁴⁷. (6) Instrument complexity. To deploy the CRISPR-based diagnosis at the point of care, it is preferred to perform the assay with simple, easy-to-use, and cost-effective instruments ^34–35. Both LAMP and RPA are isothermal assays that could be performed using simple equipment ⁶⁰ or even equipment-free ^61–63. On the other hand, the PCR method relies on thermal cycling, making the instrumentation more complex.

Figure 2.

Different strategies to reduce the FOM and improve the CRISPR nucleic acid-sensing performance. (a) Qualitative comparison of three common preamplification methods. (b) The reported catalytic rate constant (k_cat) of CRISPR effectors activated by different activators (double- and single-stranded DNAs or RNAs). (c) Schematic of using multiple crRNAs in the CRISPR assay. Introducing n different crRNAs in the assay results in n times more activated Cas in the system and thus increasing the cleavage activity. (d) Effect of digitalization on the product (cleaved reporter) concentration. Reducing the reaction volume effectively increases the signal concentration for a fixed CRISPR reaction time. (e) Comparison of the typical detection limit of various readout methods (C_min). (f) Back-of-the-envelope calculation of FOM improvement ratio using different strategies.

Use of Cas proteins with higher k_cat

According to Eq. 5, FOM has a reverse relation with the activated Cas catalytic rate (k_cat). Assuming all other factors remain the same, Cas proteins with higher k_cat would decrease the FOM of the CRISPR system. Different Cas proteins have shown different trans-cleavage activity with various catalytic rates ^{21, 64–69}. Figure 2b presents the k_cat of different CRISPR effectors reported by different groups ^{21, 37, 52, 64, 66–70}. It should be noted that these results do not cover all discovered Cas proteins. Further studies are needed to explore the kinetics of various uncharacterized Cas proteins. We observed four interesting features from these data. First, different Cas proteins have distinct k_cat. Cas13 effectors generally have a higher cleavage rate. For example, the average k_cat of LbuCas13a is around 1861 s⁻¹, much higher than the 279 s⁻¹ for LbCas12a with the dsDNA activator. Second, similar Cas proteins from different bacteria show different cleavage activity where the average reported k_cat for LbCas12a is two orders of magnitude larger than AsCas12a. Third, different activators would result in different cleavage activities. In the case of Lbcas12a, the average k_cat of dsDNA activator cases are around 100 times higher than ssDNA activators. Forth, we observed a significant dispersion between the reported k_cat for a specific Cas protein. For instance, the k_cat of Lbcas12a with a dsDNA activator ranges from 0.08 to 1089 s⁻¹. This result shows that the combination of identical Cas proteins with different sequences of crRNAs would result in different trans-cleavage speeds. In addition, Nguyen et al.⁶⁴ showed that crRNA extensions could also affect the Cas trans-cleavage activity. Their finding showed that adding a 7-mer ssDNA extension to the 3’-end of crRNA would improve the trans-cleavage activity of LbCas12a proteins (more than two times). It should be noted that all the k_cat values presented here are at the optimal temperature for the Cas proteins trans-cleavage activity (around 37 °C) ⁷¹. We believe changing the temperature would affect the k_cat of the Cas proteins, which alters the system’s FOM. The results from Figure 2b suggest that different combinations of Cas proteins, target activators, and crRNAs should be optimized to obtain the highest k_cat. From these reported data in Figure 2b, selecting an optimal enzyme could reduce the FOM up to 3 orders of magnitude.

Use of multiple crRNA in the reaction

Another strategy to reduce the FOM of CRISPR systems is the use of multiple crRNAs. Combining different crRNAs with the Cas proteins would enhance the population of Cas/crRNA binary complex in the same reaction. Consequently, one target would activate multiple Cas proteins in the assay (Figure 2c). Considering that different crRNAs would have different kinetics properties (K_M and k_cat), the reaction speed with multiple crRNA can be written as:

$$v=N_{1}A{\sum }_{i=1}^n\frac{{k_{cat}}_i\left[S\right]}{K_{M_i}+\left[S\right]}$$ (6)

where n is the number of crRNAs in the assay. Based on Eq. 6, increasing the number of crRNA could increase the cleavage rate.

Recent studies have utilized this technique to improve the CRISPR sensing performance. Fozouni et al. used three different crRNAs in developing an amplification-free method for detecting SARS-CoV-2 with CRISPR-Cas13a ⁶⁹. They showed that the LOD was improved 100-fold with the same CRISPR reaction time. In another study, Son et al. ⁷² utilized 26 different crRNAs in a Cas13a assay and improved the LOD 5 times. It is clear that utilizing multiple crRNAs could decrease the FOM value and improve the system performance. Nevertheless, the enhancement of the performance using this strategy is additive in nature (Eq. 6) and is unlikely to offer more than 2 orders of magnitude improvements. In addition, utilizing multiple crRNAs could complicate the assay design and increase the cost significantly.

Use of digital CRISPR

The FOM model also suggests that the CRISPR assay performance has a reverse relation with the reaction volume. Decreasing the reaction volume from microliter-scale to sub-nanoliter would improve the FOM of the system. In digital assays, bulk reaction volumes (~μL) are partitioned into thousands or millions of small reaction chambers with pL to fL volumes ⁷³. Figure 2d depicts the effect of reaction volume reduction on the product (cleaved reporter) concentration. As shown, the concentration of the product could increase up to 9 orders of magnitude. A few recent studies have utilized digital CRISPR to improve the performance of the assay. For instance, Tian et al. improved the LOD by five orders of magnitude by reducing the reaction volume to 15 pL⁷⁴. Besides enhancing the FOM, another advantage of digitalized assays is the ability of absolute target quantification without the need for a standard curve ^{65, 75–76}. Using Poisson statistics, the sample concentration can be estimated by −ln(1-p), where p is the ratio of the positive partitions over total partitions. Compared to other strategies, digital CRISPR could improve the FOM significantly (more than six orders of magnitude).

Use of sensitive readout system

Another parameter to improve the CRISPR FOM is the readout system’s limit of detection C_min. Sensitive readout systems with lower C_min could help achieve lower FOM and better sensing performance (Eq. 5). While the majority of Cas12 or Cas13-based sensing systems were based on fluorescence signal ^{11, 16, 77}, colorimetric ^{15, 78}, electrochemical ^{14, 21}, and electronic readout ^{45, 79} were also explored for signal readout. Figure 2e compares the reported C_min of the different readout systems ^80–84. Among the optical methods, while simple signal readout systems such as the naked eye and portable fluorescent reader do not offer high sensitivity compared to other methods, they are appealing in developing cost-effective point of care devices. In addition, electrical systems such as the field-effect transistor (FET) biosensors ⁸¹ and nanopore sensors ⁸⁴ offer a lower limit of detection (lower than 1 pM) and the potential for developing an integrated system.

Comparison of FOM improvement strategies

Figure 2f summarizes the FOM improvement ratio using these strategies. The improvement ratio was estimated by using the FOM model (Eq.5) with reported LOD and CRISPR reaction times of previous studies ^{8–9, 64, 69, 85–86}. As shown, preamplification and digital assays are most effective in improving the FOM. They could significantly improve the FOM by orders of magnitude (~10⁶ to 10⁹) if used individually. Nevertheless, it should be noted that combining the preamplification and digital assays together would not significantly improve the FOM. This is because the amplified products are not tested in a single reaction volume of $V_{0\mathrm{ }}$_, but rather aliquoted into thousands to millions of smaller chambers. Each of these chambers only has 0 or 1 amplified product (i.e., digital assays). As a result, factors A and $V_0$ in Eq. 5 are not multipliable when combining the preamplification and digital methods.

As also shown in Figure 2f, utilizing a sensitive readout system could improve the FOM by 3 to 5 orders of magnitude compared to a simple readout like using a naked eye. In comparison, utilizing multiple crRNA or different Cas proteins is less effective, although they can still improve the FOM by about two orders of magnitude. It is noteworthy that multiple strategies could be implemented in one system to achieve lower FOM compared to individual strategies. For instance, Son et al. ⁷² combined digitalization and multiple crRNA in a single system and reduced the FOM by more than 6 orders of magnitude compared to the non-amplified Sherlock system ⁹.

To guide the implementation of improvement strategies for different applications, we summarized the advantages and disadvantages of each strategy in Table 1. While preamplification, digital assays, and sensitive readout have a high impact on the FOM (more than 4-fold), they would increase the cost and complexity of the systems. On the other hand, utilization of multiple crRNA and Cas proteins with higher k_cat is easy to implement in the system; however, they have a lower impact on the FOM (less than 3-fold). One should carefully balance the tradeoff between the cost and the performance when implementing these strategies to meet their testing goals.

Table 1.

Comparisons of pros and cons of different strategies.

Strategy	Advantages	Disadvantages
Preamplification	High impact on the FOM (more than 6-fold)	Longer or multi-step assay Higher cost
Cas proteins with higher kcat	Easy to implement	Limited discovered Cas proteins
Multiple crRNA	Easy to implement	Low impact on the FOM (less than 2-fold) Higher cost
Digital CRISPR	High impact on the FOM (more than 6-fold) Absolute quantification capability	Partitioning needed
Sensitive readout system	Medium impact on the FOM (more than 4-fold)	Sophisticated instrument Higher cost

PERFORMANCE BENCHMARKING

Numerous CRISPR-based nucleic acid-sensing systems were reported in the past several years ^{35, 40, 87}. The FOM model described in Eq. 5 provides us with a tool to benchmark the performance of these different systems. We studied a total of 57 published works (Table 2) related to CRISPR-based nucleic acid-sensing up to this date (Feb. 2022) ^{8–9, 14–16, 19–21, 23–28, 39, 45–46, 48, 52, 65, 69, 72, 74, 83, 85–86, 88–118}. It is noteworthy that while many more CRISPR-based sensing studies have been published in the past few years, we only include those with the LOD and CRISPR reaction time available. It should be mentioned that Cas 9 ^{4, 6}, and Cas 14 ¹¹⁹ have been utilized for diagnostics. However, in this study, we look into Cas12 and Cas13-based systems since they are more common and parameters for comparison are available.

Table 2.

Summary of the reported CRISPR-based diagnostics with LOD and CRISPR reaction time available.

Pathogen	Target	Effector	Readout System	Amplification	Amplification time (min)	CRISPR reaction time (min)	LOD (aM)	FOM (aM.min)	Ref.
Ensemble without amplification
African Swine Fever	DNA	LbCas12a	Fluorescence	None	None	480	1e6	4.8e8	110
African Swine Fever	DNA	LbCas12a	Fluorescence	None	None	1440	1e5	1.4e8	110
Pseudorabies virus	DNA	LbCas12a	Fluorescence	None	None	15	1e8	1.5e9	8
Liver cancer	DNA	LbCas12a	Colorimetric	None	None	60	2e8	1.2e10	15
HPV	RNA	LbCas12a	Electrochemical	None	None	60	3e7	1.8e9	105
Zika virus	RNA	LwCas13a	Fluorescence	None	None	60	5e5	3e7	9
SARS-CoV-2	RNA	LbuCas13a	Fluorescence	None	None	120	1.6e4	1.9e6	69
Synthesized target	RNA	LbuCas13a	Fluorescence	None	None	120	1e6	1.2e8	104
Synthesized target	RNA	LbuCas13a	Fluorescence	None	None	20	3.7e9	7.4e10	83
HPV	DNA	LbCas12a	Electrochemical	None	None	60	5e7	3e9	20
miR-19b and miR-20a	mRNA	LwaCas13a	Electrochemical	None	None	15	1e7	1.5e8	14
DENV-4	DNA	AsCas12a	Electrochemical	None	None	120	1e5	1.2e7	94
BRCA-1	DNA	AsCas12a	Fluorescence	None	None	30	1e3	3e4	93
HPV	DNA	LbCas12a	Fluorescence	None	None	60	1e4	6e5	92
Bacillus anthracis gene	DNA	LbCas12a	Fluorescence	None	None	15	1e7	1.5e8	91
Synthesized target	DNA	LbCas12a	Fluorescence	None	None	60	1e5	6e6	90
Ensemble with amplification
Citrus greening disease	DNA	LbCas12a	Fluorescence	LAMP	40	5	16.6	83	112
African Swine Fever (ASF)	DNA	LbCas12a	Fluorescence	LAMP	40	20	3.6	72	111
HPV	DNA	LbCas12a	Fluorescence	RPA	15	60	16.6	1e3	109
ASF	DNA	LbCas12a	Fluorescence	RPA	30	60	10	600	108
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA	30	30	16.6	498	116
HPV	DNA	LbCas12a	Fluorescence	RPA	10	60	10	600	52
Pseudorabies virus	DNA	LbCas12a	Fluorescence	PCR	45	15	10	150	8
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA (one pot)	None	40	80.3	3.2e3	25
P.aeruginosa	DNA	LbCas12a	Colorimetric	LAMP	15	30	3.4	102	107
HPV	DNA	LbCas12a	Colorimetric	PCR	50	30	240	7.2e3	106
Ebola virus	RNA	LbCas12a	Fluorescence	RPA	40	240	10	2.4e3	46
Synthesized target	RNA	AacCas12b	Fluorescence	LAMP	30	30	10	300	16
SARS-CoV-2	RNA	AacCas12b	Fluorescence	RAA	30	30	16.6	498	26
Zika virus	RNA	LwCas13a	Fluorescence	RPA	120	60	2	120	9
Zika Virus	RNA	LbuCas13a	Fluorescence	RPA	20	60	6	360	19
Cytomegalovirus	DNA	LwCas13a	Fluorescence	RPA	50	180	0.6	108	103
White Spot Syndrome	RNA	Cas13a	Colorimetric	RPA	40	180	1.6	288	102
Various tumor cells	mRNA	LbuCas13a	Electrochemical	EXPAR	30	30	1e3	3e4	21
SARS-CoV-2	RNA	AsCas12a	Nanopore	PCR	30	30	22.5	675	45
SARS-CoV-2	RNA	LbCas12a	Fluorescence	LAMP	30	10	16.6	166	23
SARS-CoV-2	RNA	AsCas12a	Fluorescence	LAMP	30	30	8.3	249	24
SARS-CoV-2	RNA	LwaCas13a	Fluorescence	RPA	20	60	16.6	996	28
SARS-CoV-2	RNA	LbCas12a	Fluorescence	LAMP	20	15	16.6	249	101
HPV	DNA	AaCas12b	Fluorescence	RPA	10	180	1	180	100
SARS-CoV-2	RNA	AapCas12b	Fluorescence	LAMP (one pot)	None	60	3.3	198	48
Different Viruses	RNA	LwaCas13a	Fluorescence	PCR or RPA	20	180	0.9	162	99
SARS-CoV-2	RNA	LbCas12a	Colorimetric	RPA	30	20	8.3	166	98
Listeria monocytogenes	DNA	LbCas12a	Electrochemical	RAA	30	90	0.68	61.2	97
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA	30	10	16.6	166	96
SARS-CoV-2	RNA	LwaCas13a	Fluorescence	PCR	22	30	332	1e4	27
SARS-CoV-2	RNA	LbCas12a	Colorimetric	RPA	20	60	1.6	96	95
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA	15	25	83	2.1e3	89
Staphylococcus aureus	DNA	LbCas12a	Colorimetric	RAA	20	30	1	30	88
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA (one pot)	None	60	2	120	113
SARS-CoV-2	RNA	LbCas12a	Fluorescence	LAMP	10	25	6.5	162.5	114
SARS-CoV-2	RNA	LbCas12a	Fluorescence	LAMP	40	10	26	260	115
Ensemble using multiple crRNA
SARS-CoV-2	RNA	LbuCas13a	Fluorescence	None	None	120	166	2e4	69
Synthesized target	DNA	LbCas12a	Fluorescence	None	None	30	310	9.3e3	117
Digital without amplification
ASF	DNA	LbCas12a	Fluorescence	None	None	60	30	1.8e3	65
SARS-CoV-2	RNA	LbuCas13a	Fluorescence	None	None	60	10	600	74
SARS-CoV-2	RNA	LbuCas13a	Fluorescence	None	None	15	8.3	124.5	72
Digital with amplification
SARS-CoV-2	RNA	LbCas12a	Fluorescence	RPA (one pot)	None	60	1.5	90	86
SARS-CoV-2	RNA	Cas12a	Fluorescence	RPA (one pot)	None	60	1.6	96	85
SARS-CoV-2	RNA	LbCas12a	Fluorescence	DAMP (one pot)	None	50	8.3	415	39
SARS-CoV-2	RNA	AapCas12b	Fluorescence	LAMP (one pot)	None	120	23	2.7e3	118
Digital using multiple crRNA
SARS-CoV-2	RNA	LbuCas13a	Fluorescence	None	None	15	1.6	24	72

Figure 3a shows the LOD versus CRISPR reaction time scattering plots along with the FOM-equivalent dash lines from 10⁻⁶ to 10⁻¹⁸ M·min. Note that the upper right corner represents a smaller FOM value and is thus preferred since it means lower LODs can be achieved by shorter CRISPR reaction times. We observed three important features in Figure 3a. First, these data points were divided into six categories based on the strategies they used (shown as oval in Figure 3a): (1) ensemble without amplification, (2) ensemble with amplification, (3) ensemble using multiple crRNA, (4) digital without amplification, (5) digital with amplification, and (6) digital using multiple crRNA. To benchmark these categories, we plotted the FOM values for each category (Figure 3b). As shown, the category of the ensemble without amplification represents the plain vanilla version of the CRISPR-based sensing. The data points within this category show the worst (highest) FOM (with an average of 5.8×10¹⁹ aM.min). The data points from all other categories show significant FOM improvements. For example, ensemble with amplification, ensemble using multiple crRNA, digital without amplification, digital with amplification, and digital using multiple crRNA strategies in average improved the FOM by 6, 5, 6, 6, and 8 orders of magnitude, respectively. These improvement results are consistent with the predictions in Figure 2f.

Figure 3.

(a) Scattering plot of the limit of detection versus CRISPR reaction time for a total of 57 CRISPR-based sensing studies, along with the FOM equivalent dash lines from 10⁻¹⁸ to 10⁻⁶ M·min. The data points were divided into six categories separated by the ovals in the figure. The top right side of the figure indicates a lower FOM and thus a better CRISPR sensing performance. Within each category, the data points do not perfectly reside on a single line (LOD×T = Constant). This is because the used Cas protein, crRNA, target, amplification method, and readout system could vary within each category. (b) A box graph presents the FOM range of each category. IQR stands for the interquartile range of the FOM data.

Second, as shown in Figure 3b, FOM in the order of 1 fM·min to 10 fM·min could be achieved within the digitalization categories with or without preamplification. This means that a target concentration of 100 aM to 1 fM could be obtained in 10 min CRISPR reaction time using digital assays without amplification, which was experimental validated ^{39, 70, 85}. The best FOM performance was observed by combining digital assay and multiple crRNA cases where FOM deceased to 24 aM·min ⁷². As a result, digital CRISPR assay provides the most appealing method for amplification-free CRISPR-based nucleic acid-sensing. Since digital CRISPR-based sensing is a new trend, limited data is available and more studies in the future would improve this evaluation.

Third, we observed a general reverse relation between the LOD and reaction time. Based on Eq. 4, the logarithmic LOD (log LOD) and logarithmic reaction time (log T) are expected to have a relationship of −1 within each category in which reaction parameters are similar. To test this prediction, we examined the categories of ensemble assays with amplification and ensemble assays without amplification, as both categories have sufficient data points to establish meaningful statistics. A linear fitting revealed the slope in the ensemble assays with and without amplification is −0.9±0.3 and −1.2±0.5, respectively, consistent with the model predictions (−1).

CONCLUSIONS

In summary, we proposed and developed a figure of merit (FOM) for cleavage-based CRISPR nucleic acid-sensing systems to quantitatively benchmark different methods and explore the performance improvement strategies. The CRISPR-based nucleic acid-sensing FOM, defined as the product of the LOD and CRISPR reaction time, is analytically established by connecting the LOD and reaction time to various reaction setup properties. Based on the developed model, we found that the CRISPR sensing FOM was linked to the reaction volume, the sensitivity of the readout system, preamplification efficiency, and Cas protein enzymatic properties. We evaluated different strategies to reduce the FOM and improve the performance of the CRISPR systems, including the use of preamplification, novel Cas proteins with higher k_cat, multiple crRNA, digital CRISPR, and sensitive readout systems. Comparison of FOM improvement strategies showed that preamplification and digital CRISPR have the highest impact on the FOM (up to 9 orders of magnitude). We benchmarked the FOM performances of 57 existing studies and found that the effectiveness of these strategies on improving the FOM is consistent with the model prediction. In particular, we found that digitalization is the most promising amplification-free method for achieving comparable FOM performances (~1 fM·min) as those using preamplification.