Significance
Strategies to stabilize and suppress the latent cell reservoir of HIV have been proposed and need to be carefully examined. We demonstrate the use of time-lapse fluorescence microscopy to quantify HIV gene-expression dynamics and detect several latency-promoting agents (LPAs) that would be overlooked when screening for mean gene expression alone. These LPAs are structurally and functionally related to inhibitors of the thioredoxin/thioredoxin reductase redox pathway, which has been suggested as a promising HIV target. Some LPAs are Food and Drug Administration–approved and commercially available and can expand the currently limited LPA repertoire. This study provides a foundation to research suppression mechanisms of HIV gene expression, alternative latency-promoting therapies, and ultimately remove the need for antiretroviral therapy in patients.
Keywords: HIV latency, microscopy, drug screen, latency-promoting agents, biological noise
Abstract
Upon treatment removal, spontaneous reactivation of latently infected T cells remains a major barrier toward curing HIV. Therapies that reactivate and clear the latent reservoir are only partially effective, while latency-promoting agents (LPAs) used to suppress reactivation and stabilize latency are understudied and lack diversity in their mechanisms of action. Here, we identify additional LPAs using a screen for gene-expression fluctuations (or “noise”) that drive cell-fate specification and control HIV reactivation from latency. Single-cell protein dynamics of a minimal HIV gene circuit were monitored with time-lapse fluorescence microscopy. We screened 1,806 drugs, out of which 279 modulate noise magnitude or half autocorrelation time. Next, we tested the strongest noise modulators in a Jurkat T cell latency model and discovered three LPAs that would be overlooked by quantifying their mean expression levels alone. The LPAs reduced reactivation of latency in both Jurkat and primary cell models when challenged by synergistic and potent combinations of HIV activators. The two strongest LPAs, NSC 401005 and NSC 400938, are structurally and functionally related to inhibitors of thioredoxin reductase, a protein involved in maintaining redox balance in host cells. Experiments with multiple functional analogs revealed two additional LPAs, PX12 and tiopronin, and suggest a potential LPA family, within which some are commercially available and Food and Drug Administration–approved. The LPAs presented here may provide new strategies to complement antiretroviral treatments. Screening for gene expression noise holds the potential for drug discovery in other diseases.
Upon infection, HIV can enter a prolonged and inactive latent state. This latent state may persist until random fluctuations in gene expression or the presence of activating cytokines or antigen triggers reactivation of the virus (1, 2). The pool of latently infected T cells, called the latent reservoir, is considered the major hurdle thwarting a cure for HIV. Research efforts have focused on screening for latency-reversal agents (LRAs) (3) and using them in conjunction with antiretroviral therapy (ART) to completely reactivate and kill the latent reservoir while protecting uninfected bystander cells (the “shock and kill” strategy) (4). Shock and kill faces many obstacles: 1) LRAs impair cytotoxic T cell function (5–7), 2) many of the currently available LRAs exhibit toxic effects and affect global gene expression (8), and 3) LRAs activate a limited percentage of the total reservoir on each exposure (9).
Transcriptional inhibition (or “block and lock”) has been proposed as an alternative strategy to circumvent the disadvantages of LRAs but has not been the focus of research (10–12). Currently, known latency-promoting agents (LPAs) include didehydro-Cortistatin A (dCA) (10, 13), kinase inhibitors (12), and manidipine hydrochloride (14). While preliminary studies exist (3, 10, 15), compounds that promote HIV latency have been limited in their mechanisms of action (16, 17). Research also indicates that around 8% of the human genome is composed of benign endogenous retroviruses (18), suggesting that silent proviruses are harmless. Therefore, the possibility of complete suppression of HIV gene expression through the “block and lock” strategy presents an opportunity for alternative treatment. Here we seek to identify additional compounds that inhibit latent reactivation and advance therapeutic efforts for HIV.
Gene expression fluctuations (or “noise”) underlie diversity in cellular decision-making across species (19, 20). To actively bias biological systems like HIV that are known for noisy and excitable cell-fate decisions (1, 2), screening for gene-expression noise allows for the discovery of chemical compounds toward the stochastic design of cell fate (14, 21). In a previous gene expression noise drug screen, high-throughput flow cytometry was used to probe for changes in noise magnitude of the HIV long terminal repeat (LTR) promoter (14), measured by the coefficient of variation squared of the protein product (CV2). Compounds that increased noise magnitude were enriched with synergizers of traditional LRAs of HIV latency, enhancing their reactivation ability. These compounds may be utilized in the “shock and kill” strategy and would be overlooked by traditional screens for mean gene expression alone. However, the previous study only discovered one latency-promoting compound that suppresses HIV reactivation (manidipine hydrochloride) (14), suggesting that HIV LPAs are less easily found than synergizers of LRAs. The current study sets out to perform an advanced screen for LPAs.
Another characteristic of gene expression noise is its dynamics, measured by noise autocorrelation half-time of the protein product (τ1/2). It has been previously studied in Escherichia coli and mammalian systems (1, 22–24), but such characterization requires high-throughput measurements with fine temporal resolution that are challenging to acquire (1, 22). Although used in other contexts such as fluorescence correlation spectroscopy at shorter time scales (25), quantifying mixing time in cancer cells treated with drugs (26), measuring HIV feedback strength (1), and quantifying pluripotent stem cell gene expression (27), autocorrelation half-time has not been extensively applied as a drug-screening metric.
This study expands upon the previously reported flow-cytometry-based drug screen on the HIV LTR promoter (14) and utilizes automated time-lapse fluorescence microscopy, capable of measuring both noise magnitude (CV2) and noise autocorrelation half-time (τ1/2) on a minimal HIV positively autoregulated gene circuit (1, 2). Compounds were screened for their modulation of CV2 and τ1/2, two orthogonal attributes to mean gene expression level and extracted from their time-dependent expression signatures (22, 28). Next, noise-modulating compounds were tested on a latently infected full-length HIV construct with flow cytometry to gauge their potential at promoting latency. We discovered three LPAs, out of which two have been shown to be related to the inhibition of the thioredoxin/thioredoxin reductase (Trx/TrxR) redox pathway (29, 30). We then tested multiple compounds inhibiting the Trx/TrxR pathway and discovered two additional LPA candidates. In total we present five HIV-suppressing compounds to broaden the palette of LPAs available to the research community. One out of the five LPAs discovered is Food and Drug Administration (FDA)-approved (SI Appendix, Table S1) and commercially available.
Results
Time-Lapse Fluorescence Microscopy.
We carried out 48-h time-lapse fluorescence microscopy on a clonal Jurkat cell line infected with a minimal HIV feedback vector, the LTR-d2GFP-IRES-Tat vector (Ld2GIT; Fig. 1A, Top and SI Appendix, Fig. S1) (31, 32). This construct consists of the HIV LTR promoter driving a destabilized green fluorescent protein (d2GFP), internal ribosomal entry site (IRES), and HIV transactivator of transcription (Tat). Here, HIV Tat activates the LTR promoter and constitutes a positive feedback loop. Ld2GIT was chosen to minimize drug interactions with other viral proteins and focus on the HIV Tat positive feedback loop. At the start of the experiments, cells were attached to the bottom of a 384-well glass-bottom imaging plate (Materials and Methods), and each well was treated with one of the 1,806 drugs at 10 μM from a diverse set of a compound library obtained from the National Cancer Institute (NCI). Control drugs consist of tumor necrosis factor α [TNF-α, a potent LRA that activates the LTR promoter (33, 34)] as the positive control and alsterpaullone [Alp, a potent inhibitor of HIV through CDK2 inhibition (35)] as the negative control. Images from hours 1 to 19 of each treated cell population were processed and converted to single-cell time-dependent fluorescence trajectories (Fig. 1A, Left) (Materials and Methods and SI Appendix, Fig. S1). The 1-to-19 h constraint is necessary to allow the cells to adapt to the plate environment and avoid quantifying cells that detach from the imaging surface after about 20 to 24 h due to movements of free-floating daughter cells generated from cell doubling. The trajectories were further processed to yield high-frequency noise magnitude (measured in HF-CV2, abbreviated to CV2 hereafter; Fig. 1A, Center) and high-frequency noise autocorrelation half-time (HF-τ1/2, abbreviated to τ1/2 hereafter; Fig. 1A, Right) (1, 22). Noise components of lower frequencies were filtered out to focus on high-frequency fluctuations caused by drug perturbations. The result is a three-dimensional noise coordinate for each of the 1,806 drugs tested (Fig. 1B). Two two-dimensional representations of the full dataset are shown in Fig. 1 C and D. The drug screen was carried out on seven different days due to throughput constraints. Daily untreated control values are shown as black squares, with SDs displayed as error bars. Between 20 and 30 untreated samples were measured per plate on each day of experiment (Materials and Methods). Previous studies have shown that positive autoregulation increases CV2 and τ1/2 compared to its unregulated counterpart (1, 28, 36). As a control, we added nonfeedback model lines generated from two measurements of a polyclonal LTR-d2GFP (Ld2G) population (Fig. 1 C and D). Consistent with these predictions, most compounds have their noise coordinates landing above the nonfeedback model lines.
A drug was considered a noise modulator if its CV2 and/or τ1/2 deviates at least two standard deviations (SDs) from the untreated control on the same day of experiment (red dots, Fig. 1 C and D) (14). Among 279 modulators, the top 115 modulators, ranked according to their modulation strengths, were selected for further analysis. They were classified into seven different categories according to their effects on CV2 and τ1/2 (Fig. 2A legend).
Latency-Reversal Assay for Noise-Modulating Compounds Identifies Three LPAs.
Noise modulators have previously been shown to affect latency stability (14). To investigate the effects of detected noise modulators on latent HIV infection, we performed a latency-reversal assay on a Jurkat cell line latently infected with a full-length HIV gene circuit (JLat) (37). The JLat construct contains a full-length HIV-1 vector with a deletion of env and a replacement of nef reading frame by GFP (Fig. 2, Top Right). Flow cytometry was used to measure two clonal populations with unique integration sites, JLat 9.2 and 15.4 (14, 37). This assay aims to challenge the noise modulators with 10 ng/mL of TNF-α (34), with additional LRAs tested later. Noise modulators were added at 10 μM with TNF-α (blue/orange/white/gray bars) or without (black bars) to the cell population (Fig. 2A and SI Appendix, Fig. S2A). Treatments were done in duplicate (n = 2), and those that decreased cell viability more than 50% of TNF-α control are colored gray. Latency reversal, or reactivation, is defined as a substantial increase of GFP fluorescence compared to autofluorescence of the untreated sample (see SI Appendix, Fig. S3 for gating strategies). Flow cytometry was performed 24 h after drug treatment (34), and the resulting percentage of GFP positive cells is plotted (Fig. 2A and SI Appendix, Fig. S2A, % Reactivation). Interactions between TNF-α and the noise modulators were determined by the Bliss independence model (38) (Materials and Methods). An excess over Bliss (EoB) score, defined as the excess effect of administering a noise modulator and TNF-α together compared to the sum of their effects when administered separately, was calculated for each of the noise modulators tested in the reactivation assay (Materials and Methods). A modulator with a positive EoB score synergizes with TNF-α at reversing latency, while one with a negative score antagonizes TNF-α and promotes latency. EoB values were thresholded for each day of experiment (Fig. 2A, SI Appendix, Fig. S2B, separated by vertical dashed lines, and Materials and Methods). A total of 34 LRA synergizers and 3 LPAs were discovered among the 115 noise modulators tested (orange bars and blue bars, Fig. 2A). Correlation analysis between the two clonal JLat cell populations yielded linear regressions with high R2 values, indicating an overall consistent behavior of the tested drugs on different integration sites (SI Appendix, Fig. S4 A and B). Visualization of distributions of EoB score and Δ % reactivation (defined as the difference between percent reactivation of treatments with TNF-α and noise modulator together and TNF-α alone) indicates that the three LPAs discovered consistently land on the lower end of the distributions (SI Appendix, Fig. S4 C–F). In addition, 78 nonnoise-modulating compounds from the drug screen were tested on both JLat cell lines, and no enrichment in compounds interacting with TNF was found (SI Appendix, Fig. S5).
Next, the three LPAs (D35 a.k.a. NSC 400938, D75 a.k.a. NSC 401005, and D106 a.k.a. NSC 155703) were challenged with three more known LRAs of HIV, including synergistic combinations with TNF-α (Fig. 2B and SI Appendix, Fig. S6; see Materials and Methods for LRA concentrations). LPAs were administered at a concentration of 10 μM. In both JLat 9.2 and 15.4, NSC 401005 displayed the strongest suppression effect among all three LRAs and almost inhibited all reactivation regardless of activator combinations used (yellow bars). Of note, for TNF-α + prostratin (Pro) and TNF-α + phorbol 12-myristate 13-acetate (PMA), NSC 401005 pulls down reactivation from about 55% to about 5% (red versus yellow bars). The big difference between TNF + Pro ± D75 is likely due to integration site differences between JLat 9.2 and 15.4. For most treatments cell viability was relatively unchanged compared to control levels (SI Appendix, Fig. S6). NSC 401005 showed a consistent trend of decreasing JLat 15.4 cell viability by 10 to 30% depending on the LRAs used (SI Appendix, Fig. S6).
Latency-Reversal Assay with Drugs Functionally Similar to NSC 401005 (D75) and NSC 400938 (D35).
Two of the three LPAs are related to compounds that inhibit proteins in redox pathways. Pleurotine (PubChem CID 13994131), a compound with the same molecular formula as NSC 401005, displays high structural similarity to NSC 401005 (PubChem CID 344218), with a Tanimoto coefficient of 95% (39) [obtained from PubChem Similarity Matrix Service (40)]. Pleurotine irreversibly binds to and inhibits human thioredoxin reductase (TrxR), a major component in the Trx/TrxR redox system (41). NSC 400938 has been found to inhibit the thioredoxin glutathione reductase (30), a fusion protein of TrxR and glutathione reductase found in the human parasite Schistosoma mansoni. To further examine the effects of redox inhibition on HIV expression, an additional JLat latency-reversal assay was carried out for a subset of Trx/TrxR inhibitors, consisting of auranofin, NSC 401005, PX12, and tiopronin, for 24- and 48-h treatment durations, together with TNF-α (10 ng/mL) (Fig. 3). Cell viability was quantified using propidium iodide (PI) staining (see SI Appendix, Fig. S7 for gating strategy). In JLat 9.2, NSC 401005 10 μM, PX12 80 μM, and PX12 60 μM showed substantial suppressing effects on HIV, but all three reduced cell viability by at least 40% (Fig. 3). In contrast, tiopronin at both concentrations reduced reactivation but did not reduce cell viability. In JLat 15.4 (Fig. 3B) similar effects were observed, except for 10 μM NSC 401005, which promotes latency to a lesser degree but with increased viability compared to JLat 9.2. All treatments were done in duplicate (n = 2). Finally, an LRA challenge assay with synergistic reactivation treatments, similar to the one performed on the three LPAs discovered in the screen (Fig. 2B), was carried out for PX12 and tiopronin (SI Appendix, Fig. S8). The results show varying degrees of suppression of reactivation from different LRA combination treatments.
Combinatorial LPA Latency-Reversal Assay.
To probe the potential that combinations of different LPAs at lower concentrations provide suppression of reactivation with improved cell viability, a latency-reversal assay with combination LPA treatments was performed. We carried out a two-drug combinatorial dose–response latency-reversal assay on JLat 9.2 with 10 ng/mL TNF-α for 24 h. Six different combinations between four LPAs (PX12, NSC 400938, NSC 401005, and NSC 155703) were tested at three concentrations each. Among the four LPAs, PX12 and NSC 401005 were prime targets because they reduced cell viability in previous assays. Cell viability is gauged through gating strategies on flow cytometry data (SI Appendix, Fig. S3). All treatments were done in duplicate (n = 2). Overall, we see a reduction in the percentage of cells exhibiting latency reversal as drug concentrations rise (Fig. 4A, % Reactivation), but at the expense of cell viability (Fig. 4B, % Live). We were able to achieve substantial suppression of latent reactivation with combinations of low concentrations of two LPAs. Additionally, for combinations with NSC 155703, we see a reduction of reactivation without a significant drop in cell viability when NSC 155703 concentration was increased. An exception is the NSC 401005 + PX12 combination, where cell viability was rescued when both drugs are at their highest concentration. Collectively, this experiment yielded many combinations of drugs that provide almost complete suppression of latency with minimal cell death in JLat 9.2 (Fig. 4 A and B).
Latency-Reversal Assay with Primary CD4+ T Cells.
To examine whether latency-promoting treatments suppress HIV expression rebound upon LRA treatments during acute infection in primary cells in vitro (42), further testing was done on primary CD4+ T cells infected with a full-length HIV construct containing a destabilized GFP (d2GFP) reporter element (Materials and Methods). CD4+ T cells were isolated from donor blood using negative selection and stimulated with anti-CD3/anti-CD28 beads for 3 d before infection (Materials and Methods). One day after infection, GFP-negative cells were sorted out using fluorescence-activated cell sorting (FACS). The treatments were carried out 1 d after sorting (Fig. 5A). A combination of PMA with ionomycin (43, 44) was used as LRAs. All treatments were done in duplicate (n = 2). Resulting reactivation percentage is normalized according to PMA + ionomycin control (14, 45). Due to cell viability constraints, only concentrations where cell viability does not drop lower than 50% of the control value were tested. Cell viability was quantified with PI staining. All four compounds showed inhibition of HIV expression when cells were treated with PMA + ionomycin (Fig. 5B). Except for 0.5 μM NSC 401005, none of the drugs caused statistically significant reductions in cell viability (Fig. 5C).
Discussion
Increasing availability of high-throughput microscopy technologies capable of capturing images with fine temporal resolution may accelerate drug discovery for HIV or other diseases using gene expression dynamics and autocorrelation half-time as screening criteria. Noise magnitude (CV2) and half autocorrelation time (τ1/2) represent orthogonal axes to changes in mean expression levels (Fig. 1). As such, noise drug screens of single-cell dynamics yield compounds that would otherwise be overlooked in conventional screens studying mean expression levels and population averages. Although τ1/2 was unable to predict whether a drug is an LPA or not (SI Appendix, Supplementary Text and Fig. S10), the noise drug screen still narrowed down 115 noise modulators from a library of 1,805 drugs and greatly reduced the effort needed with latency-reversal assays. We discovered 3 LPAs from the 115 modulators investigated. Two of the three discovered LPAs are found to be connected to inhibition of the Trx/TrxR pathway: NSC 401005 (D75) displays high structural similarity to pleurotine, which has been shown to inhibit TrxR; NSC 400938 (D35) has been shown to inhibit the thioredoxin glutathione reductase, a fusion protein with TrxR active domain that is found in S. mansoni (30). This further led to the discovery of two more LPAs (PX12 and tiopronin; Fig. 3) among the Trx/TrxR inhibitor family. Of note, not all Trx/TrxR inhibitors suppressed reactivation of latency (SI Appendix, Fig. S14), but we present here an enrichment of HIV suppressors to prompt further investigation. Trx/TrxR inhibitors have also been suggested for their potential in HIV treatment in recent studies (46, 47). It remains unknown whether the discovered LPAs can suppress HIV expression epigenetically and after drug removal. In addition to the 3 LPAs, 34 noise modulators synergized reactivation from latency with TNF-α (Fig. 2A and SI Appendix, Table S4). These may extend the library of currently known LRAs and synergistic mixtures for “shock and kill” treatment strategies (3, 14, 48), but additional investigations are needed to prove their efficacy in primary cell models. In addition, as a control, nonnoise-modulators showed substantially decreased activity with latency (SI Appendix, Fig. S5), indicating that CV2 and τ1/2 are suitable metrics for drug screen efforts where noise modulators are enriched with LRA synergizers and LPAs.
In total this study uncovers five LPAs, consisting of NSC 400938, NSC 401005, NSC 155703, and two drugs of the Trx/TrxR inhibitor family, namely PX12 and tiopronin, to expand the current repertoire of LPAs (10, 12, 14, 17). Of note, NSC 401005 displays structural similarity and NSC 400938 displays functional similarity to Trx/TrxR inhibitors (30). NSC 155703 was shown to inhibit the secretion of interleukin 1 beta (49), a proinflammatory factor whose production has been shown to be enhanced by HIV infection (50). Some inhibitors of the Trx/TrxR family, including tiopronin and auranofin, are FDA-approved and commercially available. They have been used for a wide range of applications, ranging from skin care (51) to anticancer treatment (52, 53), making them attractive for advanced characterization and research. TrxR inhibitor compounds are well-studied (54), and redox imbalance has been directly linked to HIV disease progression (55). In addition, oxidative stress is associated with HIV infection and recent studies support the promise of targeting the Trx/TrxR axis (46, 47, 56). It is yet to be seen how the Trx/TrxR inhibitors perform as LPAs in patient samples on long-term ART. Despite high preoptimized treatment concentrations leading to cell death (Figs. 2 and 3), combination LPA treatments hold the potential to further lower the concentrations necessary for suppression of HIV reactivation (Fig. 4). “Block and lock” suppression will require the optimization of treatment concentrations, timing, and duration relative to the administration and/or removal of ART (57), as well as calculated and periodic treatment cycles (58).
Interestingly, the two Trx/TrxR inhibitors investigated, PX12 and tiopronin, together with NSC 401005 and NSC 400938, exhibited a dose-dependent response in percent reactivation when added together with TNF-α (SI Appendix, Fig. S11). Compounds synergize with TNF-α at lower concentrations, while higher concentrations suppress TNF-α reactivation. The Trx/TrxR redox pathway controls reduced nuclear factor κB and oxidized Tat levels, both of which influence transcriptional efficiency of the HIV LTR promoter (SI Appendix, Fig. S12) (59, 60). Under the assumption that Trx binds to TrxR with a higher affinity than Tat, we propose a theoretical model to explain this dose-dependent behavior (SI Appendix, Fig. S12). Therefore, the Trx/TrxR redox inhibitors may present new targets and opportunities for HIV research aiming to control stability of latent proviruses.
The current leading compound that rigorously inhibits HIV expression is dCA, which acts by blocking Tat–TAR interactions and inhibits Tat positive feedback (13). Although combination treatments of ART and dCA substantially delay and reduce viral rebound upon treatment removal in patient samples (10), resistant mutants capable of maintaining robust expression under dCA treatment have been discovered (61). Given that the LPAs reported here do not share the same mechanism of action, it will be interesting to test whether the administration of these LPAs in conjunction with dCA enhances suppression to further improve “block and lock” strategies. Cumulatively, the findings of this research and its expansion of a palette of “block and lock” compounds may culminate in an effective, safe, and scalable functional cure for HIV-infected individuals.
Materials and Methods
Cell Culture.
Jurkat cell lines (Ld2GIT, JLat 9.2, and JLat 15.4) were cultured in Corning RPMI 1640 with l-glutamine and phenol red, with 10% fetal bovine serum (FBS) and 1% penicillin-streptomycin (pen-strep) added. Cells were passaged twice each week, with a dilution ratio of cell culture to fresh media of about 1:4. One day before each experiment, cell concentration was calculated using hemocytometers, and subsequently diluted to about 800,000 cells per mL.
Time-Lapse Fluorescence Microscopy.
Glass-bottom 384-well imaging plates were prepared with a master mix (9.4 μL per well) containing Cell-Tak cell adhesive by Corning (22) and dried off after more than 20 min of incubation at room temperature. The master mix is prepared with a volume ratio of 291:5:4 of sodium bicarbonate (75 g/L), sodium hydroxide (40 g/L), and Cell-Tak (2.03 g/L) (22). Cells were plated with a concentration of 10,000 cells per well, suspended in 10 μL of Dulbecco’s phosphate-buffered saline (DPBS). Cells were kept in incubators for 30 to 60 min, and excess unadhered cells were carefully washed off with RPMI media (with 10% FBS and 1% pen-strep, without phenol red), and 40 μL of fresh media was added to each well after washing. Control drugs were diluted to their final concentrations in RPMI, and 40 μL of them were added to each well [final concentration: Alp at 2 μM (35) and TNF-α at 10 ng/mL (34)]. Other drugs from NCI Diversity Set III, Oncology Drugs Set I, and Natural Products Set II were added using Matrix PlateMate Plus liquid handler afterward, from drug blocks containing 10 mM concentration of drugs, dissolved in dimethyl sulfoxide (DMSO). Four hundred nanoliters of drug was added to each well to achieve a final concentration of 10 μM (14). The plate was then settled down on a stage-top incubator on a Nikon TI Eclipse microscope. Consistent focus between imaging time points was maintained by the Nikon Perfect Focus System. Cells were imaged for a total of 48 h. Each well was imaged using a 10× objective at a 15-min imaging interval. Due to the large number of wells it is not possible to further reduce the imaging interval. Movie S1 shows the image acquisition process. For each day of experiment, the numbers of untreated wells are n = 30, 21, 24, 26, 23, 24, and 24. In addition, the numbers of positive control wells (TNF) are n = 15, 10, 15, 13, 14, and 13, and the numbers of negative control wells (Alp) are n = 15, 4, 15, 11, 14, and 14.
Microscopy Image Processing and Noise Analysis.
Single cells were identified from raw images using Nikon NIS Elements software. Binary masks of cells were tracked across all images taken from the same well and cell trajectories lasting less than 18 h were discarded. Single-cell trajectories were outputted into spreadsheets with their mean intensity at each time point. Another round of cell selection filtered out any cell that did not have a gene expression intensity trajectory spanning across hours 1 to 19. The limitation imposed on the durations of single-cell trajectories was to account for cells adapting to the plating environment in the first hour and increased cellular movement due to cell doubling in bulk media after 20 to 24 h. Wells with fewer than 50 single-cell trajectories left were excluded to make sure ample cells were available for analysis. Noise processing was performed on the remaining wells. Fluorescence intensity trajectories from the same well were detrended and mean-suppressed, and the resulting trajectories were denoted as noise trajectories (1, 22, 28). Autocorrelation analysis was done on the noise trajectories, and for each single-cell noise trajectory a single-cell autocorrelation function was calculated (62). Cells containing too much white noise, quantified as a drop of more than 20% at time lag = 15 min compared to the value at time lag = 0 on its autocorrelation function, were excluded from the well, and wells with fewer than 50 cells after this round of quality control were again excluded from the dataset. The number of wells excluded from each plate ranges from 7 to 18. For the rest of the wells, an average population ensemble/composite autocorrelation function was calculated from all the cells within a single well. The mean fluorescence (<FL>) is the average of single-cell fluorescence trajectories of all cells within a well that passed quality control. Noise magnitude, measured by high-frequency coefficient of variation squared (HF-CV2, abbreviated as CV2), was calculated using the variance or the zero-lag value of their averaged/composite autocorrelation function. The high-frequency autocorrelation half-time (HF-τ1/2, abbreviated as τ1/2) was calculated as the first lag time on the composite autocorrelation function that reaches half of the maximum autocorrelation value.
The following are the detailed steps of the noise analysis and signal processing. We denote the mean fluorescence intensity trajectory of the ith cell from a given well as , where . Let m be the number of cells in this well. Then, the general trend is calculated as follows:
The mean fluorescence <FL> is calculated as
We define an intermediate variable :
Then, the single-cell noise trajectory of the ith cell, denoted as , is calculated as follows:
Let denote the single-cell noise autocorrelation function, where is the time lag value. We calculate it using the following formula:
Define as the composite autocorrelation function and it is calculated as
The CV2 of the well is then calculated as follows:
Define the set C as follows:
Let τ0 denote the smallest element of C. Then, τ1/2 is defined as the solution to , where is the continuous linear interpolation of between and .
The 115 noise modulators picked from the 279 modulators were chosen in decreasing order of magnitude of modulation in CV2 and τ1/2 for the latency-reversal assay.
HIV Latency-Reversal Assay, LRA Combination Treatments, and LPA Combination Latency-Reversal Assay.
For HIV latency-reversal assays, JLat 9.2 and 15.4 were treated with noise modulators D1 to D118 (10 μM), with and without TNF-α (10 ng/mL) (Fig. 2A). Modulators were acquired from the National Cancer Institute Drug Synthesis and Chemistry Branch. D47, D52, and D115 were excluded due to limited solubility in DMSO, resulting in 115 drugs tested. Cells were prepared in V-bottom 96-well plates, and all treatments were performed in duplicate (n = 2). The full experiment was separated into 3 d due to the large number of treatments (separated by dashed vertical lines in Fig. 2A). Samples were measured on a BD LSR Fortessa Flow Cytometry Analyzer with High Throughput Sampler. Data were analyzed with FCS Express software, and live percentage and GFP+ reactivation percentage of JLat cells was extracted from gating strategies (SI Appendix, Fig. S3).
A two-measurement variant of the latency-reversal assay was carried out on JLat 9.2 and 15.4 (Fig. 3). Cells were treated with 10 ng/mL TNF-α together with one of the LPA concentrations at time (t) = 0. Samples were collected at t = 24 h and t = 48 h, washed with DPBS twice, and stained with PI. Stained samples were measured on a BD LSR Fortessa Flow Cytometry Analyzer in tube mode and data analysis was done in a fashion similar to the previous latency-reversal assay. See SI Appendix, Fig. S7 for details on gating.
For LRA combination treatments, the same procedure for latency-reversal assay was carried out, with the LRAs of choice changed to TNF-α, Pro, PMA, trichostatin A (TSA), and combinations of TNF-α and one other LRA. Final concentrations of LRAs are 10 ng/mL for TNF-α, 3 μM for Pro, 200 ng/mL for PMA, and 400 nM for TSA (34). Cell viability was quantified using gating strategies (SI Appendix, Fig. S3).
For LPA combination treatments, the same procedure for latency-reversal assay was carried out on JLat 9.2 only, with the LPA changed to a combination of two from four LPAs (PX12, NSC 400938, NSC 401005, and NSC 155703) at three different concentrations each. Cell viability was quantified using gating strategies (SI Appendix, Fig. S3).
Statistics and EoB Calculation.
For statistical tests, we employed Welch’s t test, which is a variant of two-sample t test where we do not make assumptions about whether the two samples have equal variance or not (63). Let and denote the mean of the two samples, and and denote the unbiased estimators of the sample variances. Let and denote the number of observations in each sample. Then the t statistic is calculated as
The distribution of this t statistic is approximated as an ordinary Student’s t distribution with the following degrees of freedom (d.o.f.):
Due to the relatively small sample size, this test heavily relies on the assumption that the underlying distribution of the samples are Gaussian. We performed an experiment where 87 samples of JLat 9.2 were treated with 10 ng/mL TNF-α for 24 h, and reactivation percentages were measured with flow cytometry (SI Appendix, Fig. S13). The Gaussian distribution fit on the experimental histogram shows an R2 value of 0.9856, confirming the assumption for TNF-α–only treatments. It is then assumed that the Gaussian assumption holds true for other treatments as well.
Due to the large number of identical tests conducted for experiments in Figs. 3 and 5, we carried out a Holm–Bonferroni correction for multiple testing (64). The procedures are as follows.
-
1)
Carry out the t tests as normal. Sort P values of all tests in ascending order. Denote the sorted P values as and the null hypotheses as .
-
2)
For a given significance value , let be the minimum index such that .
-
3)
Reject and do not reject .
-
4)
If , do not reject any null hypothesis; if no such exists, reject all null hypotheses.
This method would ensure that the familywise error rate of the set of hypothesis tests is no larger than .
EoB scores (38) were calculated for each single drug to determine if its interaction with TNF-α is additive or nonadditive. Let fTNF and fD denote the reactivation percentage of the population when adding TNF-α and drug by themselves, and fTNF+D denote the reactivation when the drug and TNF-α are added together. Then the EoB is defined as the difference between the measured combined effect and the expected combined effect assuming independence between TNF-α and the drug of interest, calculated using the following formula:
Positive EoB values indicate synergy, and negative EoB values indicate antagonism. The absolute values of EoB indicate the strength of synergism/antagonism. A daily threshold on the strength of synergism/antagonism is calculated for each day of experiment, using the median of the absolute values of EoB of that day of experiment as a cutoff for significant synergy/suppression. We did not use EoB = 0 as a threshold due to concerns of elevated error rate from multiple hypothesis testing.
Primary Cell Isolation, Infection, and Drug Treatment.
Human whole-blood donations were deidentified prior to use (Innovative Research). Peripheral blood mononuclear cells (PBMCs) were isolated by Ficoll-Hypaque density gradient centrifugation of buffy coats from HIV-seronegative donors (Innovative Research). PBMCs were immediately processed to isolate CD4+ T cells using RosetteSep Human CD4+ T Cell Enrichment Kit (Stem Cell Technologies) and negative selection. Primary CD4+ T cells were stimulated with Dynabeads Human T-Activator CD3/CD28 on the same day. After 3 d of stimulation, cells were infected with concentrated lentivirus containing the full-length HIV vector for 2 h by spinoculation at room temperature (42). Next, CD4+ T cells were sorted 1 d after infection and GFP-negative cells were seeded into 48-well plates. One day after seeding, cells were treated with drugs of interest. Flow cytometry measurement of GFP expression was carried out after 24 h of incubation.
Supplementary Material
Acknowledgments
We thank W. Greene and members of the R.D.D. laboratory for feedback and fruitful discussions on the manuscript. We thank C. Zhang at the University of Illinois at Urbana–Champaign (UIUC) High-throughput Screening Facility for fruitful discussions on the experiment and drug screening design and for assistance in compound addition to plate samples. We also thank B. Pilas and B. Balhan at the UIUC Flow Cytometry core for cell sorting service. We are grateful for the National Cancer Institute Drug Synthesis and Chemistry Branch for providing the noise modulator compounds. Y.L., K.B.-W., and R.D.D. acknowledge support from NIH, National Institute of Allergy and Infectious Diseases grant AI120746. Additional research support was provided by NSF CAREER grant 1943740. JLat full-length clones (9.2 and 15.4) were obtained through the NIH AIDS Reagent Program from E. Verdin, and Ld2GIT and JLatd2GFP vectors were kindly provided by the L. Weinberger laboratory at the University of California, San Francisco.
Footnotes
The authors declare no competing interest.
This article is a PNAS Direct Submission. L.S.W. is a guest editor invited by the Editorial Board.
This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.2012191118/-/DCSupplemental.
Data Availability
The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
- 1.Weinberger L. S., Dar R. D., Simpson M. L., Transient-mediated fate determination in a transcriptional circuit of HIV. Nat. Genet. 40, 466–470 (2008). [DOI] [PubMed] [Google Scholar]
- 2.Weinberger L. S., Burnett J. C., Toettcher J. E., Arkin A. P., Schaffer D. V., Stochastic gene expression in a lentiviral positive-feedback loop: HIV-1 Tat fluctuations drive phenotypic diversity. Cell 122, 169–182 (2005). [DOI] [PubMed] [Google Scholar]
- 3.Dahabieh M. S., Battivelli E., Verdin E., Understanding HIV latency: The road to an HIV cure. Annu. Rev. Med. 66, 407–421 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Archin N. M., et al., Administration of vorinostat disrupts HIV-1 latency in patients on antiretroviral therapy. Nature 487, 482–485 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Shan L., et al., Stimulation of HIV-1-specific cytolytic T lymphocytes facilitates elimination of latent viral reservoir after virus reactivation. Immunity 36, 491–501 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Jones R. B., et al., A subset of latency-reversing agents expose HIV-infected resting CD4+ T-cells to recognition by cytotoxic T-lymphocytes. PLoS Pathog. 12, e1005545 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jones R. B., et al., Histone deacetylase inhibitors impair the elimination of HIV-infected cells by cytotoxic T-lymphocytes. PLoS Pathog. 10, e1004287 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gallinari P., Di Marco S., Jones P., Pallaoro M., Steinkühler C., HDACs, histone deacetylation and gene transcription: From molecular biology to cancer therapeutics. Cell Res. 17, 195–211 (2007). [DOI] [PubMed] [Google Scholar]
- 9.Cillo A. R., et al., Quantification of HIV-1 latency reversal in resting CD4+ T cells from patients on suppressive antiretroviral therapy. Proc. Natl. Acad. Sci. U.S.A. 111, 7078–7083 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kessing C. F., et al., In vivo suppression of HIV rebound by didehydro-cortistatin A, a “block-and-lock” strategy for HIV-1 treatment. Cell Rep. 21, 600–611 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Tyagi M., Karn J., CBF-1 promotes transcriptional silencing during the establishment of HIV-1 latency. EMBO J. 26, 4985–4995 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Vargas B., et al., Inhibitors of signaling pathways that block reversal of HIV-1 latency. Antimicrob. Agents Chemother. 63, e01744-18 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mediouni S., et al., Didehydro-cortistatin A inhibits HIV-1 by specifically binding to the unstructured basic region of Tat. mBio 10, e02662-18 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dar R. D., Hosmane N. N., Arkin M. R., Siliciano R. F., Weinberger L. S., Screening for noise in gene expression identifies drug synergies. Science 344, 1392–1396 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Balachandran A., et al., Identification of small molecule modulators of HIV-1 Tat and Rev protein accumulation. Retrovirology 14, 7 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Delannoy A., Poirier M., Bell B., Cat and mouse: HIV transcription in latency, immune evasion and cure/remission strategies. Viruses 11, E269 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jean M. J., Fiches G., Hayashi T., Zhu J., Current strategies for elimination of HIV-1 latent reservoirs using chemical compounds targeting host and viral factors. AIDS Res. Hum. Retroviruses 35, 1–24 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jern P., Coffin J. M., Effects of retroviruses on host genome function. Annu. Rev. Genet. 42, 709–732 (2008). [DOI] [PubMed] [Google Scholar]
- 19.Balázsi G., van Oudenaarden A., Collins J. J., Cellular decision making and biological noise: From microbes to mammals. Cell 144, 910–925 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Raj A., van Oudenaarden A., Nature, nurture, or chance: Stochastic gene expression and its consequences. Cell 135, 216–226 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dar R. D., Weiss R., Perspective: Engineering noise in biological systems towards predictive stochastic design. APL Bioeng. 2, 020901 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dar R. D., et al., Transcriptional burst frequency and burst size are equally modulated across the human genome. Proc. Natl. Acad. Sci. U.S.A. 109, 17454–17459 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Austin D. W., et al., Gene network shaping of inherent noise spectra. Nature 439, 608–611 (2006). [DOI] [PubMed] [Google Scholar]
- 24.Sigal A., et al., Variability and memory of protein levels in human cells. Nature 444, 643–646 (2006). [DOI] [PubMed] [Google Scholar]
- 25.Elson E. L., Fluorescence correlation spectroscopy: Past, present, future. Biophys. J. 101, 2855–2870 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cohen A. A., et al., Dynamic proteomics of individual cancer cells in response to a drug. Science 322, 1511–1516 (2008). [DOI] [PubMed] [Google Scholar]
- 27.Singer Z. S., et al., Dynamic heterogeneity and DNA methylation in embryonic stem cells. Mol. Cell 55, 319–331 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Cox C. D., McCollum J. M., Allen M. S., Dar R. D., Simpson M. L., Using noise to probe and characterize gene circuits. Proc. Natl. Acad. Sci. U.S.A. 105, 10809–10814 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Welsh S. J., et al., The thioredoxin redox inhibitors 1-methylpropyl 2-imidazolyl disulfide and pleurotin inhibit hypoxia-induced factor 1alpha and vascular endothelial growth factor formation. Mol. Cancer Ther. 2, 235–243 (2003). [PubMed] [Google Scholar]
- 30.Simeonov A., et al., Quantitative high-throughput screen identifies inhibitors of the Schistosoma mansoni redox cascade. PLoS Negl. Trop. Dis. 2, e127 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Bohn-Wippert K., et al., Cell size-based decision-making of a viral gene circuit. Cell Rep. 25, 3844–3857.e5 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hansen M. M. K., et al., A post-transcriptional feedback mechanism for noise suppression and fate stabilization. Cell 173, 1609–1621.e15 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Malinin N. L., Boldin M. P., Kovalenko A. V., Wallach D., MAP3K-related kinase involved in NF-kappaB induction by TNF, CD95 and IL-1. Nature 385, 540–544 (1997). [DOI] [PubMed] [Google Scholar]
- 34.Bohn-Wippert K., Tevonian E. N., Megaridis M. R., Dar R. D., Similarity in viral and host promoters couples viral reactivation with host cell migration. Nat. Commun. 8, 15006 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Guendel I., Agbottah E. T., Kehn-Hall K., Kashanchi F., Inhibition of human immunodeficiency virus type-1 by cdk inhibitors. AIDS Res. Ther. 7, 7 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Simpson M. L., Cox C. D., Sayler G. S., Frequency domain analysis of noise in autoregulated gene circuits. Proc. Natl. Acad. Sci. U.S.A. 100, 4551–4556 (2003). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Jordan A., Bisgrove D., Verdin E., HIV reproducibly establishes a latent infection after acute infection of T cells in vitro. EMBO J. 22, 1868–1877 (2003). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Bliss C. I., The toxicity of poisons applied jointly. Ann. Appl. Biol. 26, 585–615 (1939). [Google Scholar]
- 39.Bajusz D., Rácz A., Héberger K., Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations? J. Cheminform. 7, 20 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Kim S., Thiessen P. A., Cheng T., Yu B., Bolton E. E., An update on PUG-REST: RESTful interface for programmatic access to PubChem. Nucleic Acids Res. 46, W563–W570 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Wipf P., et al., New inhibitors of the thioredoxin-thioredoxin reductase system based on a naphthoquinone spiroketal natural product lead. Bioorg. Med. Chem. Lett. 11, 2637–2641 (2001). [DOI] [PubMed] [Google Scholar]
- 42.Lassen K. G., Hebbeler A. M., Bhattacharyya D., Lobritz M. A., Greene W. C., A flexible model of HIV-1 latency permitting evaluation of many primary CD4 T-cell reservoirs. PLoS One 7, e30176 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Clift I. C., Bamidele A. O., Rodriguez-Ramirez C., Kremer K. N., Hedin K. E., β-Arrestin1 and distinct CXCR4 structures are required for stromal derived factor-1 to downregulate CXCR4 cell-surface levels in neuroblastoma. Mol. Pharmacol. 85, 542–552 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Huang K., Kiefer C., Kamal A., Novel role for NFAT3 in ERK-mediated regulation of CXCR4. PLoS One 9, e115249 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Spina C. A., et al., An in-depth comparison of latent HIV-1 reactivation in multiple cell model systems and resting CD4+ T cells from aviremic patients. PLoS Pathog. 9, e1003834 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Gavegnano C., Savarino A., Owanikoko T., Marconi V. C., Crossroads of cancer and HIV-1: Pathways to a cure for HIV. Front. Immunol. 10, 2267 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Benhar M., Shytaj I. L., Stamler J. S., Savarino A., Dual targeting of the thioredoxin and glutathione systems in cancer and HIV. J. Clin. Invest. 126, 1630–1639 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Xing S., Siliciano R. F., Targeting HIV latency: Pharmacologic strategies toward eradication. Drug Discov. Today 18, 541–551 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Kearney S. E., et al., Canvass: A crowd-sourced, natural-product screening library for exploring biological space. ACS Cent. Sci. 4, 1727–1741 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Guo H., Gao J., Taxman D. J., Ting J. P., Su L., HIV-1 infection induces interleukin-1β production via TLR8 protein-dependent and NLRP3 inflammasome mechanisms in human monocytes. J. Biol. Chem. 289, 21716–21726 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Yodoi J., Matsuo Y., Tian H., Masutani H., Inamoto T., Anti-inflammatory thioredoxin family proteins for medicare, healthcare and aging care. Nutrients 9, E1081 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Urig S., Becker K., On the potential of thioredoxin reductase inhibitors for cancer therapy. Semin. Cancer Biol. 16, 452–465 (2006). [DOI] [PubMed] [Google Scholar]
- 53.Stafford W. C., et al., Irreversible inhibition of cytosolic thioredoxin reductase 1 as a mechanistic basis for anticancer therapy. Sci. Transl. Med. 10, eaaf7444 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Saccoccia F., et al., Thioredoxin reductase and its inhibitors. Curr. Protein Pept. Sci. 15, 621–646 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Nakamura H., Masutani H., Yodoi J., Redox imbalance and its control in HIV infection. Antioxid. Redox Signal. 4, 455–464 (2002). [DOI] [PubMed] [Google Scholar]
- 56.Shytaj I. L., et al., Alterations of redox and iron metabolism accompany the development of HIV latency. EMBO J. 39, e102209 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Rong L., Perelson A. S., Modeling latently infected cell activation: Viral and latent reservoir persistence, and viral blips in HIV-infected patients on potent therapy. PLoS Comput. Biol. 5, e1000533 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Ho Y. C., et al., Replication-competent noninduced proviruses in the latent reservoir increase barrier to HIV-1 cure. Cell 155, 540–551 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Kalantari P., et al., Thioredoxin reductase-1 negatively regulates HIV-1 transactivating protein Tat-dependent transcription in human macrophages. J. Biol. Chem. 283, 33183–33190 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Matthews J. R., Wakasugi N., Virelizier J.-L., Yodoi J., Hay R. T., Thioredoxin regulates the DNA binding activity of NF-kappa B by reduction of a disulphide bond involving cysteine 62. Nucleic Acids Res. 20, 3821–3830 (1992). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Mousseau G., et al., Resistance to the tat inhibitor didehydro-cortistatin A is mediated by heightened basal HIV-1 transcription. mBio 10, e01750-18 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Cox C. D., et al., Frequency domain analysis of noise in simple gene circuits. Chaos 16, 026102 (2006). [DOI] [PubMed] [Google Scholar]
- 63.Welch B. L., The generalisation of student’s problems when several different population variances are involved. Biometrika 34, 28–35 (1947). [DOI] [PubMed] [Google Scholar]
- 64.Holm S., A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979). [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.