Defining Stable Reference Genes in HIV Latency Reversal Experiments

Latency-reversing agents (LRAs) are ubiquitously used in the “shock-and-kill” HIV cure strategy, and their performance is often evaluated by ex vivo quantification of cell-associated HIV RNA. HIV RNA, measured by quantitative PCR (qPCR), is often normalized to internal reference genes, but the expression of these genes should not be influenced by the experimental settings.

ABSTRACT

Quantification of cell-associated HIV RNA (ca-RNA) is one of the most important and commonly used methods to evaluate the performance of latency-reversing agents (LRAs). Copies of HIV RNA measured by quantitative PCR (qPCR) are often normalized to the input RNA or cell number. However, these could be affected by biological variability and/or technical errors, which can be avoided by using an internal reference gene. To obtain reliable data, it is essential to select stable reference genes (RGs) of which the expression is not influenced by biological variability, the type of cells, or the LRAs used. However, to date, no study has carefully evaluated RG stability following LRA exposure. We analyzed the stability of six widely used RGs (GAPDH, TBP, YWHAZ, UBE2D2, HPRT1, and RPL27A) in human peripheral blood mononuclear cells (PBMCs) and CD4⁺ T cells. LRA exposure significantly influenced the stability of these RGs. Overall, TBP, UBE2D2, and RPL27A were the most stable RGs under all tested conditions. TBP was generally the most stable RG, whereas GAPDH varied the most. Finally, we evaluated the impact of applying different RG normalizers to host genes and HIV ca-RNA data. Altered results were observed both in host and HIV gene expression when unstable RGs were used. Our data underline the importance of testing the stability of RGs utilized to evaluate LRA-induced HIV ca-RNA expression. To our knowledge, this is the first careful evaluation of the stability of RGs after LRA exposure and will significantly contribute to the quality of data analysis in regard to gene expression.

IMPORTANCE Latency-reversing agents (LRAs) are ubiquitously used in the “shock-and-kill” HIV cure strategy, and their performance is often evaluated by ex vivo quantification of cell-associated HIV RNA. HIV RNA, measured by quantitative PCR (qPCR), is often normalized to internal reference genes, but the expression of these genes should not be influenced by the experimental settings. We found that treatment of human peripheral blood mononuclear cells (PBMCs) and CD4⁺ T cells with LRAs significantly altered the expression of several commonly used reference genes, such as GAPDH. Finally, we evaluate the impact of different reference genes on the normalization of host genes and HIV cell-associated RNA expression and demonstrated that using unstable reference genes dramatically altered experimental outcome. Our data highlight the importance of using reference genes that are unaffected by LRAs under study to correctly evaluate host gene and cell-associated HIV RNA expression induced by latency-reversing agents.

INTRODUCTION

The latent HIV reservoir, maintained by multiple mechanisms, is the major barrier toward achieving a cure for HIV-1 infection. The use of latency-reversing agents (LRAs) to reverse latency combined with augmentation of the immune response to enhance clearance of infected cells is one of the proposed HIV cure strategies. LRAs either eliminate HIV transcriptional blocks at the level of chromatin or interfere with cellular signaling pathways to induce HIV RNA expression (1). Measurement of HIV cell-associated RNA (ca-RNA) is one of the most widely used methods to evaluate the performance of LRAs in vitro, ex vivo, and in vivo. ca-RNA detection using quantitative PCR (qPCR) performed on bulk cells (2, 3) is a high-throughput and fairly inexpensive method to assess global gene expression and evaluate LRA activity. Evaluation of host gene expression during latency or following LRA exposure may provide useful insights toward understanding the mechanisms that govern HIV latency or developing new antilatency strategies (4–6).

Both HIV ca-RNA and host gene expression by qPCR require normalization of the data to a stable normalizer. The most straightforward strategy is to normalize the acquired PCR data to the input number of cells. However, this strategy is subject to bias introduced by technical errors or culture conditions (7). Another commonly used normalization approach is to use total input RNA, but this requires an accurate quantification method and equal RNA quality among samples (8). Additionally, normalizing to total RNA can be fraught with problems. Errors could be introduced during the extraction procedure, and interfering molecules contained within the extraction could influence the reverse transcription-PCR (RT-PCR) or qPCR efficiency. Moreover, total RNA is predominately composed of rRNA (80%), and the rRNA/mRNA ratio could vary under the experimental conditions (7). Normalizing on DNA could overcome these limitations but requires coextraction of both RNA and DNA from the same sample, a procedure that is generally less efficient and more complicated (7). Another suggested normalization method is the incorporation of external RNA molecules (RNA spike) in the extraction process (8–11). Although this method can be used to monitor extraction efficiency and PCR inhibition, sampling variation cannot be fully controlled and spiking-in could introduce technical errors. Those aspects, together with their high cost, make spiked RNA a rarely used and not fully validated normalization strategy in gene expression studies.

The use of endogenous reference genes (RGs) overcomes most of the limitations of the previously mentioned methods (7, 12). From the nucleic acid extraction process to the qPCR, RGs are subjected to the same processing steps as the genes of interest (GOIs). However, to obtain reliable GOI data, it is essential to select a stable RG, of which the expressions is not influenced by biological variability, the type of cells, nor the treatment condition. Using an unstable RG could introduce unintended bias in the results with serious implications on reliability and interpretation. Moreover, the minimum information for publication of quantitative real-time PCR experiments (MIQE) guidelines emphasized the need for RG validation and clear evidence of stability under analyzed experimental conditions (13). Classical RGs, such as GAPDH, β-actin, 18S rRNA, and HPRT1, are continuously used in relative gene expression analyses, despite the increasing evidence of their instability (14, 15). To date, no study has carefully evaluated RG stability in HIV latency-reversing strategies involving LRAs.

The goal of this study was to construct a guidepost of RGs suitable for analyzing HIV and host RNA expression following LRA exposure. We evaluated the impact of six widely used LRAs on cell viability, on RNA recovery, and on the stability of RGs in peripheral blood mononuclear cells (PBMCs) and CD4⁺ T cells after 6 h and 24 h of exposure. The mechanisms of action of the six selected LRAs are as follows: histone posttranslational modification modulators (vorinostat and romidepsin), bromodomain and extraterminal domain (BET) inhibitor (iBET-151), canonical NF-κB pathway stimulators via protein kinase C activation (PEP005 or ingenol-3-angelate), noncanonical NF-κB pathway stimulators through inhibition of inhibitors of apoptosis proteins (AZD-5582), and double-stranded DNA sensor and stimulator of interferon genes (STING agonist). Additionally, we included phytohemagglutinin (PHA) and phorbol myristate acetate-ionomycin (PMAi) treatment, since they are commonly used as positive controls in in vitro LRA studies. We selected 6 RGs, GAPDH, TBP, YWHAZ, UBE2D2, HPRT1, and RPL27A, based on their frequency of reported usage in the literature and on previous validation in PBMCs and CD4⁺ T cells (6, 15–18). These genes belong to different functional families to avoid coregulations that could interfere with downstream stability analyses and inadvertently identify false-stable RGs. We measured the fold change (FC) in expression of these genes and assessed the most stable RGs utilizing four widely used algorithms: geNorm, NormFinder, BestKeeper, and the comparative delta-C_T (ΔC_T) method. We incorporated the RefFinder tool, which integrates all of the aforementioned algorithms, in our analysis. Finally, we show the impact of the RG of choice on the normalization of the genes of interest (both HIV and host genes) and demonstrate that unstable RGs could mislead relative expression interpretations.

RESULTS

Impact of treatment on viability and total RNA recovery.

We first evaluated the viability of PBMCs and CD4⁺ T cells isolated from healthy donors after 6 h and 24 h of treatment with either dimethyl sulfoxide (DMSO; vehicle control), PHA, PMA/ionomycin (PMAi), or the following LRAs: vorinostat (VOR), romidepsin (ROM), PEP005, AZD-5582 (AZD), iBET-151 (iBET), or STING agonist (STINGa). PMAi exposure resulted in a decrease in viability of PBMCs during the first 6 h of treatment (57% ± 7%), which persisted over time (70% ± 5%). After 24 h, we observed a marked decrease in the viability of PBMCs exposed to STINGa (56% ± 5%) and a negligible decrease under the other conditions. In contrast to that for the PBMCs, for CD4⁺ T cells, we did not observe any decreases in viability for any of the conditions or time points (Fig. 1A).

FIG 1.

Viability and total RNA recovery in PBMCs and CD4⁺ T cells after LRA exposure. (A) Viability after LRA exposure. The alamarBlue assay was used to evaluate the viability in PBMCs (n = 4) and CD4⁺ T cells (n = 4). alamarBlue reagent was added after 6 h or after 24 h, and fluorescence was detected after 6 h of incubation. (B) RNA yields normalized to that with DMSO. Total RNA recovery normalized to the DMSO condition from 1 × 10⁶ PBMCs and CD4⁺ T cells after 6-h or 24-h LRA exposure. (C) RNA concentrations. Total RNA concentrations expressed as nanograms per microliter from PBMCs (n = 4) and CD4⁺ T cells (n = 4). (D) RNA purity; 260/280-nm absorbance ratios for PBMCs (n = 4) and CD4⁺ T cells (n = 4). Data are expressed as means ± SDs. Six-hour time point in black and 24-h timepoint in red.

After 6 h or 24 h, one million cells were collected, and total RNA was extracted. Comparable to viability, we observed a similar decrease in RNA recovery under PMAi and STINGa conditions compared to DMSO (Fig. 1B). Total RNA recovery was higher from CD4⁺ T cells (54.56 ± 13.01 ng/μl) than from PBMCs (33.79 ± 12.80 ng/μl) (Fig. 1C). No differences in total RNA purity was observed among the conditions and between PBMCs and CD4⁺ T cells (Fig. 1D).

LRAs influenced the expression of reference genes.

The expression of six commonly used RGs (GAPDH, TBP, YWHAZ, UBE2D2, HPRT1, and RPL27A) were measured after 6 h and 24 h in the control (DMSO) and LRA-treated samples. The 2^−ΔC_T method (19) was employed to evaluate whether the expression of these RGs varied under the various LRA conditions. In drug treatment experiments, a fold decrease of greater than 2 was previously reported to be indicative of an inappropriate RG (20). We therefore hypothesize that a fold increase higher than 2 could also denote an RG that is inappropriate to use as a normalizer. We propose the use of this threshold (2-fold increase/decrease) as a preliminary parameter to define the stability of RGs. From this, we define the RG stability zone to be when the fold change in expression falls between 0.5 and 2 (corresponding to a 2-fold decrease and increase, respectively). RGs were considered stable when both the upper and lower limits of the 90% confidence interval (CI) for the fold change were inside the stability zone and highly unstable when both the limits were outside the stability zone. This approach for establishing equivalency is analogous to conducting two one-sided tests, each with a type I error rate of 0.05 (21). RGs with one CI limit outside the 0.5 to 2 range were considered modestly regulated. Detailed results are summarized in Tables 1 and 2 for PBMCs and CD4⁺ T samples, respectively.

TABLE 1.

RG fold change and stability parameters of the six tested RGs in PBMCs^a

Condition	RG	6h					24h
		Fold change			geNorm M	BestKeeper SD	Fold change			geNorm M	BestKeeper SD
		Mean	Upper 90% CI	Lower 90% CI			Mean	Upper 90% CI	Lower 90% CI
PHA/DMSO	TBP	0.76	0.98	0.59	0.37	0.25	1.98	2.55	1.54	0.46	0.79
	RPL27A	0.70	0.88	0.56	0.36	0.36	1.70	2.08	1.39	0.56	0.61
	UBE2D2	0.70	0.92	0.52	0.38	0.37	2.59	3.66	1.84	0.42	0.77
	HPRT1	1.84	2.66	1.27	0.78	0.44	3.80	6.88	2.10	0.52	1.08
	YWHAZ	0.49	0.65	0.36	0.53	0.52	2.19	3.37	1.42	0.46	0.79
	GAPDH	0.91	1.07	0.77	0.37	0.28	6.17	12.13	3.14	0.81	1.40
PMAi/DMSO	TBP	0.79	1.18	0.53	0.70	0.22	2.37	5.33	1.05	0.44	0.75
	RPL27A	0.91	1.11	0.75	0.68	0.17	2.00	4.18	0.95	0.52	0.58
	UBE2D2	0.49	0.67	0.36	0.78	0.52	2.49	5.86	1.06	0.44	0.75
	HPRT1	2.66	3.39	2.09	1.17	0.71	4.68	15.91	1.38	0.65	1.19
	YWHAZ	0.19	0.23	0.16	1.33	1.20	1.49	3.72	0.60	0.67	0.66
	GAPDH	1.31	2.01	0.85	0.77	0.36	6.23	19.82	1.96	0.80	1.40
VOR/DMSO	TBP	0.92	1.35	0.63	0.25	0.22	0.74	1.28	0.43	0.32	0.86
	RPL27A	0.91	1.17	0.70	0.27	0.18	0.98	1.29	0.74	0.46	0.65
	UBE2D2	1.16	1.73	0.78	0.24	0.27	0.88	1.56	0.50	0.30	0.78
	HPRT1	1.04	1.81	0.59	0.21	0.32	0.96	1.80	0.51	0.32	1.02
	YWHAZ	1.00	1.49	0.68	0.18	0.28	0.83	1.69	0.41	0.31	0.97
	GAPDH	1.26	1.91	0.84	0.26	0.39	1.08	2.01	0.58	0.40	1.05
ROM/DMSO	TBP	0.60	1.09	0.33	0.33	0.46	0.69	1.73	0.27	0.44	0.98
	RPL27A	0.73	1.18	0.45	0.23	0.26	0.68	1.49	0.31	0.49	0.81
	UBE2D2	0.82	1.21	0.55	0.28	0.29	0.57	1.18	0.28	0.47	0.80
	HPRT1	0.60	1.04	0.35	0.29	0.38	0.70	1.99	0.24	0.49	1.19
	YWHAZ	0.72	1.06	0.49	0.23	0.25	0.28	0.75	0.10	0.81	1.11
	GAPDH	0.80	1.47	0.43	0.26	0.31	1.07	2.44	0.47	0.63	1.15
PEP005/DMSO	TBP	0.80	1.16	0.55	0.42	0.25	1.12	1.82	0.69	0.45	0.75
	RPL27A	0.84	1.08	0.66	0.36	0.16	1.35	2.12	0.85	0.46	0.74
	UBE2D2	0.80	0.98	0.66	0.38	0.25	1.45	2.05	1.02	0.37	0.71
	HPRT1	1.44	1.99	1.04	0.46	0.27	1.72	3.03	0.97	0.40	0.94
	YWHAZ	0.75	0.94	0.60	0.37	0.24	1.72	3.18	0.93	0.35	0.89
	GAPDH	1.89	2.46	1.45	0.61	0.47	3.09	5.05	1.90	0.67	1.18
AZD/DMSO	TBP	0.72	1.30	0.40	0.27	0.41	0.56	1.09	0.29	0.32	0.87
	RPL27A	0.71	1.42	0.36	0.25	0.36	0.78	1.82	0.33	0.44	0.85
	UBE2D2	0.79	1.22	0.51	0.28	0.27	0.63	1.19	0.34	0.40	0.77
	HPRT1	0.69	1.41	0.34	0.23	0.41	0.64	1.46	0.28	0.35	1.05
	YWHAZ	0.69	1.20	0.40	0.22	0.34	0.66	1.46	0.30	0.32	0.94
	GAPDH	0.87	1.49	0.51	0.25	0.29	0.58	1.45	0.23	0.49	1.12
IBET/DMSO	TBP	0.60	0.97	0.37	0.34	0.41	0.70	1.15	0.43	0.25	0.85
	RPL27A	0.60	1.09	0.33	0.36	0.41	0.73	1.17	0.46	0.35	0.71
	UBE2D2	0.66	1.03	0.43	0.36	0.34	0.75	1.17	0.49	0.33	0.73
	HPRT1	0.71	1.30	0.38	0.34	0.35	0.77	1.39	0.43	0.29	0.95
	YWHAZ	0.30	0.56	0.16	0.64	0.86	0.70	1.26	0.39	0.29	0.97
	GAPDH	0.79	1.59	0.39	0.39	0.35	0.80	1.36	0.47	0.38	1.00
STINGa/DMSO	TBP	0.59	0.78	0.45	0.31	0.38	0.38	0.48	0.30	0.34	0.89
	RPL27A	0.65	0.76	0.56	0.35	0.31	0.47	0.57	0.38	0.48	0.71
	UBE2D2	0.49	0.65	0.37	0.30	0.52	0.35	0.41	0.31	0.40	0.79
	HPRT1	0.65	0.89	0.48	0.33	0.33	0.34	0.39	0.31	0.40	1.11
	YWHAZ	0.35	0.46	0.27	0.46	0.75	0.38	0.53	0.27	0.36	0.97
	GAPDH	0.49	0.71	0.34	0.30	0.51	0.32	0.45	0.23	0.59	1.24

^aRelated to Fig. 2A. RG fold change expressed as average and 90% confidence interval (CI) upper and lower limits under LRA conditions relative to that in DMSO in PBMC (n = 4) after 6 h and 24 h. M stability value of geNorm and standard deviation (SD) value of BestKeeper in data set composed of two conditions (DMSO plus one LRA) from PBMC after 6 h or after 24 h. Fold change of >2 or <0.5, M value of >1.5, and SD of >1 indicate unstable RGs (highlighted in boldface font).

TABLE 2.

RG fold change and stability parameters of the six tested RGs in CD4⁺ T cells^a

Condition	RG	6h					24h
		Fold change			geNorm M	BestKeeper SD	Fold change			geNorm M	BestKeeper SD
		Mean	Upper 90% CI	Lower 90% CI			Mean	Upper 90% CI	Lower 90% CI
PHA/DMSO	TBP	1.08	1.27	0.93	0.30	0.13	1.55	1.87	1.29	0.55	0.32
	RPL27A	0.98	1.11	0.86	0.34	0.10	1.48	1.90	1.15	0.58	0.28
	UBE2D2	1.22	1.45	1.03	0.30	0.16	2.26	2.76	1.86	0.56	0.59
	HPRT1	2.18	2.41	1.97	0.58	0.56	4.08	5.37	3.09	0.64	1.01
	YWHAZ	0.84	1.08	0.66	0.42	0.18	2.06	2.27	1.87	0.53	0.52
	GAPDH	1.71	2.09	1.40	0.41	0.42	7.57	10.73	5.35	1.01	1.46
PMAi/DMSO	TBP	1.39	1.55	1.25	0.83	0.25	2.96	3.38	2.59	0.64	0.78
	RPL27A	1.06	1.06	1.05	0.81	0.14	1.86	2.03	1.70	0.81	0.45
	UBE2D2	0.77	0.90	0.66	0.91	0.23	4.16	4.92	3.51	0.69	1.03
	HPRT1	5.31	6.35	4.44	1.58	1.20	7.35	10.98	4.92	0.83	1.44
	YWHAZ	0.10	0.11	0.08	2.17	1.68	1.32	1.75	1.00	0.99	0.23
	GAPDH	1.65	2.01	1.36	0.92	0.43	12.18	16.91	8.76	1.12	1.80
VOR/DMSO	TBP	1.00	1.13	0.90	0.17	0.18	0.92	1.02	0.84	0.26	0.07
	RPL27A	1.01	1.14	0.89	0.23	0.13	1.05	1.20	0.91	0.20	0.10
	UBE2D2	1.24	1.33	1.16	0.21	0.25	1.19	1.40	1.01	0.32	0.17
	HPRT1	1.09	1.22	0.98	0.25	0.31	1.14	1.20	1.09	0.22	0.18
	YWHAZ	1.00	1.17	0.85	0.20	0.25	1.02	1.06	0.97	0.21	0.12
	GAPDH	1.25	1.44	1.08	0.22	0.33	1.58	1.77	1.41	0.37	0.35
ROM/DMSO	TBP	0.70	0.79	0.63	0.30	0.26	1.27	1.54	1.05	0.46	0.21
	RPL27A	0.97	1.06	0.88	0.24	0.17	0.99	1.11	0.89	0.46	0.08
	UBE2D2	0.99	1.12	0.87	0.23	0.20	0.92	1.02	0.84	0.55	0.16
	HPRT1	0.78	0.87	0.69	0.36	0.34	1.25	1.42	1.10	0.48	0.24
	YWHAZ	0.98	1.06	0.90	0.22	0.23	0.32	0.34	0.29	1.11	0.83
	GAPDH	1.01	1.14	0.89	0.27	0.31	2.49	2.97	2.10	0.86	0.66
PEP005/DMSO	TBP	1.01	1.06	0.95	0.43	0.16	1.06	1.15	0.98	0.45	0.07
	RPL27A	0.89	1.07	0.74	0.51	0.17	1.06	1.18	0.95	0.45	0.06
	UBE2D2	1.47	1.79	1.21	0.38	0.28	1.70	2.12	1.37	0.44	0.38
	HPRT1	2.01	2.48	1.62	0.49	0.50	1.50	1.75	1.29	0.38	0.30
	YWHAZ	1.17	1.34	1.02	0.39	0.26	1.53	1.89	1.24	0.38	0.33
	GAPDH	3.05	3.71	2.51	0.74	0.81	3.61	4.69	2.78	0.81	0.93
AZD/DMSO	TBP	0.92	1.07	0.79	0.17	0.17	0.97	1.07	0.87	0.17	0.06
	RPL27A	0.96	1.10	0.84	0.20	0.16	0.98	1.11	0.86	0.16	0.10
	UBE2D2	0.92	1.21	0.70	0.19	0.24	1.00	1.12	0.89	0.30	0.12
	HPRT1	0.82	0.94	0.71	0.31	0.24	1.00	1.26	0.79	0.20	0.22
	YWHAZ	0.89	1.01	0.79	0.18	0.24	0.99	1.08	0.90	0.16	0.13
	GAPDH	0.98	1.12	0.86	0.24	0.30	1.04	1.18	0.92	0.17	0.17
IBET/DMSO	TBP	0.89	1.01	0.79	0.30	0.14	0.85	1.04	0.70	0.19	0.16
	RPL27A	0.92	1.17	0.73	0.29	0.22	0.89	1.01	0.79	0.19	0.12
	UBE2D2	0.95	1.19	0.76	0.25	0.25	1.00	1.20	0.84	0.32	0.15
	HPRT1	0.93	1.20	0.71	0.28	0.29	0.84	1.00	0.71	0.24	0.22
	YWHAZ	0.50	0.63	0.39	0.54	0.55	0.70	0.79	0.62	0.28	0.26
	GAPDH	1.11	1.21	1.02	0.34	0.30	1.14	1.23	1.06	0.30	0.16
STINGa/DMSO	TBP	0.85	0.87	0.82	0.33	0.18	0.81	0.95	0.70	0.25	0.15
	RPL27A	0.81	0.96	0.68	0.33	0.22	0.67	0.80	0.56	0.19	0.29
	UBE2D2	0.65	0.70	0.59	0.28	0.33	0.77	0.85	0.71	0.31	0.24
	HPRT1	0.75	0.86	0.65	0.32	0.31	0.65	0.77	0.54	0.20	0.31
	YWHAZ	0.39	0.44	0.35	0.54	0.68	0.61	0.67	0.55	0.22	0.36
	GAPDH	0.63	0.73	0.55	0.33	0.41	0.56	0.74	0.42	0.29	0.42

^aRelated to Fig. 2B. RG fold change expressed as average and 90% CI upper and lower limits under LRA conditions relative to that in DMSO in CD4⁺ T cells (n = 4) after 6 h and 24 h. M stability value of geNorm and SD value of BestKeeper in data set composed of two conditions (DMSO plus one LRA) from CD4⁺ T cells after 6 h or after 24 h. Fold change of >2 or <0.5, M value of >1.5, and SD of >1 indicate unstable RGs (highlighted in boldface font).

In PBMCs treated with PHA for 24 h (Fig. 2A; Table 1), we observed that GAPDH (6.17-fold change; 90% CI, 3.14 to 12.13) and HPRT1 (3.80-fold change; 90% CI, 2.10 to 6.88) were highly unstable. Treatment with PMAi for 6 h resulted in a 0.19-fold change (90% CI, 0.16 to 0.23) in YWHAZ and a 2.66-fold change in HPRT1 (90% CI, 2.09 to 3.39), denoting high instability. After exposure to STINGa for 24 h, GAPDH, TBP, UBE2D2, and HPRT1 were highly unstable, with 0.32-fold, 0.38-fold, 0.35-fold, and 0.34-fold changes, respectively (90% CIs, 0.23 to 0.45, 0.30 to 0.48, 0.31 to 0.41, and 0.31 to 0.39, respectively). Treatment of PBMCs with VOR for 6 h was the only condition where there appeared to be no significant fold change in expression across all the RGs tested.

FIG 2.

Reference gene fold change expression in PBMCs and CD4⁺ T cells after LRA exposure. The RG fold change expression was calculated using the 2^−ΔC_T method (19) under LRA conditions relative to the DMSO control in PBMCs (n = 4) (A) and CD4⁺ T cells (n = 4) (B) and after 6 h (black) or 24 h (red) of exposure. Each data point represents one donor. The 90% CI was compared to the 0.5- to 2-fold change range (stability zone), corresponding to 2-fold decrease and increase, respectively. The equal sign (=) indicates fold change statistically equal to the stability zone (0.5 to 2). The up (↑) and down (↓) arrows indicate fold change statistically outside the stability zone, up and downregulated, respectively; RGs with only one 90% CI outside the 0.5 to 2 range were considered modestly regulated, and no symbol was applied.

In CD4⁺ T cells (Fig. 2B; Table 2), similarly to in PBMCs, GAPDH and HPRT1 were highly unstable, with a 7.57-fold change (90% CI, 5.35 to 10.73) and a 4.08-fold change (90% CI, 3.09 to 5.37), respectively, after 24 h of PHA exposure. Likewise, we observed a 0.10-fold change (90% CI, 0.08 to 0.11) in YWHAZ and a 5.31-fold change (90% CI, 4.44 to 6.35) in HPRT1 after 6 h of PMAi treatment. Interestingly, after 24 h of exposure to PMAi, YWHAZ was the only reference gene not modulated, denoting stability at that time point (1.32-fold change; 90% CI, 1.00 to 1.75), whereas GAPDH, TBP, UBE2D2, and HPRT1 were upregulated with 12.18-fold, 2.96-fold, 4.16-fold, and 7.35-fold changes, respectively (90% CIs, 8.76 to 16.91, 2.59 to 3.38, 3.51 to 4.92, and 4.92 to 10.98, respectively). GAPDH and YWHAZ were the only genes modulated by ROM treatment after 24 h, with a 2.49-fold change (90% CI, 2.10 to 2.97) and 0.32-fold change (90% CI, 0.29 to 0.34), respectively. In PEP005-treated CD4⁺ T cells, GAPDH was unstable after both 6 h and 24 h, with 3.05- and 3.61-fold changes, respectively (90% CIs, 2.51 to 3.71 and 2.78 to 4.69, respectively). YWHAZ was downregulated in STINGa-treated cells after 6 h, with a 0.39-fold change (90% CI, 0.35 to 0.44). In CD4⁺ T cells, none of the tested RGs were modulated by VOR after either 6 h or 24 h, by ROM after 6 h, by AZD after 6 h or 24 h, or by iBET after 24 h. Moreover, the fold changes expression of TBP, RPL27A, and UBE2D2 were well within the stability range under all of the tested conditions after 6 h.

Overall, GAPDH, HPRT1, and YWHAZ varied the most in both PBMCs and CD4⁺ T cells following LRA exposure. RPL27A was the only RG that was never highly unstable, because both CI limits were never outside the stability zone under all of the tested conditions, in both PBMCs and CD4⁺ T cells, and at both the tested time points. In particular, in CD4⁺ T cells, RPL27A was consistently within the stability zone with only a minor upregulation (1.86-fold change; 90% CI, 1.70 to 2.03) under the PMAi condition after 24 h (Fig. 2 and Tables 1 and 2).

Higher instability of YWHAZ, HPRT1, and GAPDH detected by geNorm and BestKeeper.

The geNorm M stability value of >1.5 (22) and the BestKeeper standard deviation (SD) of >1 (23) were previously suggested as indicators of RG instability. We compared those parameters with the fold change stability zone approach in treatment and control conditions (as conducted for the fold change calculations) for PBMCs or CD4⁺ T cells and at 6 h or 24 h. See Tables 1 and 2 for detailed results in PBMC and CD4⁺ T cell samples. The M stability values of geNorm were >1.5 only in YWHAZ and HPRT1 in CD4⁺ T cells after 6 h in DMSO/PMAi data sets. Both of these RGs were also indicated as being highly unstable after PMAi treatment by the fold change analyses. However, the geNorm threshold (M value > 1.5) failed to detect any other RG instability identified by the fold change threshold described above. This suggests that the geNorm threshold is more tolerant than the 0.5- to 2-fold change stability zone we proposed.

The BestKeeper SD stability values of TBP and RPL27A in all the analyses were <1, denoting stability of those RGs. As expected, higher SD values were observed at 24 h than at 6 h. Of note, the SD values of GAPDH were above the threshold in all the analyses in the PBMC data sets and in the CD4⁺ T cell DMSO/PHA and DMSO/PMAi data sets at 24 h. Compared to the fold change threshold of 0.5 to 2, the SD stability value of BestKeeper failed to detect instability in 13 highly unstable and 27 modestly regulated RGs detected by the fold change analyses. However, one instability was detected by the SD threshold and not by the fold change threshold of 0.5 to 2. This corresponded to HPRT1 in PBMCs after 24 h under VOR conditions where both the SD and fold change results were close to the respective threshold limits (SD = 1.02; 0.96-fold change; 90% CI, 0.51 to 1.80).

Concordance in the stability ranking among algorithms and RefFinder.

To determine the most stable RGs, the most commonly used algorithms are geNorm (22), NormFinder (24), BestKeeper (23), and the comparative ΔC_T method (25). Results can vary significantly between these algorithms due to mathematical methods and assumptions used by each approach. It is therefore advised that at least three of these tools be used to evaluate RG stability (26). Of note, all four algorithms were combined into a free online tool, called RefFinder (27), that calculates a final ranking of RG stability based on the geometric mean of the rankings from each program.

In HIV latency reversal experiments using LRAs, at least three conditions are frequently included: a vehicular control condition (usually DMSO), one positive-control treatment condition (usually PHA or PMAi), and the LRA condition. Therefore, data sets composed of quantification cycle (C_q) values of three combined conditions were created, including C_q values from only PBMCs, CD4⁺ T cells, or both and from only one of the time points (6 h or 24 h) or both. See File S1A in the supplemental material for detailed ranking results of each analysis.

The ranking results were subdivided based on the analyzed group of cells and were summarized by calculating the number of times (expressed as percentage on the x axis) that each RG was in the first position (black), the top two positions (blue), and the top three positions (red), as displayed in Fig. 3A to C (PBMCs) and Fig. 3D to F (CD4⁺ T cells). Differences in the ranking positions were observed among the algorithms, but overall, TBP, RPL27A, and UBE2D2 were the most frequent RGs in the top three ranking positions in PBMCs and CD4⁺ T cells (Fig. 3A and D). Concordance among the tools was stronger when the frequencies were calculated using ranking results from data sets composed of both PBMCs and CD4⁺ T cells (data not shown; File S1A) rather than separately. In addition, the frequencies were calculated using ranking results of subgroups of data sets composed of PHA or PMAi (Fig. 3B and E) or 6 h, 24 h, or both (Fig. 3C and F). As previously, we observed similar results among the programs and comparable to those from RefFinder.

FIG 3.

Summary of the reference gene stability using different algorithms and RefFinder tools in PBMCs or CD4⁺ T cells. Frequency (%) of each RG ranking position using RefFinder, geNorm, NormFinder, BestKeeper and the comparative ΔC_T method when three conditions (DMSO plus PHA/PMAi plus one LRA) were analyzed. Frequency calculated analyses of all PBMC data sets (A), of PBMC data set including PHA or PMAi (B), of PBMC data set including 6 h, 24 h, or both (C), of all CD4⁺ T cell data sets (D), of CD4⁺ T cell data set including PHA or PMAi (E), and of CD4⁺ T cell data set including 6 h, 24 h, or both (F). x axis, frequency that each RG is in the first position (black), in the top two positions (blue), and in the top three positions (red). Numbers in parentheses indicate the total number of analyses.

TBP, RPL27A, and UBE2D2 are frequently the best RGs.

With three conditions (vehicle control, positive control, and one LRA), we clearly demonstrated that TBP, RPL27A, and UBE2D2 are frequently found in the top three positions of the RefFinder results (Fig. 3). However, experiments are often performed with more than one LRA in order to compare their performances. We analyzed data sets composed of more than one LRA, subdividing rankings based on the analyzed group of cells, as presented in Fig. 4A to C (PBMCs) and Fig. 4D to F (CD4⁺ T cells). See File S1B for detailed ranking results of each analysis.

FIG 4.

Summary of the reference gene stability in PBMCs or CD4⁺ T cells when one or more LRAs are analyzed. Frequency (%) of each RG ranking position using RefFinder when three or more conditions (from top to bottom) were analyzed and calculated on the total analyses of all PBMC data sets (A), of PBMC data set including PHA or PMAi (B), of PBMC data set including 6 h, 24 h, or both (C), of all CD4⁺ T cell data sets (D), of CD4⁺ T cell data set including PHA or PMAi (E), and of CD4⁺ T cell data set including 6 h, 24 h, or both (F). x axis, frequency that each RG is in the first position (black), in the top two positions (blue), and in the top three positions (red). Numbers in parentheses indicate the total number of analyses.

Overall, TBP, RPL27A, and UBE2D2 were confirmed as the most frequent RGs in the top three ranking positions (Fig. 4A and D). The RefFinder tools indicated TBP as the most stable RG (first ranking position) in 51% of all the analyses (DMSO plus PHA/PMAi plus 1 to 6 LRAs) (data not shown), in 43% for the PBMC subgroup (bottom row of Fig. 4A), in 51% for the CD4⁺ T cell subgroup (bottom row of Fig. 4D), and 58% for the PBMC plus CD4 subgroup (data not shown; File S1B). Moreover, TBP was found in the top three ranking positions in 98% of all the analyses (DMSO plus PHA/PMAi plus 1 to 6 LRAs) (data not shown), in 98% for the PBMC subgroup (bottom row of Fig. 4A), in 98% for the CD4⁺ T cell subgroup (bottom row of Fig. 4D), and 100% for the PBMC plus CD4 subgroup (data not shown; File S1B).

Of note, HPRT1 never ranked as the most stable RG, and it occupied the top three positions in only 0.6% of all the analyses (data not shown). GAPDH and YWHAZ were indicated as the most stable RG in 1.8% and 3.8%, respectively, in all the analyses (data not shown). This was mostly observed in a data set composed of conditions where GAPDH or YWHAZ did not show a fold change outside the 0.5 to 2 range (compare File S1B with Fig. 2 and Tables 1 and 2).

We observed a relationship between the complexity of the data set and the ranking of RGs. Adding more than one LRA resulted in increased frequency of TBP, RPL27A, and UBE2D2 in the top three positions, for both PBMCs and CD4⁺ T cells (from top to bottom of Fig. 4A and D). Similarly, TBP, RPL27A, and UBE2D2 were found in the top three positions in all the analyses when both PBMCs and CD4⁺ T cells were included (data not shown; File S1B).

When evaluating subgrouped ranking results, YWHAZ was more frequently found in the top three positions in PHA data sets than in PMAi data sets, in which YWHAZ was never found in the top three positions (Fig. 4B and E). This result confirms the higher instability of YWHAZ in PMAi than in PHA that we previously observed, with a higher YWHAZ fold change expression in PMAi than in PHA (Fig. 2; Tables 1 and 2). Similarly, our initial analysis showed a higher fold change expression of YWHAZ after 6 h of PMAi exposure than after 24 h (Fig. 2; Tables 1 and 2). This resulted in a higher frequency of YWHAZ in the top three positions in the 24-h subgroup than in the 6-h subgroup (Fig. 4C and F). A similar but opposite trend was observed for GAPDH, with its expression often upregulated after 24 h but not after 6 h (Fig. 2; Tables 1 and 2). GAPDH was more frequently in the top three positions in the 6-h subgroup than in the 24-h subgroup or when data were pooled across time points (Fig. 4C and F), in particular, in PBMCs.

Rank aggregation analyses and consensus order.

Rank aggregation was used to determine a consensus order of RGs across experiments and subgroups and was implemented using the cross-entropy Monte Carlo algorithm in the RankAggreg package in R (28). The RefFinder rankings, obtained from each data set, were aggregated across experiments to form composite ranks of the six potential RGs from most stable (1) to least stable (6). The rank aggregation method does not allow for ties in the data set, but RefFinder produced ties for 193 (17%) of the analyses. To facilitate rank aggregation, ties were broken using linear mixed-effect modeling, as further described in Materials and Methods. Detailed new ranking results are available in File S1B.

TBP, RPL27A, and UBE2D2 were always found in the top three positions of the consensus lists when all the RefFinder rankings were included and in subgroups (Fig. 5). Overall, TBP represented the most stable RG and GAPDH the least stable RG when all the ranking results were analyzed. No differences in the top three positions were observed among the subgroups of cells (PBMC, CD4⁺ T cells, or PBMC and CD4⁺ T cells) or between PHA and PMAi subgroups. Small variations within the top three positions were observed among the time point subgroups and their combinations. RPL27A was the most stable RG in the 6-h subgroup, UBE2D2 was the most stable RG in the 24-h subgroup, and TBP was the most stable RG when the two time points were analyzed together (Fig. 5).

FIG 5.

Rank aggregation results. Rank aggregation consensus order (x axis) using the cross-entropy method across all the experiments (all, n = 1134) and the followinug subgroups: PHA (n = 567), PMAi (n = 567), PBMCs only (n = 378), CD4⁺ T cells only (n = 378), PBMCs and CD4⁺ T cells (n = 378), 6 h only (n = 378), 24 h only (n = 378), and 6 and 24 h (n = 378). The RefFinder rank results were used to rank the six potential reference genes from most stable (1) to least stable (6) (y axis).

As a sensitivity analysis, rank aggregation was repeated using the genetic algorithm in the RankAggreg package, and identical consensus lists were obtained (data not shown).

Highly altered IFI6 and ca-HIV gag RNA expression profile when normalized on unstable RGs.

As previously mentioned, the use of an unstable RG in qPCR analysis could unintentionally create artifactual GOI results. We provide examples of the consequences of applying highly unstable RGs on ca-HIV gag RNA and host gene expression, with IFI6 chosen as an example of a host gene. The comparative C_T method (also known as the 2^−ΔΔC_T method) (19) was performed using each single RG and the combination of two or three of the most stable RGs previously identified: TBP, RPL27A, and UBE2D2. The GOI fold change results are shown in Fig. 6, where red indicates a significant (P value < 0.05) difference in expression compared to the control (DMSO value = 1). The detailed raw data and RG fold changes are included in File S1C and D and Table 3, respectively.

FIG 6.

Alteration of HIV ca-RNA and IFI6 expression profiles when normalized to different RGs. (A) HIV ca-RNA fold expression, relative to that with DMSO, after 6-h exposure with PHA in rCD4⁺ cells from HIV-infected volunteers. IFI6 fold gene expression, relative to that with DMSO, in PBMCs (n = 7) (B) and in CD4⁺ T cells (n = 7) (C) after 24-h exposure to PHA. IFI6 fold gene expression, relative to that with DMSO, in PBMCs (n = 8) after 6 h (D) and 24 h (E) of exposure to STINGa. (F) HIV ca-RNA fold expression, relative to that with DMSO, after 6 h exposure to VOR in rCD4⁺ cells from HIV-infected volunteers. IFI6 fold gene expression, relative to that with DMSO, in PBMCs (n = 7) (G) and in CD4⁺ T cells (n = 7) (H) after 24 h of exposure to PHA. Fold change reported as geometric mean ± standard error of the mean (SEM). Statistical significance was calculated using Wilcoxon test compared to DMSO, and data with a P value of <0.05 are indicated in red. The letters on the x axis of each graph indicate the gene or gene combinations used for normalization. GAPDH (G), TBP (T), RPL27A (R), UBE2D2 (U), YWHAZ (Y), HPRT1 (H), TBP/RPL27A (TR), TBP/UBE2D2 (TU), RPL27A/UBE2D2 (RU), TBP/RPL27A/UBE2D2 (TRU). ^‡RG FC, RG fold change is indicated below each RG. The 90% CI RG fold change was compared to the 0.5- to 2-fold change range (stability zone). Highly regulated, modestly regulated, and stable RGs are indicated in red, black underline, and in black, respectively.

TABLE 3.

RG fold change and stability parameters of the six tested RGs in PBMCs and CD4⁺ T cells^a

Condition	RG	Fold change
		Mean	Upper 90% CI	Lower 90% CI
PHA/DMSO
PBMC 24h	TBP	2.26	2.80	1.83
	RPL27A	1.97	2.33	1.66
	UBE2D2	2.95	3.87	2.26
	HPRT1	4.48	6.45	3.11
	YWHAZ	2.58	3.36	1.98
	GAPDH	8.54	13.58	5.37
CD4 24h	TBP	1.61	1.81	1.44
	RPL27A	1.66	1.96	1.40
	UBE2D2	2.37	2.64	2.12
	HPRT1	4.29	4.98	3.70
	YWHAZ	2.13	2.27	1.99
	GAPDH	8.15	9.71	6.84
VOR/DMSO
PBMC 24h	TBP	0.84	1.09	0.64
	RPL27A	1.14	1.47	0.89
	UBE2D2	0.95	1.23	0.73
	HPRT1	1.04	1.40	0.78
	YWHAZ	0.92	1.27	0.66
	GAPDH	1.19	1.59	0.89
CD4 24h	TBP	0.90	0.96	0.85
	RPL27A	1.02	1.11	0.94
	UBE2D2	1.10	1.24	0.97
	HPRT1	1.15	1.20	1.10
	YWHAZ	0.91	1.06	0.78
	GAPDH	1.56	1.68	1.45
STINGa/DMSO
PBMC 6h	TBP	0.61	0.69	0.54
	RPL27A	0.74	0.86	0.64
	UBE2D2	0.49	0.57	0.42
	HPRT1	0.70	0.82	0.60
	YWHAZ	0.39	0.48	0.31
	GAPDH	0.46	0.67	0.32
PBMC 24h	TBP	0.49	0.62	0.39
	RPL27A	0.53	0.64	0.44
	UBE2D2	0.46	0.58	0.37
	HPRT1	0.44	0.54	0.36
	YWHAZ	0.50	0.65	0.38
	GAPDH	0.37	0.46	0.30

^aRelated to Fig. 6B to E, G, and H. RG fold change expressed as average and 90% CI upper and lower limits under LRA conditions relative to that in DMSO in PBMCs (n = 7) and CD4⁺ T cells (n = 7) under PHA and VOR conditions after 24 h and in PBMCs (n = 8) under STINGa condition after 6 h and 24 h. Fold change of >2 or <0.5 highlighted in boldface font.

We evaluated the impact of applying different normalizers to ca-HIV gag expression in resting CD4⁺ T (rCD4⁺) cells from an antiretroviral therapy (ART)-suppressed donor stimulated for 6 h with PHA. HIV ca-RNA was significantly upregulated when normalized to all the RGs, except when normalized to the unstable HPRT1. The 2.92-fold increase of HPRT1 altered the HIV ca-RNA fold expression, leading to an erroneously significant downregulation (Fig. 6A).

Similarly, we compared IFI6 gene expression profiles in uninfected PBMCs and CD4⁺ T cells after 24 h of PHA exposure. In PBMCs (Fig. 6B), IFI6 is not modulated if normalized to the most stable RGs or their combinations. However, the application of the highly unstable GAPDH (8.54-fold change; 90% CI, 5.37 to 13.58) resulted in an artificial significant fold decrease for IFI6 (0.34-fold change, P < 0.05). Similarly, in CD4⁺ T cells (Fig. 6C), the high upregulation of GAPDH (8.15-fold change; 90% CI, 6.84 to 9.71) resulted in an opposite interpretation of IFI6 expression compared to when normalized to stable TBP and RPL27A (1.61-fold change [90% CI, 1.44 to 1.81] and 1.66-fold change [90% CI, 1.40 to 1.96], respectively). In fact, IFI6 was significantly downregulated (0.38-fold change, P < 0.05) when normalized to GAPDH, but it was significantly upregulated (1.93 and 1.88-fold change, respectively; P < 0.05) when normalized to the stable TBP and RPL27A. Moreover, the significant upregulation of IFI6 by PHA treatment was artificially lost when IFI6 was normalized to the modestly upregulated YWHAZ (2.13-fold change; 90% CI, 1.99 to 2.27) and to the highly upregulated UBE2D2 and HPRT1 (2.37-fold change [90% CI, 2.21 to 2.64] and 4.29-fold change [90% CI, 3.70 to 4.98], respectively).

Accepted level of RG variability could influence small GOI fold change.

Differences in GOI expression profiles were sometime observed not only when the RGs where highly unstable (outside the 0.5 to 2 stability zone) but also when the RG expressions were within the stability zone. As the perfect and ideal RG does not exist (7), small variations in RG expression are accepted as they could be due to inherent RG variability, interdonor variability, and technical errors. However, the acceptable level of variation in RG expression depends on the GOI fold change resolution required (7). The RG fold change should be smaller than the expected GOI fold change. Therefore, it is important to define the RG fold change and to evaluate the GOI results in parallel.

For example, in PBMCs, under the STINGa conditions after 6 h and 24 h (Fig. 6D and E), the downregulation of the RGs (especially YWHAZ and GAPDH) did not affect the high fold change of IFI6. Therefore, even if an RG is not ideal, because of modest instability, its usage will not compromise the interpretation of the results if the GOI fold change is very high. On the other hand, small changes in GOI expression are more susceptible to the influence of even modest fluctuation in RG expression. In CD4⁺ T cells (Fig. 6H), the variability of GAPDH in VOR after 24 h lead to an artificially significant fold decrease of IFI6 (0.80-fold change, P < 0.05) even when the GAPDH fold change was within the range of stability (1.56-fold change; 90% CI, 1.45 to 1.68). Similarly, in VOR after 6 h (Fig. 6F), the small fold increases of GAPDH and UBE2D2 (1.79- and 1.57-fold change, respectively) resulted in loss of significant upregulation of HIV ca-RNA.

A parallel GOI and RG fold change evaluation is suggested to prevent misleading results and interpretation, in particular, when small changes in GOI expression are expected.

More than one RG should be used on expected small GOI fold change.

As previously mentioned, a small observed GOI fold change could be an artificial result of small RG variations. The use of a single RG is often discouraged when small differences in GOI expression are expected, and only normalization on multiple RGs can provide accurate results (7) in such situations.

The geNorm algorithm indicates the optimal number of reference genes for data normalization through the pairwise variation (V) calculation between two sequential normalization factors containing an increasing number of genes (V_n/n+1). A V_n/n+1 value of <0.15 and an observed V downward trend were proposed as parameters for which the inclusion of an additional RG is not required (22). We analyzed data sets composed of three conditions, as previously explained, and found that normalization to two or three RGs was sufficient in the majority of analyses (Table 4).

TABLE 4.

Optimal number of RGs for normalization in data set composed of three conditions^a

Condition
	PBMC						CD4+ T cells						PBMC+CD4+ T cells
	6h		24h		6h+24h		6h		24h		6h+24h		6h		24h		6h+24h
DMSO, PHA, VOR	V2/3	0.081	V2/3	0.081	V2/3	0.112	V2/3	0.062	V2/3	0.093	V2/3	0.090	V2/3	0.082	V2/3	0.119	V2/3	0.103
	V3/4	0.060	V3/4	0.093	V3/4	0.090	V3/4	0.051	V3/4	0.081	V3/4	0.085	V3/4	0.069	V3/4	0.087	V3/4	0.089
	V4/5	0.067	V4/5	0.095	V4/5	0.117	V4/5	0.069	V4/5	0.113	V4/5	0.120	V4/5	0.106	V4/5	0.100	V4/5	0.122
	V5/6	0.118	V5/6	0.113	V5/6	0.094	V5/6	0.086	V5/6	0.136	V5/6	0.100	V5/6	0.102	V5/6	0.130	V5/6	0.111
DMSO, PHA, ROM	V2/3	0.077	V2/3	0.129	V2/3	0.123	V2/3	0.079	V2/3	0.150	V2/3	0.119	V2/3	0.089	V2/3	0.153	V2/3	0.121
	V3/4	0.072	V3/4	0.129	V3/4	0.132	V3/4	0.072	V3/4	0.146	V3/4	0.168	V3/4	0.088	V3/4	0.135	V3/4	0.167
	V4/5	0.071	V4/5	0.126	V4/5	0.106	V4/5	0.074	V4/5	0.161	V4/5	0.117	V4/5	0.105	V4/5	0.147	V4/5	0.119
	V5/6	0.132	V5/6	0.107	V5/6	0.107	V5/6	0.101	V5/6	0.156	V5/6	0.122	V5/6	0.115	V5/6	0.133	V5/6	0.112
DMSO, PHA, PEP005	V2/3	0.085	V2/3	0.101	V2/3	0.132	V2/3	0.092	V2/3	0.117	V2/3	0.111	V2/3	0.109	V2/3	0.118	V2/3	0.121
	V3/4	0.074	V3/4	0.074	V3/4	0.095	V3/4	0.065	V3/4	0.086	V3/4	0.097	V3/4	0.075	V3/4	0.086	V3/4	0.102
	V4/5	0.109	V4/5	0.088	V4/5	0.110	V4/5	0.107	V4/5	0.112	V4/5	0.109	V4/5	0.122	V4/5	0.101	V4/5	0.111
	V5/6	0.103	V5/6	0.111	V5/6	0.104	V5/6	0.097	V5/6	0.141	V5/6	0.120	V5/6	0.107	V5/6	0.131	V5/6	0.122
DMSO, PHA, AZD	V2/3	0.074	V2/3	0.089	V2/3	0.118	V2/3	0.057	V2/3	0.095	V2/3	0.093	V2/3	0.079	V2/3	0.104	V2/3	0.112
	V3/4	0.063	V3/4	0.100	V3/4	0.088	V3/4	0.053	V3/4	0.085	V3/4	0.085	V3/4	0.070	V3/4	0.098	V3/4	0.091
	V4/5	0.067	V4/5	0.099	V4/5	0.129	V4/5	0.076	V4/5	0.120	V4/5	0.130	V4/5	0.110	V4/5	0.108	V4/5	0.131
	V5/6	0.123	V5/6	0.136	V5/6	0.109	V5/6	0.097	V5/6	0.149	V5/6	0.111	V5/6	0.109	V5/6	0.146	V5/6	0.123
DMSO, PHA, IBET	V2/3	0.078	V2/3	0.093	V2/3	0.122	V2/3	0.064	V2/3	0.113	V2/3	0.088	V2/3	0.078	V2/3	0.115	V2/3	0.107
	V3/4	0.064	V3/4	0.075	V3/4	0.125	V3/4	0.080	V3/4	0.090	V3/4	0.124	V3/4	0.114	V3/4	0.090	V3/4	0.128
	V4/5	0.101	V4/5	0.089	V4/5	0.113	V4/5	0.085	V4/5	0.121	V4/5	0.116	V4/5	0.110	V4/5	0.106	V4/5	0.117
	V5/6	0.108	V5/6	0.119	V5/6	0.101	V5/6	0.080	V5/6	0.138	V5/6	0.104	V5/6	0.095	V5/6	0.134	V5/6	0.116
DMSO, PHA, STINGa	V2/3	0.087	V2/3	0.096	V2/3	0.124	V2/3	0.087	V2/3	0.114	V2/3	0.142	V2/3	0.094	V2/3	0.123	V2/3	0.124
	V3/4	0.067	V3/4	0.098	V3/4	0.120	V3/4	0.067	V3/4	0.086	V3/4	0.106	V3/4	0.103	V3/4	0.091	V3/4	0.120
	V4/5	0.079	V4/5	0.111	V4/5	0.121	V4/5	0.079	V4/5	0.125	V4/5	0.122	V4/5	0.109	V4/5	0.116	V4/5	0.121
	V5/6	0.107	V5/6	0.146	V5/6	0.132	V5/6	0.107	V5/6	0.168	V5/6	0.112	V5/6	0.093	V5/6	0.163	V5/6	0.132
DMSO, PMAi, VOR	V2/3	0.119	V2/3	0.116	V2/3	0.158	V2/3	0.099	V2/3	0.168	V2/3	0.153	V2/3	0.160	V2/3	0.145	V2/3	0.154
	V3/4	0.143	V3/4	0.098	V3/4	0.163	V3/4	0.118	V3/4	0.138	V3/4	0.176	V3/4	0.162	V3/4	0.119	V3/4	0.191
	V4/5	0.171	V4/5	0.122	V4/5	0.136	V4/5	0.221	V4/5	0.160	V4/5	0.166	V4/5	0.195	V4/5	0.139	V4/5	0.151
	V5/6	0.200	V5/6	0.110	V5/6	0.160	V5/6	0.325	V5/6	0.152	V5/6	0.260	V5/6	0.265	V5/6	0.135	V5/6	0.214
DMSO, PMAi, ROM	V2/3	0.120	V2/3	0.091	V2/3	0.158	V2/3	0.113	V2/3	0.189	V2/3	0.164	V2/3	0.157	V2/3	0.150	V2/3	0.160
	V3/4	0.147	V3/4	0.150	V3/4	0.148	V3/4	0.115	V3/4	0.164	V3/4	0.176	V3/4	0.159	V3/4	0.154	V3/4	0.181
	V4/5	0.186	V4/5	0.111	V4/5	0.149	V4/5	0.233	V4/5	0.157	V4/5	0.175	V4/5	0.208	V4/5	0.138	V4/5	0.162
	V5/6	0.205	V5/6	0.110	V5/6	0.161	V5/6	0.338	V5/6	0.159	V5/6	0.260	V5/6	0.274	V5/6	0.134	V5/6	0.214
DMSO, PMAi, PEP005	V2/3	0.149	V2/3	0.089	V2/3	0.169	V2/3	0.187	V2/3	0.151	V2/3	0.175	V2/3	0.175	V2/3	0.122	V2/3	0.171
	V3/4	0.143	V3/4	0.122	V3/4	0.172	V3/4	0.171	V3/4	0.167	V3/4	0.206	V3/4	0.190	V3/4	0.144	V3/4	0.191
	V4/5	0.152	V4/5	0.118	V4/5	0.123	V4/5	0.199	V4/5	0.159	V4/5	0.154	V4/5	0.175	V4/5	0.138	V4/5	0.149
	V5/6	0.185	V5/6	0.105	V5/6	0.160	V5/6	0.318	V5/6	0.153	V5/6	0.261	V5/6	0.254	V5/6	0.135	V5/6	0.214
DMSO, PMAi, AZD	V2/3	0.122	V2/3	0.124	V2/3	0.163	V2/3	0.109	V2/3	0.156	V2/3	0.147	V2/3	0.140	V2/3	0.147	V2/3	0.154
	V3/4	0.138	V3/4	0.098	V3/4	0.189	V3/4	0.101	V3/4	0.149	V3/4	0.192	V3/4	0.165	V3/4	0.120	V3/4	0.210
	V4/5	0.178	V4/5	0.132	V4/5	0.142	V4/5	0.231	V4/5	0.166	V4/5	0.174	V4/5	0.203	V4/5	0.147	V4/5	0.157
	V5/6	0.198	V5/6	0.133	V5/6	0.163	V5/6	0.330	V5/6	0.166	V5/6	0.265	V5/6	0.267	V5/6	0.152	V5/6	0.218
DMSO, PMAi, IBET	V2/3	0.117	V2/3	0.075	V2/3	0.161	V2/3	0.107	V2/3	0.145	V2/3	0.147	V2/3	0.142	V2/3	0.130	V2/3	0.153
	V3/4	0.137	V3/4	0.096	V3/4	0.172	V3/4	0.103	V3/4	0.137	V3/4	0.182	V3/4	0.161	V3/4	0.109	V3/4	0.198
	V4/5	0.166	V4/5	0.123	V4/5	0.131	V4/5	0.221	V4/5	0.168	V4/5	0.169	V4/5	0.192	V4/5	0.145	V4/5	0.150
	V5/6	0.176	V5/6	0.117	V5/6	0.160	V5/6	0.301	V5/6	0.155	V5/6	0.250	V5/6	0.241	V5/6	0.139	V5/6	0.208
DMSO, PMAi, STINGa	V2/3	0.133	V2/3	0.108	V2/3	0.161	V2/3	0.101	V2/3	0.135	V2/3	0.150	V2/3	0.117	V2/3	0.130	V2/3	0.154
	V3/4	0.135	V3/4	0.095	V3/4	0.197	V3/4	0.122	V3/4	0.141	V3/4	0.221	V3/4	0.177	V3/4	0.111	V3/4	0.218
	V4/5	0.161	V4/5	0.143	V4/5	0.135	V4/5	0.217	V4/5	0.173	V4/5	0.164	V4/5	0.187	V4/5	0.155	V4/5	0.158
	V5/6	0.184	V5/6	0.144	V5/6	0.161	V5/6	0.302	V5/6	0.185	V5/6	0.255	V5/6	0.245	V5/6	0.169	V5/6	0.211

^aPairwise variation of geNorm (V_n/n+1) in data set composed of three conditions (DMSO plus PHA/PMAi plus one LRA) from PBMCs, CD4⁺ T cells, or both, and after 6 h, after 24 h, or both. A V_n/n+1 value of <0.15 indicates that the inclusion of an additional RG is not required, together with the evaluation of the downward trend. V_n/n+1 chosen is highlighted in boldface font.

Minimal variation in RG expression could induce substantially different results for small GOI fold change which can be mitigated by incorporating multiple stable RGs in the analyses. Except for GAPDH (previously discussed), after 24 h of VOR exposure, all the tested RG fold changes were considered acceptable and minimal. Nevertheless, different IFI6 results were obtained when normalized to the single RGs or their combinations (Fig. 6G and H). In this case, only normalization to multiple RGs provided an accurate result: the absence of up- or downregulation of IFI6. Supporting this point, interferon 2a (IFN-2a) production, which upregulates the expression of IFI6, was confirmed to be absent in the supernatant of VOR-treated cells by quantitative enzyme-linked immunosorbent assay (ELISA) compared to elevated levels of production under STINGa conditions (data not shown). Of note, all the VOR induced differences in IFI6 expression observed were less than 2-fold, highlighting the concept that a small GOI fold change should be cautiously interpreted and, as previously suggested, substantiated with other methods of evaluation (29).

DISCUSSION

In the HIV shock-and-kill cure strategy, small molecules known as LRAs are used to force the reactivation of HIV provirus. The feasibility of this approach has been demonstrated in vitro and in vivo, although to date, no LRAs have led to a significant depletion of the HIV reservoir (1). HIV ca-RNA is one of the most widely used techniques to evaluate HIV RNA induction following LRA exposure and to assess the size of the inducible reservoir (2, 3). Additionally, our knowledge of the mechanisms of HIV latency could be increased by profiling host gene transcriptome modification after LRA treatment and evaluating its correlation with HIV reactivation (4–6).

In both host gene and HIV ca-RNA measurements, the data are normalized to some biological parameters, such as the number of cells, total RNA or DNA input, and exogenous (RNA spike) or endogenous (reference gene) normalizers. The final results are tabulated as a relative expression and the quality of the reported data is dependent upon the stability of the denominator used (or normalizer) among different experimental conditions (30). LRAs used in HIV shock-and-kill cure strategies are not specific for the HIV genome but induce HIV reactivation by targeting host factors (1), interfering with chromatin structure or with host signaling pathways. An impact on the reference gene used as a normalizer is inevitable, and some toxicity effects are expected. Thus, the normalizer should be cautiously selected and its stability carefully evaluated to avoid erroneous data interpretation.

In most HIV latency studies, qPCR data are normalized to total cells or the RNA input, with poor demonstrable evidence of accurate measurements, efficiency, and reproducibility. The number of cells is frequently quantified at the beginning of the experiment and it is assumed to be stable for the duration of the experiment and across LRA treatments. Subsequently, total RNA is extracted from an approximate cell input. Similarly, normalizing to the total RNA extracted does not take into consideration variation in the RNA expressed within the cell due to the effect of the LRAs on various cellular mechanisms. In this study, we demonstrated that LRA treatment can influence cell viability and total RNA recovery. Thus, such parameters as normalizer should be chosen only if accurately quantified, with no evident differences across treatment conditions. Even then, normalizing to the number of cells or total RNA input could be insufficient, as the various steps, from RNA extraction to qPCR, could be subjected to technical errors, differences in RNA extraction, and reverse transcription-PCR efficiency. Such assay variations could be accounted for only if an endogenous control (a reference gene) is employed (7).

RGs are defined as genes whose expression levels are not affected by the experimental conditions, including the type of cells, culture methods, time, treatments, and so on (29, 31). According to the MIQE (minimum information for publication of quantitative real-time PCR experiments) guidelines, authors must justify the number and choice of RGs (13). In many latency reversal studies, authors often choose classical RGs (such as GAPDH) without any provided validation process. In other cases, also incorrectly, authors choose RGs based on their validation in the same type of cells as of interest without considering the potential impact of the LRA treatment on the RGs. While a few studies have hinted at the influence of LRA on RG stability (6, 32, 33), careful evaluation in the HIV field is missing.

We carefully evaluated the expression of commonly used RGs following exposure to various LRAs with the aim of defining RGs that are better suited for latency reversal studies. We first evaluated the RG fold change expression in treated versus that in untreated samples, as previously suggested (19, 20). We proposed a fold change between 0.5 and 2 (called the stability zone) as an acceptable RG fold change and preliminary indication of stability. We compared this stability zone gauge to the previously suggested geNorm and BestKeeper criteria and demonstrated that our criteria were almost always more conservative. Overall, among the tested RGs, TBP, RPL27A, and UBE2D2 were the most stable, while GAPDH, HPRT1, and YWHAZ varied the most. Classical and widely used RGs, such as GAPDH, were highly influenced by some LRA treatments. Thus, we strongly discourage GAPDH usage, unless it is used collectively with other RGs (to confirm) and clearly shown as stable. While, in most cases, we observed that the stability of RGs decreased with longer exposure to LRAs, this was not the general trend across the board. For example, YWHAZ was downregulated after 6 h, and a partial reversion was observed after 24 h. Moreover, we observed different RG responsiveness following exposure to different LRAs that are within the same family. For instance, the histone deacetylase inhibitor (HDACi) romidepsin, but not vorinostat, highly downregulated YWHAZ after 24 h. This demonstrates that the stability under one condition (time points or treatment) cannot be generalized and should always be evaluated.

Although RG fold change (between two conditions) inside the stability zone can be a good preliminary indicator of stability, it was necessary to include additional metrics to identify the most stable RGs in the context of common HIV LRA experimental designs that include, at a minimum, 3 conditions (a vehicle control, a positive control, and the LRA[s] of interest). In our experimental setting, different tools (geNorm, NormFinder, BestKeeper, and the comparative ΔC_T method) resulted in similar RG rankings, allowing us to use the integrated RefFinder tool. As expected, different RGs or their combinations were appropriate for different conditions and time points (detailed results are provided in File S1A and B in the supplemental material), but overall, TBP, RPL27A, and UBE2D2 were the most stable RGs. The obtained results were compiled using rank aggregation (28). With this approach, a consensus list across all the stability ranking results was created, and TBP was identified as the most stable RG across all experiments. Conversely, GAPDH, HPRT1, and YWHAZ were always in the least stable ranking positions.

Finally, we demonstrated that normalizing on an unstable RG could alter the GOI expression results. Highly regulated RGs (such as GAPDH, YWHAZ, and HPRT1) dramatically distorted the IFI6 and HIV ca-RNA fold expression results under some LRA conditions. More consistent results were obtained by using one of the identified stable RGs (TBP, RPL27A, or UBE2D2) or by using a combination of two or three RGs, a good practice emphasized by several authors (13, 22, 24, 34). However, small RG fold changes could also lead to variation in GOI fold expression. Since variations in gene expression, in general, are not always avoidable due to biological and technical variability, selecting the most stable RGs greatly helps avoid misinterpretation of results. In this context, we highly recommended to evaluate small GOI fold changes in association with the RG fold change and to confirm the GOI fold change results by using multiple RGs. A parallel evaluation of GOI fold change and RG fold change could be particularly useful under highly modulating conditions, such as prolonged exposure to PHA and PMAi.

Our study provides the first careful evaluation of the stability of RGs after exposure to LRAs, widely applied in the HIV shock-and-kill cure strategy. The results of our study will provide a fundamental basis to improve the quality of both gene expression and HIV ca-RNA data. However, we emphasize that the RG results reported here cannot be extrapolated and applied to different experimental settings without (re)validation. For example, it should not be assumed that a validated RG will be stable in a different time point, type of cell, or subset or using different LRA concentrations. As continuously advised and required in the MIQE guidelines, blind RG selection is not acceptable (7, 13, 20, 34). Each author should (re)validate selected RGs in their own experimental design. Similarly, reviewers should consider RG selection a crucial aspect in qPCR analyses, and increased scrutiny should be applied in cases where RGs are not properly chosen or their stability not proven (13, 29, 35).

To facilitate RG selection and the validation process, we provide a flowchart with useful steps and criteria (Fig. 7). First, authors should research previously validated RGs suitable for the experimental conditions of interest (i.e., previously validated under the same exact conditions as those of interest). However, it is highly recommended that the stability of the chosen RGs is confirmed independently by the authors (13, 29). This could be easily performed through the fold change calculation between treated and untreated conditions (or, depending on the aim of the studies, between different time points or treatment concentrations) as demonstrated in this study. If the conditions are not exactly the same, it will be necessary to proceed with a revalidation of the selected RGs or a new validation study accordingly. A new preliminary study to detect the stable RGs must be performed when critical differences are included and be available at least to the reviewer. Although RG (re)validation should be performed in the same samples as that of interest, this is not always possible. In the cases of studies involving HIV latency reversal, low availability of cells and the low frequency of latently infected cells could pose a challenge. In those cases, at least a preliminary study from healthy donors should be performed using the same experimental conditions planned for the sample of interest. In addition to our suggestions, authors should follow the MIQE guidelines and include important information, such as detailed RNA extraction, cDNA and qPCR protocols, and primer information and efficiency (13).

FIG 7.

Flow chart for reference gene selection and (re)validation.

Our study is not without its limitations. We included only six RGs and we could potentially have unintentionally missed other undiscovered more stable RGs. Furthermore, the evaluation of other reference genes commonly used in the literature, including the β-actin gene, IPO8, and others, was not performed in our study. Additionally, other normalization strategies such as the use of exogenous RNA molecules (RNA spike) were omitted. This strategy was not included in our study because of its reported limitations (29) and its minimal usage in HIV studies. Lastly, although 18S RNA has been used and suggested as a normalizer in HIV ca-RNA studies (36), we excluded the 18S RNA from our RG selection because there are many difficulties associated with its usage. Due to its higher expression than HIV ca-RNA, the use of 18S as an RG will require cDNA dilution that may eventually lead to significant technical errors. Moreover, as for rRNA, 18S is transcribed by RNA polymerase I, and its stability may not correspond to that of the mRNA of interest, transcribed by RNA polymerase II (12, 14). Furthermore, it has demonstrably poor performances compared to those of other RGs in human T cells (16). Another limitation of this study is that the gene modulation changes measured reflect dynamics within the bulk population of cells. Various levels of c-Myc expression across different cell types can affect the levels of total RNA produced within a cell, with high c-Myc expression causing 2- to 3-times higher levels of total RNA production (37). The effect of this transcriptional amplification phenomenon (37) was not fully assessed in our study. Nevertheless, as we were able to detect several stable reference genes between treated and untreated conditions, this would suggest that, at least at the bulk population level, similar quantities of some RNAs are transcribed in cells.

In conclusion, qPCR is an excellent and widely used technology to measure host gene mRNA and, in HIV studies, the expression of the provirus. The reliability and reproducibility of the data start from the selection of appropriate normalizers in the experimental setting. The use of RGs as normalizers allows controls for the variations and technical errors in the extraction, reverse transcription, and qPCRs. RGs can be affected by the experimental conditions, and using a validated and stable RG is necessary to avoid misinterpretation of results. Our findings will be useful for the proper selection of RGs to obtain accurate and high-quality data from gene expression analyses.

MATERIALS AND METHODS

Cell isolation.

Peripheral blood mononuclear cells (PBMCs) from healthy donors were obtained by Ficoll gradient from whole blood obtained from the New York Blood Center (Long Island City, NY, USA). CD4⁺ T cells were isolated from PBMCs by negative selection (EasySep human CD4⁺ T cell enrichment kit; StemCell Technologies). After isolation, the cells were cultured overnight at 10 × 10⁶ cells/ml in Iscove’s modified Dulbecco’s medium (IMDM) without phenol red, supplemented with 10% heat-inactivated charcoal-stripped fetal bovine serum (FBS), 100 U/ml penicillin, 100 μg/ml streptomycin, and 20 U/ml of interleukin-2 (IL-2) (37°C, 5% CO₂). IMDM without phenol red and with charcoal-stripped FBS was used because of the necessity to eliminate the presence of sex hormones for a different study conducted in parallel. Of note, we did not observe any difference in viability and RG expression in cultures containing charcoal-stripped FBS compared to those with regular FBS.

Resting CD4⁺ T (rCD4⁺) cells from ART-treated stably suppressed HIV-infected participants were isolated as previously described (38). Cells were maintained in culture overnight in IMDM supplemented with 10% heat-inactivated FBS, 100 U/ml penicillin, 100 μg/ml streptomycin, and 4 μM abacavir (Selleckchem) and 1 μM of raltegravir to prevent the spread of spontaneously reactivated virus.

Latency-reversing agent treatment conditions.

PBMCs or CD4⁺ T cells were counted and cultured in 24-well culture plates at 1 × 10⁶ cells/ml in 1 ml final volume of complete medium and one of the following latency-reversing agents: PHA/IL-2 (2 μg/ml plus 100 U/ml), PMA plus ionomycin (16 nM plus 0.5 μM), VOR (340 nM), ROM (20 nM), PEP005 (12 nM), AZD-5582 (100 nM), iBET-151 (1,000 nM), STINGa (39) (100 nM), or complete medium plus 0.1% DMSO (37°C, 5% CO₂). The media for all conditions (except PHA/IL-2) were also supplemented with 5 U/ml of IL-2. All the LRA concentrations used in this study are values reported in the literature (32, 40, 41). The final concentration of DMSO for all conditions was approximately 0.01%, except for STINGa, where the final concentration was 0.1%. For AZD and STINGa stimulations, the medium containing the drugs was replaced with fresh medium without LRAs after 3 h and 6 h, respectively. For each condition, in each time point, 3 replicate wells were used. rCD4⁺ cells were treated for 6 h with PHA or VOR as previously reported (42).

alamarBlue cell viability assay.

The alamarBlue cell viability assay (G-Biosciences) was performed in parallel for all treatment conditions following the manufacturer’s instructions. Each condition was cultured immediately in a 96-well culture plate at a final volume of 100 μl. A minimum of 2 technical replicates per condition were performed. AZD and STINGa conditions were plated after the medium replacement. alamarBlue reagent was added at 10% of the volume in the well immediately after 6 h or after 24 h. The cells were incubated with the alamarBlue reagent at 37°C, 5% CO₂ in the dark, and fluorescence was detected after 6 h with an excitation wavelength at 530 nm and emission wavelength at 590 nm using a SpectraMax M3 plate reader (Molecular Devices). The percentage difference between treated cells and untreated controls were calculated by dividing the average of the relative fluorescence unit (RFU) value for the experimental condition by that for the DMSO control condition. The STINGa and AZD conditions were compared to a distinct DMSO control that was also subjected to a medium replacement in order to limit bias due to loss of cells in the medium replacement process.

RNA extraction, quantification, and quality control.

After 6 h or 24 h, PBMCs and CD4⁺ T cells were collected, washed with PBS, and pelleted by centrifugation at 4°C. The pellets were immediately stored at −80°C, and total RNA was extracted within 3 days using the KingFisher Flex purification system (Thermo Fisher), adapting the NucleoMag RNA (TaKaRa) protocol for PBMC and CD4⁺ T samples, and the MagJet RNA kit (Thermo Fisher) protocol for rCD4⁺ cells. Total RNA concentration was measured from duplicates using a NanoDrop 8000 spectrophotometer (Thermo Fisher), and purity was estimated by the 260-nm/280-nm absorbance ratio.

cDNA synthesis and quantitative real-time PCR.

cDNA was synthesized using the Maxima first strand cDNA synthesis kit for RT-qPCR, with dsDNase (to ensure absence of DNA contamination) (Thermo Fisher) according to the manufacturer’s instructions. A total of 100 ng of RNA was reverse transcribed, and cDNA reaction mixtures were diluted in RNase-free water at a final concentration of 2.5 ng/μl corresponding total RNA. Quantitative real-time PCRs (qPCRs) were performed in 10-μl final volumes using the QuantiTect Multiplex PCR NoROX master mix (Qiagen), previously mixed with ROX passive reference dye (Bio-Rad), and primer/probes at a final concentration of 900 nM and 250 nM, respectively. We used 5 ng corresponding total RNA input for each reaction. qPCRs were conducted on the CFX384 Touch real-time PCR detection system (Bio-Rad), with sample maximization strategy, with the following protocol: 50°C for 2 min, 95°C for 15 min, followed by 42 cycles of 94°C for 1 min and 60°C for 1 min. C_q values were determined using the regression analysis in CFX Manager Software 3.1. Two technical replicates were performed for each sample. Each qPCR run included at least one no-template control (NTC), one no-reverse transcriptase control (NRT), and two no-amplification controls (NACs). For each donor and each condition, final C_q values (see File S1D in the supplemental material) are expressed as the means from two technical replicates, followed by the mean from three experimental replicates. Primers/probes were obtained from IDT (Integrated DNA Technologies), and detailed information is shown in Table 5.

TABLE 5.

Primers and probes used for gene expression qPCR^a

Assay	Species or virus	Primer/probe	Sequence (5′→3′)	RefSeq no.	Detect all variants?	Exon location	IDT assay ID	Source	Amplicon size (bp)	Efficiency
										%	r2
TBP	Homo sapiens	F	CAGTGAATCTTGGTTGTAAACTTGA	NM_003194	Yes	4–5	Hs.PT.58. 20792004	IDT	113	99.49	0.9975
		R	TCGTGGCTCTCTTATCCTCAT
		P	CGCAGCAAACCGCTTGGGATTAT
RPL27A	Homo sapiens	F	AACATGCCATCCAGACTGAG	NM_000990	Yes	1–4	Hs.PT.58. 38463594	IDT	148	98.28	0.9973
		R	GGTATTTGTCGAAGTTGATCCG
		P	CAGACCACCAGCATTACCGCG
UBE2D2	Homo sapiens	F	GTACTCTTGTCCATCTGTTCTCTG	NM_181838	Yes	6–8	Hs.PT.58. 622887	IDT	120	99.89	0.9989
		R	CCATTCCCGAGCTATTCTGTT
		P	CCGAGCAATCTCAGGCACTAAAGGA
HPRT1	Homo sapiens	F	TTGTTGTAGGATATGCCCTTGA	NM_000194	Yes	8–9	Hs.PT.58v. 45621572	IDT	149	100.71	0.9959
		R	GCGATGTCAATAGGACTCCAG
		P	AGCCTAAGATGAGAGTTCAAGTTGAGTTTGG
YWHAZ	Homo sapiens	F	TCCCTCAAACCTTGCTTCTAG	NM_003406	Yes	1–2	Hs.PT.39a. 22214858	IDT	135	98.00	0.9988
		R	TCAGTTACAGACTTCATGCAGG
		P	AGTTTGGCCTTCTGAACCAGCTCATT
GAPDH	Homo sapiens	F	ACATCGCTCAGACACCATG	NM_002046	Yes	2–3	Hs.PT.39a. 22214836	IDT	143	100.72	0.9943
		R	TGTAGTTGAGGTCAATGAAGGG
		P	AAGGTCGGAGTCAACGGATTTGGTC
IFI6	Homo sapiens	F	CTGCTGTGCCCATCTATCAG	NM_002038	Yes	1–2	Hs.PT.58. 4407609	IDT	129	99.41	0.9970
		R	GTAGCACAAGAAAAGCGATACC
		P	CCAAGGTCTAGTGACGGAGCCC
HIV GAG	HIV-1	F	TACTGACGCTCTCGCACC	K03455.1	NA	NA	NA	IDT	127	102.20	0.9890
		R	TCTCGACGCAGGACTCG
		P	CTCTCTCCTTCTAGCCTCCGCTAGT

^aDetailed information on primers and probes used in this study. F, forward primer; R, reverse primer; P, probe; ID identifier; NA, not available.

Efficiency of all primer/probe sets was evaluated, in at least 5 independent dilutions and runs, from pooled cDNA samples using 10-fold and 2-fold serial dilutions (1:1, 1:10, 1:100, 1:200, 1:1,000, 1:2,000, and 1:10,000), including the experiment condition tested and expanded at least 20% (43). Three technical replicates were performed for each dilution point. Efficiency was estimated from the slope of the linear relationship between C_q and the logarithm of initial template concentration. Efficiency was calculated as $-1+{10}^{-1/\text{slope}}$ and expressed as percentual. Overall, the primer efficiencies were between 95 and 105% with an r² of >0.990.

Ca-HIV qPCR was performed, as described above, using previously published primers and probe for group M HIV-1 (44) at final concentrations of 400 nM and 200 nM, respectively, and 12 ng corresponding total RNA for each reaction in triplicates.

Fold change reference gene analysis.

Six widely used RGs were selected: GAPDH, TBP, RPL27A, UBE2D2, YWHAZ, and HPRT1. For each donor, the fold change expression was assessed by comparing each LRA treatment to the untreated control (DMSO) at the same time point, according to the 2^−ΔC_T method (19, 20). Estimated means and their corresponding Wald 90% CIs were calculated from the log₂-transformed normalized values (34). The estimated mean fold change and corresponding 90% CI were then computed using antilog transformations, with results displayed in Tables 1 and 2. The endpoints of the 90% CIs were compared to the fold change range of 0.5 to 2 (stability zone). RGs were considered stable when both the 90% CI upper and low limits were inside the stability range and highly unstable when both the limits were outside the stability range. This approach for establishing equivalency is analogous to conducting two one-sided tests, each with a type I error rate of 0.05 (21). RGs with one CI limit outside the 0.5 to 2 range were considered modestly regulated.

Reference gene stability determination.

The evaluation of RG stability was performed in several data sets composed of the C_q values of the untreated control condition (DMSO), either PHA or PMAi condition (commonly used as positive control in HIV LRA experiments), and one or more LRA condition. The type of cells and the time points were also taken into consideration in the data set creation. Data from PBMCs and CD4⁺ T cells at 6 h and 24 h were analyzed separately or together. All the analyses were carried out using the integrated tool RefFinder (27), which combines, in a final rank, the results of four common algorithms: geNorm, NormFinder, BestKeeper, and the comparative ΔC_T method. The RefFinder rank is based on the geometric mean of each ranking across the four algorithms with the lowest value corresponding to the most stable RG. Additionally, data sets composed of three conditions (DMSO, PHA or PMAi, and one LRA) were evaluated using each of the 4 algorithms independently to assess the concordance among the algorithms. The rank stability of each program were created using, from the smallest to the highest, the M stability value after stepwise exclusion (the two most stable genes were reported with rank “1”) for geNorm (version 3.4) (22), the stability value for NormFinder (v0.953) (24), the stability value called “standard deviation value” (SD) for BestKeeper (version 1) (23), and the mean standard deviation value for the comparative ΔC_T method (25). Based on the analyses performed with RefFinder or the individual algorithms, ranking results are summarized and expressed as frequency of ranking in the first position, in the top two positions, or in the top three positions. Finally, the M stability values calculated by geNorm and the SD calculated by BestKeeper were considered parameters of stability when <1.5 and <1, respectively. The pairwise variation V (V_n/n+1) calculated in geNorm was used to determinate the optimal number of RGs needed.

Rank aggregation analysis.

The rank aggregation R package, called RankAggreg, was used to determine a consensus order of the RGs across all the RefFinder ranking results and their subgroups. Ties in the RefFinder rankings were broken by fitting linear mixed-effect models using the lmer function in R version 4.0.2. The C_q data were linearized by applying the transformation $\mathit{linCq}={10}^8\times 2^{-\mathit{Cq}}$. For each tied gene, linCq was regressed on the categorical condition variable, modeled as a fixed effect, and the donor identification variable, modeled as a random effect. The P value associated with the F test for the condition fixed effect was stored. Ties in the rankings were broken by comparing these P values from their associated mixed effect models, and the gene with the larger P value was given the lower ranking (indicating more stability). Detailed new ranking results are available in File S1B. After all ties were broken, rank aggregation was conducted with the RankAggreg function in R based on the cross-entropy (CE) Monte Carlo algorithm with the Spearman footrule distance function (28). This method uses a stochastic search algorithm to minimize the differences among the ranks across multiple sets of rankings. Importance weights were applied, equal to the square root of the number of experiments associated with each ranking. This gave more weight to rankings based on larger numbers of experiments than those based on fewer experiments when computing the aggregated ranks. Rank aggregation was conducted overall (for all 1,134 RefFinder ranking results) as well as for the following ranking RefFinder subsets (based on the RefFinder data set composition): PBMCs only, CD4⁺ T cells only, PBMC and CD4⁺ T cells, PHA, PMAi, 6 h only, 24 h only, and 6 h plus 24 h. As a sensitivity analysis, the tuning parameters for the CE method were adjusted, and the genetic algorithm (GA) was implemented with various values of the tuning parameters. Aggregated ranks were consistent regardless of the method (CE or GA) or tuning parameter values for the overall rankings as well as each of the eight subsets.

Analyses of genes of interest expression profiles.

To evaluate the impact of applying stable or unstable reference genes as normalizers, the IFI6 gene fold change expression was evaluated in PBMCs and CD4⁺ T cells after 24 h of PHA and VOR exposures and in PBMCs after 6 h and 24 h of STINGa exposure. Cell isolation, latency reversal treatment, cDNA synthesis, qPCR, and RG fold change evaluation were performed as previously mentioned. Gene expressions were analyzed using the comparative ΔC_T method (19). Statistical analyses were performed on the log-transformed relative normalized expression, and expression levels are reported as the geometric mean ± standard error of mean after a linear transformation, as indicated in reference 34. The Wilcoxon test was used to determinate statistical significance based on a type I error rate of 0.05. No corrections were made for multiple comparisons.

Quantification and statistical analyses.

Statistical analyses of gene expression data were conducted using GraphPad Prism 8 (version 8.4.2). P values were determined using the nonparametric Wilcoxon test. R version 4.0.2 was used to perform rank aggregation analysis.