Blood transcriptomic signatures for symptomatic tuberculosis in an African multicohort study

Graphical abstract

Overview of the study. We performed a nested case–control study in symptomatic patients from six African countries to assess the performance of 20 blood transcriptomic tuberculosis (TB) signatures as triage tests for TB.

Abstract

Background

Multiple host blood transcriptional signatures have been developed as non-sputum triage tests for tuberculosis (TB). We aimed to compare the diagnostic performance of 20 blood transcriptomic TB signatures for differentiating between symptomatic patients who have TB versus other respiratory diseases (ORD).

Methods

As part of a nested case–control study, individuals presenting with respiratory symptoms at primary healthcare clinics in Ethiopia, Malawi, Namibia, Uganda, South Africa and The Gambia were enrolled. TB was diagnosed based on clinical, microbiological and radiological findings. Transcriptomic signatures were measured in whole blood using microfluidic real-time quantitative PCR. Diagnostic performance was benchmarked against the World Health Organization Target Product Profile (TPP) for a non-sputum TB triage test.

Results

Among 579 participants, 158 had definite, microbiologically confirmed TB, 32 had probable TB, while 389 participants had ORD. Nine signatures differentiated between ORD and TB with equivalent performance (Satproedprai7: area under the curve 0.83 (95% CI 0.79–0.87); Jacobsen3: 0.83 (95% CI 0.79–0.86); Suliman2: 0.82 (95% CI 0.78–0.86); Roe1: 0.82 (95% CI 0.78–0.86); Kaforou22: 0.82 (95% CI 0.78–0.86); Sambarey10: 0.81 (95% CI 0.77–0.85); Duffy9: 0.81 (95% CI 0.76–0.86); Gliddon3: 0.8 (95% CI 0.75–0.85); Suliman4 0.79 (95% CI 0.75–0.84)). Benchmarked against a 90% sensitivity, these signatures achieved specificities between 44% (95% CI 38–49%) and 54% (95% CI 49–59%), not meeting the TPP criteria. Signature scores significantly varied by HIV status and country. In country-specific analyses, several signatures, such as Satproedprai7 and Penn-Nicholson6, met the minimal TPP criteria for a triage test in Ethiopia, Malawi and South Africa.

Conclusion

No signatures met the TPP criteria in a pooled analysis of all countries, but several signatures met the minimum criteria for a non-sputum TB triage test in some countries.

Shareable abstract

In a nested case–control study of blood transcriptomic TB signatures in symptomatic African patients, no signatures met benchmark criteria for a non-sputum TB triage test when all countries were pooled, but several met the minimum criteria in some countries https://bit.ly/4bawFdZ

Introduction

Tuberculosis (TB) is among the top 13 causes of death globally and the leading cause of death from a single infectious agent [1]. In 2021, an estimated 10.6 million people had TB and 1.6 million people died. The African continent accounted for ∼25% of this global burden and almost a third of global deaths due to TB. Many people with infectious TB remain undiagnosed and cannot access care and treatment. Improvements in identification and diagnosis of individuals with TB disease are urgently needed. However, current strategies to control the global TB epidemic largely rely on those with symptoms to self-present to clinics or other facilities for investigation and care.

Routine diagnosis of TB is based on identification of Mycobacterium tuberculosis complex (MTBC) species in expectorated sputum, by sputum smear microscopy, liquid culture or nucleic acid amplification tests (NAATs). However, a considerable proportion of individuals, and especially young children and those with immunocompromise, cannot expectorate sputum [2]. The diagnostic accuracy of NAATs, e.g. Xpert MTB/RIF, decreases considerably in people living with HIV, children with paucibacillary TB and individuals with smear-negative disease [3]. In addition, MTBC load in sputum can be highly variable. It is estimated that approximately 4.2 million people with TB did not reach healthcare facilities or receive a diagnosis and consequently remained untreated [1]. It is critical that better tests, ideally not based on sputum, be developed to broaden diagnosis of pulmonary TB in all patients. Accessibility at the point of care (POC) would facilitate more rapid decision making and TB treatment provision and expand case-finding strategies to include TB screening in communities.

Host biomarkers hold promise as non-sputum-based diagnostics with fast turnaround times that could be deployed as POC triage tests to either rule in those with evidence of TB or rule out those that do not require further investigation [4]. A large number of blood transcriptomic signatures, based on mRNA transcripts that are modulated in those with TB, have been developed as potential tests for TB (reviewed in [5, 6]). A frequent strategy to discover such host biomarkers is to profile blood transcripts in a case–control cohort of patients with microbiologically confirmed TB and healthy control participants to identify transcripts that best discriminate the two groups. This strategy has led to the development of dozens of transcriptomic signatures that have good diagnostic performance in independent validation cohorts [7–10]. Most signatures include common transcripts, which largely map to interferon-stimulated gene (ISG) pathways, illustrating that these signatures capture common inflammatory processes characteristic of TB. This underpins confidence that transcriptomic TB signatures developed by different investigators in diverse settings reproducibly detect evidence of TB disease in peripheral blood. The feasibility of transforming a blood transcriptomic signature into a POC test has been demonstrated by the Xpert MTB Host Response (Xpert-MTB-HR) triage test [11]. This test, developed by Sweeney et al. [12], evaluates a three-transcript blood transcriptomic signature which performs well even when using capillary blood obtained through fingerprick [13].

A key limitation in many studies that have developed or validated diagnostic or triage test performance of transcriptomic signatures, and can influence diagnostic accuracy, is that case–control studies have been small, based on a single geographic setting and/or not represented a clinically relevant population [4]. In most routine healthcare settings, TB management requires a triage test that can be used to determine who, among those presenting for healthcare with symptoms suggestive of TB, should be referred for intensive investigation. Such individuals may have TB or other infectious or non-infectious respiratory diseases, the appropriate management and treatment of which is dependent on a definitive diagnosis. Respiratory diseases other than TB, such as viral infections, may also induce inflammatory responses that modulate blood transcriptomic signatures. It thus stands to reason that analysis of the diagnostic performance of transcriptomic signatures should be assessed in clinically relevant cohorts of symptomatic individuals seeking care, ideally from diverse geographic and epidemiological settings [3, 4].

Here, we sought to determine the performance of 20 previously published, parsimonious blood transcriptomic signatures for diagnosing TB in patients who presented for care at clinics in six African countries with presumptive TB. Gene expression was measured by optimised, microfluidic real-time quantitative PCR (RT-qPCR). For each signature the performance for differentiating between TB and other respiratory diseases (ORD) was benchmarked against the Target Product Profile (TPP) criteria for a non-sputum TB triage test, developed by the World Health Organization (WHO) [14]. The use-case for symptomatic patients who seek care would be a test that allows triage of test-positive individuals for TB confirmation and, if confirmed, chemotherapy. Such a triage test must therefore have excellent sensitivity to ensure that those with TB have the best opportunity to receive the appropriate diagnosis and treatment. Under this scenario, a triage test with a sensitivity of at least 90% should achieve a specificity of at least 70% to meet the TPP criteria.

Methods

Study design

We conducted a nested case–control study that included participants from two previously completed, multisite phase III diagnostic cohorts: the Evaluation of Host Biomarker-based Point-of-care Tests for Targeted Screening for Active TB (ScreenTB) [15] and African European Tuberculosis Consortium (AE-TBC) [16–18] studies (figure 1). Common eligibility criteria were used to enrol adults (18–70 years) with or without HIV, seeking care at primary healthcare centres in Ethiopia, Malawi (AE-TBC only), Namibia, South Africa, Uganda and The Gambia. Individuals presenting with cough for >2 weeks, and at least one other feature suggestive of TB disease (close contact with a TB patient, fever, malaise, weight loss, night sweats, haemoptysis, chest pain or loss of appetite), were prospectively recruited. Patients were excluded if they were on TB treatment or received isoniazid prophylaxis in the last 90 days, reported quinolone or aminoglycoside antibiotic use in the last 60 days, were pregnant or breastfeeding, had a haemoglobin <9 g·dL⁻¹, were without permanent residence or had not been residing in the study area for >3 months.

FIGURE 1.

Flow diagram of participant enrolment, allocation and analyses. Participants in the research study that were previously enrolled into either the ScreenTB [15] or AE-TBC [16–18] studies. TB: tuberculosis; ORD: other respiratory diseases; QC: quality control.

At enrolment, participants provided sputum samples for M. tuberculosis culture, smear microscopy and Xpert MTB/RIF (ScreenTB only), underwent chest radiography and provided venous blood in PAXgene blood RNA tubes (Qiagen, Hilden, Germany). Harmonised protocols were implemented across study sites for collection and cryopreservation of whole-blood samples in the PAXgene tubes. Participants were required to attend follow-up visits from week 2 up to week 24 for further sample collection and to assess their response to treatment.

An outcome classification committee, comprised of clinician investigators from the participating sites, classified each participant as having either definite TB (microbiologically confirmed), probable TB, possible TB or no TB (ORD controls). Case definitions were based on a composite reference standard of microbiological, clinical and radiological findings (supplementary table S1). Microbiological confirmation was by TB sputum culture, Xpert MTB/RIF (ScreenTB only) or smear microscopy, while clinical diagnosis was based on chest radiography results, and clinical response to antibiotics or TB treatment, if prescribed. Participants with possible TB were not analysed due to insufficient evidence of TB disease and uncertainty about which group to allocate them to.

ORD controls with no evidence of TB were selected at a 2:1 ratio to participants with definite or probable TB (cases). We used a propensity score method to verify that cases and controls included in this substudy were matched (supplementary methods). Logistic regression on TB status was performed using study site, HIV, sex and age.

Ethics statement

Study protocols for the ScreenTB and AE-TBC studies were approved by the Health Research Ethics Committee (HREC) of Stellenbosch University (Cape Town, South Africa), and at all participating sites. All participants provided written, informed consent in accordance with the Declaration of Helsinki. The transcriptomic signatures substudy protocol was reviewed and approved by the HREC of the University of Cape Town (Cape Town, South Africa) (589/2019).

Transcriptomic signatures

Diagnostic, prognostic and treatment response monitoring signatures were considered for inclusion. Transcriptomic signatures were selected (table 1 and supplementary methods) based on diagnostic performance in recently published systematic reviews and head-to-head comparisons of published RNA signatures [19–21], a systemic review of transcriptomic signatures by our group [5] and consultation with experts in the field.

TABLE 1.

Characteristics of whole-blood transcriptomic signatures^# evaluated as possible biomarkers for point-of-care diagnostic development

Signature name	Signature model	Discovery population	Discovery country	Intended application
da Costa3 [29]	Random forest	Adults	Brazil	Diagnostic; TB versus ORD
de Araujo1 [30]	Standardised expression of NPC2	Adults	Brazil	Diagnostic; TB versus LTBI and HC
Duffy9 (10)¶ [7]	Random forest	Adults	Publicly available datasets from South Africa and Malawi [31, 32]	Diagnostic; TB versus LTBI and ORD
Francisco2 [33]	Random forest	Adults	China	Diagnostic; TB versus ORD and HC
Gjøen7 [34]	LASSO regression	Children	India	Diagnostic; TB versus ORD
Gliddon3 [35]	Disease risk score	Adults	South Africa, Malawi	Diagnostic; TB versus LTBI
Gliddon4 [35]	Disease risk score	Adults	South Africa, Malawi	Diagnostic; TB versus ORD
Jacobsen3 [36]	Linear discriminant analysis	Adults	Germany	Diagnostic; TB versus HC
Kaforou22 (27)+ [31]	Disease risk score	Adults	South Africa, Malawi	Diagnostic; TB versus LTBI and ORD
Maertzdorf4 [8]	Pair ratio algorithm adopted from Suliman et al. [37]	Adults	India	Diagnostic; TB versus LTBI and HC
Penn-Nicholson6 [9]	Pair ratio algorithm	Adolescents	South Africa	Prognostic; TB progressors versus non-progressors
Rajan5 [38]	Unsigned sums	Adults	Uganda	Diagnostic; HIV+ TB versus HIV+ HC
Roe1 [39]	Standardised expression of BATF2	Adults	UK	Diagnostic; TB versus HC and TB
Roe3 [40]	SVMs (linear kernel)	Adults	UK	Prognostic; TB progressors versus non-progressors
Sambarey10 [41]	Linear discriminant analysis	Adults	India	Diagnostic; TB versus LTBI and ORD
Satproedprai7 [10]	LASSO regression	Adults	Thailand	Diagnostic; TB versus HC and previous TB patients
Suliman2 [37]	Pair ratio algorithm	Adults	The Gambia, South Africa, Ethiopia	Prognostic; TB progressors versus non-progressors
Suliman4 [37]	Pair ratio algorithm	Adults	The Gambia, South Africa	Prognostic; TB progressors versus non-progressors
Sweeney3 [12]	TB score (GBP5+DUSP3)/2–KLF2	Adults	Meta-analysis (France, Malawi, South Africa, UK, USA)	Diagnostic; TB versus HC, LTBI and ORD
Thompson5 [42]	Pair ratio approach	Adults	South Africa	Monitoring response to TB treatment

TB: tuberculosis; ORD: other respiratory diseases; LTBI: latent tuberculosis infection; HC: healthy controls; LASSO: least absolute shrinkage and selection operator; SVM: support vector machine. ^#: signature names combine the surname of the first author of the original publication and the number of constituent transcripts used in our real-time quantitative PCR analyses. ^¶: Duffy9 originally had 10 transcripts: one primer–probe assay representing the transcript CERKL1 failed during panel optimisation and was omitted. Signature scores were calculated based on the model described in the corresponding publications. ⁺: Kaforou22 had 27 transcripts in the original signature. Three transcripts, C1QC, GBP6 and C1QB, were excluded from analysis due to high primer–probe assays failure rates, while primer–probe assays for two FCGR1B transcripts in the original signature could not be identified since the microarray probe was a duplicate of the same FCGR1B transcript. LOC728744 was discontinued and had all information withdrawn from both the NCBI and RefSeq databases.

Since we aimed to compare multiple signatures using a microfluidic RT-qPCR platform that can measure a maximum of 96 mRNA transcripts simultaneously, another key aspect was that each signature should incorporate a small number of transcripts. Other aspects included the ability to reconstruct the signatures based on availability of published gene targets for RT-qPCR. Signatures discovered using RNA sequencing or microarray platforms were reparameterised for application in RT-qPCR (supplementary methods).

Measurement of signatures

Blood signatures were measured as previously described [22, 23]. Whole-blood samples from all participants were thawed once to perform RNA extraction and processed in the same manner using standardised protocols. RNA was extracted from cryopreserved PAXgene blood RNA tubes using the Maxwell HT simplyRNA Custom Kit (Promega, Madison, WI, USA) and cDNA synthesis performed using EpiScript reverse transcriptase (Lucigen, Hoddesdon, UK), in a semiautomated manner in 96-well plate format on the Freedom EVO 150 robotics platform (Tecan, Männedorf, Switzerland). Non-template (water) and no reverse transcriptase controls were included in all runs. Following cDNA synthesis, 93 transcripts of interest and three reference (“housekeeping”) genes were pre-amplified using a pool of TaqMan primer–probe assays (supplementary table S2). Measurement of transcript expression was performed by RT-qPCR using 96.96 microfluidic gene expression chips (Standard BioTools, South San Francisco, CA, USA) on the Biomark HD system (Standard BioTools), which multiplexes 96 samples with the 96 TaqMan primer–probe assays (supplementary table S2). An internal positive control sample was included on all chips to monitor batch variability. Gene expression data quality control, including evaluation of control samples, and calculation of signature scores was performed using a locked-down R script (supplementary methods). Signature score calculations were based on the machine learning models and statistical methods described in the original publications, unless otherwise stated (table 1). Batch correction was performed to correct for differences in signature score distribution between Fluidigm chip runs using a quantile regression method with the adjust_batch function in the batchtma R package [24] (R package version 0.1.6 2021). This method unifies lower quantiles (25th percentile) and ranges between the lower and upper quantile (75th percentile) between batches, allowing for differences in both parameters due to confounders. Thereafter, signature scores were “winsorized” (clipped) to account for outliers (Winsorize function; DescTools R package [25]; R package version 0.99.49 2019) log transformed to normalise score distribution (log function; base R package), converted to z-scores (scale function; base R package) and compared using the cumulative distribution function (pnorm function; stats R package).

Blinding

The ScreenTB and AE-TBC clinical teams determined TB end-point status prior to measurement of signature scores. The laboratory teams were blinded to the TB status of participants during sample processing and signature score generation. Clinical and demographic data (including TB disease status) were shared by collaborators only after all laboratory work was completed and signature scores had been calculated. Statistical analyses were agreed upon and signed off before any data analysis officially began.

Statistical analysis

Sample size was determined based on the availability of PAXgene RNA samples from TB cases and controls enrolled in the ScreenTB and AE-TBC cohorts. All statistical analyses were carried out in R version 4.1.3 (www.r-project.org). A p-value or, to adjust for multiple testing, adjusted q-value (Benjamini–Hochberg q-value) of <0.05 was considered significant.

The primary diagnostic analyses were performed on individuals classified as having either definite or probable TB, compared to ORD controls. The pROC package was used to calculate the area under the receiver operating characteristic (ROC) curve (AUC) [26]. The DeLong method was used to calculate 95% confidence intervals and compare AUCs of each candidate signature in a pairwise approach [27]. The false discovery rate was controlled using the Benjamini–Hochberg method to calculate q-values [28].

Positive and negative predictive values, sensitivity and specificity for each candidate signature were calculated using routine formulae and benchmarked against the minimal (sensitivity 90% and specificity 70%) and optimal (sensitivity 80% and specificity 95%) WHO TPP criteria for a non-sputum TB triage test [14]. The binomial Wilson method was used to calculate 95% confidence intervals for all performance metrics.

The Wilcoxon rank-sum test was used to test for differences in signature scores between groups. Multivariable linear regression was performed to determine the effect of participant clinical and demographic characteristics at enrolment on signature scores. Missing data points, including signature scores due to failed primer–probe reactions, were assumed missing at random and excluded from the multivariable regression (i.e. complete case analysis only) and pairwise signature comparisons. Spearman's rank correlation coefficient was used to test association between signatures.

Results

Selection and matching of study groups

Among 2130 participants enrolled into the ScreenTB and AE-TBC studies, 425 participants with definite TB, 94 with probable TB and 1128 with ORD were eligible for inclusion in this substudy, of which 158 with definite TB, 32 with probable TB and 389 ORD controls were randomised to inclusion (figure 1). Transcriptomic signature scores were obtained from 150 definite TB cases, 30 probable TB cases and 361 ORD controls. We compared demographic and clinical variables among TB cases and ORD controls (table 2). No differences in sex (p=0.17), age (p=0.28), CD4 count (p=0.28), proportion on antiretroviral therapy (p=0.65), history of prior TB (p=0.37) or smoking (p=0.30) were observed. The distribution of cases and controls across countries was different (p=0.007). Definite TB cases had a lower body mass index (median 18.5 versus 19.4 versus 20.8 kg·m⁻²; p<0.001), were more likely to be interferon-γ release assay (IGRA) positive (84.3% versus 45.5 versus 42.7%; p<0.001), present with haemoptysis (p=0.042) and report subjective weight loss (p<0.001) than probable TB cases or ORD controls, respectively.

TABLE 2.

Participant characteristics

	Definite TB (n=150)	Probable TB (n=30)	ORD (n=361)	p-value#
Male	99 (66.0)	15 (50.0)	214 (59.3)	0.17
Age, years	32.0 (25.0–41.8)	34.5 (29.0–44.0)	32.0 (26.0–41.0)	0.28
Previous TB	32 (21.3)	5 (16.7)	58 (16.2)	0.37
Missing, n	0	0	2
Smoking history				0.30
Current smoker	38 (27.0)	5 (20.8)	63 (19.0)
Ex-smoker	14 (9.9)	1 (4.2)	28 (8.4)
Never-smoker	89 (63.1)	18 (75.0)	241 (72.6)
Missing	9	6	29
BMI, kg·m−2	18.5 (17.2–20.2)	19.4 (18.1–21.6)	20.8 (18.8–23.1)	<0.001
Missing	10	6	31
IGRA result				<0.001
Positive	86 (84.3)	5 (45.5)	94 (42.7)
Negative	14 (13.7)	6 (54.5)	109 (49.5)
Indeterminate	2 (2.0)	0 (0.0)	17 (7.7)
Missing	48	19	141
HIV-positive	33 (22.0)	14 (46.7)	96 (26.6)	0.020
Antiretroviral therapy	16/33 (48.5)	5/14 (35.7)	29/96 (30.2)	0.654
CD4 count in HIV-infected, cells·mm−3 blood	176.0 (95.0–304.0)	220.0 (219.5–233.5)	355.0 (127.0–634.0)	0.28
Missing/not available	135	27	320
Country				0.007
Ethiopia	41 (27.3)	11 (36.7)	105 (29.1)
Malawi	8 (5.3)	6 (20.0)	29 (8.0)
Namibia	26 (17.3)	5 (16.7)	55 (15.2)
South Africa	30 (20.0)	0 (0.0)	35 (9.7)
The Gambia	4 (2.7)	0 (0.0)	7 (1.9)
Uganda	41 (27.3)	8 (26.7)	130 (36.0)
Study				<0.001
AE-TBC	57 (38.0)	24 (80.0)	195 (54.0)
ScreenTB	93 (62.0)	6 (20.0)	166 (46.0)
Haemoptysis	24 (37.5)	4 (20.0)	45 (22.1)	0.042
Missing	86	10	157
Shortness of breath	118 (83.1)	15 (62.5)	233 (70.2)	0.006
Missing	8	6	29
Weight loss				<0.001
Yes	109 (76.8)	17 (70.8)	166 (50.0)
No	29 (20.4)	7 (29.2)	155 (46.7)
Unsure	4 (2.8)	0 (0.0)	11 (3.3)
Missing	8	6	29
Night sweats				0.090
Yes	82 (57.7)	18 (75.0)	165 (49.7)
No	56 (39.4)	6 (25.0)	160 (48.2)
Unsure	4 (2.8)	0 (0.0)	7 (2.1)
Missing	8	6	29
Cough duration, months	0.7 (0.6–1.0)	1.0 (0.7–2.3)	0.7 (0.7–1.0)	0.091
Missing	16	6	44

Data are presented as n (%), n/N (%), median (interquartile range) or n, unless otherwise stated. TB: tuberculosis; ORD: other respiratory diseases; BMI: body mass index; IGRA: interferon-γ release assay. ^#: Pearson's Chi-squared test; Kruskal–Wallis rank-sum test; Fisher's exact test for count data with simulated p-value (based on 2000 replicates).

Nine signatures had indistinguishable performance in discriminating TB cases and ORD controls

All 20 transcriptomic signatures significantly differentiated between all TB cases and ORD controls with AUC values of 0.68 (95% CI 0.63–0.73) or above (figure 2 and table 3). Notably, all signatures had higher diagnostic performance for definite TB than for probable TB (supplementary table S3). We report signature performance for definite and probable TB cases combined, rather than separately, in the following analyses to reflect more realistic signature performance in real-world settings, where people with unconfirmed TB are not excluded. Satproedprai7 had the highest AUC (0.83, 95% CI 0.79–0.87) for distinguishing between all TB cases and controls (figure 2). The diagnostic performance of eight other signatures, Jacobsen3, Kaforou22, Roe1, Suliman2, Sambarey10, Duffy9, Gliddon3 and Suliman4, was not statistically different to Satproedprai7 (table 3 and supplementary figure S1). It should be noted that analysis of inter-signature performance required pairwise comparisons between Satproedprai7 and each other signature, and thus the sample size for each comparison was limited to the samples that had scores for both signatures. For example, the comparison of Satproedprai7 (AUC 0.83, 95% CI 0.79–0.87) with Duffy9 (AUC 0.81, 95% CI 0.76–0.86) was based on 127 paired TB cases and 230 paired controls, whereas the comparison of Satproedprai7 with Penn-Nicholson6 (AUC 0.81, 95% CI 0.77–0.84), with a near-identical AUC to Duffy9, was based on a much larger sample size of 167 paired TB cases and 333 paired controls. In this case the AUC for Duffy9 was found to be statistically not different to Satproedprai7, whereas the AUC for Penn-Nicholson6 was lower than that for Satproedprai7 (table 3 and supplementary figure S1). Scores for most signatures were highly correlated, with Spearman rank correlation coefficients ranging from 0.28 to 0.96 (supplementary figure S2). Data batch correction and normalisation processing steps had a negligible effect on the diagnostic performance of transcriptomic signatures (supplementary figure S3).

FIGURE 2.

Signature score distributions and diagnostic performance of transcriptomic signatures in tuberculosis (TB) cases and controls with other respiratory diseases (ORD). a) Box-and-whisker plot of the signature with the best diagnostic performance, based on the area under the receiver operating characteristic (ROC) curve (AUC) (Satproedprai7). The plot shows the signature score distribution (each dot represents a participant) in definite TB cases (n=139), probable TB cases (n=28) and controls with ORD (n=334). p-values for comparison of median signature scores between groups were calculated using the Mann–Whitney U-test. Boxes depict the interquartile range (IQR), the midline represent the median and whiskers represent 1.5×IQR. b) ROC curve depicting diagnostic performance (AUC with 95% confidence interval) of the Satproedprai7 signature in definite and probable TB cases separately, and in all TB cases combined, versus ORD controls. The shaded areas represent 95% confidence intervals. The solid box represents the optimal criteria (95% sensitivity and 80% specificity) and the dashed box represents the minimal criteria (90% sensitivity and 70% specificity) set out in the World Health Organization Target Product Profile for a TB triage test. c) Summary of signature diagnostic performance, expressed as AUC for all TB cases versus ORD. Error bars depict 95% confidence intervals. The diagnostic AUCs for definite TB cases versus ORD and probable TB versus ORD are also shown. Signatures were ranked according to AUC for definite TB cases versus ORD. A table of AUCs for all signatures is shown in supplementary table S3.

TABLE 3.

Diagnostic performance of 20 published host response tuberculosis (TB) transcriptomic signatures, ordered by area under the receiver operating characteristic curve (AUC) for all TB patients versus other respiratory diseases (ORD) controls

Signature	TB# versus ORD AUC (95% CI)	Sensitivity (95% CI) (at 70% specificity¶)	Specificity (95% CI) (at 90% sensitivity+)	PPV (95% CI)§	NPV (95% CI)§	p-value versus signature with highest AUC	q-valueƒ versus signature with highest AUC
Satproedprai7	0.83 (0.79–0.87)	0.81 (0.74–0.86)	0.48 (0.43–0.53)	0.46 (0.41–0.52)	0.91 (0.85–0.94)
Jacobsen3	0.83 (0.79–0.86)	0.80 (0.73–0.85)	0.52 (0.46–0.57)	0.48 (0.42–0.53)	0.91 (0.87–0.95)	0.316	0.441
Kaforou22	0.82 (0.78–0.86)	0.77 (0.69–0.83)	0.51 (0.45–0.57)	0.48 (0.42–0.54)	0.91 (0.85–0.94)	0.226	0.350
Roe1	0.82 (0.78–0.86)	0.80 (0.73–0.85)	0.49 (0.44–0.54)	0.48 (0.42–0.53)	0.91 (0.86–0.94)	0.051	0.115
Suliman2	0.82 (0.78–0.86)	0.78 (0.71–0.83)	0.49 (0.44–0.55)	0.47 (0.42–0.53)	0.91 (0.86–0.94)	0.263	0.385
Sambarey10	0.81 (0.77–0.85)	0.81 (0.74–0.86)	0.54 (0.49–0.59)	0.49 (0.44–0.55)	0.92 (0.87–0.95)	0.029	0.074
Penn-Nicholson6	0.81 (0.77–0.84)	0.80 (0.73–0.85)	0.45 (0.40–0.50)	0.45 (0.40–0.50)	0.90 (0.85–0.94)	0.002	0.008
Duffy9	0.81 (0.76–0.86)	0.81 (0.73–0.87)	0.52 (0.46–0.59)	0.51 (0.45–0.58)	0.90 (0.84–0.94)	0.079	0.155
Gliddon3	0.80 (0.75–0.85)	0.78 (0.71–0.85)	0.51 (0.45–0.57)	0.48 (0.41–0.54)	0.91 (0.85–0.95)	0.080	0.157
Suliman4	0.79 (0.75–0.84)	0.77 (0.70–0.83)	0.44 (0.38–0.49)	0.44 (0.39–0.50)	0.90 (0.84–0.94)	0.020	0.052
da Costa3	0.79 (0.75–0.83)	0.75 (0.68–0.81)	0.39 (0.34–0.44)	0.43 (0.38–0.48)	0.89 (0.83–0.93)	0.003	0.012
Maertzdorf4	0.79 (0.75–0.83)	0.78 (0.71–0.83)	0.49 (0.43–0.54)	0.46 (0.41–0.51)	0.91 (0.86–0.94)	0.001	0.002
Roe3	0.79 (0.75–0.83)	0.77 (0.70–0.83)	0.37 (0.32–0.42)	0.42 (0.37–0.47)	0.88 (0.82–0.92)	0.001	0.001
Gjøen7	0.79 (0.74–0.83)	0.76 (0.69–0.82)	0.32 (0.28–0.37)	0.40 (0.35–0.45)	0.87 (0.80–0.91)	0.001	0.004
Sweeney3	0.79 (0.74–0.83)	0.75 (0.68–0.81)	0.40 (0.35–0.45)	0.43 (0.38–0.48)	0.89 (0.83–0.93)	0.003	0.012
Rajan5	0.76 (0.71–0.81)	0.68 (0.59–0.75)	0.41 (0.35–0.47)	0.45 (0.39–0.51)	0.88 (0.81–0.93)	0.001	0.001
Thompson5	0.76 (0.71–0.80)	0.69 (0.62–0.76)	0.40 (0.35–0.45)	0.42 (0.37–0.47)	0.89 (0.83–0.93)	0.003	0.001
Francisco2	0.74 (0.69–0.78)	0.71 (0.64–0.77)	0.26 (0.22–0.31)	0.38 (0.33–0.43)	0.84 (0.76–0.90)	0.001	0.001
Gliddon4	0.73 (0.68–0.79)	0.71 (0.62–0.78)	0.26 (0.21–0.32)	0.40 (0.34–0.45)	0.83 (0.73–0.90)	0.001	0.001
de Araujo1	0.68 (0.63–0.73)	0.61 (0.54–0.68)	0.21 (0.17–0.26)	0.36 (0.32–0.41)	0.81 (0.72–0.88)	0.001	0.001

PPV: positive predictive value; NPV: negative predictive value. ^#: definite and probable TB combined; ^¶: minimum specificity criterion of the World Health Organization (WHO) Target Product Profile (TPP) for a triage test (70%); ⁺: minimum sensitivity criterion of the WHO TPP for a triage test (90%); ^§: based on a prevalence of 30% and the specificity at a sensitivity of 90%; ^ƒ: Benjamini–Hochberg method.

No signatures met the minimal WHO TPP benchmarks when cohorts were pooled

We benchmarked signature performance against the minimal WHO TPP criteria specified for a non-sputum-based triage test. At a fixed specificity of 70%, sensitivities ranged from 0.61 (95% CI 0.54–0.68) for the de Araujo1 signature to 0.81 (95% CI 0.74–0.86) for Satproedprai7; none met the minimum criterion threshold of 90% sensitivity (table 3). At a fixed sensitivity of 90%, specificities ranged from 0.21 (95% CI 0.17–0.26) for the de Araujo1 signature to 0.54 (95% CI 0.49–0.59) for Sambarey10; none met the minimum criterion threshold of 70% specificity (table 3). In this study population with a 30% TB prevalence and a sensitivity of 90%, positive predictive values ranged from 0.36 (95% CI 0.32–0.41) to 0.51 (95% CI 0.45–0.58) and negative predictive values ranged from 0.81 (95% CI 0.72–0.88) to 0.91 (95% CI 0.87–0.95).

HIV status and country were associated with signature scores

Multivariable linear regression was used to identify clinical and demographic features associated with signature scores. HIV positivity and definite or probable TB disease were independently associated with higher signature scores for most signatures; among the top nine performing signatures, only Jacobsen3 scores were not significantly associated with HIV status (table 4). When disaggregated by HIV status, AUC values for all signatures were higher in HIV-negative than HIV-positive participants (supplementary tables S4 and S5).

TABLE 4.

Regression analyses determining clinical and demographic variables associated with signature scores

	Duffy9	Gliddon3	Jacobsen3	Kaforou22	Penn-Nicholson6	Roe1	Sambarey10	Satproedprai7	Suliman2	Suliman4
Age (per 10 years)	0.00 (−0.02–0.03); 0.92	0.00 (−0.02–0.02); 0.90	0.00 (−0.03–0.02); 0.94	0.01 (−0.01–0.03); 0.79	−0.01 (−0.03–0.01); 0.42	−0.01 (−0.03–0.01); 0.65	0.00 (−0.02–0.03); 0.90	−0.01 (−0.03–0.02); 0.85	0.00 (−0.02–0.02); 0.99	0.01 (−0.02–0.03); 0.79
Sex
Female	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Male	0.02 (−0.04–0.08); 0.74	−0.02 (−0.07–0.03); 0.67	−0.07 (−0.13– −0.01); 0.05	−0.01 (−0.06–0.03); 0.79	−0.03 (−0.08–0.01); 0.37	−0.04 (−0.08–0.01); 0.31	−0.06 (−0.12–0.00); 0.13	−0.06 (−0.12– −0.01); 0.06	0.00 (−0.05–0.04); 0.93	−0.02 (−0.07–0.03); 0.74
HIV status
Negative	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Positive	0.12 (0.05–0.19); 0.01	0.10 (0.05–0.16); <0.001	0.07 (0.01–0.14); 0.11	0.09 (0.03–0.14); 0.01	0.10 (0.04–0.15); <0.001	0.08 (0.03–0.13); 0.01	0.09 (0.02–0.16); 0.04	0.13 (0.07–0.19); <0.001	0.09 (0.04–0.14); 0.01	0.08 (0.01–0.14); 0.06
TB status
ORD controls	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Definite TB	0.30 (0.23–0.36); <0.001	0.25 (0.20–0.30); <0.001	0.30 (0.24–0.36); <0.001	0.27 (0.22–0.32); <0.001	0.27 (0.22–0.31); <0.001	0.33 (0.28–0.38); <0.001	0.36 (0.30–0.42); <0.001	0.34 (0.29–0.40); <0.001	0.26 (0.22–0.31); <0.001	0.27 (0.22–0.32); <0.001
Probable TB	0.21 (0.08–0.34); 0.01	0.15 (0.06–0.25); 0.01	0.20 (0.08–0.32); 0.01	0.18 (0.09–0.28); <0.001	0.16 (0.06–0.25); 0.01	0.17 (0.07–0.27); <0.001	0.20 (0.08–0.32); 0.01	0.21 (0.10–0.32); <0.001	0.18 (0.09–0.27); <0.001	0.17 (0.07–0.28); 0.01
Smoking history
Never-smoker	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Current smoker	−0.04 (−0.14–0.05); 0.67	0.02 (−0.06–0.09); 0.86	0.00 (−0.08–0.08); 1.00	0.03 (−0.04–0.10); 0.68	0.00 (−0.07–0.07); 0.99	−0.02 (−0.08–0.05); 0.86	−0.04 (−0.12–0.05); 0.67	0.01 (−0.07–0.09); 0.93	−0.01 (−0.08–0.05); 0.86	0.01 (−0.07–0.09); 0.90
Ex-smoker	−0.04 (−0.16–0.08); 0.77	−0.06 (−0.16–0.03); 0.41	0.01 (−0.09–0.10); 0.95	−0.01 (−0.09–0.06); 0.90	−0.02 (−0.10–0.06); 0.84	−0.05 (−0.13–0.02); 0.41	−0.01 (−0.10–0.09); 0.95	0.01 (−0.08–0.09); 0.94	−0.03 (−0.11–0.04); 0.67	0.01 (−0.08–0.10); 0.90
Previous TB
No	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Yes	0.01 (−0.07–0.09); 0.93	0.00 (−0.07–0.06); 0.99	0.00 (−0.08–0.07); 0.99	−0.02 (−0.08–0.04); 0.85	−0.01 (−0.07–0.05); 0.89	0.02 (−0.04–0.08); 0.77	0.01 (−0.06–0.08); 0.90	0.01 (−0.05–0.08); 0.86	0.00 (−0.06–0.06); 0.99	0.00 (−0.07–0.07); 0.99
Country
Uganda#	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference	Reference
Ethiopia	0.02 (−0.05–0.08); 0.85	0.00 (−0.05–0.05); 0.98	0.04 (−0.02–0.10); 0.46	0.02 (−0.02–0.07); 0.65	0.03 (−0.02–0.08); 0.47	0.00 (−0.06–0.05); 0.93	0.00 (−0.06–0.06); 0.99	−0.01 (−0.07–0.05); 0.90	0.06 (0.01–0.10); 0.07	0.06 (0.01–0.12); 0.07
Namibia	0.17 (0.07–0.26); <0.001	0.10 (0.03–0.18); 0.03	0.18 (0.10–0.27); <0.001	0.18 (0.11–0.25); <0.001	0.19 (0.12–0.25); <0.001	0.14 (0.07–0.20); <0.001	0.16 (0.07–0.24); <0.001	0.13 (0.06–0.21); <0.001	0.14 (0.08–0.21); <0.001	0.19 (0.11–0.27); <0.001
South Africa	0.03 (−0.09–0.15); 0.85	−0.01 (−0.10–0.08); 0.93	0.03 (−0.08–0.14); 0.85	−0.01 (−0.10–0.09); 0.95	0.00 (−0.09–0.09); 0.98	−0.02 (−0.11–0.06); 0.83	0.04 (−0.07–0.15); 0.74	−0.05 (−0.15–0.05); 0.65	0.03 (−0.05–0.12); 0.70	0.01 (−0.09–0.11); 0.91
The Gambia	−0.13 (−0.33–0.08); 0.50	−0.10 (−0.27–0.07); 0.54	−0.12 (−0.30–0.06); 0.42	−0.08 (−0.22–0.06); 0.57	−0.04 (−0.18–0.09); 0.78	−0.05 (−0.19–0.10); 0.77	−0.10 (−0.27–0.07); 0.53	−0.04 (−0.21–0.13); 0.86	−0.03 (−0.16–0.11); 0.88	−0.11 (−0.26–0.04); 0.35

Data are presented as β-coefficient (95% CI); adjusted p-value. Signatures are listed in alphabetical order. ^#: Uganda was chosen as the reference because it has the highest number of both cases and controls.

Country was also a strong predictor of signatures scores. We therefore sought to investigate signature scores and performance when disaggregated by country. Due to the low numbers of TB cases (n=4) and ORD controls (n=7), data from The Gambia were excluded from these analyses (table 2). It was notable that signature scores differed considerably by country, in particular ORD controls from Namibia had significantly higher scores for most signatures than those from other countries (figure 3). This influenced diagnostic performance and most signatures had higher AUCs in Ethiopia, Malawi and South Africa than Uganda and Namibia (figure 3 and supplementary tables S6 and S7). Consistent with this, several signatures met the minimum TPP criteria for a triage test in Ethiopia, Malawi or South Africa. For example, Gliddon3 and Penn-Nicholson6 met the TPP criteria in Ethiopia, Malawi and South Africa; Satproedprai7 met these criteria in Ethiopia and Malawi; and Sambarey10 met these criteria in Ethiopia (figure 3).

FIGURE 3.

Signature score distributions and diagnostic performance of transcriptomic signatures for all tuberculosis (TB) cases and controls with other respiratory diseases (ORD), stratified by country. a, c, e, g) Receiver operating characteristic (ROC) curves and b, d, f, h) box-and-whisker plots for a, b) Gliddon3, c, d) Penn-Nicholson6, e, f) Satproedprai7 and g, h) Sambarey10 for distinguishing between all TB cases and ORD controls from Ethiopia, Malawi, Namibia, South Africa (SA) and Uganda. The Gambia was excluded from individual country analyses due to very small sample size (n=11) (supplementary table S5). Diagnostic performance (area under the curve with 95% confidence interval) disaggregated by country is shown in the ROC plots and for all signatures in supplementary table S4. Box-and-whisker plots show signature score distribution (each dot represents a participant) between TB cases and controls in Ethiopia, Malawi, Namibia, South Africa and Uganda. Boxes depict the interquartile range (IQR), the midline represents the median and the whiskers represent 1.5×IQR. The tables in b, d, f, h) show q-values (Benjamini–Hochberg adjusted p-values) for comparison of signature scores in control groups between the countries, calculated using the Mann–Whitney U-test.

Discussion

We sought to determine the diagnostic accuracy of 20 candidate blood transcriptomic TB signatures in symptomatic individuals presenting at primary care facilities. Our results show that nine transcriptomic signatures had equivalent diagnostic performance in differentiating between patients with TB from those who presented with ORD. All signatures had reduced performance for clinically defined, probable TB, than for microbiologically confirmed definite TB. In pooled analysis of all countries, none of the signatures met the minimum TPP criteria for a triage test for TB. However, multiple signatures met these criteria in certain countries. Among the clinical and demographic factors explored, HIV positivity, independent of TB classification, was associated with higher signature scores, leading to reduced diagnostic performance of transcriptomic signatures.

We found that, compared to Satproedprai7, the signature with the highest AUC, eight other signatures had statistically indistinguishable diagnostic performance. These included signatures trained on diverse cohorts from different settings, derived using different training strategies and distinct scoring algorithms and signatures that range from a single mRNA transcript to 22 transcripts. This finding suggests that it is unlikely that there is a single “best-performing” signature or strategy. It also calls into question the value of developing yet more signatures for pulmonary TB, especially if such signature discovery approaches follow the approach that has been taken in most studies, where the most differentially expressed genes are selected. More innovative statistical models, such as multinomial signatures, may also perform better. It therefore remains to be established exactly where existing signatures may add most value.

Diagnostic biomarkers that can identify those with TB who do not have a definitive diagnosis based on traditional diagnostic tests, who cannot expectorate sputum (including young children), who have extrapulmonary TB or who have microbiologically negative subclinical TB are needed. Many such individuals traditionally do not receive a diagnosis, yet would benefit from TB treatment and some forms of difficult-to-diagnose TB may transmit M. tuberculosis to others [43]. Reduced diagnostic performance of TB transcriptomic signatures in those with probable TB has been reported previously [44]. Since transcriptomic signatures reflect the host inflammatory response consistent with TB, the lower diagnostic performance of transcriptomic signatures in unconfirmed TB likely reflects an earlier or less pronounced pathological or inflammatory state, as has also been reported for asymptomatic TB [22, 23, 45]. It is possible that diagnostic pathogen markers may offer better diagnostic performance than host response markers [44], although it is likely that M. tuberculosis load in such difficult-to-identify cases is lower than in sputum test-positive, symptomatic cases of pulmonary TB.

A prospective, observational study that compared the diagnostic accuracy of 27 transcriptomic signatures in symptomatic adults who presented to a TB clinic found that four signatures met the minimum TPP criteria, but not the optimum criteria for a triage test; these signatures also did not meet the minimum criteria for a confirmatory test [20]. When assessed as an add-on confirmatory test for patients that had positive Xpert MTB/RIF results, transcriptomic signatures improved specificity [20], illustrating one use-case for signatures for TB diagnosis under certain conditions.

The level of correlation between signature scores was not surprising, given the extent of common transcripts, predominated by ISGs [46, 47], that constitute these signatures. For example, FCGR1A is present in six signatures [10, 29, 31, 35, 36, 41] and GBP5 in five of the 20 signatures [12, 29, 33, 34, 39]. Our findings further underpin the consistent involvement of type I interferons in TB pathogenesis [48]. Several other infections and conditions, most notably viral infections [49, 50], including HIV [51, 52], induce expression of ISGs, leading to erosion of the specificity of ISG-based TB signatures. Capturing a wider scope of immune responses and specifically TB-associated genes that are not interferon-induced may lead to better specificity of signatures [52, 53].

As most signatures on our panel predominantly included ISGs, the effect of HIV infection on signature scores and diagnostic performance was not surprising. Although plasma viral load was not included in our regression analysis, due to a high number of missing values, others have shown an association between detectable HIV viral loads and elevated scores for ISG-based signatures [9, 23, 51].

The identification of other biological pathways and transcripts less affected by HIV may have better utility in HIV-infected individuals and predominantly HIV-infected populations. Genes involved in the complement pathway are promising biomarkers for TB [52, 54–56]. It was notable that Thompson5 [42], a treatment response signature that contains no ISGs, showed only a minimal reduction in AUC in HIV-positive versus HIV-negative participants. This is consistent with findings of our previous study that Thompson5 scores were not affected by HIV viral load [23]. One transcript within the Thompson signature belongs to the category of long non-coding RNAs. These transcripts are known to modulate immune responses [57, 58], and have now also been highlighted as potential biomarkers for TB [59].

Satproedprai7 [10], the signature with the best diagnostic performance in these African cohorts, was discovered in an adult population from India. Many of the signatures evaluated here, alongside many others not included, have either been developed or validated predominantly in South African populations [9, 31, 35, 37, 41, 42] or using publicly available datasets [31, 32, 46, 60] generated from studies conducted within South Africa [7, 8, 10, 12]. This likely accounts for the good performance of most transcriptomic signatures, especially those that met the minimal WHO TPP, in South Africa. It was puzzling, however, that signature performance in neighbouring Namibia was consistently poorer than in South Africa. We do not know the underlying reason(s), but speculate that there may be differences in illness and severity thereof in the ORD group, which we expect to influence signature scores. For example, all recruitment in South Africa was done at primary healthcare clinics. By contrast, Namibia performed hospital-based recruitment and therefore it is likely that a more ill population was enrolled. The ScreenTB and AE-TBC studies were designed to minimise this by using highly standardised protocols and predefined TB classification algorithms across countries, but implementation and programme settings may have been different. We also cannot rule out the possibility that more controls in some settings could have had undiagnosed TB and may have been misclassified or missed as TB cases during follow-up. Season is unlikely to have affected performance since study participants, across all sites, were recruited within the same period across several seasons. Overall, what is clear is that differences present across African countries may be attributed to more than just variations in TB burden and host genetics. Well-designed multisite biomarker studies that address these underlying factors are needed to understand this better.

Strengths of our study were the unbiased selection and evaluation of candidate transcriptomic signatures, inclusion of cohorts from multiple African countries representing diverse geographies and populations, and a diagnostic study design that compared TB cases to appropriate controls within the real-world setting where POC tests are needed. Inclusion of probable TB cases was important to ascertain performance of signatures in those who are difficult to diagnose.

Limitations of our study included insufficient information on the cause of respiratory or other infections or diseases in the control group or in the cases, or comorbidities such as diabetes, and to what degree this may have differed by country setting. It is likely that these may have contributed to the differences in signature performance at the different sites. A limitation of the analysis done to compare the performance of different signatures is that failed quantification of transcripts in some signatures for some samples led to very different sample sizes for certain signatures, ranging from 360 samples for Duffy9 to 541 samples for de Araujo1. Because inter-signature performance required pairwise comparisons between Satproedprai7, which had the highest AUC, and each other signature, the sample sizes for each comparison were different. Consequently, it is important to interpret the results of the statistical comparisons of signature AUCs with caution. We also cannot rule out that diagnostic performance of Gliddon4 [35], Duffy9 [7] and Rajan5 [38] may have been biased by poor performance of the primer–probe assays for GBP6, which failed in a considerable number of samples. Although the primer–probe assay that represented GBP6 had been published and used in by Gliddon et al. [35], it did not meet our assay qualification pass criteria.

In conclusion, we report a head-to-head comparison of multiple signatures for TB, demonstrating their utility within a wider African context. It is critical that transcriptomic signatures be assessed in larger studies of unconfirmed TB, and well-designed studies of extrapulmonary and paediatric TB, where there is an urgent need for better diagnostics.

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material ERJ-00153-2024.Supplement ^{(1.1MB, pdf)}

Shareable PDF

This one-page PDF can be shared freely online.

Shareable PDF ERJ-00153-2024.Shareable ^{(1,006.2KB, pdf)}

Data availability

De-identified signature scores, PCR probe data, clinical metadata and TB end-point data have been deposited in ZivaHub (https://doi.org/10.25375/uct.25650933.v1), an open access data repository hosted by the University of Cape Town's institutional data repository powered by Figshare for Institutions.