Bacterial versus non-bacterial infections: a methodology to support use-case-driven product development of diagnostics

Abstract

Acute febrile illness (AFI) is one of the most common reasons for seeking medical care in low-income and middle-income countries. Bacterial infections account for a relatively small proportion of AFIs; however, in the absence of a simple diagnostic test to guide clinical decisions, healthcare professionals often presume that a non-malarial febrile illness is bacterial in origin, potentially resulting in inappropriate antibiotic use. An accurate differential diagnostic tool for AFIs is thus essential, to both limit antibiotic use to bacterial infections and address the antimicrobial resistance crisis that is emerging globally, without resorting to multiple or complex pathogen-specific assays. The Biomarker for Fever-Diagnostic (BFF-Dx) study is one of the largest fever biomarker studies ever undertaken. We collected samples and classified disease aetiology in more than 1900 individuals, distributed among enrolment centres in three countries on two continents. Identical protocols were followed at each study site, and the same analyses were conducted in each setting, enabling like-with-like comparisons to be made among the large sample set generated. The BFF-Dx methodology can act as a model for other researchers, facilitating wider utility of the work in the future. The established sample collection is now accessible to researchers and companies and will facilitate the development of future fever-related diagnostic tests. Here, we outline the methodology used to determine the sample populations and to differentiate bacterial versus non-bacterial AFIs. Future publications will set out in more detail the study’s demographics, the causes of fever identified and the performance of selected biomarkers.

Summary box.

• Acute febrile illness (AFI) is one of the most common reasons for seeking medical care in low-income and middle-income countries (LMICs).

• The adoption of malaria rapid diagnostic tests to guide antimalarial treatment has led to reduced use of antimalarials; however, in many malaria-endemic regions there has been an increase in antibiotics given to those who test negative for malaria.

• Although bacterial infections account for a relatively small proportion of AFIs in LMICs, in the absence of a simple diagnostic test clinicians often presume that an AFI is bacterial in origin, which can potentially lead to the inappropriate use of antibiotics.

• Here, we outline the methodology of the Biomarker for Fever-Diagnostic (BFF-Dx) study, one of the largest fever biomarker studies ever undertaken, which enables like-with-like comparisons to be made among epidemiologically different settings and has generated a well-characterised sample set that can be used for future research and development of biomarkers and diagnostic tools.

• The BFF-Dx methodology facilitates the evaluation of the usefulness of biomarkers in differentiating AFIs of bacterial versus non-bacterial origin, the results of which will contribute to efforts to provide appropriate care, reduce the overuse of antibiotics and help curb the threat posed by antimicrobial resistance.

Introduction

Acute febrile illness (AFI) is one of the most common reasons for seeking medical care in low-income and middle-income countries (LMICs). The diseases underlying AFIs, including malaria, typhoid, leptospirosis, rickettsial illnesses and many illnesses caused by viruses, such as arboviruses, are a major cause of morbidity and mortality, especially among children.¹ The global roll-out of simple, rapid diagnostic tests (RDTs) for malaria has improved our understanding of the role malaria plays in AFIs and led to an awareness that malaria is responsible for a much smaller fraction of fever cases than was once thought.^{2 3} In Africa alone, it is estimated that more than 90 million children present to health facilities annually with non-malarial fevers.^{4 5} However, information about the causes of fever in LMICs is scarce.^{6 7} Recent studies conducted in Latin America have shown that viruses, including arboviruses and respiratory viruses, are the most frequently reported causative agents of febrile illness.^{8 9} A study in Tanzania showed up to 70% of all paediatric patients who present with gastroenteritis, respiratory symptoms or bloodstream infections are infected by viral agents and suggested bacterial agents are implicated in fewer than 25% of AFI cases.¹⁰ Another study of adult and paediatric patients with fever conducted in northern Tanzania identified malaria as the cause of fever in just 1.6% of patients.¹¹ These studies, and another conducted in South-east Asia,¹² also show great heterogeneity in the causes of febrile illness across regions and even within a country. In such a complex and poorly characterised epidemiological context and in the absence of a simple diagnostic test to guide clinical treatment, especially for cases malaria-negative by RDT, many healthcare professionals prescribe antibiotics as a precaution, since they fear undertreating life-threatening bacterial infections such as pneumonia.^{2 13} Therefore, an accurate differential diagnostic tool for AFIs is essential to improve the targeted use of antibiotics and help address the emerging global crisis of antimicrobial resistance,¹⁴ in a context where the primary causes of fever remain unknown, and costly, pathogen-specific detection tools are not available.

Host biomarkers have been suggested as an appropriate means of meeting the challenge of differentiating bacterial from non-bacterial infections.¹⁵ C reactive protein (CRP) and procalcitonin are long-established biomarkers used to guide clinical decisions in hospitals in high-income countries (HICs).^{15 16} However, the use of such biomarkers was until recently mostly restricted to hospital-based care and therefore not easily transferable to a decentralised testing approach in LMICs. To define more clearly the needs of LMICs, a consortium of experts in global health and diagnostics developed a target product profile (TPP), which identified the need for an assay to distinguish bacterial from non-bacterial infections in low-resource settings (eg, corresponding to community-based healthcare settings as well as primary care centres) to support evidence-based treatment guidance.¹⁷ From this consensus effort, the ideal characteristics for such a test were defined and the target population was identified as the general febrile population and included all age groups. To determine how effectively any potential solution meets these TPP priorities, it is essential that potential biomarkers are investigated within the intended target population. To date, most biomarker studies have been conducted in HICs and have focused on severe and/or hospitalised patients¹⁸ (also Fernandez et al, in preparation). Data that address the challenges of the TPP (eg, target setting, target population) are therefore urgently needed, not least because the health priorities and operational challenges faced in less well-resourced settings differ considerably from those faced in HICs, and the performance of biomarkers may also differ considerably in these settings.¹⁹

To address this data gap, which until now has impeded targeted diagnostic development to address the emerging needs in LMICs, we conducted the Biomarker for Fever-Diagnostic (BFF-Dx) study; one of the largest fever biomarker studies ever undertaken and one that involved extensive laboratory testing. The primary objective of BFF-Dx was to evaluate the performance in differentiating bacterial versus non-bacterial infections of various host biomarkers across multiple settings in Africa and South America, the intended-use settings of any potential fever biomarker tests. Here, we outline the overall BFF-Dx methodology adopted: the protocols used to determine the BFF-Dx sample populations, how bacterial versus non-bacterial AFIs were differentiated and how the various analytical tools used were employed.

Biomarker for Fever-Diagnostic study: overall approach

Study sites

Several potential study locations were identified based on the following factors: geographical location, type of health facility, endemic pathogen profile, logistical and operational characteristics, laboratory and recruitment capacity and expected study population. An initial assessment led to eight sites being identified for a subsequent site visit. Based on the findings of these on-site assessments, four sites were shortlisted, with three sites finally selected to participate in recruitment for BFF-Dx (table 1).

Table 1.

Participating study site settings and corresponding ethical boards that approved BFF-Dx

Country	Brazil	Gabon	Malawi
Institute	Instituto Nacional de Infectologia Evandro Chagas (INI), FIOCRUZ, Rio de Janeiro	Center of Medical Research Lambaréné (CERMEL)	Malawi Epidemiology and Intervention Research Unit (MEIRU)
Enrolment site	UPA Rocha Miranda, UPA Manguinhos and Family Health Clinics Armando Palhares	Clinical trials unit, CERMEL	MEIRU, Chilumba campus
Enrolment setting	Primary healthcare facility in an urban area (favela)	Hospital in a semirural setting	Primary healthcare facility in a rural setting
Enrolment period	October 2018 to July 2019	May 2019 to November 2019	April 2017 to April 2018
Main causes of fever (expected)	Circulation of arboviruses, including dengue, Zika and chikungunya viruses	Endemic Plasmodium falciparum, dengue virus and chikungunya virus	Endemic Plasmodium falciparum and possibly chikungunya virus

BFF-Dx, Biomarker for Fever-Diagnostic.

table 1 The study was conducted in full compliance with the principles of both the Declaration of Helsinki and the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use guidelines. All participants or their parent/guardian gave written informed consent prior to their participation in the study.

Sample size

The sample size was determined according to previously published formulae,²⁰ taking into account available performance data for selected fever biomarkers and making the following assumptions:

• Estimate a sensitivity and specificity of 80% and 80%, respectively, based on published reports of the performance of the human neutrophil lipocalin ELISA,²¹ the FebriDx RDT²² and the TRAIL/IP-10/CRP combination²³ in HICs.

• Significance level alpha=0.05 (used for the derivation of CIs).

• Expected width of the 95% CI of the point estimates of sensitivity and specificity, M =±10%.

• An estimated prevalence of 10% bacterial infections in patients presenting with AFI at an outpatient department (based on estimates from the literature and consultation with on-site infectious disease clinicians).¹⁰

• Power to detect estimates of sensitivity and specificity with a CI of width M: 80%; power of sampling the necessary number of patients with bacterial infections based on the reported prevalence: 90%.

Based on the above assumptions, the minimum sample size required for the primary discovery cohort was calculated to be 1380 participants; this was rounded up to 1500 participants, that is, 500 participants per study site.

Study design

This was a cross-sectional, observational study that used a convenience sample of children and adults who had clinical signs of AFI and no signs of severe illness. All patients enrolled in BFF-Dx continued to be clinically managed according to local standards of care. Inclusion and exclusion criteria for the study were based on the target population previously identified in the TPP¹⁷; patients diagnosed with chronic disease were enrolled only when their fever was a new and separate symptom (table 2). Investigators used case report forms (CRFs) for data capture, tailored to local needs. Data items captured included enrolment information, clinical signs and symptoms, laboratory results and patient follow-up details. Templates of CRFs are provided in online supplemental appendix 1. Participant follow-up visits were conducted 14–28 days after their initial healthcare-seeking appointment to allow convalescent samples to be taken for selected confirmatory tests (IgM/IgG testing for dengue, Zika, chikungunya, Rickettsia spp and Leptospira spp). Based on a participant’s clinical presentation, their samples were sent for symptom-based panels of laboratory tests. A standard panel of tests was performed for all participants; other tests were performed only if specific signs or symptoms were present (figure 1). A table listing all tests and sample types used for each panel is provided in online supplemental appendix 2.

Table 2.

Inclusion and exclusion criteria at the enrolment sites

Study site	Rio de Janeiro (Brazil)		Lambaréné (Gabon)		Karonga (Malawi)
Criteria	Inclusion	Exclusion	Inclusion	Exclusion	Inclusion	Exclusion
Acute fever	History of fever, last 7 days	History of fever, more than 7 days previously*	History of fever, last 7 days	History of fever, more than 7 days previously*	On presentation†	More than 7 days*
Age (years)	2–65		2–17‡		2–65
Patient condition	Outpatient only	Critical condition	Outpatient only	Critical condition	Outpatient only	Critical condition
Informed consent/assent	Yes		Yes		Yes
Prepared to have follow-up at 2 weeks	Yes		Yes		Yes
Pregnant		No exclusion		No exclusion		Yes §

*Exclusion of patients with a history of fever of more than 7 days excludes the majority of presumptive tuberculosis cases, who usually present with a fever that has lasted for over 2 weeks.

†In Malawi, patients were unlikely to self-medicate with antipyretics prior to their clinic visit, as was the case in Brazil and Gabon, and therefore history of fever was not added to the inclusion criteria.

‡Children only, due to the setting and to counter the lower rates of child enrolment experienced in Brazil.

§National Health Science Research Committee (NHSRC) requirement. Women of childbearing age were asked about the possibility of pregnancy and offered a urine-based pregnancy test for confirmation.

Figure 1.

Symptom-based panel of tests. MAT, microscopic agglutination test; NS1, non-structural protein 1; RDT, rapid diagnostic test.

Most laboratory tests were performed daily onsite, with further characterisation performed on batched samples. For batched samples from Malawi and Gabon, this characterisation was conducted in a specialised clinical laboratory (Limbach Gruppe SE, Heidelberg, Germany); for samples from Brazil, it was performed by reference laboratories at FIOCRUZ, Rio de Janeiro, Brazil.

Sample transport and storage

Blood and urine samples were collected from all participants on enrolment. Stool, oropharyngeal swab, aspirate and pleural fluid, cerebrospinal fluid and skin swab samples were collected according to the criteria shown in figure 1. Standardised guidance for sample transport and storage prior to laboratory evaluation was provided to all sites (online supplemental appendix 3). All samples for biomarker testing or reference testing were stored at −20°C until being tested at a reference laboratory. Samples collected for the sample collection were stored at −80°C. All shipments were undertaken via World Courier and followed international shipping requirements.

Data collation and quality control

Data captured using CRFs were added to a secure database (Brazil/Gabon: OpenClinica Enterprise 34, managed by the Foundation for Innovative New Diagnostics (FIND); Malawi: local Microsoft Access project database). PCR and ELISA reference testing yielded qualitative results that were generated as electronic files and directly transferred to the FIND data management team, who reviewed them to ensure consistency with the standard format prior to importing them into the database.

Good clinical practice and good clinical laboratory practice standards were observed at all stages of BFF-Dx. Detailed site initiation, monitoring and close-out visits were undertaken. All paper forms, logbooks and sample containers were labelled with a unique identification number and barcode. Data cleaning was conducted both during the enrolment period and at the end of it; this cleaning comprised five components: (1) during data entry, in response to detailed electronic data capture system logic and range checks; (2) by adopting a double data entry procedure; (3) by site supervisor monitoring of local data managers; (4) by preprogrammed cross-form or other complex checks performed by the FIND data management team and (5) by checking the data for inconsistencies, which was performed by statisticians before they conducted statistical analyses.

Classification of patients with bacterial and non-bacterial causes of fever

We opted for a two-step approach, described in a previous publication¹⁹ as the best method for differentiating patients as having either bacterial-caused or non-bacterial-caused fever. First, an electronic classification was applied; second, there was an expert clinical review of unclassified patient files (figure 2).

Figure 2.

The two-step approach used to differentiate causes of fever: (A) electronic classification, (B) expert clinical panel classification and (C) the final classification categories.

The electronic classification was based on predefined and widely accepted laboratory parameters, including direct pathogen detection, a fourfold increase in antibody titre, or a positive PCR or antigen RDT result. The case definitions are listed in figure 3. The classification system prioritised bacterial infections such that in cases of AFIs where both bacterial and non-bacterial criteria were met, the output category would be ‘bacterial’. The rationale was that the clinical practice adopted for dealing with bacterial and non-bacterial coinfections would necessarily involve treatment with antibiotics.

Figure 3.

Microbiological criteria used to differentiate bacterial versus non-bacterial causes of AFI. Tests that were performed but do not appear in the figure were not considered for the electronic classification step. However, all test results were communicated to the clinical panel reviewers.

Cases that could not be assigned in the first step were converted into a summary case file that included the patient’s history, clinical data and laboratory findings (online supplemental appendix 4). All case files were reviewed by a panel of three clinicians who were independent from the study and possessed at least 5 years’ relevant experience in the geographical area of the study site concerned. Each clinical panel member reviewed all patient files, blinded to the assessment results of other members, and assigned them to one of the three overarching categories: bacterial infection, non-bacterial infection or undetermined cause of fever. Their adjudications were compared and, depending on the level of agreement between each clinical reviewer, the cases were classified into a final category (figure 2). For patients with AFI where two of the three panel members gave a classification of ‘bacterial’ or ‘non-bacterial’, these patients were considered to have ‘probable bacterial infection’ or ‘probable non-bacterial infection’, respectively, for analysis purposes; the analyses were then performed both with and without these cases included in the bacterial and non-bacterial classification.

Sample collection

BFF-Dx provided a unique opportunity to establish a sample collection of extensively characterised biological samples from patients with febrile illness from different settings in Africa and South America. Samples were processed and aliquoted within 8 hours of sample collection and stored onsite at −80°C until they could be shipped, on dry ice, to a central location (ZeptoMetrix, Franklin, Massachusetts, USA). The samples, together with information regarding sample types, volumes and numbers of related aliquots are available on request to product developers and researchers (https://www.finddx.org/specimen-bank/specimens-fev/); this sample collection will allow for further comparative analyses.

Biomarker tests and analysis

Previously identified, promising host fever biomarkers¹⁸ were selected to be part of an initial panel for evaluation (online supplemental appendix 5). Qualitative biomarker data will be analysed using standard two-by-two tables to assess the sensitivity, specificity and negative and positive predictive values for bacterial infections, based on local disease prevalence. Receiver operating characteristic (ROC) analysis will be carried out using the quantitative biomarker data to assess various diagnostic characteristics (area under the curve, sensitivity and specificity) at different cut-off points. The ROC analysis will be used to determine the optimal cut-off values for the various biomarkers in the different study settings. Detailed results of this biomarker analysis, from both individual and combined cohorts, will be made available in forthcoming publications.

Benefits of the Biomarker for Fever-Diagnostic study

BFF-Dx is one of the largest studies ever undertaken of fever biomarkers in patients with non-severe AFI in outpatient settings in LMICs. It involved extensive laboratory testing and an aetiologic classification system applied to more than 1900 individuals from enrolment centres in three countries across two continents. Of particular importance was the need to identify biomarkers that could be used to distinguish bacterial from non-bacterial AFIs in the large proportion of patients that presents at health facilities. It was essential that this distinction was valid among outpatients without severe illness, who comprise the majority population in outpatient settings in LMICs. One of the problems previously encountered when evaluating host fever biomarkers has been the lack of comparable reference tests to enable comparative analyses of biomarkers.¹⁸ BFF-Dx affords a uniform recruitment and analysis protocol that allows direct comparisons to be made among various cohorts from very different settings. A further longer-term benefit arising from BFF-Dx is the sample collection we have established; these samples are available to researchers and companies beyond those already collaborating with FIND. These samples, together with their related clinical and microbiological data, are providing unrivalled opportunities for the identification of novel diagnostic targets and for advancing the development and evaluation of new diagnostic tests intended to guide the management of patients with AFIs. BFF-Dx has also enhanced local knowledge among our collaborators, revealing the circulation of previously undocumented pathogens and helping health professionals to improve estimates of the causes of fever in their local areas. The study data constitute a valuable ongoing resource for local teams and will contribute to efforts aimed at improving local research and planning activities. Collaborating colleagues from the study sites report that the multidisciplinary nature of BFF-Dx has led to improvements in several aspects of their work, including the coordination of sample dispatch, processes for collecting results from multiple laboratories and forging new links with laboratory and clinical teams. The study has also contributed to scientific capacity building, both through the provision of laboratory equipment and storage capacity and by facilitating local PhD studies into multiple aspects of AFIs.

Limitations

Despite our best efforts, there were several challenges and limitations associated with BFF-Dx. First, not having a control group meant we had no baseline data for the biomarkers or for the carriage of respiratory pathogens in the healthy population. Second, considering that patients with severe illness were excluded from the study, the inclusion of patients with central nervous system (CNS) symptoms should have been removed from the study design. In the event, however, no patients with CNS symptoms were recruited to the study from any of the three enrolment sites, confirming that all patients with severe symptoms were excluded. Third, the studies in Brazil and Gabon lasted for less than 1 year; therefore, any seasonal effects on the causes of fever in these locations could not be fully observed. Fourth, while we anticipate that the epidemiology of AFIs in Brazil, Gabon and Malawi will be broadly representative, pathogens that are geographically focal, especially in Asia, will not be represented in the samples we collected. Fifth, no perfect method exists for classifying AFI cases into those of bacterial or non-bacterial aetiology. When all other factors were taken into consideration, the approach we adopted was the most appropriate for determining bacterial/non-bacterial cases. For example, we could have chosen to classify AFIs of bacterial origin as only those cases that were microbiologically confirmed. However, given the limited percentage of microbiologically confirmed cases obtained even in the most comprehensive aetiological studies,^{10 24} restricting the analysis to this subgroup would not have truly represented the intended-use population for which the new test was expected to be used. Sixth, in the Malawi study, fever at presentation was an inclusion criterion, and not history of fever, as patients were not likely to self-medicate with antipyretics, as was the case in Gabon and Brazil. However, as several fever-causing infections present with intermittent fever, we consider that history of fever in the last 7 days should be part of the systematic inclusion criteria of any such study. Finally, the overall classification process may have been refined and improved by adding additional tests and parameters or expanding the clinical panel. However, technical solutions, financial resources and local capacities were limited, and the project methodology was designed to make the best use of available resources.

Conclusion

This study and related activities (eg, systematic reviews, TPP development, technology reviews), along with the use of biomarkers to guide evidence-based decision making, were initiated 5 years ago as part of a concerted effort by the global health community to reduce the overuse of antibiotics and to curb the threat posed by antimicrobial resistance.¹⁴ Now, we have completed one of the most extensive studies ever designed to address this very specific challenge. Despite the study’s limitations, we believe that the approach adopted and the outputs achieved (from a standardised methodology to a sample collection), which have been made openly available, can move the dial on ambitious goals such as improving patient care and reducing the overuse of antibiotics worldwide. Therefore, BFF-Dx provides a positive exemplar for global collaborations that aim to improve healthcare for all in response to ongoing public health challenges.