Integration of population-level data sources into an individual-level clinical prediction model for dengue virus test positivity

Abstract

The differentiation of dengue virus (DENV) infection, a major cause of acute febrile illness in tropical regions, from other etiologies, may help prioritize laboratory testing and limit the inappropriate use of antibiotics. While traditional clinical prediction models focus on individual patient-level parameters, we hypothesize that for infectious diseases, population-level data sources may improve predictive ability. To create a clinical prediction model that integrates patient-extrinsic data for identifying DENV among febrile patients presenting to a hospital in Thailand, we fit random forest classifiers combining clinical data with climate and population-level epidemiologic data. In cross validation, compared to a parsimonious model with the top clinical predictors, a model with the addition of climate data, reconstructed susceptibility estimates, force of infection estimates, and a recent case clustering metric, significantly improved model performance.

Introduction

Acute febrile illness (AFI) is a common reason for seeking healthcare in low- and middle-income countries (LMICs) (1). Determination of AFI etiology is often limited by diagnostic testing capacity, given the wide spectrum of potential infectious agents. Inappropriate use of testing and treatment resources may result in poor outcomes, such as the high case fatality rates seen in admitted AFI patients (5–20%) (2–7). Dengue virus (DENV) is a major cause of AFI in LMICs, accounting for an estimated 390 million infections, 96 million illnesses, 2 million severe cases, and 21,000 deaths per year (8). The differentiation between dengue and other common causes of febrile illness is important to avoid misdiagnosis, which can lead to delays in initiation of effective treatment, and inappropriate use of antibiotics (9). Due to the lack of pathognomonic clinical features that reliably distinguish dengue from other febrile illnesses, virological or serological laboratory confirmation is required for definitive diagnosis. While multiplexed tests that can quickly identify the causative pathogen are ideal, they are often unavailable in LMICs due to cost and insufficient laboratory infrastructure. Even rapid, point-of-care tests may be cost-prohibitive in LMICs (10). Accurate and cost-effective tools to better determine etiology of fever at the point-of-care are greatly needed to guide the use of diagnostics and therapeutics, conserving scarce healthcare resources.

Clinical Decision-Support Systems (CDSS) incorporating prediction models may offer a solution to better management of infectious diseases in low resource settings. CDSSs, such as applications on smartphone devices, can gather data from a range of online sources and implement sophisticated clinical prediction models that would be impractical for clinicians to calculate manually. CDSS have proven effective at improving therapeutic management and reducing unnecessary diagnostic tests in both high-income countries (HICs) (11) and LMICs (12–14). In Bangladesh, an electronic CDSS was shown to improve clinical dehydration assessment and WHO diarrhea guideline adherence, as well as reduce non-indicated antibiotic use in children under five by 29% (12). Traditional predictive models generally incorporate clinical information that is obtained solely from the presenting patient. Predictive models that incorporate additional information – such as seasonal or climate predictors, location-specific historical prevalence, characteristics of prior patients – have been shown to increase diagnostic accuracy and limit inappropriate antibiotic use (14–16).

The underlying probability of being infected by DENV varies by both space and time. The risk of DENV transmission depends on conditions that promote mosquito breeding, including when temperatures are warmer (17–19), and the risk of infection is influenced by local population immunity, as large outbreak years are typically followed by periods of low transmission (20–22). As most DENV transmission is highly focal, it means that population susceptibility profiles can be spatially heterogeneous at any time (21, 23–25). Thus, our objective is to develop an improved clinical prediction model for dengue by integrating temporal and spatial (location-specific) parameters including climate data, clustering of recent cases, and population susceptibility estimates derived from seroprevalence or hospital data in the surrounding community. We demonstrate the potential for integrating location- and population-specific data sources into clinical prediction models, with the potential to inform the development of improved tools to aid clinicians in diagnostic and therapeutic decision making for patients presenting with suspected dengue.

Methods

Location

Kamphaeng Phet is a province in north-central Thailand that is located 350 km north of Bangkok and has a population of 725,000 people in a mostly rural and semirural setting (26, 27). We used data collected from patients presenting to Kamphaeng Phet Provincial Hospital (KPPH), a large, tertiary care hospital in the province to identify clinical predictors that could discriminate between DENV-infected and uninfected patients (26, 27).

Hospital-based suspected dengue patient data

We used data on over 12,000 patients presenting to KPPH with suspected dengue between August 2007-December 2021. The data was collected by the United States Army Medical Directorate-Armed Forces Research Institute of Medical Sciences (USAMD-AFRIMS). As DENV testing in this hospital is provided free of charge and this is a highly DENV-endemic region, individuals will be tested for DENV infection if there is any suspicion of dengue, however minor. This provides an excellent test case to understand whether individual or location-specific risk factors are associated with testing positive for DENV.

For all suspected dengue cases, we used demographic and clinical information including patient age, sex, home village, admission diagnosis, date of admission, presenting symptoms, and DENV PCR status. The following signs and symptom were recorded as binary variables: fever, chills, malaise, rhinitis, rash, sore throat, seizure, cough, nuchal rigidity, eye pain, nausea, headaches, vomiting, joint pain, abnormal movements, anorexia, myalgias, diarrhea, dark urine, abdominal pain, and bleeding. DENV infection was evaluated using RT-PCR. We recorded the residence of each patient to the district (Amphoe) level using detailed base maps of the region.

Climate variables using National Oceanic and Atmospheric Administration (NOAA) data

Climate and seasonal factors such as temperature, precipitation, and humidity influence vector populations and DENV transmission (17–19, 28). We employed the R package GSODR to gather climate data from the central most NOAA weather station in the province of Kamphaeng Phet, Thailand, which included mean daily temperature, precipitation, dewpoint, relative humidity, sea level pressure, visibility, and windspeed. To better reflect seasonal trends, we aggregated data in 14-day increments prior to the day of the DENV infection prediction. As climate can alter vector feeding behavior (19, 29), we used aggregated climate predictors in the two weeks prior to case presentation. Additionally, climate in the months prior to outbreaks can influence both vector population dynamics as well as viral replication (19, 28). To determine the appropriate lag time for each climate variable, we constructed a random forest classifier with climate variables lagged at one, two, and three months. Using the R package, “vip”, we calculated each Variable of Importance by AUC and used the best performing lag time for each climate variable.

Estimates of temporal changes in population susceptibility using national surveillance system data

We estimate population susceptibility data using age-specific case data from the national surveillance system using data from Kamphaeng Phet province only. We note that most of the cases in this dataset are suspected DENV cases (i.e., without confirmatory testing). We have previously developed models to explicitly link underlying infection risks to the observed age distribution of cases by age and year to estimate annual age-specific force of infection in provinces of Thailand up until 2017 (30). The estimates can be used to reconstruct the buildup of immunity in populations by age. Here, we reconstruct population susceptibilities in Kamphaeng Phet going into each year, using only data prior to the year, to mimic the real-world use, where only prior years’ data is available. As dengue disease severity is greatest for secondary infections, we consider two alternative formulations to define susceptibility to disease. Firstly, we consider complete susceptibility, where we use the estimates of the proportion of individuals of an age group and year that are completely seronaive. Second, we consider the proportion of individuals of an age group and year that have experience one prior infection, and are therefore at risk of increased risk of severe disease.

Estimates of spatial differences in the underlying force of infection using seroprevalence data from a cohort study

To estimate underlying spatial differences in the force of infection in the province, we make use of a DENV cohort study in the region, where healthy individuals of all ages from throughout Kamphaeng Phet province have provided blood (31). The cohort is ongoing. We use data from samples collected during baseline blood draws, that occurred between 2015 and 2021. Hemagglutination inhibition assays were used to characterize immunity to the four DENV serotypes; individuals were considered seropositive if they had a titer of 10 or greater to any serotype. We have previously used this seroprevalence data to estimate the underlying mean force of infection, and the proportion of the population that are susceptible to DENV infection in different subdistricts in the province (32). Here, we use this subdistrict specific estimates to characterize underlying heterogeneity in the force of infection in the province. As the cohort data comes from 2015–2021, however, much of the hospital case data we are working with comes from prior to the cohort, we are assuming that the force of infection is stable in time within any location.

Spatial clustering of positive cases based on prior patients presenting to the hospital

The local clustering of positive cases from a single area, may signal local ongoing transmission. To assess for a temporal and spatial relationship between cases, we stratified cases that presented to KPP hospital by both district and province and then summed the number of positive cases in the 30 days prior to presentation divided by the total cases over the study period from that area.

Statistical Analysis and Modeling

We fit random forest classifiers to predict DENV infection. Random forests are a machine learning method which constructs a multitude of decision trees and averages over them to obtain a prediction robust to nonlinearities and interactions between covariates, and has been widely applied to biomedical sciences for both classification and regression (33, 34).

We initially identified the subset of clinical symptoms that were most informative of true infection status._To do this we fit random forest models using only clinical predictors and then used the R package “vip” to calculate the Variable of Importance by AUC for each clinical variable. We determined a variable’s importance by calculating the change in AUC after permuting, or randomly shuffling each predictor. To attempt to achieve the most parsimonious prediction rule (i.e., the best predictive model requiring the fewest variables to be input by clinicians), we fit random forest and logistic regression models using training data with consecutively increasing clinical predictor set sizes based on the order of importance and applied this to the test set to determine the smallest model with the best performance. Next, we incorporated the patient extrinsic factors. We fit each random forest classifier using 1000 decision trees and used the default number of variables to be randomly considered at each node split (mtry = square root of number of candidate variables). In the construction of our predictive models, we input climate predictors, age, susceptibility estimates, and the case clustering metric as continuous variables and we input the optimized clinical predictors as binary presence or absence categorical variables. Missing predictor data was imputed using the R package ‘RandomForest’.

We used logistic regression for each predictor to create a univariate comparison between DENV-positive and DENV-negative cases. We fit multiple logistic regression models to compare the performance of parsimonious models with a random forest classifier using the same number of predictors.

To assess predictive performance for both random forest and logistic regression models, we used repeated cross-validation using 80% training/20% testing splits with 100 iterations. No testing data was used when training the model. In each iteration, predictions on the test set were produced and corresponding measures of performance obtained. To determine overall model performance, we averaged the area under the receiver operator characteristic curve (AUC) and confidence intervals for the 100 iterations. To determine statistical significance between models we used a bootstrap method over 100 iterations, which involves resampling the data with replacement multiple times, creating bootstrap samples. For each bootstrap sample, receiver operating characteristic (ROC) curves were generated and the differences between the curves were computed. All analyses were completed using R version 4.2.0, and model development/validation was completed in accordance with the TRIPOD checklist (Supplement Table S1).

Ethical considerations

This study was approved by the institutional review boards of the Thai Ministry of Public Health and Walter Reed Army Institute of Research (WRAIR #2119), and the University of Utah (IRB_00150106)

Results

Of the 12,833 participants in the clinical data set, 5731 (45%) were confirmed to have DENV infection by PCR. DENV-positive patients were significantly younger (18 vs 22 years, p<0.001, Table 1). Nearly all cases (97.8%) came from the 11 districts within Kamphaeng Phet province (Table 1). There was no significant difference between the probability of testing positive for males and females (p=0.07); no other genders were reported. The probability of testing positive differed substantially by age, ranging from 26% for those < 4 years to 58% for those 15–19 years of age (Table 2). Patients between the ages of 10–14 years, 15–19 years, and 5–9 years comprised the largest proportion of cases (23%, 18%, 16% respectively) while older patients comprised a much smaller proportion of cases (30–34 years 5%, 35–39 years 4%).

Table 1:

Age, gender, and top discriminative symptoms by DENV positivity. Locations listed are the eleven provinces in Kamphaeng Phet.

	Overall, N = 12,8331	DENV Negative, N = 7,1021	DENV Positive, N = 5,7311	p-value2
Age (mean, sd)	21 (15)	22 (18)	18 (11)	<0.001
Female	6,401 (50)	3,491 (49)	2,910 (51)	0.068
Symptoms
Cough	4,741 (37)	3,057 (43)	1,684 (29)	<0.001
Nausea	6,227 (49)	3,051 (43)	3,176 (55)	<0.001
Fever	11,467 (89)	6,129 (86)	5,338 (93)	<0.001
Headache	9,146 (71)	4,797 (68)	4,349 (76)	<0.001
Rhinitis	2,165 (17)	1,455 (20)	710 (12)	<0.001
Pharyngitis	3,534 (28)	2,113 (30)	1,421 (25)	<0.001
Location
District				<0.001
Bueng Samakkhi	226 (1.8)	166 (2.3)	60 (1.0)
Khanu Woralaksaburi	910 (7.1)	522 (7.4)	388 (6.8)
Khlong Khlung	733 (5.7)	397 (5.6)	336 (5.9)
Khlong Lan	945 (7.4)	645 (9.1)	300 (5.2)
Kosamphi Nakhon	750 (5.8)	407 (5.7)	343 (6.0)
Lan Krabue	556 (4.3)	333 (4.7)	223 (3.9)
Mueang Kamphaeng Phet	5,780 (45)	2,910 (41)	2,870 (50)
Pang Sila Thong	571 (4.4)	324 (4.6)	247 (4.3)
Phran Kratai	1,186 (9.2)	684 (9.6)	502 (8.8)
Sai Ngam	609 (4.7)	363 (5.1)	246 (4.3)
Sai Thong Watthana	288 (2.2)	178 (2.5)	110 (1.9)
Province
Kamphaeng Phet	12,554 (97.8)	6,929 (97.5)	5,625 (98.2)

¹Mean (SD); n (%)

²Wilcoxon rank sum test; Pearson’s Chi-squared test

Table 2.

The AUCs and confidence intervals by base model, compared to base model plus inclusion of additional data sources. ‘Clinical’ indicates the inclusion of the top three clinical predictors, ‘Climate’ indicates the inclusion of climate predictors, ‘RS’ indicates the inclusion of reconstructed susceptibility estimates derived using national surveillance data, ‘FOI’ indicates the inclusion of force of infection estimates derived using cohort data, ‘Cluster’ indicates the recent case cluster metric.

Model	AUC (%)	95% CI
Clinical*Climate*RS*FoI*Cluster	70.0	67.9–71.9
Clinical*Climate*RS*Cluster	69.5	67.5–71.5
Clinical*Climate*FoI*Cluster	69.2	67.2–71.2
Clinical*Climate*Cluster	68.8	66.8–70.8
Clinical*Climate*RS*FoI	68.7	66.7–70.7
Clinical*Cluster	68.7	66.7–70.7
Clinical*FoI*Cluster	68.5	66.5–70.6
Clinical*Climate*RS	68.4	66.4–70.5
Clinical*RS*FoI*Cluster	68.4	66.4–70.4
Clinical*RS*Cluster	68.2	66.1–70.2
Clinical*Climate*FoI	68.1	66.1–70.1
Clinical*FoI	67.7	65.7–69.8
Clinical*RS*FoI	67.6	65.5–69.6
Climate*RS*FoI*Cluster	67.5	65.5–69.6
Clinical*RS	67.5	65.4–69.5
Clinical*Climate	67.2	65.2–69.3
Clinical	67.0	65–69.1
Climate*RS*Cluster	66.8	64.8–68.9
Climate*RS	65.7	63.6–67.8
RS*Cluster	65.7	63.6–67.7
RS	65.6	63.5–67.7
Climate*FoI*Cluster	64.7	62.6–66.8
Climate*Cluster	60.5	58.3–62.7
Climate	58.7	56.5–60.9
Cluster	56.4	54.2–58.6
FoI	57.0	54.8–59.2

We found that there were significant differences in the clinical symptoms between DENV positive and negative patients. Table 1 lists the top discriminative symptoms between the groups based on random forest and logistic regression. The most common symptom reported was fever, followed by headache. In univariate analysis, we found that individuals with fever, chills, malaise, retro-orbital pain, nausea, headache, and vomiting were significantly more likely to test positive for DENV, and individuals with cough, rhinitis, pharyngitis were significantly less likely to test positive for DENV (Supplementary Table S2).

When we examined the proportion of positive cases to total cases by year and month, we found that both total and positive cases significantly increased in the months between June and September (p<0.001). The proportion of positive cases differed substantially by year (p< 0.001), ranging from 19% in 2016 to 90% in 2017. The period of lowest test-positivity in 2016 and 2017, coincided with the Zika virus epidemic in the country (Figure 1).

Figure 1.

Dengue virus (DENV) cases at Kamphaeng Phet Provincial Hospital, Thailand, 2007–2021. The number of DENV cases (green) over total cases (blue) as proportion of AFI cases by year (A) and month (C) and the percentage of positive cases by year (B) and month (D) over the study period. A map of Kamphaeng Phet Province and its 11 districts. Colors indicate the number of positive cases (E) and the annual case rate per 100,000 persons (F) within each district between 2007–2021.

Derivation of a model using clinical parameters alone resulted in a parsimonious model that achieved moderate predictive performance.

We first assessed the performance of the model using a traditional clinical prediction model which only includes the presenting patient’s information. A random forest classifier using all 23 clinical features resulted in an average AUC of 69.5% (95%CI: 67.5–71.5) from repeated cross-validation. To determine the optimal number of variables for a parsimonious prediction model, we used a random forest classifier to analyze the improvement in model performance with each additional clinical variable included. Figure 2 shows the improvement in AUC with each additional variable using two random forest classifiers – one with all other predictors and the other using only clinical data – as well as a logistic regression model using only clinical variables. Performance levelled off with three clinical variables: age, cough, and nausea. Using a model with only these three predictors, we achieve an average AUC of 67.0% (95%CI: 65.0–69.1). Supplementary Table S3 shows the relative frequency of these variables by age group. We demonstrate the direction and magnitude of the effect of the top predictors by generating partial dependence plots from random forest and logistic regression classifiers (Supplementary Figure S1).

Figure 2.

Average AUC and 95% CIs from cross-validation (100 iterations) for Random Forest (RF) and Logistic Regression (LR) models. The red line indicates an RF model with all other predictors (climate, reconstructed susceptibilities estimates, force of infection estimates, prior patients) included. The green line indicates an RF model which includes only clinical predictors. The blue line indicates an LR model with only clinical predictors included. The dotted lines indicate CIs.

Addition of climate data to the clinical parameters model resulted in an improved area under the curve

Next, we fit models using climate data. To appropriately adjust lag time for each climate variable, we fit a random forest classifier using only climate variables and assessed the Variables of Importance by AUC. A random forest model with recent and lagged aggregated climate data without clinical predictors resulted in an AUC of 58.7% (95% CI: 56.5–60.9). We found the best performing climate variables were visibility, relative humidity, wind speed, and precipitation, all lagged by 3 months. For each climate predictor, Supplementary Table S4 lists the odds ratio and compares the mean of each predictor by DENV-positive or negative groups. Figure 3 shows the relationship between visibility, relative humidity, and the proportion of positive cases each month. When combined with the top three clinical variables, climate data performed similarly (median p = 0.60, 2% p-values <0.05) as clinical data alone. Table 2 shows the AUCs for the clinical base model, compared to the base model plus the inclusion of additional data sources.

Figure 3.

The monthly relative humidity (orange) and visibility (blue) in Thailand over the study period, compared with rates of DENV (green). For each case, we gathered the nearest NOAA weather station’s climate data, lagged by three months, and averaged that data for each month.

Addition of reconstructed susceptibility (RS) estimates to the clinical parameters model resulted in an improved area under the curve.

Using historical hospital case data from the province, we obtained estimates of the size of the susceptible population by age for each year (across all subdistricts in the province). In our predictive model we used the prior year’s RS estimates. Using logistic regression, we found secondary RS estimates performed better than primary RS estimates [60.7% (95%CI: 58.6–62.9) vs 52.3% (95%CI:50.1–54.6)]. When added to a random forest classifier with climate and/or clinical predictors, the inclusion of RS estimates consistently resulted in higher AUCs (Table 2). When added to the top 3 clinical parameters alone, RS estimates non-significantly improved model performance from an AUC of 67.0% (95%CI: 65.0–68.8) to an AUC of 67.5% (95%CI: 65.4–69.5), (median p=0.40, 9% p-values < .05). Finally, a model including all predictors resulted in higher AUCs than a model without RS (median p=0.09, 32% p-values < 0.05).

Addition of subdistrict-specific Force of Infection (FoI) estimates to the clinical parameters model resulted in an improved area under the curve.

We incorporated FoI estimates for each age by subdistrict using data from a local cohort study. This assumes that the underlying differences in the force of infection are constant in time. Using logistic regression, FoI estimates had an AUC of 57.0% (95%CI: 54.8–59.2). The inclusion of FoI estimates lead to increases in AUC when added to the top clinical predictors, when added to clinical predictors and climate data, and when added to clinical predictors, climate predictors, and RS estimates (Table 2). When included with all other predictors, a model with FoI estimates non-significantly improved performance compared to a model without FoI estimates (median p=0.30, 23% p-values < 0.05)

Addition of the case clustering metric to the clinical parameters model resulted in an improved area under the curve

Finally, we fit a model that assessed for clustering of recent cases based on prior patients presenting to the KPP hospital. Using logistic regression, we found the case clustering metric (the number of positive cases in the subdistrict over last 30 days divided by the total number of cases from that subdistrict in the study period) had an AUC of 56.4% (95%CI: 54.2–58.6). We found that the use of the case clustering metric consistently improved model performance. Stratifying by the finer spatial size of subdistrict consistently outperformed models with prior patients stratified by province. When added to the top performing clinical variables, model performance significantly improved (median p= 0.02, 60% of p-values < 0.05). When compared to a model with all predictors except cluster of recent cases, the inclusion of this predictor significantly improved model performance (median p= 0.007, 79% p-values <0.05).

Finally, when comparing a model including all predictors with a model including only the top clinical predictors model performance improved from an AUC of 67.0% (95%CI: 65.0–69.1) to an AUC of 70.0% [(95%CI: 67.9–71.9) (median p=0.006, 87% p-values < 0.05)].

Discussion

The management of AFI in LMICs often requires clinical decision making with limited availability of diagnostic testing. The differential diagnosis of AFI is broad and clinicians must decide on appropriate use of antibiotics as well as patient disposition. If diagnostics are available, clinicians must consider if the benefits of the information obtained outweighs the cost of the test. CDSSs can augment clinical decision making at minimal cost to the clinician and have proven effective at improving therapeutic management and reducing unnecessary diagnostic tests in LMIC settings (12–14). Historically, CDSSs use only clinical and demographic information from the presenting patient. Here, we present a predictive model for DENV infection that integrates multiple sources of information both intrinsic and extrinsic to the patient, including climate data, clinical data, seroprevalence-based susceptibility estimates, and historical information from prior patients, which results in improved predictive performance.

DENV transmission can exhibit significant temporal and geographical heterogeneity even at fine spatial scales, with variations observed even among neighboring villages (27, 35, 36). We thus used patient-extrinsic (location-specific) data sources in our models. Although modest, the improvement in model performance with finer spatial units suggests that population-level spatial heterogeneity exists at the district level and can be applied to individual-level clinical prediction. We expect further improvements in predictive performance if finer-scale location became routinely available for case data, such as to the community level. The improvement with the use of either the province or district level case clustering metric highlights the utility of temporal predictors in clinical prediction DENV models. We also show that reconstructed susceptibility estimates, which reflect the transmission dynamics of disease and the susceptible proportion of a population, improve individual level clinical prediction on their own. Given that reconstructed susceptibility estimates may be more difficult to obtain across different settings, we favor use of the other location-specific data sources. Moreover, reconstructed susceptibility estimates may not serve as a reliable indicator of protection against DENV, as they represent a mixed concept – immunity may reflect protection due to herd immunity or may indicate increased risk of dengue infection, as higher levels of immunity may reflect higher viral circulation of the multiple DENV serotypes with significant immunologic cross-reactivity.

Transmission of DENV occurs in a seasonal pattern, and several climate variables have been found to increase DENV transmission and/or vector populations (17–19, 28, 29). We found visibility and relative humidity 3 months prior to presentation to be the most important predictors of DENV infection in Kamphaeng Phet, Thailand. Our findings suggest that site-specific climate variables aid in site-specific models to predict DENV infection. Appropriate lag times would need to be tuned to different sites. For use in a clinical decision support tool, the most recent climate variables could be gathered from online weather sources, based on smartphone-based detection of GPS location. An optimal utilization of this model would be through a smartphone application, as there is a scarcity of electronic medical record availability in LMICs. This would necessitate access to a smart phone device and internet connection; however, clinicians and frontline healthcare workers increasingly have access to smartphone devices, even in remote areas of LMICs (37).

We found the use of clinical data alone provided moderate discrimination between DENV-positive and DENV-negative patients. There were significant differences between DENV-positive and -negative patients in 16 of the 22 clinical symptoms collected on presentation, consistent with features known to distinguish dengue from other illnesses (38, 39). To minimize clinician input requirements (40), we used random forest regression to identify the optimal variables to derive a parsimonious model. We were able to achieve near-optimal performance with only three clinical variables – age, nausea, and cough. It should be noted that the input of as little as one clinical variable – age – along with other predictors can provide useful clinical information (AUC 67.9%, 95%CI: 65.6–70.0), especially in cases where other symptoms cannot be easily obtained, such as in infants, and nonverbal or comatose patients.

Our study has several limitations. First, our model was constructed using data from a single center and testing was limited to patients suspected of having dengue infection, potentially hindering the model’s generalizability to a broader population. Similarly, as there was inherent heuristic bias in the patients selected for testing, the clinical components of the model reflect this specific population, meaning other important predictors of dengue infection, such as fever, were already included in the clinician’s decision making. Our results were limited to internal cross-validation; further studies for external validation are necessary. Finally, our assessment of the use of spatial dynamics in DENV transmission was limited as cases were only matched to each district rather than sub-district or village. In the future, models that integrate cases based on a finer spatial scale may better assess the role of a patient’s residing location in prediction. Despite these limitations, we demonstrate that predictive models that include patient-extrinsic location-specific elements can improve prediction and allow for parsimonious models that minimize clinician input and should be considered in future work on clinical prediction and decision support tools.

Figure 4.

The AUCs and confidence intervals by base model, compared to base model plus inclusion of additional data sources. ‘Clinical’ indicates the inclusion of the top three clinical predictors, ‘Climate’ indicates the inclusion of climate predictors, ‘RS’ indicates the inclusion of reconstructed susceptibility estimates derived using national surveillance data, ‘FOI’ indicates the inclusion of force of infection estimates derived using cohort data, ‘Cluster’ indicates the recent case cluster metric.