Abstract
Although clinical immunity to malaria eventually develops among children living in endemic settings, the underlying immunologic mechanisms are not known. The Vδ2+ subset of γδ T cells possess intrinsic reactivity to malaria antigens, can mediate killing of P. falciparum merozoites, and expand markedly in vivo following malaria infection in previously naïve hosts, but their role in mediating immunity in children repeatedly exposed to malaria is unclear. We evaluated γδ T cell responses to malaria among 4-year-old children enrolled in a longitudinal study in Uganda. We found that repeated malaria was associated with reduced percentages of Vδ2+ γδ T cells in peripheral blood, decreased proliferation and cytokine production in response to malaria antigens, and increased expression of immunoregulatory genes. Further, loss and dysfunction of pro-inflammatory Vδ2+ γδ T cells was associated with a reduced likelihood of symptoms upon subsequent P. falciparum infection. Together, these results suggest that repeated malaria infection during childhood results in progressive loss and dysfunction of Vδ2+ γδ T cells that may facilitate immunological tolerance of the parasite.
Introduction
Children living in endemic settings eventually develop “clinical immunity” to malaria, characterized by a decline in symptomatic malaria episodes and an increasing proportion of P. falciparum infections that are asymptomatic. However, the immunologic mechanisms underlying the acquisition of clinical immunity are not known. Because most individuals fail to develop true sterilizing immunity (i.e. protection against parasitemia) and remain vulnerable to asymptomatic parasitemia into adulthood (1), it has been suggested that clinical immunity may be due in part to immune tolerance of the parasite, rather than resulting entirely from an adaptive immune response that effectively protects against P. falciparum infection (2, 3).
Several studies have shown that a subset of γδ lymphocytes, Vγ9+Vδ2+ T cells (also known as Vγ2+Vδ2+), become rapidly activated upon stimulation with P. falciparum antigens during the blood stage of infection (4, 5). These Vδ2+ T cells react specifically to phosphoantigens produced by the Plasmodial apicoplast and mediate killing of the invasive blood stage merozoites via release of cytotoxic granules containing granulysin, a process that is TCR-dependent but does not require processing or presentation by professional APCs (6-8). Vδ2+ T cells have been shown to expand markedly in vivo following malaria infection in previously naïve hosts (9-11), reaching percentages of up to 30% of circulating T cells (5). Following in vitro stimulation with P. falciparum-infected red blood cells (iRBC), Vδ2+ T cells produce IFNγ and TNF and proliferate vigorously (4, 12-15). These findings suggest that Vδ2+ T cells may function as an innate-like immune response to malaria that may be important in control of P. falciparum infection.
Notably, most prior studies of antimalarial γδ T cell function have been performed using cells obtained from malaria-naïve adults and individuals from low-exposure settings; few studies have explored the role of Vδ2+ cells in the natural acquisition of immunity to malaria among children residing in highly endemic regions. Two small studies of individuals in malaria endemic settings found no evidence of elevated T cell frequencies or in vivo expansion following malaria infection (16, 17), raising the possibility that the antimalarial responsiveness of this population may be attenuated in the setting of chronic exposure. Further, although some have suggested that γδ T cells play a beneficial role in protection from malaria (18), other data suggest that production of pro-inflammatory cytokines by these cells may be implicated in the pathology of malaria (19, 20). To better understand the role of T cells in the development of immunity to malaria, we examined a cohort of children in rural eastern Uganda with heavy year-round exposure to P. falciparum. We hypothesized that chronic and repeated malaria exposure may lead to downregulation of the Vδ2+ T cell response. Here, we show that Vδ2+ cells decline in both number and function following repeated malaria exposure, and that loss of this pro-inflammatory response may be associated with clinical tolerance to malaria.
Results
Study cohort and clinical outcomes
The study cohort consisted of 78 HIV-uninfected children enrolled prior to one year of age. Complete clinical histories, including documentation of all symptomatic malaria episodes and frequent screening for asymptomatic parasitemia, were available through 5 years of age. The incidence of symptomatic malaria in this cohort was high but variable (5.4 episodes per person year (ppy), IQR 3.2-7.0) and peaked at 25 months of age with a subsequent gradual decline in malaria episodes and a corresponding increase in asymptomatic parasitemia (Fig. 1). For this study, blood was drawn from all subjects at four years of age, and again at five years in a longitudinal subset, enabling assessment of δ T cell correlates of both prior malaria exposure and prospective protection. One child had symptomatic malaria (parasitemia with a fever requiring treatment) at the time of the four-year blood draw, and 17 (22%) had blood smears demonstrating parasitemia.
Loss of Vδ2+ γδlow cells in setting of heavy prior malaria
We measured peripheral blood percentages of T cells from all subjects at four years of age, and observed two distinct populations of δ T cells (δhigh and δlow) corresponding to the two major subsets, Vδ1+ and Vδ2+(Fig. 2A-C and fig. S1). The majority of Vδ2+ T cells expressed the Vγ9 chain (median 95%). We noted a striking inverse association between percentages of Vδ2+ cells and the prior cumulative incidence of malaria (rs = −0.39, P=0.003, Fig. 2D). A similar relationship was observed between percentages of Vδ2+ cells and the total number of prior parasitemic months (rs = −0.41, P=0.002, n=55). However, there was no significant relationship between Vδ1+ T cell percentages (or total γδ T cells percentages) and the prior incidence of malaria (Fig. 2E, fig. S1). Despite prior studies describing rapid in vivo expansion and high circulating frequencies of Vδ2+cells following infection of malaria-naïve adults (9), we did not observe a significant relationship between Vδ2+ T cell percentages and concurrent parasitemia, nor duration since last malaria infection, among these chronically malaria-exposed children. Based on these results, longitudinal measurements were performed at 5 years of age in a subset of children with particularly high or low malaria exposure between 4 and 5 years of age, in order to test the hypothesis that repeated malaria episodes led to sustained and specific depletion of Vδ2+ T cells from peripheral blood. Vδ2+cell percentages declined significantly among children with ≥8 intercurrent episodes of symptomatic malaria ppy, in contrast to stable or rising Vδ2+cell percentages among children with 0-1 intercurrent malaria episodes ppy (median fold change 0.68 vs. 1.27, respectively, P=0.007, Fig. 2F and fig. S2). Together these findings suggest that heavy malaria exposure results in the loss of Vδ2+ γδ T cells from peripheral blood.
Dysfunction of Vδ2+ γδlow cells following repeated malaria infection
We next investigated the functional responsiveness of γδ T cells to malaria antigens and its relationship to prior malaria exposure. Prior studies have demonstrated robust proliferation and production of inflammatory cytokines in response to stimulation with malaria parasites among malaria-naïve individuals and those with low prior exposure (4, 9), suggestive of an innate-like immune response. We observed that in vitro stimulation with P. falciparum-infected red blood cells (iRBCs) resulted in robust, but variable, production of IFNγ and TNF, with a subset of these cells co-producing IL2, but no appreciable production of IL17 or IL10 (Fig 3a-b). Most γδ T cells responding to iRBC stimulation were of the γδlow subset corresponding to Vδ2+ cells (fig S1), and were predominantly CD45RA−, CCR7− “effector-memory”-like γδ T-cells (21) though expression of these memory markers did not significantly differ based on prior malaria exposure. We found that the functional responsiveness of Vδ2+ γδ T cells to malaria declined in parallel with the quantitative decline in Vδ2+ percentages. Malaria-specific production of inflammatory cytokines (IFNγ and TNFα) by δlow cells was inversely associated with increasing prior malaria incidence (rs = −0.41, P=0.0002, Fig. 3C). A similar relationship was observed between percentages of cytokine-producing γδlow cells and the total number of prior parasitemic months (rs = −0.43, P=0.0001, n=78). Concurrent parasitemia did not influence the percentage of IFNγ+ or TNF+ γδlow cells (P=0.69 and 0.89 respectively, n=78, Wilcoxon rank-sum). As an additional measure of Vδ2+ γδ T cell function, we assessed proliferation in response to malaria antigen stimulation in a CFSE dilution assay. Again, a strong inverse correlation was observed between Vδ2+ T cell proliferation and cumulative prior malaria incidence (Fig. 3D-E), suggesting that heavy malaria antigen exposure may result in a proliferative defect in Vδ2+ T cells. Together these results indicate that children who have survived repeated clinical malaria episodes exhibit dysfunction as well as loss of Vδ2+ cells.
Increased expression of immunoregulatory pathways in Vδ2+ T cells from heavily exposed children
Given the diminished malaria-specific responsiveness of Vδ2+ T cells from children with numerous prior malaria episodes, we hypothesized that repeated infection may lead to the upregulation of immunoregulatory pathways that dampen the innate Vδ2 inflammatory response, as a means of evading host immunopathology. To test this hypothesis, we compared basal gene expression patterns of sorted, unstimulated Vδ2+ T cells from children with high (≥8 ppy, n=4) and low (<2 ppy, n=4) prior malaria incidence using whole transcriptome analysis. We identified 48 differentially expressed genes (FDR ≤0.05, fold change ≥ 2, Fig. 4A), many with known roles in immunomodulation. For each of these genes, expression was higher among individuals with high prior exposure to malaria. These include: HAVCR2, which encodes the receptor Tim-3 (T cell immunoglobulin and mucin domain-containing 3), a cell surface marker that has been implicated in tolerance and exhaustion of Th1 cells (22-26); FCRL6, which encodes the ITIM-containing Fc receptor-like protein 6 (27); LYN, which negatively regulates BCR signaling through phosphorylation of ITIM-containing receptors (28); BATF, a transcription factor upregulated in exhausted CD8+ T cells (29); and B3GAT1, which encodes CD57, a cell surface protein which has been associated with replicative senescence of CD8+ T cells in the setting of chronic antigen exposure (30). Confirmatory analyses were performed by flow cytometry in children with high or low prior incidence, showing higher Tim3, FcRL6, CD57, and CD16 on Vδ2+ γδ T cells from children with numerous prior malaria infections (Fig. 4B and table S1).
We next compared induction of gene expression by sort-purified Vδ2+ T cells following 4 hour in vitro stimulation with malaria-infected red blood cells (iRBC). In children with <2 prior episodes ppy (n=4), 171 genes were upregulated following in vitro stimulation with iRBC (Fig 5a-b). In contrast, only 17 of these genes were upregulated in children with ≥8 episodes (n=3, Fig. 5A-B and table S2). Several of these differentially induced genes encode cytokines, including IFNG, IL2, IL3, IL4, CSF2 (encodes GM-CSF), and LTA (Fig. 5C). In children with ≥8 prior episodes ppy, mRNA expression of these cytokine-encoding genes was significantly lower following in vitro stimulation in comparison to children with <2 prior episodes ppy (Fig. 5D). Together, these data strongly suggest that chronic recurrent malaria infection leads to upregulation of immunoregulatory pathways that dampen the immune response.
Loss and dysfunction of Vδ2+ T cells associated with clinical immunity to malaria
Finally, we evaluated the relationship between the frequency and function of Vδ2+ T cells and protection from symptomatic malaria, as assessed during one year of prospective observation (age 4-5 years). Percentages of malaria-responsive γδlow T cells were significantly lower in children who experienced at least one episode of asymptomatic infection in the year of follow-up compared to children who developed fever with each subsequent malaria infection (P=0.008, Fig 6). While children with lower percentages of cytokine-producing γδlow T cells had a higher risk of parasitemia throughout the study (Fig 7a), these highly exposed children had a significantly lower probability of exhibiting symptoms if they became parasitemic during the year of follow-up (OR 0.27, P=0.036, Fig 7b). This reduced risk of symptoms remained significant after adjusting the analysis for prior or current malaria exposure (OR 0.14, 95% CI 0.04-0.5, P=0.0025). Furthermore, children in the lowest tertile of malaria-responsive γδlow T cell responses experienced a significant decline in their overall incidence of symptomatic malaria in the year of follow-up, whereas children in the highest tertile did not experience a decline in the incidence of symptomatic malaria (Fig 7c). Together, these findings suggest that T cells do not protect against P. falciparum infection, but that with progressive loss of malaria-responsive Vδ2+ γδlow T cells there is an increased likelihood that infections will be asymptomatic, consistent with the development of immunologic tolerance to malaria parasites.
Discussion
Our results demonstrate that, in contrast to the expansion of Vδ2+ cells following malaria infection of previously naïve individuals(9-11), repeated malaria exposure is associated with reduced percentages of Vδ2+ γδ T cells in peripheral blood, decreased proliferation and cytokine production in response to malaria antigens, and increased expression of immunoregulatory genes. Further, in this cohort, loss and dysfunction of pro-inflammatory Vδ2+ γδ T cells was associated with a reduced likelihood of symptoms upon subsequent P. falciparum infection. These findings support the hypothesis that the clinical immunity to malaria observed in young children living in endemic regions may not be mediated entirely by an adaptive immune response to the parasite, but rather, at least in part, by attenuation of the pro-inflammatory response of semi-innate γδ T cells resulting in tolerance of P. falciparum infection, a hypothesis forwarded in the early 1990s by Goodier et. al. (17).
Our results are consistent with several recent studies indicating that malaria infection results in attenuation of pro-inflammatory responses, as well as dysfunction and/or exhaustion of numerous immune cell subsets, including αβ T cells (31-35), B cells (31, 36), and myeloid cells (37). Portugal et. al. recently demonstrated a decrease in iRBC-stimulated expression of genes encoding pro-inflammatory cytokines and chemokines in children after their first malaria infection of the season (3, 19). Because these analyses were performed on bulk unfractionated PBMCs, the cellular subsets responsible for this blunted inflammatory response were not identified. We have previously shown, in the same cohort of children described here, that repeated malaria infection leads to a decline in the frequency of TNF-producing CD4+ T cells and an expansion of malaria-specific, autoregulating Th1 CD4+ cells that co-produce IFNγ along with IL-10, an anti-inflammatory cytokine(38). Frequencies of IL10-producing CD4+ T cells were higher in children with current or recent parasitemia(3, 38), but were not associated with prospective protection from malaria nor with a reduced likelihood of symptoms upon reinfection (38). Thus while several lines of evidence suggest that T cell production of pro-inflammatory cytokines may decrease with prior malaria exposure, these prior studies were not able to demonstrate a relationship with prospective clinical outcomes such as febrile malaria, as we have shown here for Vδ2+ T cells. In mice, malaria infection leads to upregulation of the T cell inhibitory receptors PD1 and LAG3 on T cells, resulting in T cell “exhaustion” which contributes to an inability to clear the infection (32, 34). Upregulation of PD1 and LAG-3 on CD4+ T cells has also been reported in children chronically exposed to malaria (31, 32). However, PD1 and LAG-3 were not differentially expressed by Vδ2+ T cells from high and low exposure children in our cohort. Rather, we found that Vδ2+ γδ T cells upregulate alternate immunomodulatory pathways, such as Tim-3. The fact that γδ T cells use distinct immunomodulatory pathways (compared to conventional αβ T cells) is perhaps not surprising given the semi-innate role of γδ T cells and their frequent co-expression of surface markers conventionally associated with NK cells. In light of our novel observation that loss and dysfunction of Vδ2+cells is associated with a reduced likelihood of symptoms upon subsequent P. falciparum infection, such pathways may represent important mechanisms by which repeated malaria infection leads to blunted inflammatory responses by Vδ2+ T cells.
As sampling for most assays was performed at a single time point, unmeasured confounders – including genetic, environmental, or other immunologic factors - limit our ability to infer causality from these observations. Nonetheless, the sustained and progressive decline in Vδ2+ T cells observed in children with numerous intercurrent malaria episodes, in contrast to stable to rising percentages in those with few or no episodes, strongly support our conclusion that repeated malaria exposure is associated with a loss of Vδ2+ γδ T cells. While the loss and dysfunction of Vδ2+ T cells was statistically associated with the development of asymptomatic infection, it remains to be shown that this cell subset is causally responsible for reduced symptomatology; many immune effector populations are regulated in concert, any of which may contribute to the clinical symptoms (or lack of symptoms) of malaria. Larger prospective studies incorporating longitudinal measurements, as well as careful prospective measures of malaria incidence and exposure, will be needed to further elucidate the relationship between P falciparum exposure, γδ T cell function, and clinical immunity to malaria.
Given the intrinsic reactivity of Vγ9+Vδ2+ T cells to P falciparum and their direct anti-merozoite effects in vitro (4-6, 8), it is likely that these cells may play a beneficial role as ready-made effectors during primary acute malaria infection of infants (39). However, our data suggest that they do not protect against reinfection or parasitemia, at least in the young children living in this high transmission setting. While the progressive loss and dysfunction of Vδ2+ γδ T cells is associated with reduced symptoms, it may conceivably interfere with effective clearance of the infection, allowing persistence of parasitemia and propagation of P. falciparum to the next host. These findings have important implications in understanding the immunopathogenesis of malaria in childhood, and suggest a novel mechanism of clinical immunity to repeated malaria infection in high incidence settings.
Materials and Methods
Ethical approval
Informed consent was obtained from the parent or guardian of all study participants. The study protocol was approved by the Uganda National Council of Science and Technology and the institutional review boards of the University of California, San Francisco, Makerere University and the Centers for Disease Control and Prevention.
Study site, participants, and clinical outcomes
Samples for this study were obtained from a subset of children enrolled in the Tororo Child Cohort (TCC) in Tororo, Uganda, a rural district in southeastern Uganda with an entomological inoculation rate (EIR) estimated at 310 infective bites per person year (40). Details of the larger TCC study and eligibility criteria used for enrollment have been previously described (41, 42). The subjects in this report only includes children who were born to HIV-uninfected mothers, further described in (43), and included all children who were still enrolled in the study at the time of blood sampling at 4 years of age (n=78.) Briefly, children were enrolled in infancy (median 5.5 months of age) from the postnatal clinic at Tororo District Hospital, and were eligible for enrollment if they lived within a 30 km radius of the study clinic and agreed to come to the study clinic for all medical care. Children were followed for all medical problems at a dedicated study clinic, with monthly assessments to perform routine blood smears and ensure compliance with study protocols. Children who presented with a documented fever (≥38.0°C) or history of fever in the previous 24 hours had blood obtained by finger prick for a thick smear. If the thick smear was positive for malaria parasites, the patient was diagnosed with malaria and given artemisinin-based combination therapy. At the time of their first episode of uncomplicated malaria, study participants were randomly assigned to receive open-label artemether-lumefantrine (AL) or dihydroartemisinin-piperaquine (DP). Study participants received the assigned treatment for all subsequent episodes of uncomplicated malaria. Both treatments were shown to be efficacious for treatment of malaria infection (41); treatment with DP was associated with a 15% less risk of malaria over the 5-year period of the study, due to a delayed time to recurrent malaria (42). Children with asymptomatic parasitemia were not provided antimalarial therapy, in accord with local guidelines.
Incident episodes of malaria were defined as all febrile episodes accompanied by any parasitemia requiring treatment, but not preceded by another treatment in the prior 14 days (43). The incidence of malaria was calculated as the number of episodes per person years (ppy) at risk, with prior cumulative incidence defined as the incidence ppy from study enrolment until the time of the blood draw. Asymptomatic parasitemia was defined as a positive routine blood smear in the absence of fever that was not followed by the diagnosis of malaria in the subsequent seven days. The number of parasitemic months was calculated from routine monthly blood smears, regardless of the presence or absence of symptoms. The conditional probability of developing malaria once parasitemic was calculated as a binary outcome (malaria or asymptomatic parasitemia, as defined above) assessed at monthly intervals, in accordance with published methods (44).
Sample collection and processing
At 4 years of age, 6-10 mls of whole blood was obtained from each subject in acid citrate dextrose tubes. In a subset of children (n=12) whole blood was also obtained at 5 years of age. Peripheral blood mononuclear cells (PBMCs) were isolated by density gradient centrifugation (Ficoll-Histopaque; GE Life Sciences). PBMCs were cryopreserved in liquid nitrogen and shipped to our laboratory in San Francisco for evaluation. Analysis of cell viability using Guava Viacount (Millipore) consistently demonstrated >80% viability after thaw.
Malaria antigens
Plasmodium falciparum blood-stage 3D7 parasites were grown by standard methods and harvested at 5-10% parasitemia. Red blood cells infected with mature asexual stages (iRBC) were purified magnetically, washed, and cryopreserved in glycerolyte prior to use as previously described(38). Uninfected RBC (uRBC) were used as controls. Schizont extracts (SE) were prepared by 3 freeze-thaw cycles of iRBC in liquid N2 for freezing and 37°C water bath for thawing, then resuspended in R10 media and stored at −20°C until use.
Intracellular Cytokine Staining
Thawed PBMC were rested overnight in 10% fetal bovine serum (Gibco) and counted prior to stimulation with uRBC, iRBC, media, or phorbol miristate acetate/calcium ionophore (PMA/Io) at 1×106 cells/condition. An E:T ratio of 1:3 was used with uRBC and iRBC. Anti-CD28 and – CD49d were added for costimulation (0.5 μg/ml, BD Pharmingen). Brefeldin-A and Monensin (BD Pharmingen) were added at 6 hours of incubation at a final concentration of 10 μg/ml to inhibit cytokine secretion. At 24 hours of incubation, cells were washed, fixed and permeabilized per standard protocols (Invitrogen/Caltag).
Surface and/or intracellular staining of PBMC was done with standard protocols using the following antibodies for the primary analysis: Brilliant violet 650-conjugated CD4, APC anti-γδ (clone B1), Brilliant violet 421-conjugated anti-IL-2 (Biolegend), PerCP–conjugated anti-CD3, APC-H7-conjugated CD8, PE-Cy7-conjugated IFNγ PE-conjugated anti-IL-10, and FITC-conjugated TNFα (BD Pharmingen). Alexa 700-conjugated CD14 and CD19 and Live/dead aqua amine (Invitrogen) were included as exclusion gates to reduce unwanted nonspecific antibody binding when measuring antigen-specific T cell populations. Additional experiments utilized PE-conjugated Vδ2, FITC-conjugated Vδ1, Brilliant violet 605-conjugated CD45RA, FITC-conjugated CCR7, PE-conjugated FCRL6, FITC-conjugated CD57, Pacific blue-conjugated CD16 (Biolegend), and PE-conjugated Tim-3 (R&D).
CFSE Proliferation assay
Thawed PBMC were rested for one hour, washed in 10% Human AB media (Gemini), and 3-6 × 106 PBMC were labeled with 1ml of 1.25μM 5,6-carboxyfluorescein diacetate succinimidyl ester (CFSE; Molecular Probes) for seven minutes. CFSE-labeled PBMC were incubated in 96-well, deep-well culture plates (Nunc) at a density of 106 PBMC per well at a final volume of 1ml for 7 days. Antigens tested included media, phytohemagglutinin (PHA; 5 μg/mL; Sigma-Aldrich), uRBC, or PfSE at an E:T ratio of 1:3 schizont equivalents. At day 7 cells were treated with 100 units DNase I (Invitrogen) in culture medium at 37°C for 10 min, washed, and stained with surface antibodies (PerCP–conjugated anti-CD3, APC-H7-conjugated CD8 (BD Pharmingen), Brilliant violet 650-conjugated CD4, Alexa 700-conjugated CD14 and CD19, PE-conjugated anti-Vδ2, and APC-conjugated anti-γδ (Biolegend) before acquisition.
Cell Sorting
Thawed PBMC were rested overnight and left unstimulated or stimulated with iRBC for 4 hours. Cells were promptly stained for cell surface markers, and ~1 × 104 to 3 × 104 cells (>99% pure) were double-sorted directly into RNA lysis buffer (Ambion) with a FACSAria (BD) and stored at −80 until RNA amplification.
Flow Cytometry Data Analysis
Flow cytometry profiles were gated on CD3+ lymphocytes, and 200,000 to 300,000 events were collected (Figure S2). Samples were analyzed on an LSR2 three laser flow cytometer (Becton Dickinson) with FACSDiva software. Color compensations were performed for each patient’s PBMC using beads or samples single stained for each of the fluorochromes used. Data were analyzed using FlowJo (Tree Star) and Pestle (version 1.7)/SPICE (version 5.3; M. Roederer, Vaccine Research Center, National Institute of Allergy and Infectious Diseases, National Institutes of Health). Percentages of iRBC-stimulated cytokine producing γδ-T cells (alone or in combination) are reported after background subtraction of the frequency of the identically gated population of cells from the same sample stimulated with control (uRBC). In experiments with CFSE-labeled cells, the percentage of divided γδ+ T cells following PfSE stimulation was calculated and reported after subtraction of the percentage of divided γδ+ T cells following uRBC-stimulation, using the FlowJo Proliferation Platform.
Whole transcriptome microarray analysis
RNA was isolated using an RNAqueous Micro kit (Life Technologies) and amplified in two rounds using the Aminoallyl MessageAmp II kit (Life Technologies). Amplified RNA was covalently coupled to Cy3and hybridized to SurePrint G3 Human Gene Expression 8×60K v2 gene expression microarrays (Agilent) according to the manufacturer’s protocol. Arrays were scanned on an Agilent microarray scanner, and raw signal intensities were extracted with Agilent Feature Extraction software.
Raw intensities for 58,717 features represented on the microarray were log2 transformed and quantile normalized using the R package limma (45). 38,511 unexpressed probes were removed from the dataset, with expression being defined as a probe having a log2 normalized intensity of greater than 7 for all samples within at least one sample group. Expression values for each probe were centered to the median across samples in order to enhance between-sample differences when visualized as heat maps (relative yellow to blue). Differentially expressed genes were identified using Significance Analysis of Microarrays (SAM). For comparisons of unstimulated samples between low and high prior incidence groups, two-class unpaired comparisons were performed. For comparisons of iRBC-stimulated vs. unstimulated samples, paired comparisons were performed. The fold change following stimulation was calculated as the difference in the means of log2 gene expression between stimulated and unstimulated samples. A 5% false discovery rate (FDR) and 2-fold change cutoff was observed to identify differentially expressed genes (46).
Statistical Methods
All statistical analyses were performed using Prism 4.0 (GraphPad), STATA version 12 (College Station), SPICE v.5.3 (NIAID), or R version 3.0.2. We first examined associations between immunologic outcomes with exposure variables including prior cumulative incidence, concurrent parasitemia at the time of the assay, and antimalarial treatment allocation. Comparisons of cellular percentages between groups were performed using the Wilcoxon rank-sum and/or t test, and the Wilcoxon signed-rank and/or paired t test was used to compare paired data. Associations between continuous variables were assessed using Spearman’s rank correlation (rs). Concurrent parasitemia and antimalarial treatment assignment did not significantly alter the immunologic assessments reported in this study.
Associations between the absence or presence of any asymptomatic infection during the year of follow-up and percentages of cytokine-producing γδ T cells and were made using the Wilcoxon rank sum test, and were conditional on the presence of any parasitemia during this period. 44/47 children with asymptomatic infection during follow-up also had at least one symptomatic infection. Generalized additive models with smoothing splines were used to visualize the effects of age and γδ T cell responses on the monthly risk of parasitemia, probability of symptoms if parasitemic, and incidence of malaria (47). Associations between the highest and lowest tertiles of γδ T cell responses and the monthly risk of parasitemia, probability of symptoms if parasitemic, and incidence of malaria, stratified by year of age, were evaluated using generalized estimating equations with robust standard errors accounting for repeated measures in the same patient (48). Multivariate models assessing the probability of symptoms if infected in the year following the assay included antimalarial treatment allocation, presence or absence of concurrent parasitemia, and prior malaria exposure, defined as total months with parasite infection. Two-sided p-values were calculated for all test statistics and P < 0.05 was considered significant.
Supplementary Material
Acknowledgements
We are grateful to all the parents and guardians for kindly giving their consent and to the study participants for their cooperation. We thank all the members of the study team for their tireless effort and excellent work. We also thank P. Rosenthal, J.M. McCune, and M. Boyle for technical support and valuable discussions.
Funding:
Support for this work was provided by the Centers for Disease Control and Prevention (Cooperative Agreement No U62P024421); NIH/NIAID R01AI093615 (MEF), UCSF Centers for AIDS Research (Supplement to MEF, P30AI027763), NIH/NIAID U19AI089674 (GD), NIH/NIAID K23 AI100949 (PJ), Doris Duke Charitable Foundation (clinical scientist development award to BG), and Burroughs Wellcome Fund/American Society of Tropical Medicine and Hygiene (PJ). The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
Footnotes
Fig. S1. Gating strategy and relationship of γδ T cell subsets with prior malaria.
Fig. S2. Longitudinal measurements of Vδ2+ γδ T cells at 4 and 5 years of age in a subset of children with high or low malaria exposure.
Table S1. Flow cytometric expression of immunoregulatory proteins identified by whole transcriptome analysis.
Table S2. Microarray-based gene expression fold change analysis of in vitro stimulated purified Vδ2+ T cells.
Author Contributions: P.J. designed, performed, and analyzed all experiments and drafted the paper. C.C.K. designed and analyzed microarray experiments listed in Figs. 5 and 6 and edited the manuscript. B.G. performed regression modeling associated with Figs. 1 and 7, provided statistical guidance for all figures, and edited the manuscript. F.N. coordinated, collected and processed patient samples for all experiments. K.B. performed experiments listed in Figs. 2 and 3. I.E.J. performed experiments listed in Figs. 2 and 3. M.K.M. provided patient care and collected patient data. E.A. enrolled patients, provided patient care, and collected patient data. J.W.T. developed, funded, and coordinated the clinical study. M.R.K. developed, funded, and coordinated the clinical study. G.D. developed, funded, and coordinated the clinical study, and evaluated the statistical analyses reported in figures 1 and 7. M.E.F. designed and coordinated the project, evaluated the data, provided funding for all experiments, and prepared the manuscript. All authors reviewed and approved the manuscript.
Competing Interests: The authors report no competing interests.
Data and materials availability: The microarray data have been deposited in the Gene Expression Omnibus under accession number GSE55843.
References and Notes
- 1.Tran TM, Li S, Doumbo S, Doumtabe D, Huang CY, Dia S, Bathily A, Sangala J, Kone Y, Traore A, Niangaly M, Dara C, Kayentao K, Ongoiba A, Doumbo OK, Traore B, Crompton PD. An intensive longitudinal cohort study of Malian children and adults reveals no evidence of acquired immunity to Plasmodium falciparum infection. Clin. Infect. Dis. 2013;57:40–47. doi: 10.1093/cid/cit174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schofield L, Grau GE. Immunological processes in malaria pathogenesis. Nat Rev Immunol. 2005;5:722–735. doi: 10.1038/nri1686. [DOI] [PubMed] [Google Scholar]
- 3.Portugal S, Moebius J, Skinner J, Doumbo S, Doumtabe D, Kone Y, Dia S, Kanakabandi K, Sturdevant DE, Virtaneva K, Porcella SF, Li S, Doumbo OK, Kayentao K, Ongoiba A, Traore B, Crompton PD. Exposure-dependent control of malaria-induced inflammation in children. PLoS Pathog. 2014;10:e1004079. doi: 10.1371/journal.ppat.1004079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Behr C, Dubois P. Preferential expansion of V gamma 9 V delta 2 T cells following stimulation of peripheral blood lymphocytes with extracts of Plasmodium falciparum. Int. Immunol. 1992;4:361–366. doi: 10.1093/intimm/4.3.361. [DOI] [PubMed] [Google Scholar]
- 5.Ho M, Webster HK, Tongtawe P, Pattanapanyasat K, Weidanz WP. Increased gamma delta T cells in acute Plasmodium falciparum malaria. Immunol. Lett. 1990;25:139–141. doi: 10.1016/0165-2478(90)90105-y. [DOI] [PubMed] [Google Scholar]
- 6.Behr C, Poupot R, Peyrat MA, Poquet Y, Constant P, Dubois P, Bonneville M, Fournie JJ. Plasmodium falciparum stimuli for human gammadelta T cells are related to phosphorylated antigens of mycobacteria. Infect Immun. 1996;64:2892–2896. doi: 10.1128/iai.64.8.2892-2896.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Elloso MM, van der Heyde HC, vande Waa JA, Manning DD, Weidanz WP. Inhibition of Plasmodium falciparum in vitro by human gamma delta T cells. J Immunol. 1994;153:1187–1194. [PubMed] [Google Scholar]
- 8.Costa G, Loizon S, Guenot M, Mocan I, Halary F, de Saint-Basile G, Pitard V, Dechanet-Merville J, Moreau JF, Troye-Blomberg M, Mercereau-Puijalon O, Behr C. Control of Plasmodium falciparum erythrocytic cycle: gammadelta T cells target the red blood cell-invasive merozoites. Blood. 2011;118:6952–6962. doi: 10.1182/blood-2011-08-376111. [DOI] [PubMed] [Google Scholar]
- 9.Teirlinck AC, McCall MB, Roestenberg M, Scholzen A, Woestenenk R, de Mast Q, van der Ven AJ, Hermsen CC, Luty AJ, Sauerwein RW. Longevity and composition of cellular immune responses following experimental Plasmodium falciparum malaria infection in humans. PLoS Pathog. 2011;7:e1002389. doi: 10.1371/journal.ppat.1002389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Seder RA, Chang LJ, Enama ME, Zephir KL, Sarwar UN, Gordon IJ, Holman LA, James ER, Billingsley PF, Gunasekera A, Richman A, Chakravarty S, Manoj A, Velmurugan S, Li M, Ruben AJ, Li T, Eappen AG, Stafford RE, Plummer SH, Hendel CS, Novik L, Costner PJ, Mendoza FH, Saunders JG, Nason MC, Richardson JH, Murphy J, Davidson SA, Richie TL, Sedegah M, Sutamihardja A, Fahle GA, Lyke KE, Laurens MB, Roederer M, Tewari K, Epstein JE, Sim BK, Ledgerwood JE, Graham BS, Hoffman SL. Protection against malaria by intravenous immunization with a nonreplicating sporozoite vaccine. Science. 2013;341:1359–1365. doi: 10.1126/science.1241800. [DOI] [PubMed] [Google Scholar]
- 11.Roussilhon C, Agrapart M, Ballet JJ, Bensussan A. T lymphocytes bearing the gamma delta T cell receptor in patients with acute Plasmodium falciparum malaria. J Infect Dis. 1990;162:283–285. doi: 10.1093/infdis/162.1.283-a. [DOI] [PubMed] [Google Scholar]
- 12.Jones SM, Goodier MR, Langhorne J. The response of gamma delta T cells to Plasmodium falciparum is dependent on activated CD4+ T cells and the recognition of MHC class I molecules. Immunology. 1996;89:405–412. doi: 10.1046/j.1365-2567.1996.d01-762.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.D’Ombrain MC, Hansen DS, Simpson KM, Schofield L. gammadelta-T cells expressing NK receptors predominate over NK cells and conventional T cells in the innate IFN-gamma response to Plasmodium falciparum malaria. Eur J Immunol. 2007;37:1864–1873. doi: 10.1002/eji.200636889. [DOI] [PubMed] [Google Scholar]
- 14.Goodier MR, Lundqvist C, Hammarstrom ML, Troye-Blomberg M, Langhorne J. Cytokine profiles for human V gamma 9+ T cells stimulated by Plasmodium falciparum. Parasite Immunol. 1995;17:413–423. doi: 10.1111/j.1365-3024.1995.tb00909.x. [DOI] [PubMed] [Google Scholar]
- 15.Ribot JC, Debarros A, Mancio-Silva L, Pamplona A, Silva-Santos B. B7-CD28 costimulatory signals control the survival and proliferation of murine and human gammadelta T cells via IL-2 production. J Immunol. 2012;189:1202–1208. doi: 10.4049/jimmunol.1200268. [DOI] [PubMed] [Google Scholar]
- 16.Hviid L, Kurtzhals JA, Dodoo D, Rodrigues O, Ronn A, Commey JO, Nkrumah FK, Theander TG. The gamma/delta T-cell response to Plasmodium falciparum malaria in a population in which malaria is endemic. Infect Immun. 1996;64:4359–4362. doi: 10.1128/iai.64.10.4359-4362.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Goodier M, Krause-Jauer M, Sanni A, Massougbodji A, Sadeler BC, Mitchell GH, Modolell M, Eichmann K, Langhorne J. Gamma delta T cells in the peripheral blood of individuals from an area of holoendemic Plasmodium falciparum transmission. Trans R Soc Trop Med Hyg. 1993;87:692–696. doi: 10.1016/0035-9203(93)90299-6. [DOI] [PubMed] [Google Scholar]
- 18.D’Ombrain MC, Robinson LJ, Stanisic DI, Taraika J, Bernard N, Michon P, Mueller I, Schofield L. Association of early interferon-gamma production with immunity to clinical malaria: a longitudinal study among Papua New Guinean children. Clin. Infect. Dis. 2008;47:1380–1387. doi: 10.1086/592971. [DOI] [PubMed] [Google Scholar]
- 19.Stanisic DI, Cutts J, Eriksson E, Fowkes FJ, Rosanas-Urgell A, Siba P, Laman M, Davis TM, Manning L, Mueller I, Schofield L. gammadelta T cells and CD14+ Monocytes Are Predominant Cellular Sources of Cytokines and Chemokines Associated With Severe Malaria. J Infect Dis. 2014 doi: 10.1093/infdis/jiu083. [DOI] [PubMed] [Google Scholar]
- 20.Langhorne J, Ndungu FM, Sponaas AM, Marsh K. Immunity to malaria: more questions than answers. Nat Immunol. 2008;9:725–732. doi: 10.1038/ni.f.205. [DOI] [PubMed] [Google Scholar]
- 21.Qin G, Liu Y, Zheng J, Xiang Z, Ng IH, Malik Peiris JS, Lau YL, Tu W. Phenotypic and functional characterization of human gammadelta T-cell subsets in response to influenza A viruses. J Infect Dis. 2012;205:1646–1653. doi: 10.1093/infdis/jis253. [DOI] [PubMed] [Google Scholar]
- 22.Sabatos CA, Chakravarti S, Cha E, Schubart A, Sanchez-Fueyo A, Zheng XX, Coyle AJ, Strom TB, Freeman GJ, Kuchroo VK. Interaction of Tim-3 and Tim-3 ligand regulates T helper type 1 responses and induction of peripheral tolerance. Nat Immunol. 2003;4:1102–1110. doi: 10.1038/ni988. [DOI] [PubMed] [Google Scholar]
- 23.Sanchez-Fueyo A, Tian J, Picarella D, Domenig C, Zheng XX, Sabatos CA, Manlongat N, Bender O, Kamradt T, Kuchroo VK, Gutierrez-Ramos JC, Coyle AJ, Strom TB. Tim-3 inhibits T helper type 1-mediated auto- and alloimmune responses and promotes immunological tolerance. Nat Immunol. 2003;4:1093–1101. doi: 10.1038/ni987. [DOI] [PubMed] [Google Scholar]
- 24.Golden-Mason L, Palmer BE, Kassam N, Townshend-Bulson L, Livingston S, McMahon BJ, Castelblanco N, Kuchroo V, Gretch DR, Rosen HR. Negative immune regulator Tim-3 is overexpressed on T cells in hepatitis C virus infection and its blockade rescues dysfunctional CD4+ and CD8+ T cells. J Virol. 2009;83:9122–9130. doi: 10.1128/JVI.00639-09. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.McMahan RH, Golden-Mason L, Nishimura MI, McMahon BJ, Kemper M, Allen TM, Gretch DR, Rosen HR. Tim-3 expression on PD-1+ HCV-specific human CTLs is associated with viral persistence, and its blockade restores hepatocyte-directed in vitro cytotoxicity. J Clin Invest. 2010;120:4546–4557. doi: 10.1172/JCI43127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jones RB, Ndhlovu LC, Barbour JD, Sheth PM, Jha AR, Long BR, Wong JC, Satkunarajah M, Schweneker M, Chapman JM, Gyenes G, Vali B, Hyrcza MD, Yue FY, Kovacs C, Sassi A, Loutfy M, Halpenny R, Persad D, Spotts G, Hecht FM, Chun TW, McCune JM, Kaul R, Rini JM, Nixon DF, Ostrowski MA. Tim-3 expression defines a novel population of dysfunctional T cells with highly elevated frequencies in progressive HIV-1 infection. J Exp Med. 2008;205:2763–2779. doi: 10.1084/jem.20081398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wilson TJ, Presti RM, Tassi I, Overton ET, Cella M, Colonna M. FcRL6, a new ITIM-bearing receptor on cytolytic cells, is broadly expressed by lymphocytes following HIV-1 infection. Blood. 2007;109:3786–3793. doi: 10.1182/blood-2006-06-030023. [DOI] [PubMed] [Google Scholar]
- 28.Xu Y, Harder KW, Huntington ND, Hibbs ML, Tarlinton DM. Lyn tyrosine kinase: accentuating the positive and the negative. Immunity. 2005;22:9–18. doi: 10.1016/j.immuni.2004.12.004. [DOI] [PubMed] [Google Scholar]
- 29.Quigley M, Pereyra F, Nilsson B, Porichis F, Fonseca C, Eichbaum Q, Julg B, Jesneck JL, Brosnahan K, Imam S, Russell K, Toth I, Piechocka-Trocha A, Dolfi D, Angelosanto J, Crawford A, Shin H, Kwon DS, Zupkosky J, Francisco L, Freeman GJ, Wherry EJ, Kaufmann DE, Walker BD, Ebert B, Haining WN. Transcriptional analysis of HIV-specific CD8+ T cells shows that PD-1 inhibits T cell function by upregulating BATF. Nat Med. 2010;16:1147–1151. doi: 10.1038/nm.2232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Brenchley JM, Karandikar NJ, Betts MR, Ambrozak DR, Hill BJ, Crotty LE, Casazza JP, Kuruppu J, Migueles SA, Connors M, Roederer M, Douek DC, Koup RA. Expression of CD57 defines replicative senescence and antigen-induced apoptotic death of CD8+ T cells. Blood. 2003;101:2711–2720. doi: 10.1182/blood-2002-07-2103. [DOI] [PubMed] [Google Scholar]
- 31.Illingworth J, Butler NS, Roetynck S, Mwacharo J, Pierce SK, Bejon P, Crompton PD, Marsh K, Ndungu FM. Chronic exposure to Plasmodium falciparum is associated with phenotypic evidence of B and T cell exhaustion. J Immunol. 2013;190:1038–1047. doi: 10.4049/jimmunol.1202438. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Butler NS, Moebius J, Pewe LL, Traore B, Doumbo OK, Tygrett LT, Waldschmidt TJ, Crompton PD, Harty JT. Therapeutic blockade of PD-L1 and LAG-3 rapidly clears established blood-stage Plasmodium infection. Nat Immunol. 2012;13:188–195. doi: 10.1038/ni.2180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hafalla JC, Claser C, Couper KN, Grau GE, Renia L, de Souza JB, Riley EM. The CTLA-4 and PD-1/PD-L1 inhibitory pathways independently regulate host resistance to Plasmodium-induced acute immune pathology. PLoS Pathog. 2012;8:e1002504. doi: 10.1371/journal.ppat.1002504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Horne-Debets JM, Faleiro R, Karunarathne DS, Liu XQ, Lineburg KE, Poh CM, Grotenbreg GM, Hill GR, MacDonald KP, Good MF, Renia L, Ahmed R, Sharpe AH, Wykes MN. PD-1 dependent exhaustion of CD8+ T cells drives chronic malaria. Cell reports. 2013;5:1204–1213. doi: 10.1016/j.celrep.2013.11.002. [DOI] [PubMed] [Google Scholar]
- 35.Chandele A, Mukerjee P, Das G, Ahmed R, Chauhan VS. Phenotypic and functional profiling of malaria-induced CD8 and CD4 T cells during blood-stage infection with Plasmodium yoelii. Immunology. 2011;132:273–286. doi: 10.1111/j.1365-2567.2010.03363.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Weiss GE, Crompton PD, Li S, Walsh LA, Moir S, Traore B, Kayentao K, Ongoiba A, Doumbo OK, Pierce SK. Atypical memory B cells are greatly expanded in individuals living in a malaria-endemic area. J Immunol. 2009;183:2176–2182. doi: 10.4049/jimmunol.0901297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Pinzon-Charry A, Woodberry T, Kienzle V, McPhun V, Minigo G, Lampah DA, Kenangalem E, Engwerda C, Lopez JA, Anstey NM, Good MF. Apoptosis and dysfunction of blood dendritic cells in patients with falciparum and vivax malaria. J Exp Med. 2013;210:1635–1646. doi: 10.1084/jem.20121972. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Jagannathan P, Eccles-James I, Bowen K, Nankya F, Auma A, Wamala S, Ebusu C, Muhindo MK, Arinaitwe E, Briggs J, Greenhouse B, Tappero JW, Kamya MR, Dorsey G, Feeney ME. IFNgamma/IL-10 Co-producing Cells Dominate the CD4 Response to Malaria in Highly Exposed Children. PLoS Pathog. 2014;10:e1003864. doi: 10.1371/journal.ppat.1003864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Cairo C, Longinaker N, Cappelli G, Leke RG, Ondo MM, Djokam R, Fogako J, Leke RJ, Sagnia B, Sosso S, Colizzi V, Pauza CD. Cord Blood Vgamma2Vdelta2 T Cells Provide a Molecular Marker for the Influence of Pregnancy-Associated Malaria on Neonatal Immunity. J Infect Dis. 2014 doi: 10.1093/infdis/jit802. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Kilama M, Smith DL, Hutchinson R, Kigozi R, Yeka A, Lavoy G, Kamya MR, Staedke SG, Donnelly MJ, Drakeley C, Greenhouse B, Dorsey G, Lindsay SW. Estimating the annual entomological inoculation rate for Plasmodium falciparum transmitted by Anopheles gambiae s.l. using three sampling methods in three sites in Uganda. Malar J. 2014;13:111. doi: 10.1186/1475-2875-13-111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Arinaitwe E, Sandison TG, Wanzira H, Kakuru A, Homsy J, Kalamya J, Kamya MR, Vora N, Greenhouse B, Rosenthal PJ, Tappero J, Dorsey G. Artemether-lumefantrine versus dihydroartemisinin-piperaquine for falciparum malaria: a longitudinal, randomized trial in young Ugandan children. Clin Infect Dis. 2009;49:1629–1637. doi: 10.1086/647946. [DOI] [PubMed] [Google Scholar]
- 42.Wanzira H, Kakuru A, Arinaitwe E, Bigira V, Muhindo MK, Conrad M, Rosenthal PJ, Kamya MR, Tappero JW, Dorsey G. Longitudinal Outcomes in a Cohort of Ugandan Children Randomized to Artemether-lumefantrine Versus Dihydroartemisinin-piperaquine for the Treatment of Malaria. Clin. Infect. Dis. 2014 doi: 10.1093/cid/ciu353. [DOI] [PubMed] [Google Scholar]
- 43.Jagannathan P, Muhindo MK, Kakuru A, Arinaitwe E, Greenhouse B, Tappero J, Rosenthal PJ, Kaharuza F, Kamya MR, Dorsey G. Increasing incidence of malaria in children despite insecticide-treated bed nets and prompt anti-malarial therapy in Tororo, Uganda. Malar J. 2012;11:435. doi: 10.1186/1475-2875-11-435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Greenhouse B, Ho B, Hubbard A, Njama-Meya D, Narum DL, Lanar DE, Dutta S, Rosenthal PJ, Dorsey G, John CC. Antibodies to Plasmodium falciparum antigens predict a higher risk of malaria but protection from symptoms once parasitemic. J Infect Dis. 2011;204:19–26. doi: 10.1093/infdis/jir223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Smyth GK, Speed T. Normalization of cDNA microarray data. Methods. 2003;31:265–273. doi: 10.1016/s1046-2023(03)00155-5. [DOI] [PubMed] [Google Scholar]
- 46.Tusher VG, Tibshirani R, Chu G. Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci U S A. 2001;98:5116–5121. doi: 10.1073/pnas.091062498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Hastie T, Tibshirani R. Monographs on statistics and applied probability 43. ed. 1st Chapman and Hall; London; New York: 1990. Generalized additive models; p. xv.p. 335. [Google Scholar]
- 48.Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42:121–130. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.