The Short-Term Reliability and Long-Term Stability of Salivary Immune Markers

Abstract

Salivary markers of immune function are increasingly commonly used in studies of human health. Yet, few studies have examined the short-term or long-term reliability or stability of these biomarkers, making their measurement properties unclear. We addressed this issue in the present study by collecting two saliva samples, two hours apart, from 426 adolescent girls during a baseline laboratory visit. Then, eighteen months later, we collected the same samples again from a subset of these participants (n=113). The correlations between the two samples collected at each session were generally high (mean r=.67). In contrast, although single saliva samples were only weakly correlated across 18 month (mean rs=.18), averaging the two quantifications within a session considerably improved the reliability (mean r=.27). In short, salivary immune markers show strong short-term test-retest correlations, and averaging across multiple assessments notably improved long-term test-retest correlations. Future research is needed to establish the health relevance and mechanisms underlying these potentially useful, non-invasive biomarkers.

1. Introduction

Substantial research has shown that psychosocial factors can influence various components of the immune system (for reviews, see Dantzer and Kelley, 2007; Marsland et al., 2017; Slavich and Irwin, 2014). These findings have in turn prompted numerous biobehavioral researchers to assess immune function, often focusing on inflammatory biomarkers. Inflammatory biomarkers are frequently examined in blood, which is generally regarded as the gold standard method for assessing inflammatory activity. However, there are also drawbacks to this approach. For example, some individuals refuse to provide blood samples. Additionally, it can be difficult or impossible to collect blood samples in some study designs (e.g., at-home studies).

An increasingly common alternative to collecting blood involves measuring inflammatory biomarkers in saliva. Salivary inflammatory biomarkers are believed to reflect an interaction between systemic and local immune activity as well as oral hygiene (Slavish et al., 2015). Numerous studies have assessed salivary immune markers in recent years, and this work has suggested that these biomarkers may index important aspects of immune function, such as stress responsivity and biological resource redistribution (e.g., Shields et al., 2016; Slavish et al., 2015). However, few studies have examined the measurement characteristics of salivary inflammatory biomarkers. Therefore, their basic methodological characteristics and measurement properties remain relatively unknown. This is problematic because in order to have utility, a biomarker must be able to be assessed reliably.

Riis and colleagues (2014) examined the reliability of inflammatory biomarkers at three assessment points separated by one year each. They found that intercorrelations among salivary inflammatory biomarkers were high in a baseline sample, but correlations within each inflammatory biomarker from one timepoint to the next (i.e., over a one-year period) were often nonsignificant. This analysis represented a critical first-step in documenting the long-term (un)reliability of salivary inflammatory biomarkers, but because all samples were separated by one year, it remains unclear to what extent these results are due to measurement error versus changes in inflammation over time. Moreover, there are presently no guidelines for designing studies to optimize the reliability of salivary immune markers in psychoneuroimmunology research.

One strategy that could improve the long-term reliability of salivary inflammatory biomarker assays is averaging two or more samples (i.e., versus a single sample, per usual in studies of inflammatory biomarkers). Although most studies assay samples in duplicate, this is only a partial solution. Assaying in duplicate helps address measurement error introduced by the assay and technician; however, unlike taking an average of two independent samples, assaying in duplicate does not help address measurement error introduced by the sample taker, variability in storage procedures between participants (e.g., collection-to-freezing time), or rapid changes in the environment of the mouth.

Psychometric research has long established that single-item measures show poor reliability (Gliem and Gliem, 2003), and averaging two or more samples has been used to improve reliability in prior medical research (Jensen and McFarland, 1993). Moreover, with two samples, it is possible to utilize the Spearman-Brown prophecy formula—a statistical method that projects the reliability of a test if the number of items change—to determine the number of samples required to achieve a desired reliability for each analyte and calculate disattenuated correlations (i.e., correlations correcting for measurement error). Although averaging two samples collapses within-person variance, enhancing the temporal stability of these biomarkers may offer important applications, such as a better ability to predict the onset of depression or distinguish subtypes within a heterogeneous disorder, such as schizophrenia. To date, however, no study has examined whether creating a composite from multiple same-day samples can improve the long-term stability of salivary immune markers.

To address these issues, we recruited a large sample of adolescents and quantified their salivary levels of nine commonly investigated immune markers in two samples (separated by two hours) at both a baseline and follow-up assessment (18 months later). The biomarkers were chosen based on a comprehensive literature review of biobehavioral research and included tumor necrosis factor-α (TNF-α), interleukin (IL)-1β, IL-6, IL-8, IL-10, IL-18, IL-33, monocyte chemoattractant protein 1 (MCP-1), and C-reactive protein (CRP). First, we characterized both the short-term (same-session) reliability and long-term (longitudinal) test-retest correlations of the analytes. Second, we examined whether creating composite values from the two samples within each session improved the long-term test-retest correlations of the analytes. Finally, we calculated disattenuated correlations to determine the stability of these salivary immune markers over 18 months, correcting for measurement error.

2. Method

2.1. Participants

Participants were 426 adolescent girls who completed Wave 3 of the Adolescent Development of Emotions and Personality Traits (ADEPT) project (M_age=15.84 years; SD=0.63), ADEPT is a longitudinal study examining factors affecting female adolescent wellbeing and depression risk. Inclusion criteria for enrollment into ADEPT were English fluency, ability to read and comprehend questionnaires, age between 13.5–15.5 years old, and a biological parent consenting to participate in the study. Exclusion criteria were a lifetime history of a major depressive episode (MDE) or dysthymia, or intellectual disability. A diagnosis of an autoimmune disorder was not part of the exclusion criteria for this study, and approximately 2% (eight participants) reported being diagnosed by a doctor with an autoimmune disorder. Excluding these participants did not alter any reliability or stability outcome.

Salivary immune biomarker data were collected at ADEPT assessments Wave 3 and Wave 5; as such, Wave 3 will hereafter be referred to as “Baseline” and Wave 5 as “Follow-up”. Participants were predominately White (81.2%), followed by Hispanic (10.1%), Black (5.2%), Asian (2.6%), American Indian (0.2%), and other (0.7%). Of this cohort of 426 girls, 113 were also assayed at the follow-up assessment (i.e., Wave 5) 18 months later. The retention rate for the larger cohort was over 90%, but we assayed only a randomly sampled subset of these participants due to limited funding. Participants whose samples were randomly selected to be assayed did not differ from unselected participants with respect to age, BMI, parent years of education, race, or oral health at Follow-up, ps>.268.

2.2. Materials and Procedure

Participants provided two saliva samples—120min apart—at both the baseline assessment (i.e., Baseline Sample 1 and Baseline Sample 2) and follow-up assessment that occurred 18 months later (i.e., Follow-up Sample 1 and Follow-up Sample 2). Nearly all saliva samples were provided between 3pm and 8pm; participants who were unable to attend the study during that time at Baseline were assessed at approximately the same time during the Follow-up assessment if possible. After providing the first saliva sample (i.e., Sample 1), participants remained in the lab for 120min, during which time they completed unrelated measures—none of which were inherently stressful. Participants were not allowed to eat anything during this time. After 120min had elapsed, participants provided the second saliva sample (i.e., Sample 2). Saliva was collected via passive drool and immediately stored in a −80°C freezer until batch assayed at the UNC Cytokine and Biomarker Analysis Facility.

2.2.1. Assays.

Salivary levels of inflammatory biomarkers were determined using multiplex immunoassay kits purchased from R&D Systems (Minneapolis, MN) with a Bio-Plex 200 (Luminex) instrument. Assays were conducted following manufacturer instructions. The mean fluorescence intra-assay coefficient of variation (CV) was 2.99%, inter-assay CV was 10.27%, and the average percent of observed to expected values of known concentration was 99.7%. All values are given in pg/mL.

2.2.2. Oral health.

Participants completed an interview examining oral health/hygiene at each assessment (see Supplemental Material).

2.3. Data Analysis

Pearson correlations and Spearman-Browne reliabilities were used in analyses. Additional information on the analytic strategy is available in the Supplemental Material.

3. Results

3.1. Detection Rates

Detection rates were very good for nearly all analytes. Analytes with a detection rate below 80% in either baseline sample were IL-10 and IL-33. These detection rates were essentially equivalent or better at the follow-up assessment (see Table 1). Due to poor detection rates, we do not consider IL-10 and IL-33 further.

Table 1.

Descriptive Statistics, Mean Differences, and Correlations of Each Sample for Each Analyte at Each Assessment

	Sample 1				Sample 2				Analyses
	N	Detection Rate	Mean	SD	N	Detection Rate	Mean	SD	Cohen’s d	Paired N	r	Reliability of Sum
Baseline
TNF-α	417	97.9%	3.11	4.42	414	97.2%	3.37	3.37	0.13	391	.65	0.79
IL-1β	426	100.0%	137.26	174.72	416	97.7%	191.14	233.36	0.36	403	.70	0.82
IL-6	422	99.1%	4.18	8.76	413	96.9%	4.33	8.76	0.01	401	.72	0.84
IL-8	413	96.9%	528.55	401.61	389	91.3%	711.27	437.34	0.51	379	.51	0.67
IL-18	378	88.7%	7.74	12.95	385	90.4%	8.65	14.26	0.08	344	.55	0.71
CRP	405	95.1%	112.98	269.43	393	92.3%	128.58	308.95	0.02	381	.81	0.90
MCP-1	422	99.1%	374.17	724.25	415	97.4%	389.62	651.22	0.07	403	.64	0.78
Poor Detection
IL-10	159	37.3%	1.08	0.98	176	41.3%	1.13	1.07	0.06	97	.25	0.41
IL-33	247	58.0%	1.36	1.45	270	63.4%	1.30	1.41	−0.01	177	.45	0.62
Follow-up
TNF-α	113	100.0%	8.17	3.90	113	100.0%	8.42	4.39	−0.04	110	.54	0.70
IL-1β	113	100.0%	235.76	264.46	113	100.0%	280.48	297.45	0.17	108	.61	0.76
IL-6	108	95.6%	8.97	14.75	111	98.2%	9.00	18.69	−0.08	104	.66	0.79
IL-8	112	99.1%	792.49	601.28	109	96.5%	947.75	642.20	0.26	106	.39	0.56
IL-18	102	90.3%	14.13	17.89	105	92.9%	15.15	15.80	0.16	91	.59	0.74
CRP	113	100.0%	153.76	419.41	113	100.0%	230.37	842.64	0.03	111	.72	0.84
MCP-1	113	100.0%	288.29	270.43	113	100.0%	359.31	386.78	0.18	108	.49	0.66

Note: For analyses, values greater than three SDs ± the mean were removed. A positive Cohen’s d indicates greater values at Sample 2. Significant (p<.05) differences and/or correlations are indicated by boldface font.

3.2. Short-Term Reliability

Descriptive statistics, differences between means, and correlations between samples for each analyte (derived from the two samples taken two hours apart at each of the assessments) are presented in Table 1. Correlations between Baseline Sample 1 and Baseline Sample 2 were strong, rs> 50, ps< 001, Correlations between Follow-up Sample 1 and Follow-up Sample 2 were essentially equivalent, rs>.44, ps<001. At Baseline, the mean short-term reliability (i.e., the average test-retest correlation between each analyte’s Sample 1 and Sample 2) was r=.67; at Follow-up, the mean short-term reliability was r=.58. We used Spearman-Brown prophecy formula to estimate the number of samples per person needed to achieve reliability of .80 for each analyte, and it ranged from two to four (see Table 2).

Table 2.

Short-Term (Same Session) Estimated Reliability of Analyte by Number of Samples

Number of Samples	1	2	3	4
Variable	Reliability
TNF-α	.65	.79	.85	.88
IL-1β	.70	.82	.87	.90
IL-6	.72	.84	.88	.91
IL-8	.51	.67	.76	.81
IL-18	.55	.71	.79	.83
CRP	.81	.90	.93	.95
MCP	.64	.78	.84	.88

Note: Estimated short-term reliability calculated by the Spearman-Brown formula from same-session samples (i.e., separated by two hours). Reliabilities greater than .80 are bolded.

As for within-person changes, nearly all of the immune markers exhibited negligible within-person changes between Sample 1 and Sample 2 at both assessments (i.e., |d|s< 14). The primary exceptions were IL-1β and IL-8, which increased from Baseline Sample 1 to Baseline Sample 2 (d=.36 and d=.51, respectively) and to a lesser extent from Follow-up Sample 1 to Follow-up Sample 2 (d=.17 and d=.26, respectively). In sum, most of the salivary inflammatory biomarkers assessed showed significant within- and between-person reliability and negligible changes over a two-hour period.

3.3. Long-Term Test-Retest Correlations

Next, we examined the test-retest correlations of these salivary immune markers over 18 months. We first correlated individual samples—namely, Baseline Sample 1 with Follow-up Sample 1, and Baseline Sample 2 with Follow-up Sample 2. All of the correlations were positive, ranging from .04 to .32 (mean rs=.18 for both Baseline Sample 1 with Follow-up Sample 1 and Baseline Sample 2 with Follow-up Sample 2, ps>. 05), and five of the fourteen correlations were significant (see Table 3).

Table 3.

Correlations Within Analytes Between Each Assessment (i.e., Baseline and Follow-up)

Variable	Correlation Between Baseline and Follow-up
	Sample 1	Sample 2	Composite	Partial	Disattenuated
TNF-α	.07	.17†	.22*	.22*	.30**
IL-1β	.18†	.19*	.30**	.26**	.38***
IL-6	.04	.19†	.10	.12	.13
IL-8	.17†	.23*	.27**	.27**	.44***
IL-18	.18†	.11	.37***	.30**	.51***
CRP	.31***	.20*	.31***	.30**	.36***
MCP	.32***	.18†	.31**	.34***	.43***

Note:

^†p<.10,

^*p<.05,

^**p<.01,

^***p<.001.

Columns provide the correlation between Baseline and Follow-up for the given correlation (e.g., Baseline Sample 1 with Follow-up Sample 1). Composite represents the correlation between variables after averaging both samples at each assessment. Partial represents the correlation between the composite variables after covarying baseline and follow-up oral health as well as baseline and follow-up sample collection time. Disattenuated represents the correlation between the composite variables after correcting for measurement error.

3.4. Enhancing Long-Term Test-Retest Correlations and Stability

To attempt to improve the long-term test-retest correlations of these analytes, we created a composite that averaged the values of Sample 1 and Sample 2 for each analyte at both Baseline and Follow-up. Doing so greatly improved the test-retest correlations of these biomarkers (see Table 3), with correlations now ranging from .10 to .37 (mean r=.27, p=.004). Moreover, the long-term test-retest correlations of all analytes were significant except for IL-6. Therefore, using a composite score from two samples substantially increases the long-term test-retest correlations of salivary immune markers.

Because oral health/hygiene and sample collection time can both strongly influence salivary inflammatory biomarkers, we conducted analyses of the composite correlation (i.e., averaging Sample 1 and Sample 2 for use at both Baseline and Follow-up) controlling for oral health/hygiene at baseline and follow-up as well as sample collection time at both baseline and follow-up. As shown in Table 3, controlling for these covariates did little to influence this composite: the mean difference in magnitudes between these correlations was r=.01, and the largest difference was r=.07. Therefore, oral health/hygiene and time of sample did not exert strong influences on relative changes in these inflammatory biomarkers in our sample.

Finally, to estimate the 18-month stability of these markers, we corrected the long-term correlations for attenuation (i.e., measurement error). The disattenuated correlations ranged .13 to .51 (mean r=.37), indicating that the long-term stability of these salivary immune markers is moderate, on average, when correcting for measurement error (see Table 3).

Intraclass correlation coefficients (ICCs) are presented within the Supplemental Material.

4. Discussion

Despite growing interest in salivary immune markers, very little is known about their measurement properties. To address this issue, we assessed the short-term reliability of nine salivary markers of inflammation and the long-term stability of seven. Most of these salivary biomarkers (7/9) were highly detectable, with nearly all showing detectability rates of greater than 80%. On average, the short-term reliability of these markers was strong (i.e., mean r=.67). The long-term test-retest correlations were weaker, but here we show that taking the average of two samples substantially improves the long-term test-retest correlations of salivary immune markers. Inflammatory biomarkers showing the greatest long-term stability were IL-18, IL-8, and MCP; in contrast, IL-6 was very unstable.

One of the most interesting results obtained was that, despite similar short-term reliability, CRP showed relatively higher long-term stability than most salivary immune markers, whereas IL-6 showed relatively lower long-term stability. This difference in relative stability may thus suggest that salivary IL-6 is more sensitive than CRP to state-related factors, such as acute stress. Some research supports this idea, with acute stress effects on CRP being smaller than IL-6 (e.g., Marsland et al., 2017). Future research should examine this potential explanation and other potential reasons to better understand the relative instability of IL-6 relative to CRP.

Although we did not measure serum immune markers, concentrations of the measured salivary immune markers were generally in agreement with prior literature. This research has found that salivary concentrations of these markers are roughly equivalent with serum, with the exceptions of IL-1β and IL-8, which are higher in saliva than serum, and CRP, which is lower in saliva (Byrne et al., 2013; Riis et al., 2014). High salivary concentrations of IL-1β and IL-8 are thought to reflect the importance of neutrophils in oral health, as these cytokines attract and activate neutrophils (Riis et al., 2014); low CRP concentrations are due to the fact that this protein is primarily made in the liver and cannot easily pass into saliva (Byrne et al., 2013). It is also worth speculating that non-local production of CRP may be responsible for its relatively greater stability in this study.

Because we did not record the time each participant took to fill their vials, adjusting for flow rate was not possible. Crucially, however, salivary IL-1β (Salimetrics, 2017a), IL-6 (Izawa et al., 2013b), and CRP (Salimetrics, 2017b) are known to be independent of flow rate. To our knowledge, no study has examined whether salivary TNF-α, IL-8, IL-18, or MCP-1 are flow-rate dependent. Importantly, though, flow-rate dependency of these salivary inflammatory biomarkers would lower their reliabilities; because the reliabilities of these analytes was similar to the flow-rate independent analytes, we do not believe that flow-rate adjustment would have substantially altered the results.

More broadly, these results replicate the findings of Riis et al. (2014), who found that salivary inflammatory biomarkers exhibit small-to-moderate test-retest correlations over an 18-month period. They also extend these findings, though, by showing that using a composite of samples obtained at different times within an assessment produces much better long-term test-retest correlations than obtaining only one sample.

These results have several implications for research employing salivary immune markers. For example, they provide evidence that most of the salivary biomarkers being measured are highly detectable and reliably index immunological function. Most importantly, they also suggest that longitudinal studies utilizing salivary measures that are interested in examining changes or differences in, or the stability of, immune function over time would benefit from collecting multiple samples per session.

Several study limitations should be noted. First, the sample was young and female, and additional research is needed to examine the generalizability of these findings to other populations. Second, we examined the results of only one type of assay kit, and it is possible that different kits would yield different results. Third, we did not test the potential advantages of obtaining more than two samples per session or of shortening the interval between samples, both of which could further enhance the reliability estimates. Fourth, we did not assess these biomarkers in blood, so we could not compare the stability of these markers in saliva to blood. Finally, this study was not designed to identify factors or processes that could have influenced changes in immune function between the baseline and follow-up assessments, such as diet, sleep, stress, and health behaviors.

In conclusion, although serum-based immune markers have played a major role in psychoneuroimmunology and health research to date, salivary inflammatory biomarkers are becoming increasingly used due to their many advantages (e.g., less expensive and invasive, easier to obtain, etc.). Here, we show that despite existing criticism of this sampling technique, salivary immune markers are highly detectable and exhibit an average long-term stability of r=.37. Moreover, by using a composite of two samples, multiple salivary immune markers—namely, TNF-α, IL-Iβ, IL-8, IL-18, CRP, and MCP—demonstrate significant test-retest correlations over 18 months, therefore providing evidence of their suitability for use in studies assessing immune function.

Highlights.

• We examined the short-term reliability and long-term stability of salivary immune markers

• All of the analytes achieved excellent short-term reliability by averaging four samples

• Long-term stability of the analytes was substantially improved by averaging two or more samples

• Salivary immune markers may thus be more stable than previously thought