Test-Retest Reliability and Stability of the Nicotine Metabolite Ratio Among Treatment-Seeking Smokers

Abstract

Introduction:

The nicotine metabolite ratio (NMR), the ratio of 3-hydroxycotinine to cotinine, is a biomarker used in smoking cessation research, with several retrospective studies suggesting that NMR predicts treatment outcome. To be maximally useful in tailoring treatment, estimates of NMR should be stable over time. The present study is the first to examine the short-term test-retest reliability of NMR among treatment-seeking smokers.

Methods:

Blood NMR was assessed at two time points, approximately 2–3 weeks apart and prior to intervention, among 72 healthy adult smokers (49% female; 35% non-White) enrolled in a cessation trial (http://ClinicalTrials.gov ID: NCT01314001).

Results:

Mean NMR was stable from Time-1 to Time-2, with no significant change between assessments; test-retest reliability for NMR values was excellent (ICC[2,1] = 0.87). Test-retest reliability remained acceptable to high when NMR was categorized, as in recent clinical trials. Classification of participants as slow (quartile 1, NMR ≤ 0.24) or normal/fast NMR (quartiles 2–4, NMR ≥ 0.25) was consistent from Time-1 to Time-2 for 96% of participants (κ = 0.89). Though classification of participants into NMR quartiles was less consistent from Time-1 to Time-2 (67% agreement; weighted κ = 0.73), all reclassifications occurred between adjacent quartiles.

Conclusions:

Overall, these data support the use of a single NMR assessment for association studies with smoking phenotypes and in smokers seeking to quit, and they encourage large-scale efforts to determine optimal NMR cutpoints for tailoring treatment selection.

Introduction

Smoking remains the leading cause of preventable death in the United States, resulting in more than 480 000 deaths each year.¹ Though there are several effective cessation therapies,² all give average 6-month abstinence rates below 40%.³ Identification of robust moderators of response to specific treatments may improve abstinence rates by using the moderator to select the best treatment for a particular smoker. Biomarkers are receiving increasing attention in this respect, and the nicotine metabolite ratio (NMR) is emerging as a leading candidate for predicting cessation and personalizing treatment.⁴

The NMR, the ratio of 3-hydroxycotinine to cotinine, reflects individual differences in the speed with which nicotine is cleared from the body. Nicotine is predominantly metabolized to cotinine by CYP2A6⁵; CYP2A6 is also exclusively responsible for metabolizing cotinine to 3-hydroxycotinine.^5,6 CYP2A6 is encoded by the highly polymorphic CYP2A6 gene. Although CYP2A6 genotype is strongly associated with the rate of nicotine clearance and smoking behaviors,^7–10 NMR has the additional advantages over CYP2A6 genotype of accounting for other genetic, environmental (eg, diet) and demographic (eg, sex, race) correlates of nicotine metabolism,^11–15 although many of these appear to represent minor sources of variation.¹³

Clinical, epidemiological, and laboratory studies have often divided NMR values into quartiles^16–19 or dichotomized them as “slow” versus “normal/fast metabolizers” (usually defined as 1st vs. 2nd–4th quartile, respectively).¹⁶ Results suggest that relative to faster metabolizers (ie, those with higher NMR) slow metabolizers have higher quit rates on placebo and NRT.^16–19 In contrast, treatment outcomes with bupropion, which is not a substrate of CYP2A6, are less affected by nicotine metabolism.^18,21 Thus previous studies associating NMR retrospectively with cessation outcomes provide support for the use, and prospective testing, of NMR as a biomarker for personalized cessation treatments.

The NMR is not affected by the time of day of sampling, nor by storage at room temperature.^22,23 Additionally, the biomarker has been determined to be relatively stable among ad libitum smokers, with a reliability coefficient of 0.70 for repeated measurements over a 44-week period.^22–24 While these data are promising and support the use of a single NMR measurement, previous stability studies were exclusively from nontreatment-seeking smokers and focused on NMR as a continuous measure. It is important to document the reliability of NMR in treatment-seeking smokers, as psychometric properties can shift across contexts and samples.^25,26 For example, it is important to determine whether NMR, which is related to smoking rate,²⁷ is reliable among smokers who are actively cutting down as they prepare to quit.^28–30 It is also important to evaluate the reliability of the NMR categories typical of clinical research (quartiles and slow/fast classification). Although categorization often attenuates reliability,³¹ no prior work has reported the reliability of NMR categories. The primary objective of the current study was to establish the short-term test-retest reliability of continuous and categorical NMR in a treatment-seeking sample.

Methods

Participants

Participants were 72 adult regular smokers (≥10 cigarettes per day [CPD]) enrolled in a multisite randomized clinical trial for smoking cessation (http://ClinicalTrials.gov Identifier: NCT01314001). Participants were 49% female, 35% racial/ethnic minority, with a mean age of 47 years, reflecting the demographics of the treatment seekers screened by NMR for this trial.¹³ Detailed exclusion and inclusion criteria for the trial are described elsewhere.²⁷

Procedures and Apparatus

All procedures were approved by Institutional Review Boards at the University at Buffalo, the State University of New York, the University of Pennsylvania, the MD Anderson Cancer Center/University of Texas, and the Centre for Addiction and Mental Health and University of Toronto. Study procedures (described in Schnoll et al.²⁷) were completed during the prequit period of a multisite smoking cessation clinical trial assessing the efficacy of blood NMR in prospectively predicting cessation outcomes on placebo, transdermal nicotine, or varenicline (Chantix).

A blood sample (10mL) was collected at intake (Time-1), frozen and shipped to the University of Toronto, for assay by liquid chromatography-tandem mass spectrometry.²³ Those participants deemed eligible, based on their NMR at intake, attended a prequit session 2–3 weeks later during which blood samples for the Time-2 NMR assessment were obtained.

For this sub-study of the clinical trial, 123 participants provided Time-1 blood samples. Of these, 92 were deemed NMR-eligible and provided a Time-2 blood sample at the prequit visit. Because the clinical trial oversampled slow metabolizers, as decided a priori, we selected an equal number of participants from each of the Time-1 NMR quartile ranges from subjects screened to date in the trial (n = 868; Schnoll et al.²⁷; quartile ranges were Q1 = 0.01–0.24, Q2 = 0.25–0.35, Q3 = 0.36–0.48, Q4 = 0.49–1.39). Quartile 4 had the lowest number of participants (n = 18); thus 18 participants were selected for assessment of Time-2 NMR from each of the other quartiles, matching for sex and race to the composition of those screened for the trial²⁷ because females and Caucasians have faster metabolism rates.^13,14 NMR quartiles were generally comparable on basic demographic and baseline smoking characteristics (Supplementary Table 1).

Data Analysis

Stability (mean change across assessments) and test-retest reliability of NMR values were estimated within an intraclass correlation (ICC; specifically ICC[2,1]) framework applied to Assessment × Subjects random-effects analysis of variance^25,32,33 conducted in SPSS (IBM; Version 21) reliability procedure. We measured test-retest reliability for data categorized as “slow” (quartile 1) versus “normal” (quartiles 2–4) by the simple Cohen’s kappa statistic and categorized into quartiles by the weighted kappa with Cicchetti-Allison weights.³⁴ We calculated kappa statistics in SAS Proc Freq (SAS Version 9.3; SAS Institute, Inc., Cary, NC).

Results

Mean NMR was stable over time (means [SEs] = 0 .37 [0.02] and 0.36 [0.03]), time F(1,71) = 0.08, P = .79. As illustrated in Figure 1, the test-retest reliability of raw NMR values was excellent (ICC[2,1] = 0.87; 95% CI = 0.80%–0.92%). The standard error of measurement (SEM = sqrt[MSE]) was 0.077, indicating that the true NMR for a given smoker is 95% likely to fall within ±0.15 (±1.96 × SEM) of the NMR obtained from a single plasma sample.

Figure 1.

Scatterplot of test-retest nicotine metabolite ratio data.

Reliability was also high for NMR quartile assignment (weighted κ = 0.73; 95% CI = 0.64%–0.83%), although only 67% of participants had the same quartile membership at both time points (Table 1, A and B). Test-retest reliability of “slow” (quartile 1) versus “normal/fast” metabolizers (quartiles 2–4) was excellent and comparable to that observed for raw NMR values and NMR quartile assignment (κ = 0.89; 95% CI = 0.77%–1.00%), with consistent classification as “slow” versus “normal/fast” across assessments for 96% of the sample (Table 1, A and B).

Table 1.

Classification of NMR According to Population-Derived Quartile Membership (Panel A; r = 0.87) and Slow vs. Normal/Fast NMR Classification (Population-Derived Quartile 1 vs. Quartiles 2–4; Panel B, r = 0.89)

	NMR quartile at Time-1
	Q1	Q2	Q3	Q4
Panel A
NMR quartile at Time-2
Q1	17	2
Q2	1	11	7
Q3		5	7	5
Q4			4	13

	NMR at Time-1
	Slow (Q1)	Normal/fast (Q2–Q4)
Panel B
NMR at Time-2
Slow (Q1)	17	2
Normal/fast (Q2–Q4)	1	52

NMR = nicotine metabolite ratio; Q = quartile. Q1 = 0.01–0.24; Q2 = 0.25–0.35; Q3 = 0.36–0.48; Q4 = 0.49–1.39.

Supplementary Analyses

Although the high reliability of NMR in the present sample suggests changes in smoking behavior do not attenuate overall NMR reliability, we explored NMR reliability in participant sub-groups based on changes in smoking rates obtained with timeline follow-back interviews at each visit (data were missing for one participant, n = 71). Overall, there was no change in smoking rate from intake to prequit (means[SDs] = 18.1[6.8] and 17.7[6.4] CPD, respectively), F < 1. However, 23% of the sample reduced their smoking at least 2 CPD (mean[SD] change = −4.8[2.8]), and 17% increased their smoking rate at least 2 CPD (mean[SD] change = 4.3[2.8]). Test-retest reliability of the NMR was excellent in all three subsamples, ICCs(2,1) = 0.81, 0.88, and 0.94 for the CPD reducers, CPD increasers, and stable CPD participants, respectively (similar results were obtained with other cutpoints). Moreover, CPD change scores were not significantly correlated with NMR change scores, r = −0.17, P = .15.

In addition to self-reported changes in smoking rate, cotinine is a common biomarker of smoking exposure. Cotinine was stable from intake to prequit in the overall sample (means[SDs] = 254[115] and 254[100] ng/mL, respectively), time F < 1 and ICC(2,1) = 0.79. Although CPD change scores were modestly correlated with (residualized) cotinine change scores, r = 0.33, P = .005 in the sample overall, the CPD reducer, CPD increaser, and stable CPD sub-groups did not differ in cotinine change, CPD Group × Time F < 1 (mean[SD] intake and prequit COT = 205[92] and 194[74], 258[115] and 261[100], and 303[124] and 311[91], respectively). Therefore, we also examined NMR reliability as a function of change in cotinine from intake to prequit: the test-retest reliability of NMR was excellent in participants with stable cotinine (57%), participants with ≥30ng/mL decreases in cotinine (18% of sample), and participants with cotinine increases of ≥30ng/mL (25% of sample; ICCs[2,1] = 0.90, 0.85, and 0.86; results were similar with other cotinine cutpoints).

Though raw NMR has been used in clinical studies, log-transformed NMR is common in the broader literature,^{15,17,24,27,35} as it can normalize the positively skewed NMR distribution. In the present sample, test-retest reliability of log-NMR was comparable to that observed for raw NMR (ICC[2,1] = 0.91, 95% CI = 0.86%–0.94%).

Discussion

The results of the present study indicate that the NMR is stable and exhibits excellent short-term test-retest reliability. These findings are consistent with several recent studies of ad libitum smokers^23,24,36; however, the present study makes an important extension of earlier work by providing the first NMR reliability data from treatment-seeking smokers. The time frame in which test-retest analyses were conducted parallels the time frame of a planned quit attempt in the real world (ie, 2–4 weeks).

The rationale for examining NMR reliability in smokers approaching a quit attempt was twofold. First, reliability and validity of any measure cannot be assumed to generalize from one context or population to another.^25,26 For example, prequit changes in smoking behavior, such as reduction in smoking rate or exposure in some participants,^28,30 could reduce the reliability of the NMR. Although supplementary tests on sub-groups with prequit changes in smoking rate or cotinine uniformly suggested strong test-retest reliability of NMR, conclusions from these tests are tempered by limited sample size and magnitude of CPD/cotinine changes observed in the current study of treatment-seeking smokers.

Second, because categorization can attenuate reliability,³¹ it was important to evaluate the reliability of typical NMR categories. In the present study, both approaches to categorizing NMR (ie, quartiles and dichotomizing Q1 vs. Q2–Q4) revealed high kappa coefficients. The reproducibility of NMR quartile assignment (67% agreement) was weaker compared to the dichotomous classification as slow (quartile 1) versus normal/fast NMR (quartiles 2–4), which was stable for 96% of the sample. The increased risk for misclassification with the quartile approach may be a function of having a greater number of cutpoints, including cutpoints in the densest part of the NMR distribution. As a result, there are simply more people near (ie, within the SEM, or 0.077 NMR units) a cutpoint in the quartile approach compared to the slow versus normal/fast classification (Figure 1).

Overall, these results suggest that both continuous NMR values and the classification of slow versus normal fast NMR have strong short-term reproducibility in treatment-seeking smokers. These findings bode well for large-scale evaluation of the NMR-treatment response function and the feasibility of integrating NMR into clinical practice.

Supplementary Material

Supplementary Table 1 can be found online at http://www.ntr.oxfordjournals.org

Funding

The study (http://ClinicalTrials.gov identifier: NCT01314001) was supported by NIH grant U01 DA020830 from the National Institute on Drug Abuse, the National Cancer Institute on General Medical Sciences, and the National Human Genome Research Institute, awarded to CL and RFT and Canadian Institutes of Health Research grants MOP86471 and TMH-109787 to RFT.

Declaration of Interests

Pfizer provided the study medication and placebo for this study free of charge but had no role in the current analyses or manuscript preparation. MCM has served on the Speaker’s Bureau for Pfizer and as the medical director of the New York State Smokers Quit Line. RFT has served as a consultant to pharmaceutical companies that make smoking cessation products. The remaining authors declare no potential competing interests.