Validating Use of Internet-Submitted Carbon Monoxide Values by Video to Determine Quit Status

Abstract

Introduction

Daily visits to biochemically verify continuous smoking abstinence via expired-air carbon monoxide (CO) may deter participation in cessation trials. One way to reduce need for daily visits while continuing to monitor abstinence success may be use of a recent procedure to verify abstinence from daily CO values via the Internet. This method requires participants submit to study staff video recordings of themselves correctly using a CO monitor. However, it has not been clearly demonstrated that those classified quit via Internet-submitted videos of CO would be reliably classified quit when assessed in lab.

Methods

Our study examined agreement in quit status from Internet-submitted CO values with quit status via CO collected in later same-day lab visits. Participants (n = 23) were from a short-term cessation study who agreed to record and submit videos of offsite CO testing, in addition to attending daily lab visits. All CO values were obtained via Bedfont pico+ Smokerlyzer monitors, with CO < 8 ppm indicating quit. During two 4-day practice quit attempts, a video was submitted before daily lab visits, up to eight videos each.

Results

Of the total of 150 videos submitted, 97 videos indicated “not quit” and 53 “quit.” Cohen’s Kappa indicated substantial agreement in quit status between assessments, 0.70, p < .001, as 85% of the videos indicating “quit” CO were also “quit” CO in lab.

Conclusions

To our knowledge, these results are the first validation of daily Internet-submitted CO values to confirm daily quit status, supporting the utility of this approach for close monitoring of continuous abstinence.

Implications

This study compared consistency between quit status from CO values submitted over the Internet and quit status via CO collected in later same-day lab visits. Findings indicate substantial agreement in quit status between these two methods of CO assessment. Our results validate the use of Internet-submitted CO values to verify daily quit status. This method can be used in future cessation trials as a means to biochemically validate continuous abstinence without the burden of daily lab visits or relying on self-report of recent smoking lapses.

Introduction

With a half-life of just 4 hours, expired-air carbon monoxide (CO) is the recommended method to biochemically validate recent (eg, 12-hour) smoking abstinence, as compared to salivary cotinine, with a 16-hour half-life allowing validation of abstinence only if longer than 3 days.^1,2 CO and cotinine may therefore be differentially appropriate for validating abstinence in smoking cessation trials, depending on the timing of assessments and duration of verified abstinence, either point prevalence or continuous.² Point prevalence abstinence measures the proportion of participants who are quit as of a fixed point in time, often assessed intermittently over months post-quit attempt, during and after active treatment is provided.³ However, point prevalence may not be sensitive to occasional smoking lapses between abstinence assessment points, raising concerns that those validated as “quit” at each of the scheduled time points may not have remained completely abstinent throughout the period between assessments.² Substantial research indicates that lapses after initiating a quit attempt predict eventual relapse back to regular smoking (ie, failure of the quit attempt).^4–6 For this reason, trials reporting rates of “continuous abstinence” as indicated by validated quit CO values across all assessment points usually rely also on participant self-reports of no smoking at all, as brief lapses are very difficult to detect from intermittent biochemical verification.^7,8

Therefore, the only way to biochemically confirm continuous abstinence from smoking without relying on self-report of recent smoking, which can be very unreliable,^9,10 is via daily assessments of CO. Such frequent assessments can pose a substantial burden on participants by requiring study visits every day to provide a CO sample. One way to reduce need for daily visits may be use of a recent procedure to verify abstinence from daily CO values submitted by participants via the Internet. Previous studies have established the feasibility of collecting CO values submitted over the Internet to biochemically validate smoking abstinence, such as from participants enrolled in a contingency management program providing escalating amounts of monetary reinforcement as continuous abstinence is maintained over longer durations.^11–13

In short, this method requires participants to electronically submit video recordings of themselves correctly using a CO monitor, including the CO value displayed on the monitor. Earlier studies with this method had participants use a study-supplied laptop computer with a web camera to record and email CO videos to the study staff.^14,15 More recent studies ask participants to record videos using an internet-connected cellphone equipped with a camera (ie, a “smartphone”), which are then uploaded directly to a secure website.¹⁶ Given the ubiquity of smartphones,¹⁷ including among smokers,^18,19 most participants in a cessation trial are able to provide Internet-obtained CO values, potentially replacing the need for most daily lab visits primarily intended to verify quit status.

However, despite increasing use of this approach, to our knowledge it has not been clearly documented that those classified daily as quit via Internet-submitted videos of CO would also be classified on those days as quit from standard in-person CO measures obtained in the lab. The present study compared quit status using CO values submitted through the Internet to quit status determined using CO values collected in subsequent same-day lab visits.

Methods

Participants

Participants (n = 23) in a larger short-term cessation study agreed to record themselves providing a CO sample on a lab-supplied monitor and submit the resulting videos to lab staff, in addition to their participation in the larger study. All were healthy adults (12 M, 11 F) who smoked ≥ 10 cigarettes per day for ≥1 year and met DSM-V nicotine dependence criteria. Mean (SD) sample characteristics were 15.7 (4.1) cigarettes per day, 34.7 (12.0) years old, and 5.1 (1.4) Fagerström Test of Cigarette Dependence score.^20,21

CO Monitor

All expired-air samples were obtained using pico+ Smokerlyzer monitors (Bedfont Scientific, Kent, United Kingdom). The Bedfont pico+ Smokerlyzer has an LCD screen to guide the participant through the expired-air CO process, facilitating collection of expired-air samples outside of the lab. This monitor has a range of measurement of 0–100 ppm and displays CO values in 1 ppm increments. The monitors were calibrated before the study began and once every 6 months, as recommended by the manufacturer. The pico+ Smokerlyzer is accurate at a level of ±2% and has been reported to have high internal consistency, with an intraclass correlation coefficient of 0.985.²²

Procedure

Video Obtained CO Samples

As part of the informed consent process for the short-term cessation study previously described elsewhere,²³ all participants were offered the opportunity to record and submit videos of themselves correctly using the CO monitor at home, prior to their scheduled lab visits. The CO videos and lab visits occurred each day from Tuesday to Friday on each of the 2 weeks during which they attempted to briefly quit smoking. Participants were told they would receive $5.00 for submitting each daily video correctly as instructed, regardless of quit status. This payment was contingent upon bringing the CO monitor to each midday study visit. Those who agreed to submit the CO videos were given a CO monitor to take home on the Monday visit of each of their two quit weeks, for up to eight daily submissions per participant. Specifically, each video was required to include a clear view of the participant, the participant stating the time of day (corroborated with time of upload), the monitor’s LCD screen before initiating the air sample collection procedure, the participant inhaling, holding their breath for 15 seconds, and exhaling fully through the device when the respective prompts are provided by the monitor (via auditory tones and visually on the LCD screen), and the LCD screen displaying the final CO value. Participants practiced recording themselves using the monitor and electronically submitting a CO video at the end of the first Monday visit in the lab, to ensure they could perform this task correctly.

Participants used their own camera-equipped smartphone or laptop to record the CO videos. Each participant was assigned a unique username and password by study staff in order to log into the video upload website. The SSL-secured website was designed by the Office of Academic Computing at the Western Psychiatric Institute and Clinic of the University of Pittsburgh Medical Center. Once participants logged into the website, they were prompted to choose a file from their device to upload. Once selected, the file was uploaded to a secure server for study staff to access. This study was approved by the University of Pittsburgh Institutional Review Board.

Lab Obtained CO Samples

Study staff oversaw all CO samples provided in lab. Participants provided in-lab samples with the same monitor used in the submitted videos. If the monitor was not brought to the lab visit by the participant, another Bedfont pico+ Smokerlyzer was used to obtain the CO sample.

Determination of Quit Status

Quit status was confirmed using the standard cutoff of CO < 8 ppm, to increase the generalizability of our results to those in most cessation trials.¹ Moreover, when using the Pico+ monitor, CO < 8 ppm was found to maximize both sensitivity and specificity for classifying smokers from nonsmokers.²⁴

Statistical Analyses

The objective of this study was to assess consistency in quit status classification between in-lab and video submitted assessments of CO. To do so, Cohen’s kappa statistic²⁵ was used. Additionally, 95% confidence intervals were calculated for the kappa statistic (κ), using the standard error of kappa (SE_κ). The following formula was used to compute the 95% confidence interval:

$$\kappa -1.96 \times S{E}_{\kappa } to \kappa +1.96 \times S{E}_{\kappa }$$

Percent agreement between assessments was also calculated and reported.²⁶ All analyses were performed using SPSS version 23.0.

Results

Figure 1 contains a scatterplot of CO values obtained via the Internet paired with CO values measured in the lab, with a solid diagonal line indicating perfect agreement and a dashed line on each axis (scaled to log base 2) representing the CO cutoff criterion for determining quit status (ie, < 8 ppm). A total of 150 videos were submitted, with a mean (SD) of 6.5 (1.9) videos per participant and 2.1 (1.8) hours between time of video and the subsequent lab visit on each day. There was a significant correlation between CO values submitted via the Internet and the subsequent CO values obtained in lab, r = 0.79, p < .001, indicating a very strong positive linear relationship in CO values between these assessment methods, as expected. Moreover, the longer the time between assessments, the generally greater decline in the assessed CO value, r = −0.15, p = .06. Overall, 97 videos indicated the participant was not quit and 53 indicated quit. Cohen’s kappa assessed agreement in quit status classification between video and lab assessments, κ = 0.70 (95% CI, 0.58–0.82), p < .001, indicating substantial agreement.²⁷ Percent agreement was similarly high, with 86% (129/150) of paired values indicating the same quit status, 87% (84 of 97) for those videos identified as not quit and 85% (45 of 53) for those videos indicating quit.

Figure 1.

Scatterplot of CO values obtained over the Internet paired with CO values measured in later same-day lab visits, with a solid line indicating perfect agreement and dashed lines representing the CO criterion for determining quit status. Axes scaled log base 2.

Because the primary utility of this procedure is to validate daily smoking abstinence, our main analysis of interest focused on the 53 videos indicating the participant was quit, 45 (85%) of which were followed by CO values also indicating quit in lab. Of the eight not quit in lab, three participants admitted to smoking between the video and in-lab assessments, and three others had CO values slightly below the 8 ppm cutoff in the video but slightly above 8 ppm in the lab (eg, 7–9 ppm). The remaining two inconsistencies were unexplained but, because the in-lab CO values were higher by 7 ppm than the video CO values, unreported smoking between the assessments was likely.

Discussion

We assessed agreement in quit status between CO values submitted through the Internet and CO values collected during subsequent same-day lab visits. Overall, there was substantial agreement in quit status between these two methods of CO assessment. Focusing only on videos submitted while quit, 85% were also quit when measured in the lab, virtually the same as the 87% rate of agreement with the videos submitted when not quit, indicating no difference in likelihood of agreement as a function of actual quit status. This consistency may be unsurprising, but documentation that CO values submitted electronically are equivalent to CO values assessed during in-person visits is still important, to validate use of this method for verifying daily quit status without requiring excessive subject burden. Moreover, longitudinal assessment of daily smoking status with this method allows close monitoring of an abstinent participant’s progress through a trial while trying to avoid relapse, rapidly identifying lapses in quitting nearly in real time. Such a manner of continuous assessment of CO may also be useful in pinpointing critical periods during the intervention (ie, early lapses, when additional help or motivation may be essential to increase the likelihood of long-term success).^28,29

Our results validate the use of Internet-submitted CO values to verify daily quit status. This method can be used in future cessation trials as a means to biochemically validate continuous abstinence without the burden of daily lab visits or relying on self-report of recent smoking lapses. Because the traditional CO cutoff of <8 ppm may still not detect minimal smoke exposure between 12 and 24 hours prior to testing, CO values may need to be obtained at multiple points during the day to confirm total abstinence from any smoking (eg, “not even a puff”³⁰). In such research, multiple daily in-person visits for CO testing would be even more burdensome than once daily, making each assessment by this video CO procedure far more practical,¹³ as well as necessary in order to attract participants without having to offer substantial compensation. A more conservative CO cutoff (ie, CO < 5 ppm) may address this issue.³¹ Finally, the CO monitor being used to collect the expired-air samples must also be taken into consideration. CO values have been found to vary between and within brands of monitors, which in turn may impact the optimum CO cutoff to use and, thus, classification of quit status.^22,24,32 Overall, our findings suggest that quit status determined using CO values submitted over the Internet is a methodologically valid alternative to quit status via CO measured in lab.

Funding

Research reported in this publication was supported by National Institutes of Health (NIH) Grants UH3 TR00958 from National Center for Advancing Translational Sciences (NCATS) KAP and T32 HL7560 (JLK). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Declaration of Interests

None declared.