A Scoping Review and Meta-analysis of the Use of Remote Biochemical Verification Methods of Smoking Status in Tobacco Research

Abstract

Introduction

Increasing digital delivery of smoking cessation interventions has resulted in the need to employ novel strategies for remote biochemical verification.

Aims and Methods

This scoping review and meta-analysis aimed to investigate best practices for remote biochemical verification of smoking status. The scientific literature was searched for studies that reported remotely obtained (not in-person) biochemical confirmation of smoking status (ie, combustible tobacco). A meta-analysis of proportions was conducted to investigate key outcomes, which included rates of returned biological samples and the ratio of biochemically verified to self-reported abstinence rates.

Results

A total of 82 studies were included. The most common samples were expired air (46%) and saliva (40% of studies), the most common biomarkers were carbon monoxide (48%) and cotinine (44%), and the most common verification methods were video confirmation (37%) and mail-in samples for lab analysis (26%). Mean sample return rates determined by random-effects meta-analysis were 70% for smoking cessation intervention studies without contingency management (CM), 77% for CM studies, and 65% for other studies (eg, feasibility and secondary analyses). Among smoking cessation intervention studies without CM, self-reported abstinence rates were 21%, biochemically verified abstinence rates were 10%, and 47% of individuals who self-reported abstinence were also biochemically confirmed as abstinent.

Conclusions

This scoping review suggests that improvements in sample return rates in remote biochemical verification studies of smoking status are needed. Recommendations for reporting standards are provided that may enhance confidence in the validity of reported abstinence rates in remote studies.

Implications

This scoping review and meta-analysis included studies using remote biochemical verification to determine smoking status. Challenges exist regarding implementation and ensuring high sample return rates. Higher self-reported compared to biochemically verified abstinence rates suggest the possibility that participants in remote studies may be misreporting abstinence or not returning samples for other reasons (eg, participant burden, inconvenience). Remote biochemical confirmation of self-reported smoking abstinence should be included in smoking cessation studies whenever feasible. However, findings should be considered in the context of challenges to sample return rates. Better reporting guidelines for future studies in this area are needed.

Introduction

Biochemically verified smoking status is widely considered the “gold standard” outcome in smoking cessation research.^1,2 However, the remote delivery of interventions and collection of cessation outcome data has become increasingly common,^3,4 and the COVID-19 pandemic with associated limitations on in-person research has only accelerated the ­importance of remote interventions. In remote studies, participants do not attend in-person sessions with study personnel and interventions are delivered and data are collected via telephone, mobile application, the Internet, social media, and/or other virtual methods.^4–8 Remote biochemical verification of abstinence in these studies presents many opportunities and challenges for tobacco researchers.

Previous recommendations suggested that biochemical verification of smoking abstinence is not necessary for remote studies.⁹ The assumption was that participants might be less pressured to provide socially desirable responses if they do not encounter study staff or treatment providers at follow-up face-to-face. However, more recent recommendations suggest the need for biochemical verification of abstinence in all cessation studies while also acknowledging that biochemical verification may not be possible for all types of study designs (eg, remotely conducted cessation trials).¹ Currently, little is known about which methods of biochemical verification are most feasible and accurate when delivered remotely, how remotely biochemically verified abstinence rates compare to self-reported abstinence, or how to improve adherence to remote biochemical collection.

Therefore, the current study authors performed a comprehensive scoping review to elucidate the best practices for using remote biochemical verification of smoking abstinence when conducting cessation research. The goal was to answer the following questions: (1) what types of samples (eg, saliva and urine) are collected for biochemical verification, (2) which biomarkers (eg, cotinine and carbon monoxide) and methods (eg, video observation and mailed in samples) of verification are used, (3) how is participant adherence to study procedures encouraged, (4) what study outcomes are obtained (eg, sample return rates, self-reported abstinence, and biochemically verified abstinence), and (5) how are study characteristics related to study outcomes?

The primary focus of this paper is on the biochemical verification of smoking abstinence as the primary study outcome. In some studies where the primary outcome was not specified, the final assessment point that included biochemical verification of smoking status was selected. Because the goal of the current study was to conduct an inclusive scoping review, the use of biochemical verification for other purposes (eg, feasibility studies) is also briefly discussed.

Methods

Design

The PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation¹⁰ were used as reporting guides. The only way in which this scoping review differed from a systematic review was that the risk of bias/quality assessment was not performed for the included articles. A scoping review rather than a systematic review was conducted because this paper sought to identify knowledge gaps regarding the use of biochemical verification in smoking cessation studies and investigate how biochemical verification is used in the conduct of research. Systematic reviews aim to identify and synthesize evidence related to a research question or questions and typically focus on a limited type of study (eg, only randomized clinical trials). The current review did not limit the types of studies reviewed and opted to be more inclusive. This enabled the current review to examine biochemical verification research methods across study types and to identify and analyze knowledge and practice gaps regarding the use of biochemical verification in remote smoking cessation research.

Eligibility Criteria

Publications were included if they reported remotely obtained (ie, not in-person) biochemical confirmation of participant smoking status. Studies were required to specifically assess combustible tobacco use. Studies that reported only on chewing tobacco, vaped products, or other forms of nicotine or tobacco were excluded, as were studies of non tobacco combustibles such as cannabis. Studies that only involved on-site staff-administered verification of smoking status were excluded.

In addition, the following study and publication types were excluded: animal studies, case reports, clinical trials with unpublished results, conference abstracts, theses and dissertations, protocol papers, opinion pieces, other reviews including systematic reviews and meta-analyses, duplicative reports of the same studies, and non-scholarly articles such as magazine or newspaper pieces. Articles not available in English were excluded.

Search Strategy

A medical librarian searched the following databases: Ovid/MEDLINE; Wiley/Cochrane Central Register of Controlled Trials (CENTRAL); Elsevier/Embase; Clarivate/Web of Science (WOS); EBSCO/Cumulative Index of Nursing and Allied Health Literature (CINAHL); and EBSCO/PsycInfo, from the dates of their inception until May 17, 2022, the date the searches were completed. An English language filter was applied to all the searches. The search strategy in each of the databases is available in Supplementary Materials, Appendix A.

Study Selection

All records identified through the database searches were exported to the reference management software EndNote Version X9 (Clarivate Analytics, Philadelphia, PA, USA), which was used to document and delete duplicate records. Using EndNote, the medical librarian also prescreened and excluded animal studies, case reports, conference abstracts, non-scholarly articles, opinion pieces, articles not available in English, protocol papers, and reviews.

The authors were divided into four teams of two people. After de-duplication and prescreening, search results were uploaded into the web-based systematic review software, DistillerSR (Evidence Partners, Ottawa, Canada), and divided among the four teams. Two independent reviewers on each team screened the titles and abstracts of each article in their set for relevance; disagreements within each team were adjudicated by consensus between the two reviewers or by a ninth reviewer. Using the in-depth inclusion/exclusion criteria outlined above, each member independently reviewed the full text of the articles in their set. Again, disagreements were resolved by consensus and/or by a ninth reviewer.

Data Extraction

A customized data extraction form was created within DistillerSR. Articles that met the full inclusion criteria were reshuffled among the four teams. Each team divided their set of articles in half so that each team member did the primary extraction for half the articles and, using DistillerSR’s built-in quality control feature, served as a checker for the other half. Disagreements were adjudicated by consensus between the two reviewers and/or by a ninth reviewer.

A total of 61 variables were extracted from the articles. However, the authors focused on a subset of 17 variables in this paper (Supplementary Table S1): Study ID, author and year, study type, RCT (yes/no), sample size, population, number of remote biochemical assessments, primary outcome (eg, point-prevalence abstinence, continuous abstinence, etc), time point (eg, 3 months, 12 months, number of days for contingency management [CM] studies, etc), type of sample collected (blood, expired air, hair/nails, saliva, and urine), biomarker used (CO, cotinine, and anabasine), verification method (app, mail-in sample lab analyzed, both mail-in and in-person sample lab analyzed, mail-in test strips, photo, and video), from whom samples were collected (all participants, those who self-reported quitting), percent of requested samples that were returned, percent of usable returned samples, percent of participants biochemically confirmed abstinent, brief description of biochemical verification protocol including approaches to increase participant adherence with returning requested samples, percent of self-reported quit rates, and reported problems with biochemical verification. If multiple outcomes were reported, only one was selected (eg, the primary outcome if the study designated one). The final follow-up time point was selected if the study reported outcomes for multiple time points.

Data Analyses

First, descriptive statistics were used to report study characteristics. Then, a series of random-effects meta-analyses of proportions were conducted to estimate the percentage rates of returned samples for all study types. Because of heterogeneity in study design, this review did not make comparisons across study types. Self-reported, biochemically verified, and the concordance between biochemically verified and self-reported abstinence rates were only investigated among smoking cessation intervention studies excluding CM, because CM studies did not report self-reported abstinence rates, and the study designs among other studies were too heterogeneous to allow for meaningful comparisons. Meta regressions were estimated to investigate relationships between study characteristics (eg, samples collected, biomarkers, or verification method) and study outcomes (eg, sample return rates). These analyses only included studies that used a single collection method, biomarker, or verification method. Moreover, outliers were removed from analyses based on Baujat plots.¹¹ Forest plots were generated for each outcome and study type. All analyses were conducted using the metafor package for RStudio with restricted maximum-likelihood estimator.¹²

Results

The PRISMA¹⁰ flow diagram displays details on the paper selection process (Figure 1). The database searches identified 9522 records. Of the 4058 records that remained after duplicates were removed and prescreening was completed, 3582 were excluded during title/abstract screening because of irrelevance to the topic. Upon full-text screening of the remaining 476 publications, 394 articles were excluded primarily for exclusive in-person data collection, no publication of results, and/or unclear methodology. Eighty-two articles met the full inclusion criteria and were included in this review. Studies were divided into three categories: (1) Smoking cessation intervention studies excluding CM studies, (2) CM studies, and (3) Other studies, including feasibility studies, secondary analyses, and validation studies. Supplementary Tables S1.1–S1.3 contain information on all included studies. Detailed information on biochemical verification procedures and problems reported in each study can be found in Supplementary Tables S2.1–S2.3. Table 1 displays descriptive statistics on studies in the different categories.

Figure 1.

Flowchart of the process of literature search and extraction of studies meeting the inclusion criteria.

Table 1.

Summary of Biochemical Verification Procedures By Study Type

Characteristic	Overall, N = 821	Contingency management, N = 261	Intervention (cessation), N = 421	Other studies, N = 141
Sample collected
Expired air	38 (46%)	24 (92%)	8 (19%)	6 (43%)
Saliva	33 (40%)	0 (0%)	30 (71%)	3 (21%)
Combinationa	7 (9%)	2 (8%)	1 (2%)	4 (29%)
Urine	3 (4%)	0 (0%)	3 (7%)	0 (0%)
Hair/nails	1 (1%)	0 (0%)	0 (0%)	1 (7%)
Biomarker used
Carbon monoxide	39 (48%)	24 (92%)	8 (19%)	7 (50%)
Cotinine	36 (44%)	0 (0%)	32 (76%)	4 (29%)
Combinationb	6 (7%)	2 (8%)	2 (5%)	2 (14%)
Not reported	1 (1%)	0 (0%)	0 (0%)	1 (7%)
Verification methods
Video	30 (37%)	22 (85%)	6 (14%)	2 (14%)
Mail-in sample (lab analyzed)	21 (26%)	0 (0%)	18 (43%)	3 (21%)
Mail-in sample (lab analyzed) and in-person sample	8 (10%)	0 (0%)	7 (17%)	1 (7%)
App	7 (9%)	0 (0%)	4 (10%)	3 (21%)
Combinationc	7 (9%)	2 (8%)	2 (5%)	3 (21%)
Photo	7 (9%)	2 (8%)	3 (7%)	2 (14%)
Mail-in test strips	2 (2%)	0 (0%)	2 (5%)	0 (0%)
Method to increase adherence
Training	52 (63%)	26 (100%)	15 (36%)	11 (79%)
Incentives	56 (68%)	26 (100%)	24 (57%)	6 (43%)
Reminders	14 (17%)	3 (12%)	10 (24%)	1 (7%)
Number of methods used
0	15 (18%)	0 (0%)	13 (31%)	2 (14%)
1	19 (23%)	0 (0%)	12 (29%)	7 (50%)
2	41 (50%)	23 (88%)	14 (33%)	4 (29%)
3	7 (9%)	3 (12%)	3 (7%)	1 (7%)

¹ n (%). Combination refers to more than one method or technique used.

^a n = 1 Blood, saliva; n = 4 expired air, saliva; n = 1 urine, saliva, wrist sensor; n = 1 resting heart rate, expired air.

^b n = 3 Cotinine, carbon monoxide; n = 2 cotinine, anabasine; n = 1 cotinine, carbon monoxide, anabasine.

^c n = 1 App, email; n = 2 app, web platform; n = 1 photo, email, or messenger; n = 1 photo, expired air/CO was verified in person (staff drove to meet participants); n = 1 video, photo, other (Youth also brought used saliva screens to weekly CBT appointments); n = 1 mail-in sample (lab analyzed) for urine, video for saliva, SmokeBeat wrist sensor detected hand to mouth movement.

Smoking Cessation Intervention Studies (Excluding CM)

Overview

There were 42 smoking cessation intervention studies included in the review.^13–54 Of these, 34 (81%) were randomized trials, 5 (12%) were pilot studies, and the remaining studies were an implementation trial (2%), a quasi-experiment (2%), and a retrospective study (2%). The primary outcome assessed most frequently was 7-day point-prevalence abstinence (n = 21).^{17,18,21,23–25,27,28,30,38–43,45,47–49,52–54} Assessments of smoking abstinence were conducted spanning different time points—ranging from 2⁵⁰ to 12 months.^{30,38,42,44,46,48,49,51,52}

Participants and Sample Sizes

Participant samples in these studies included general populations of people who smoke daily or non-daily as well as special populations, for example, pregnant people,^17,41,43,55 people who are hospitalized,^14,22,48 people who have low socioeconomic status,^15,18,47, individuals with HIV³¹ or cancer,⁵³ young adults who engaged in heavy drinking,⁴⁹ and parents.^45,46 Sample sizes ranged from 17¹³ to 5800;¹⁹ five studies (12%) had fewer than 100 participants.

Remote Verification Procedures Used

Of all smoking cessation intervention studies (excluding CM), 30 (71%) reported collecting saliva cotinine as the primary sample to remotely biochemically verify smoking status. Eight (19%) studies used expired-air carbon monoxide, three (7%) studies used urine cotinine, two (5%) studies used saliva cotinine as well as anabasine, and one (2%) study used blood as well as saliva cotinine. The most frequent verification method used was mail-in samples which were lab analyzed (43%). Other verification methods used were both mail-in and in-person samples (eg, studies used remote collection methods if participants lived far from the study site, were unable to attend study visits in person, etc) (17%), video confirmation (14%), apps (10%), photo (7%), and mail-in test strips (5%).

Four studies (10%) used a combination of remote saliva cotinine and in-person carbon monoxide testing for participants and combined outcomes (these studies were coded as remote saliva and cotinine verification). The first study⁵⁶ conducted sequential testing, using remote saliva cotinine and subsequent in-person carbon monoxide testing, among 23 participants with cotinine levels exceeding 10 ng/mL in the initial saliva test. The second study¹⁹ offered in-person carbon monoxide testing to 2 participants who refused remote saliva cotinine testing. The two remaining studies^24,53 conducted remote saliva testing and subsequent in-person carbon monoxide testing if participants reported use of nicotine replacement therapy or electronic nicotine delivery systems at follow-up (number of participants not reported).

Feasibility of Biochemical Verification

Most of the studies (n = 28; 67%) reported participant sample return rates and in these studies, between 24%¹⁷ and 100%¹³ of participants returned samples. Only 14% (n = 6) of studies reported the percentage of usable samples, with a range between 73% and 100%. Reasons for unusable samples reported in these studies included insufficient or contaminated saliva samples,⁴⁴ saliva samples that had evaporated,¹⁶ and unreadable test strips.¹³ Random-effects meta-analysis included 26 studies (62%) and showed pooled percentage return rates of 71% (95% CI 64%; 77%) (Supplementary Figure S1).

Approaches to Increase Adherence with Biochemical Verification

Approaches to increase adherence to biochemical verification methods that were reported in studies included: monetary incentives (57% of studies); participant training (36%); and reminders to provide biochemical verification samples (24%). The greatest number of studies employed either one (29%) or two (33%) methods to increase adherence, and only three studies (7%) employed all three approaches.⁴⁷

Biochemical Verification Outcomes Compared to Self-reported Outcomes

A total of 26 (62%) smoking cessation studies were included in a random-effects meta-analysis that investigated self-reported and biochemically verified smoking abstinence at the same assessment time point (Supplementary Figures S4 to S6). In all but one study,⁴² self-reported quit rates (range 2%²⁶ to 65%¹³) were higher than the biochemically confirmed abstinence rates (range 1%¹⁶ to 53%¹³). The ratio of biochemically confirmed rates to self-reported quit rates ranged from 12%⁴⁹ to 100%.⁴² Random-effects meta-analysis showed a pooled percentage of 21% (95% CI 17%; 26%) for self-reported smoking abstinence rates, 10% (95% CI 7%; 13%) for biochemically confirmed smoking abstinence rates, and 47% (95% CI 41%; 54%) for the ratio of biochemically confirmed rates to self-reported quit rates, meaning that in these studies, only 47% of individuals who self-reported abstinence were also biochemically confirmed as abstinent.

CM Studies

Overview

There were 26 CM studies identified during the review.^57–82 Most of the CM studies did not include biochemically verified abstinence at one-time point (eg, 6-month follow-up) as a primary outcome, but rather used biochemical verification at multiple time points throughout the study as part of the intervention procedures, to determine smoking/abstinence or reduction in tobacco use. Biochemical verification was obtained at several time points: At baseline for inclusion, during a run-in period to determine the level of smoking, during the intervention to determine eligibility for receiving an incentive, and at the end of the study. These studies used a variety of designs, including randomized controlled trials, cluster-randomized trials, pilot trials, and within-subjects designs. Review results are reported separately for CM studies because the distinct functions of biochemical verification in these studies are likely to influence the extent and rigor of verification procedures and participant adherence to them, limiting the comparability to other study types.

Participants and Sample Sizes

Participant samples in these studies included general populations of people who smoke as well as special populations, for example, pregnant people,^78,80 people who have low socioeconomic status,⁷⁷ individuals who smoke both cannabis and tobacco,⁵⁸ individuals with alcohol use disorder,⁶⁰ individuals in outpatient treatment for mood disorders,⁸¹ individuals with schizophrenia,⁵⁹ veterans who are experiencing homelessness,⁶¹ those who smoke heavily (defined as 20+ cigarettes per day⁶⁵), and adolescents.^67–69 Sample sizes ranged from 4 participants⁶² to 183 participants;⁶⁸ 18 studies (69%) had fewer than 50 participants.

Remote Verification Procedures Used

Twenty-four studies (92%) reported that they used expired carbon monoxide (CO) as the primary sample to verify smoking/abstinence. The number of CO samples collected remotely per participant during the studies ranged from 10 samples⁸⁰ to 168 samples.⁵⁸ In most studies, participants were asked to provide one or more samples per day throughout the study. Participants were instructed to video record themselves during the collection process in all but three studies that used either photo confirmation ^72,77 or confirmation via an app and web platform.⁸⁰

Feasibility of Biochemical Verification

Most (n = 21; 81%) of CM studies reported participant sample return rates and, in these studies, 30%⁷⁷ to 98%⁶⁵ of samples were returned. None of the studies reported the percentage of usable samples that were returned. Problems with biochemical verification procedures were reported by 9 studies (35%), most of which included technical difficulties.^{59,64,67,69,71,77,80} Random-effects meta-analysis included 20 Studies (77%) and showed pooled percentage return rates of 77% (95% CI 67%; 84%) (Supplementary Figure S2).

Approaches to Increase Adherence with Biochemical Verification

Approaches to increase adherence to biochemical verification methods that were reported in studies included: training of participants (100% of studies); monetary incentives (100% of studies); and reminders to provide biochemical verification samples (12% of studies). Most studies employed two approaches to improve adherence (88% of studies), and only three studies employed all three approaches (12% of studies).

Biochemical Verification Outcomes Compared to Self-reported Outcomes

Most CM studies (n = 15, 58%) reported the percentage of abstinent samples provided over a period of time (eg, during a 10-day abstinence phase).^{57,60,63–67,70,72–75,79–81} The percentage of abstinent samples in these studies ranged from 6%⁷³ to 100%.⁶⁰ The remaining studies reported CO reduction outcomes (n = 4, 15%)^62,68,69,76 or other biochemical abstinence outcomes (n = 5, 20%), for example, the percentage of participants with all abstinence samples for the final 7 days of treatment⁶¹ and participant CO data over time.⁶⁵ One study (4%) reported only feasibility, participant retention, and satisfaction,⁵⁸ while another study (4%) reported no feasibility outcomes and unclear biochemical verification results.⁵⁹ Self-reported abstinence corresponding to biochemically verified abstinence rates was not reported in any of the CM studies.

Other Studies

Overview

This category included 14 studies^83–96 that investigated the feasibility or validation of biochemical verification procedures, secondary analyses, a cross-sectional study, and one study conducted three sequential RCTs. Four studies^86,94–96 (29%) in this category reported biochemically verified abstinence as outcome. All remaining studies (71%) in this section reported other types of outcomes (eg, the feasibility of procedures, including returned samples, and usable samples).

Participants and Sample Sizes

Participant samples in this category of studies included general populations of people who smoke (ie, daily and non-daily), youth or other specific age groups, such as 12–17-year-olds,⁸³ 15–25-year-olds,⁸⁸ young adults,⁹² and 27–57-year-olds,⁸⁹ individuals with low income,⁹⁵ and pregnant people,⁹⁰ as well as pregnant and postpartum Medicaid members.⁹⁶ Sample sizes ranged from 15 participants⁹¹ to 579 participants;⁸⁵ seven studies^{87–91,94,96} had fewer than 100 participants.

Remote Verification Procedures Used

Biochemical verification in these studies was conducted primarily by evaluating expired-air carbon monoxide (43%), followed by saliva cotinine testing (21%). In one study (10%)⁸³ collected hair/nail samples without any biomarker or analysis reported. Studies in this category used various verification methods to confirm sample results, including combinations of multiple methods (21%), apps (21%), mail-in samples (lab analyzed) (21%), photos (14%), videos (14%), and both mail-in and in-person samples for lab analysis (7%).

Feasibility of Biochemical Verification

Of the 14 other studies, 11 (79%) reported ranges of returned samples which varied from 25%⁸⁶ to 83%.⁸³ Three of these studies also reported sample usability rates of 61%,⁹⁵ 87%,⁸⁵ and 97%.⁸⁴ Random-effects meta-analysis included 11 Studies (79%) and showed pooled percentage return rates of 65% (95% CI 52%; 76%) (Supplementary Figure S3).

Approaches to Increase Adherence with Biochemical Verification

Approaches to increase adherence to biochemical verification methods that were reported in studies included: Training of participants (79% of studies); monetary incentives (43%); and reminders to provide biochemical verification samples (7%). The greatest number of studies employed one or two approaches to improve adherence (50% and 29%, respectively), and only one study (7%) employed all three approaches.

Biochemical Verification Outcomes Compared to Self-reported Outcomes

In this category of studies, two^86,95 (14%) reported biochemically verified outcomes among those who self-reported quitting. In the first study,⁸⁶ participants were sent a CO monitor that was paired with the software on a computer. However, participants were not paid for completion of CO tests, and there was low adherence to testing procedures (25% of participants completed a CO test). Self-reported abstinence rates (15%) were higher than biochemically verified abstinence rates (3%). The other study⁹⁵ mailed NicAlert cotinine tests to participants for testing of urine. Participants were instructed to take a digital photo of the test strip and send it to the study team by text or email. The authors reported a sample return rate of 46% and 61% of usable samples, because of a substantial number of inconclusive photos. Overall, only 9% of participants, who had all self-reported abstinence, were biochemically confirmed abstinent.

Random-Effects Meta-analysis Investigating Study Outcomes

A random-effects meta-analysis, stratified by study type (eg, smoking cessation intervention without CM, CM, and other studies), was conducted to identify whether study characteristics, including approaches to improve adherence with biochemical verification measures, were associated with sample return rates. The meta-analysis also investigated whether study characteristics were associated with the ratio of ­biochemically verified self-reported abstinence rates for smoking cessation intervention studies without CM only.

Return Rates

Subgroup analyses were conducted to investigate study characteristics associated with sample return rates. Since CM studies did not have variability in study characteristics (ie, all studies included in analyses used expired-air carbon monoxide and video confirmation; see Table 1), these analyses were only conducted for non-CM intervention studies and the other study category. Among intervention studies included in analyses, there were no significant differences in sample return rates by type of sample collected (k = 26), biomarker (k = 26), or verification method used (k = 27, results not shown). Among the other studies category, there were no differences in return rates by type of sample collected or biomarker used (k = 8; results not shown). However, compared to studies that used apps for verification, higher return rates were reported among studies using mail-in samples for lab analysis (estimate = 2.1, lower CI 0.4, upper CI 3.8, p < .05) and studies using video confirmation (estimate = 2.3, lower CI 0.6, upper CI 4.1, p < .01) (k = 9).

Approaches to Increase Adherence and Return Rates

Potential relationships between methods to improve adherence and return rates among the different study types were also investigated. Stratified by study type (ie, intervention studies [k = 27], CM studies [k = 20], and other studies [k = 11]), there were no significant differences in sample return rates among studies that included participant training, incentives, or reminders (results not shown). Neither the dichotomous predictor of any versus no approaches used to improve adherence, nor the number of approaches used were significantly related to return rates (results not shown).

Ratio Between Biochemically Verified and Self-reported Outcomes

Finally, the meta-analysis investigated the ratio between biochemically verified and self-reported smoking abstinence rates as a measure of concordance for intervention studies (k = 26). There were no differences in the ratio between biochemically verified and self-reported smoking abstinence rates by type of sample collected or biomarker used (results not shown). However, compared to studies that used video confirmation, a lower ratio between biochemically verified and self-reported smoking abstinence rates was reported among studies that used photo confirmation (estimate = −2.0, lower CI −3.2, upper CI −0.8, p < .01), mail-in samples for lab analysis (estimate = −0.8, lower CI −1.6, upper CI −0.1, p < .05), or mail-in samples for lab analysis combined with in-person samples (estimate = −1.0, lower CI −1.9, upper CI −0.1, p < .05).

Discussion

The goal of the current study was to conduct a scoping review and meta-analysis of studies using remote biochemical verification of smoking status. A total of 82 studies were included. Among the 42 non-CM smoking cessation intervention studies, the most common type of sample collected was saliva (71% of studies), the most common biomarker used was cotinine (76% of studies), and the most common verification method was lab analysis of mailed samples (43% of studies). CM studies (n = 26) and other studies (eg, feasibility, secondary analyses; n = 14) most commonly collected expired air (92% of CM studies; 43% of other studies), used carbon monoxide (92% of CM studies; 50% of other studies), and video verification (85% of CM studies). Mean sample return rates determined by random-effects meta-analysis were 70% for smoking cessation intervention studies without CM, 77% for CM studies, and 65% for other studies. Approaches to increase participant adherence to returning samples reported among studies were not significantly related to higher sample return rates. Among smoking cessation intervention studies without CM included in meta-analysis, self-reported abstinence rates were 21%, and biochemically verified abstinence rates were 10%.

Overall, the current review found a mismatch between self-reported and biochemically verified abstinence rates in smoking cessation intervention studies without CM that employed remote biochemical verification. Regarding the ratio of biochemically verified to self-reported outcomes, only 47% of self-reported abstainers were confirmed in pooled random-effects meta-analysis. This ratio did not significantly vary across studies collecting different types of samples or using different biomarkers. However, studies that used video confirmation had a significantly higher ratio compared to studies that used photo confirmation, mail-in samples for lab analysis, or mail-in samples for lab analysis combined with in-person samples. Our findings on the mismatch between self-reported and biochemically verified abstinence are in line with previously reported findings. A recent study⁹⁷ combined data from five hospital-initiated smoking cessation trials and found that 60% of self-reported smoking cessation was biochemically confirmed, which is slightly higher than the confirmation rates found in the current study. In sum, these findings suggest that remotely biochemically verified abstinence rates are substantially lower than self-reported abstinence rates and are therefore not comparable across studies. The reasons why study participants who self-reported abstinence did not provide biochemical confirmation remain unknown and may plausibly include lack of convenience, additional burdensome effort, uncomfortable or tedious procedures, as well as continued smoking and/or relapse.

This review also found no significant relationships between methods to improve adherence and return rates. On the one hand, these findings suggest the need to identify ways to improve return rates of samples for remote biochemical verification across the board. For example, new, low-cost remote CO verification devices are increasingly available and could be used more widely to assess smoking abstinence.^77,89 Moreover, studies could experimentally test different biochemical verification approaches and methods to improve participant adherence. On the other hand, studies using remote biochemical verification should report in detail testing procedures and relevant data, including sample return rates and number of usable samples by study group/condition, as well as approaches used to improve participant adherence. Moving forward, improving remote biochemical verification procedures will be a critical contribution to digital and mobile health smoking cessation studies and other studies that deliver remote smoking cessation support (eg, quitlines).

Finally, not all remote verification methods can confirm that the participant provides the sample instead of a third person. Confirmation of identity is likely more important for CM studies that directly tie abstinence to distribution of rewards and thus may create an incentive for participants to ­misrepresent who provided the sample. CM studies most frequently use video confirmation of breath sample provision, for example, videos that are automatically uploaded to a platform and can be checked by research staff.⁶⁶ More recently, studies have also used photos were taken during the breath sample provision process,⁷² including automatic facial recognition technology.⁷⁷ Another strategy to confirm participant identity, used by some smoking cessation intervention studies that do not rely on frequent sampling of abstinence, includes real-time video calls with participants and project staff, which has been used for both breath CO²⁸ and saliva cotinine (using test strips)⁵⁴ monitoring. A technique that does not require real-time contact with participants includes mailing saliva cotinine test kits to participants, paired with the request to document the sample provision and test results with photos to be sent to research staff.^49,92 Thus, multiple different approaches are available to confirm if participants provide samples for biochemical verification themselves.

Limitations

The primary limitation of this scoping review is that the methods used to biochemically verify abstinence, the reasons for missing samples (eg, non-return, technical difficulties, and unusable samples), and study approaches to improve sample return rates, may not be reported comprehensively and consistently across studies, limiting the strength of conclusions that can be drawn about the causes and implications of discrepancies between self-reported abstinence and biochemically verified abstinence. In addition, the number of studies in the other studies category was low and contained both feasibility and cross-sectional studies, so findings should be interpreted cautiously. Moreover, included studies did not consistently account for the use of other tobacco/nicotine products or other products (ie, cannabis); this is an important issue given the high prevalence of electronic nicotine delivery systems and cannabis use.^98,99

Conclusions

This scoping review and meta-analysis provide an overview of studies that used remote biochemical verification of smoking status. The review found that biochemically verified abstinence rates were lower than self-reported abstinence rates for almost all studies included. However, in light of limitations to data available from included studies, it remains unclear which factors are responsible for this mismatch and if the ground truth of smoking abstinence is more closely represented by biochemically verified or self-reported rates in remote studies. In addition to recent recommendations for biochemical verification provided by our SRNT colleagues,¹ and to improve the evidence for remote biochemical verification of smoking status, the authors recommend the following reporting guidelines for future studies in this area: (1) Report sample return rates, usable samples, self-reported abstinence, biochemically verified abstinence, and the number of concordant/discordant self-reported and verified outcomes, with detailed data reported for each study subgroup/condition. (2) Report and account for other tobacco product use and cannabis use. (3) Report identity verification of who provided samples, and (4) Include and report study approaches to increase sample return rates. The results of this review suggest that improved verification methods and improved reporting standards are needed to enhance confidence in the validity of reported abstinence rates in remote studies.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://academic.oup.com/ntr.

Funding

This work was supported by the University of Arizona College of Nursing to Dr. Judith Gordon. Dr. Thrul’s work on the manuscript was supported by a grant from the National Cancer Institute (R01CA246590). Oklahoma Tobacco Settlement Endowment Trust grant R21-02 provided support for the effort of Drs. Kendzor, Businelle, Alexander, and Ra. NCI P30CA225520 provided support for the effort of Drs. Kendzor and Businelle. Manuscript preparation was additionally supported by the National Institute on Minority Health and Health Disparities (grant number 1K01MD015295-01A1 [to AA]), and the National Institute on Drug Abuse (grant number R00DA046564 [to ETH]). Dr. Heffner’s work on this manuscript was supported by grants from the National Institutes of Health (R34DA050967, R21CA236980). Funding sources had no role in the writing of this report or the decision to submit this manuscript for publication.

Declaration of Interests

Thrul reports membership on the scientific advisory board of MindCotine, Inc., which offers a smoking cessation program. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflicts of interest policies. Heffner and Kendzor have received research support from Pfizer (unrelated to the current manuscript). Businelle and Kendzor are inventors of the INSIGHT mobile health platform, though no royalties were earned related to the publication of this manuscript. The other authors declare that they have no competing interests.

Author Contributions

JT: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Writing—Original Draft, Writing—Review & Editing. CLH: Conceptualization, Data Curation, Investigation, Methodology, Project Administration, Writing—Original Draft, Writing—Review & Editing. JD: Data Curation, Formal Analysis, Investigation, Visualization, Writing—Review & Editing. AA: Investigation, Writing—Review & Editing. AMA: Investigation, Writing—Review & Editing. MSB: Investigation, Writing—Review & Editing. ETH: Investigation, Writing—Review & Editing. JLH: Investigation, Writing—Review & Editing. DEK: Investigation, Writing—Review & Editing. CKR: Investigation, Writing—Review & Editing. JSG: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Writing—Original Draft, Writing—Review & Editing.