Using wearable technology to detect prescription opioid self-administration

Abstract

Appropriate monitoring of opioid use in patients with pain conditions is paramount, yet it remains a very challenging task. The current work examined the use of a wearable sensor to detect self-administration of opioids after dental surgery using machine learning. Participants were recruited from an oral and maxillofacial surgery clinic. Participants were 46 adult patients (26 female) receiving opioids after dental surgery. Participants wore Empatica E4 sensors during the period they self-administered opioids. The E4 collected physiological parameters including accelerometer x-, y-, and z-axes, heart rate, and electrodermal activity. Four machine learning models provided validation accuracies greater than 80%, but the bagged-tree model provided the highest combination of validation accuracy (83.7%) and area under the receiver operating characteristic curve (0.92). The trained model had a validation sensitivity of 82%, a specificity of 85%, a positive predictive value of 85%, and a negative predictive value of 83%. A subsequent test of the trained model on withheld data had a sensitivity of 81%, a specificity of 88%, a positive predictive value of 87%, and a negative predictive value of 82%. Results from training and testing model of machine learning indicated that opioid self-administration could be identified with reasonable accuracy, leading to considerable possibilities of the use of wearable technology to advance prevention and treatment.

1. Introduction

In the past 20 years, the rates of prescription drug abuse have increased 250%, with prescription opioids leading this trend and being responsible for increased opioid overdose deaths.^18,25 The opioid epidemic is a multistage public health problem with many potential opportunities for intervention, including prevention programs that encourage appropriate use of opioids acquired after medical and dental procedures. Misuse of prescribed opioids (ie, using more opioid than directed and use of opioid for reasons other than pain) has been found to precede opioid use disorder (OUD).^27,29,38 Research suggests that individuals undergoing pain treatment with opioids could be at risk of OUD because opioid misuse is largely motivated by pain relief.^17,23

Wearable technology has great potential in the field of substance use research. For instance, 2 small studies using a wrist biosensor found that physiological changes related to cocaine use included increased electrodermal activity and movement and decreased skin temperature.^5,7 Similarly, bracelets and ankle sensors have been used to detect transdermal alcohol concentrations with modest success.¹¹ Even accelerometer data in smart phones have been used to detect alcohol intoxication.³⁹ Notably, participants are receptive to wearable technology, and these devices can be used to supplement substance use treatment and relapse prevention.^2,4–6,35

1.1. Opioid use detection

Appropriate monitoring of opioid use in patients with pain conditions is paramount, yet it remains a very challenging task.^13,15 Current opioid use monitoring methods include pill counts, self-report, prescription drug monitoring, and toxicology screening.^3,13,21,41 However, these methods may be inadequate because of errors, recall bias, distortion, and false-negative results.^13,21,41 Accurate and reliable detection of opioid use in medical populations may prevent opioid misuse and, ultimately, the development of OUD.

Opioid use produces physiological changes that could be detected by wearable devices, namely, decreased heart rate and psychomotor impairment.^43,44 In fact, a few studies have used wearable devices to detect opioid use. These pioneering studies have found that electrodermal activity, heart rate, skin temperature, and motion can be used as biomarkers to detect opioid use in patients with minimal opioid exposure and those who exhibit chronic use.^7–9,36 Nevertheless, these studies have been limited by the sample (ie, patients with OUD) and the highly controlled setting (ie, direct observation at emergency departments). Thus, there is a dearth of research on physiological markers that can help detect opioid use in individuals without OUD in outpatient settings. A method that accurately detects real-time opioid use would provide several advantages, including the ability to obtain environmental and behavioral contexts surrounding opioid misuse and tolerance and the potential to intervene in real time to prevent escalation to opioid dependence and OUD.

1.2. The current study

The purpose of this study was to validate the use of wearable sensor technology to detect self-administration of prescription opioids over an extended period and in an ecologically valid setting using machine learning.

2. Methods

2.1. Participants

Eligible participants were patients scheduled for dental surgery at an oral and maxillofacial surgery clinic that would be followed by opioid prescription. Inclusion criteria consisted of age of 18 years or older, had not used opioids for at least 6 months, ability to consent, and ability to speak and understand English. Exclusion criteria included pregnancy, incarceration, chronic pain, current drug or alcohol dependence, musculoskeletal pain limiting motion, inability to wear the sensor device, and presence of a developmental disability.

2.2. Procedures

The Institutional Review Board at the University of Tennessee Health Science Center approved this study. This study was registered in ClinicalTrials.gov (identifier: NCT03462797).

2.2.1. Presurgery consultation meeting

Staff at the dental clinic referred patients for eligibility evaluation conducted by the study coordinator on the date of the patient’s presurgical evaluation (usually 1 week before surgery). Interested participants were screened for eligibility in a private room at the oral and maxillofacial surgery clinic. Eligible participants provided informed consent and then completed baseline questionnaires. The study coordinator then scheduled the device drop-off.

2.2.2. Presurgery wearable device drop-off

During this meeting, generally scheduled for 1 business day before surgery, the study coordinator distributed the Empatica E4 wristband, a charger, and a handout with detailed instructions to operate the device and troubleshoot common problems. Instructions explicitly directed participants to wear the watch on their nondominant hand to prevent exacerbated motion.²⁴ Then, the study coordinator instructed participants to wear the sensor for at least 24 hours before the surgery day to establish baseline data.

2.2.3. Day of surgery

The study coordinator met with participants 30 minutes before their extraction surgery to download baseline physiological data from the wristband and to reset the device for postsurgery recording. At this time, the study coordinator provided participants opioid medication logs to record self-administration of opioid medication after the surgery. Oral and maxillofacial surgery clinic staff provided routine opioid medication and recovery instructions for all participants immediately after surgery.

2.2.4. Data download visits

In subsequent days after the surgery, the study coordinator met with participants every 2 to 4 days to collect physiological data (from the E4) and opioid self-administration data (from medication logs). Afterward, the coordinator provided additional blank medication logs until opioid self-administration was discontinued either because of completion of the medication regimen or because of the participant’s decision to discontinue the opioid. Duration of opioid self-administration was recorded. Participants received a $20 gift card to their preferred store (Walmart or local grocery) at each download visit. After opioid discontinuation, the study coordinator met with participants to collect the device, charger, and the last medication log and gave participants a $50 gift card to their preferred store.

2.3. Measures

2.3.1. Demographic information

Participants responded to questions regarding their age, sex, race and ethnicity, marital status, annual household income, and highest level of education.

2.3.2. Opioid logs

First, participants confirmed their current opioid medication, dose, the recommended frequency of use, changes to their medication, and pill count. Then, participants were provided with the daily opioid logs that contained a table with 5 columns where participants recorded the following information for each opioid self-administration: date and time, dose per opioid pill, number of opioid pills used, other medication use, and level of pain before opioid self-administration (from 0 [no pain] to 10 [worst pain]).⁴²

2.3.3. Physiological measurement

Physiological data were collected using the Empatica E4 wristband sensor (Empatica Inc, Milan, Italy). The E4 has a photoplethysmography sensor, electrodermal activity (EDA) sensor, 3-axis accelerometer, and infrared thermopile.¹² The photoplethysmography sensor has a sampling frequency of 64 Hz and uses green and red light emitting diodes that tolerate external light and a photodetector on the opposite side of the sensor.^12,32,33 The EDA sensor has a sampling frequency of 4 Hz with a range of 0.01 to 100 μSiemens and consists of 2 silver-plated electrodes placed in the inner wrist of the band.¹² The 3-axis accelerometer has a sampling frequency of 32 Hz with a range of ±2 g at a resolution of 8 bits. With these parameters, accelerometry is collected in steps of 0.016 g, which allows for high sensitivity motion detection ranging from large arm movements to minor wrist motion.¹² Finally, the infrared thermopile has a sampling frequency of 4 Hz with a range of – 40 to 115°C and an accuracy of 6 ± 0.2°C within 36 to 39°C. Overall, the E4 has a temporal resolution of 0.2 seconds. In addition, the device has an event marker button that allows researchers and participants to indicate specific changes or interventions in the data. This device can record up to 48 hours with 1 battery charge and store up to 60 hours of data or approximately 4 days. The E4 has demonstrated similar validity when compared with laboratory sensors.^24,34 Physiological data parameters included electrodermal activity (EDA in μSiemens), blood volume pulse, heart rate (in bpm), peripheral skin temperature (°C), and 3-axis motion data on the x-, y-, and z-axis planes (g).

2.4. Data analysis

Data analysis involved deriving features for machine learning models to classify opioid use periods from nonopioid use periods. Five wearable sensor signals were analyzed that included accelerometer (x-, y-, and z-axis), EDA, and heart rate data during a 30-minute period after each opioid use report based on the medication logs. These 30-minute periods were selected to check for immediate effects of opioid use on the measured signals since the onset of action from orally administered oxycodone and hydromorphone is within 30 minutes.^1,10,20,28 In addition, 30-minute intervals of baseline (ie, nonopioid use) data were analyzed. A sliding window of 5-minute duration with a 4-minute overlap was used to extract features for the 30-minute periods after each opioid use report and for each 30-minute baseline period. This process broke down each 30-minute interval into 26 five-minute windows. Previous research has compared time windows immediately before and after opioid use.^7,8 However, using postsurgery, preopioid data as negative class data in our study may present confounding variables because these segments may include residual opioid effects from a previous dose. Therefore, baseline data were selected as the negative class, and opioid use data were selected as the positive class in our models.

For each of the 5 sensor signals (accelerometer x-axis, y-axis, and z-axis, EDA, and heart rate), there were 5 features used to represent each signal: the mean, variance, gamma (distribution) shape, gamma (distribution) scale, and $D\left(\sqrt{{\text{ Shape }}^2+{\text{Scale }}^2}\right)$ parameters. The means and variances of the raw signal data for each of the 5 sensors would allow for comparisons between the general behavior of the raw signals from the 5-minute windows collected during the baseline and after opioid use events (ie, check whether the average opioid event heart rate is less than or greater than the average baseline heart rate).

Regarding the gamma shape, scale, and D features, MATLAB was used to fit the amplitudes of each sensors’ data during each 5-minute window using a Hilbert transform^9,16 to test whether the signals’ parameters could be used to detect opioid consumption. The magnitudes of each analytic signals’ data points were taken to generate the amplitudes for each point along the corresponding signal. These amplitudes were distributed in a similar manner to that of a gamma distribution with shape and scale values similar to previous research.⁹ The gamma shape, scale, and D parameters were collected to determine whether the distribution of each analytic signals’ amplitudes within a given 5-minute window varied with respect to the baseline and opioid use events (ie, determine whether an opioid event accelerometry distribution was more or less positively skewed than a baseline accelerometry distribution). This resulted in a total of 25 features that could be used to train machine learning models to distinguish between a baseline and opioid event by detecting differences in the behaviors of the signals and their distributions for both event types.

To check for underlying patterns among the collected features, principal component analyses (PCAs) with orthogonal rotation were conducted in SPSS (version 24.0) using the original data set. The sample data were initially screened for entry error, missing data, and outliers together with the distribution normality and descriptive statistics for the baseline records of the 25 feature variables.

2.4.1. Machine learning classification determination

The MATLAB Classification Learner app was used to test the binary classification case distinguishing between the baseline (ie, negative class) and opioid use report data (ie, positive class). This test was used to determine whether any of the 25 supported classification models could accurately classify the data. These models included multiple variants of decision tree, discriminant analysis, logistic regression, naive Bayes, support vector machine, nearest neighbor, and ensemble classifiers.

2.4.2. Training, validation, and testing methods

Because there were more baseline cases extracted than opioid self-administration cases, the baseline data windows were randomly sorted, and several were withheld from the training set to equalize the number of baseline (nonopioid) and opioid cases. The baseline and opioid data windows were then randomly sorted, and 30% of them were set aside to test the trained models. This left 70% of the windows for use in the model training and cross-validation procedures. In this regard, 10-fold cross-validation was used to optimize model hyperparameters, check the effects of different feature combinations on model performance, and help prevent the classifiers from overfitting the data by splitting the data set into 10 folds, training on 9 of the folds, and validating the trained model on the last fold. This process was repeated until each fold was used to validate the model, and the resulting classifier’s validation performance was the average of each of the 10 trained models’ performances.²² Once validation was complete, the final model was evaluated by observing the model’s performance when the test set was provided as input.

3. Results

A total of 80 individuals were eligible for the study at the time of the presurgical visit; however, 34 individuals were withdrawn from the study because of failure to attend their surgical visit or failure to wear the device after surgery.

A total of 46 participants had complete data that could be used in the data analysis (26 female). The final sample was diverse, with 21 (45.7%) identifying as African American, 16 (34.8%) as Anglo-American, and 16 (34.8%) with another racial or ethnic group (participants could select more than 1 category). Most participants were single (69.6%), with 23.9% reporting an annual income below $19,999 and 26.1% reporting an educational level of high school graduation/GED or less. Detailed information on demographic characteristics can be found in Table 1.

Table 1.

Demographic characteristics of maxillofacial surgery patients.

Variable	N	%
Sex
Male	20	43.5
Female	26	56.5
Race/ethnicity*
White	16	34.8
African American	21	45.7
Asian	1	2.2
Native American	1	2.2
Other	7	15.2
Hispanic/Latino	7	15.2
Marital status
Single	32	69.6
Married	7	15.2
Widowed or divorced	7	15.2
Education
GED or less	12	26.1
Some college	27	58.7
Bachelor’s or above	7	15.2
Income
$0-$19,999	11	23.9
$20,000 to $39,999	17	37.0
$40,000 or more	18	39.1
Opioid prescribed
Hydrocodone	27	58.7
Oxycodone	19	41.3
Dose
5 mg	45	97.8
10 mg	1	2.2

^*Participants could choose more than 1 category.

Participants wore sensors during waking hours from the time of surgery until opioid discontinuation. Time of opioid discontinuation ranged from 2 to 35 days (M = 8.3, SD = 6.0) from day of surgery. On average, participants wore devices for 66.9% of waking hours (11.37 h/d based upon a 17-hour waking day).

A total of 484 data records were collected for analysis, with all session records for the same participants documented as individual observations. Sessions were defined as every period that the E4 was collecting data (from turning the E4 on to turning it off). From the opioid log records collected, 199 records corresponded to individual opioid report cases. The remaining 285 records were taken as a baseline under the assumption that there were no opioids administered during these intervals.

3.1. Principal component analysis

The data were first restructured in SPSS. Specifically, the multiple baseline (nonopioid) records were averaged, and the session records for the same participants were restructured into longitudinal data instead of being treated as independent observations to avoid overestimating the variance. The restructuring resulted in a sample of 44 participants with 25 feature variables and a maximum of 20 sessions, including the baseline. Principal component analyses with parallel analyses were then conducted for the first 2 restructured sessions containing opioid administration data, as well as the baseline data.³⁰

The data screening showed no missing values at baseline and that the sample size decreased along with the sessions (Table 2). No out-of-range values were found. According to normality test results, only 2 of the 25 features were normally distributed: the y- and z-axis accelerometry means. The other 23 features were all positively skewed. The presence of this general positive skew coincided with the decision to model each sensor’s Hilbert transform data using gamma distribution parameters for the machine learning models. However, given that the PCA was exploratory in nature, no additional data transformation was performed on the data set.

Table 2.

Data restructure sample size for each session.

Session	Valid N	Missing N
1	44	0
2	37	7
3	32	12
4	26	18
5	20	24
6	17	27
7	14	30
8	11	33
9	10	34
10	9	35
11	8	36
12	8	36
13	4	40
14	2	42
15	2	42
16	2	42
17	1	43
18	1	43
19	1	43
20	1	43

The Kaiser–Meyer–Olkin (KMO) test for baseline data was 0.674, indicating acceptable sampling adequacy.¹⁹ The parallel analysis results suggested 3 components accounted for 73.82% of the total explained variance (Table 3). The first component consisted of 16 features, including the 4 parameters (variance, shape, scale, and D) of the three-axis motion variables and 4 heart rate features (mean, shape, scale, and D). The second component contained 6 features, including 1 heart rate (variance), 2 EDA (shape and D), and 3 accelerometer (x-, y-, and z-axis means) variables. The x-axis mean was a weak indicator (loading = 0.278), and the z-axis mean cross-loaded on both components, 1 (0.436) and 2 (0.484). The third component consisted of 3 EDA features (scale, variance, and mean). As aforementioned, other than the x- and z-axis means, the remaining 23 variables were loaded on a single component with no cross-loadings and strong loadings ranging between 0.70 and 0.99.

Table 3.

Factor loadings based on a principal component analysis with orthogonal rotation for 25 feature variables from the baseline sensor signal data.

Features	Component
	1	2	3
HR_Mean.1	0.964	−0.060	−0.020
ACCY_Shape.1	0.961	0.058	−0.093
ACCX_Variance.1	0.942	−0.096	−0.199
ACCX_Shape.1	0.942	−0.247	−0.257
ACCZ_Shape.1	0.912	0.031	−0.171
HR_D.1	0.905	−0.433	0.018
ACCZ_D.1	0.884	0.131	−0.013
ACCZ_Variance.1	0.869	0.049	−0.176
ACCX_D.1	0.863	0.222	0.034
ACCZ_Scale.1	0.852	0.138	0.002
ACCY_Variance.1	0.846	−0.161	0.180
HR_Shape.1	0.817	0.082	0.028
HR_Scale.1	0.814	−0.520	0.025
ACCX_Scale.1	0.780	0.300	0.083
ACCY_D.1	0.693	0.002	0.276
ACCY_Scale.1	0.603	−0.029	0.339
EDA_Shape.1	0.434	0.747	−0.166
HR_Variance.1	0.596	−0.737	0.183
EDA_D.1	0.390	0.704	0.010
ACCZ_Mean.1	0.436	0.484	0.174
ACCY_Mean.1	−0.208	0.470	−0.011
ACCX_Mean.1	−0.039	0.278	−0.032
EDA_Scale.1	−0.089	−0.051	0.995
EDA_Variance.1	−0.246	−0.227	0.975
EDA_Mean.1	0.333	0.335	0.572

Extraction method: principal component analysis. Rotation method: Promax with Kaiser normalization. Rotation converged in 5 iterations. Numbers in bold indicate high factor loadings onto each component based on the standard threshold in SPSS, unless a higher factor loading was present on a different component.

ACCX, accelerometer x-axis; ACCY, accelerometer y-axis; ACCZ, accelerometer z-axis; EDA, electrodermal activity; HR, heart rate.

Compared with the baseline component structure, the sensor data from the first 2 restructured sessions containing opioid administration data showed different patterns (Tables 4 and 5). The KMO for both sessions’ results were low (first session = 0.323; second session = 0.339) showing poor sample adequacy. However, the total explained variance was similar to the baseline data (77.4% and 70.4%, respectively). The parallel analysis suggested 6 components for the first session data and 5 for the second session, with different numbers of feature variables loaded on each component. The heart rate and EDA variables seemed to be components by themselves and remained consistent across both sessions. Some of the y- and z-axis variables also remained a component across the 2 sessions. The feature variables loaded on the remaining components showed no consistent patterns. The differences in component structure between the baseline data set and the opioid administration data sets indicated that different feature sets were more prominent in the baseline data when compared with the opioid administration data. This distinction indicated that there is a detectable difference between the baseline and opioid data sets.

Table 4.

Factor loadings from the first opioid application session data.

Features	Component
	1	2	3	4	5	6
HR_D.2	0.938	−0.065	0.093	0.092	0.131	−0.047
HR_Variance.2	0.932	−0.101	−0.005	−0.035	−0.053	0.052
HR_Scale.2	0.862	0.012	0.185	−0.141	0.323	0.013
HR_Mean.2	0.623	0.120	0.231	−0.249	−0.112	0.298
EDA_Variance.2	−0.012	0.916	−0.119	−0.080	0.018	0.022
EDA_Mean.2	−0.114	0.911	−0.232	−0.116	−0.118	−0.060
EDA_Scale.2	0.052	0.888	−0.239	−0.121	−0.125	−0.044
ACCZ_D.2	0.119	−0.208	0.840	0.005	0.316	0.041
ACCZ_Scale.2	0.170	−0.145	0.818	−0.317	0.278	0.139
ACCY_Variance.2	0.493	0.010	0.607	0.043	−0.019	0.071
ACCZ_Variance.2	0.247	−0.247	0.557	0.285	0.041	−0.017
ACCY_Mean.2	0.072	0.133	−0.443	0.141	−0.118	−0.295
ACCZ_Mean.2	0.099	0.344	−0.416	0.382	0.377	0.166
ACCX_Shape.2	−0.143	−0.094	−0.004	0.921	−0.057	−0.026
ACCZ_Shape.2	0.003	−0.205	−0.134	0.847	−0.029	−0.172
ACCY_Shape.2	−0.036	−0.019	0.253	0.598	0.013	−0.559
ACCX_D.2	0.166	−0.236	0.156	−0.201	0.820	−0.128
ACCX_Scale.2	0.213	−0.188	0.216	−0.434	0.763	−0.039
ACCX_Mean.2	−0.106	0.036	−0.028	0.316	0.705	0.161
ACCX_Variance.2	0.029	−0.119	0.373	0.293	0.614	0.124
HR_Shape.2	−0.052	−0.249	−0.175	0.370	−0.539	−0.047
ACCY_Scale.2	0.410	−0.265	0.044	−0.172	0.068	0.799
ACCY_D.2	0.412	−0.310	0.017	−0.008	0.062	0.772
EDA_Shape.2	−0.249	0.440	0.323	−0.036	0.033	0.713
EDA_D.2	−0.249	0.452	0.317	−0.038	0.031	0.711

Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalization. Rotation converged in 16 iterations.

ACCX, accelerometer x-axis; ACCY, accelerometer y-axis; ACCZ, accelerometer z-axis; EDA, electrodermal activity; HR, heart rate.

Table 5.

Factor loadings from the second opioid application session data.

Features	Component
	1	2	3	4	5
ACCZ_Variance.3	0.795	0.141	0.226	−0.018	−0.103
ACCZ_D.3	0.786	0.270	0.034	0.184	−0.263
ACCY_D.3	0.769	−0.274	0.206	−0.024	0.067
ACCZ_Scale.3	0.760	0.259	−0.018	0.205	−0.280
ACCY_Variance.3	0.742	−0.153	0.233	0.218	0.019
ACCY_Scale.3	0.735	−0.309	0.207	−0.053	0.057
ACCZ_Mean.3	−0.466	−0.131	−0.368	0.390	−0.069
EDA_Variance.3	−0.026	0.963	−0.102	0.014	−0.017
EDA_Scale.3	−0.059	0.943	−0.074	−0.070	−0.129
ACCX_Shape.3	0.161	0.904	0.040	0.056	0.161
ACCX_Variance.3	0.185	0.676	0.038	0.509	0.197
EDA_Mean.3	−0.256	0.562	−0.177	−0.095	−0.272
HR_Scale.3	0.101	−0.133	0.933	0.014	−0.032
HR_D.3	0.156	−0.171	0.876	−0.065	−0.184
HR_Variance.3	0.277	−0.151	0.838	0.079	−0.158
HR_Mean.3	0.314	0.145	0.733	0.002	−0.038
ACCZ_Shape.3	0.129	−0.035	0.467	−0.203	0.317
HR_Shape.3	0.290	−0.107	−0.382	−0.124	−0.337
ACCX_D.3	0.182	0.079	−0.123	0.883	−0.023
ACCX_Scale.3	0.165	−0.217	−0.106	0.858	−0.044
ACCX_Mean.3	−0.127	0.070	0.102	0.689	0.058
ACCY_Shape.3	0.372	0.250	0.081	0.435	0.074
EDA_Shape.3	−0.121	−0.032	−0.211	−0.074	0.857
EDA_D.3	−0.122	−0.009	−0.214	−0.074	0.857
ACCY_Mean.3	−0.010	0.020	−0.221	−0.294	−0.625

Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalization. Rotation converged in 7 iterations.

ACCX, accelerometer x-axis; ACCY, accelerometer y-axis; ACCZ, accelerometer z-axis; EDA, electrodermal activity; HR, heart rate.

When comparing the component structure from the first session containing opioid administration data with the second session containing opioid administration data, there was a difference in the order of features within each component. For the component structure of the first opioid administration session, shown in Table 4, the first component contained only heart rate features, followed by EDA features in the second component, and then mostly accelerometry features in the remaining components. For the component structure of the second opioid administration session, shown in Table 5, the first component contained solely accelerometry features, followed by mostly EDA features in the second component, and then mostly heart rate features in the third component. Because a component’s ability to account for variance in the data set decreases as the component number increases, the heart rate features seemed to play a larger role in the structure of the data set for the first administration session. In comparison, accelerometry played a greater role in the data set structure for the second administration session. These differences in the factor loading patterns from the first to the second administration sessions can be attributed to the patterns of motion being more prominent in the second session, allowing the second session data set to be more accurately modeled by the inclusion of accelerometry features.

3.2. Machine learning

When comparing the first and second opioid administration events within a given session, both events provided equally valuable information to the machine learning algorithm because the component structure for the signals after each opioid administration event was not identical to that of a preceding event. Therefore, it was necessary to include training samples from multiple administration events within a session across all sessions for the machine learning algorithms to attempt to accurately predict opioid intake because of the observed changes in the component structure shown by the PCA.

From the PCA, heart rate features and EDA features were consistent across both the first and second opioid administration sessions within the first 3 principal components, leading to their inclusion in the set of features used to train machine learning models to detect opioid use. From the second opioid administration session, several accelerometry features were present in the first principal component. In the event that a given 5-minute window exhibited a component structure similar to that of the second opioid administration session, accelerometry features would be beneficial in making predictions, so they were also included in the set of training features instead of being withheld.

After testing multiple combinations of the collected features, 4 models provided validation accuracies greater than 80%: fine KNN, weighted KNN, fine Gaussian SVM, and ensemble bagged tree. Using a combination of the first 3 principal components determined by the baseline and opioid administration PCAs (the mean, variance, and D features for three-axis accelerometry, EDA, and heart rate), the bagged-tree model provided the highest combination of validation accuracy (83.7%) and area under the receiver operating characteristic (ROC) curve (AUC = 0.92).

The bagged-tree model made predictions by aggregating the results of multiple decision trees, each of which compared the values of the features with internal thresholds as the trees were traversed. Specifically, the machine learning algorithm traversed the decision trees looking for the differences, outlined in Table 6, between all the 5-minute windows extracted after each opioid and baseline event. These differences between the feature set for baseline and opioid events allowed the algorithm to make decisions at each node of the aggregated decision trees to generate a final classification prediction.

Table 6.

Averages ± SEs and P-values (α = 0.05) of the features during baseline (N = 7373) and opioid administration events (N = 5076) across all subjects.

Feature	Baseline	Opioid	P
ACCX_Mean	−12.61 ± 0.40	−13.88 ± 0.48	0.0810
ACCY_Mean	−0.50 ± 0.28	−4.52 ± 0.33	<0.00001
ACCZ_Mean	14.61 ± 0.26	16.22 ± 0.33	<0.00001
ACCX_Variance	400.46 ± 3.79	411.65 ± 5.57	0.0005
ACCY_Variance	627.07 ± 6.39	515.52 ± 7.18	<0.00001
ACCZ_Variance	527.32 ± 4.64	479.79 ± 5.73	<0.00001
ACCX_D	7176 ± 1835	3645 ± 1552	0.0002
ACCY_D	37,598 ± 8305	20,393 ± 5714	0.0013
ACCZ_D	12,789 ± 5120	4515 ± 1156	0.0054
EDA_Mean	1.41 ± 0.05	0.72 ± 0.003	<0.00001
EDA_Variance	1.79 ± 0.34	0.45 ± 0.01	<0.00001
EDA_D	53,300 ± 16,188	1104 ± 67	<0.00001
HR_Mean	86.98 ± 0.18	81.51 ± 0.2	<0.00001
HR_Variance	102.74 ± 1.73	77.64 ± 1.65	<0.00001
HR_D	1118 ± 76	1211 ± 59	<0.00001

p-values less than 0.05 using rank-sum statistics show significant difference.

ACCX, accelerometer x-axis; ACCY, accelerometer y-axis; ACCZ, accelerometer z-axis; HR, heart rate; EDA, electrodermal activity; ACC feature units: unitless (unitless sensor data); EDA feature units: Mean = μS (micro-Siemens); Variance = μS²; D = unitless; HR feature units: Mean = BPM (beats per minute); Variance = BPM²; D = unitless.

By designating the opioid condition as the positive class, the model’s sensitivity (ie, percentage of true-positive cases correctly classified) and specificity (ie, percentage of true-negative cases correctly classified) were 82% and 85%, respectively. In addition, the positive predictive value (ie, percentage of all positive predictions correctly classified) and negative predictive value (ie, percentage of all negative predictions correctly classified) were 85% and 83%, respectively.³¹ The bagged-tree model’s performance was represented numerically and graphically by the confusion matrix and receiver operating characteristic curve in Figure 1.

Figure 1.

Bagged-tree model confusion matrix and ROC curve with the opioid report cases (class 1) designated as the positive class, and the baseline cases (class 0) designated as the negative class. In the confusion matrix on the left, the green tiles contain the number and percentage of correctly predicted values for a given class, while the red tiles contain the number and percentage of incorrectly predicted values. The ROC curve on the right graphs the performance of the bagged-tree model as the model’s parameters are varied to maximize the true-positive rate (TPR), while minimizing the false-positive rate (FPR). The current model with its TPR and FPR is marked as an orange point. AUC, Area under the ROC curve; ROC, receiver operating characteristic.

After training and validation, the remaining 30% of the data set was passed through the trained model to verify its performance during the validation phase. The model’s resulting testing accuracy was 84%, with a sensitivity and specificity of 81% and 88%, respectively. In addition, the positive predictive and negative predictive values were 87% and 82%, respectively. The features used to train the other 3 models and their resulting performance metrics were mean, variance, and distribution shape and scale. Cross-validation and testing performance metrics are shown in Table 7.

Table 7.

Cross-validation and testing performance metrics for the four selected machine learning models.

Model	Cross-validation metrics						Testing metrics
	AUC	Accuracy %	TPR, %	TNR, %	PPV, %	NPV, %	Accuracy, %	TPR, %	TNR, %	PPV, %	NPV, %
FINE KNN	0.84	84.1	85	83	84	84	85.7	88	84	84	87
WEIGHTED KNN	0.90	81.2	83	79	80	82	82.7	84	81	82	84
FINE GAUSSIAN SVM	0.90	82.2	80	84	83	81	83.0	81	85	85	82
BAGGED TREE	0.92	83.7	82	85	85	83	84.0	81	88	87	82

AUC, areas under the ROC curves; NPV, negative predictive values; PPV, positive predictive values; TPR, true-positive rates/sensitivities; TNR, true-negative rates/specificities.Numbers in bold indicate the values of the model that provided the highest combination of AUC and validation accuracy.

3.3. Changes in signal behavior

Upon completion of the machine learning analysis, the signal behaviors for the subjects were examined to observe how they changed over time. Most subjects’ signals changed in the same direction in relation to baseline after opioid administration in accordance with the comparisons outlined in Table 6. A small subset of the windows for each subject during different sessions deviated from the expected relationships. However, there were no instances where, for a given subject, the signals from an entire session deviated from the expected patterns. For the heart rate, EDA, and y-axis accelerometry signals, there is a clear difference in the baseline and periods after opioid use, shown by the baseline to opioid feature comparisons in Table 6. In the cases of heart rate and EDA, these signals tended to show decreased values in periods after opioid use, while the y-axis accelerometry signal tended to vary over a smaller range of values around a higher absolute mean value when compared with the baseline. As for the x- and z-axis accelerometry signals, there were differences, but they were less significant and more dependent on small fluctuations for certain features, such as the x-axis variance and z-axis mean.

Specifically, the samplings in the 14- to-21-minute range (on average) exhibited the most noticeable change with respect to the baseline, leading to greater prediction accuracies for the windows in this time range (windows 15–22). As shown in Figure 2, the window numbers ranging from 15 to 22 for all subjects in the test set exhibited prediction accuracies noticeably higher than the other windows, with most of the individual windows’ accuracies being greater than the testing accuracy of 84%.

Figure 2.

Average bagged-tree model prediction accuracies for each 5-minute window after the administration of an opioid in the test set (ie, the prediction accuracy for window number 1 represents the average prediction accuracy for all the of the first 5-minute windows after the opioid administrations for all subjects in the test set). The solid blue markers highlight windows 15 to 22 where most of the average prediction accuracies noticeably exceeded the bagged-tree model’s testing accuracy of 84% (represented by the red dashed line).

3.4. Postsurgery pain effects

One disadvantage in comparing baseline (nonopioid) and opioid use segments is the potential influence of pain. To explore whether the machine learning model was detecting pain when making predictions for opioid administration events, features were collected from 30-minute periods before each opioid use (ie, postsurgery, preopioid intake period; PS/PO) for a subset of participants. This subset included participants whose average self-reported pain index for their first 3 opioid doses was between 9 and 10 (on a 0–10 scale, where 10 means “most pain”). The 30-minute periods were subdivided into 26 five-minute windows in the same way as the opioid use intake data, and in this analysis, correct predictions for all windows were assumed to be baseline. The windows were then passed as input to the previously trained machine learning algorithm. By averaging the prediction accuracies across the 26 windows from the PS/PO periods for the high-pain subset, the model correctly predicted these windows as baseline with a mean accuracy of 60% and an SE of ± 1.21%. When performing the same analysis using PS/PO periods for all participants, regardless of their pain levels, the model predicted these windows as baseline with a mean accuracy of 52% and an SE of ± 0.70%.

4. Discussion

This study explored the use of a wearable device to detect opioid self-administration periods using a sample of dental patients after extraction surgery. The primary goal of this work was to detect changes in wearable sensor data between baseline (ie, nonopioid use) and opioid use periods. In addition, the secondary goal was to examine the adequacy of a wearable device to detect opioid ingestion in a naturalistic setting, thereby representing an extension of the controlled designs that explored physiological changes after a single intravenous administration.

Changes in the sensor data partially matched their expected behaviors. We expected an overall decrease in motion and heart rate/heart rate variability after opioid use.^43,44 Heart rate data seemed to follow this pattern, but the accelerometer data were more complex. There seems to be a difference in the distribution of the accelerometer over the 3 axes rather than simply a global decrease in accelerometer data.

The differences in physiological data between sessions containing opioid use events, as shown by the PCA, could be attributed to several factors. Some participants had fewer opioid events and had less data collection sessions than other participants, resulting in a smaller sample size for later sessions. This reduction in sample size could lead to session-to-session differences because the results rely on less participant data. Additional session-to-session differences may be a result of residual opioids in the participants’ systems from opioids taken during previous sessions or changes to how participants’ systems respond to the opioids and their side effects (eg, drowsiness) as more were taken. Pain may also be an underlying factor. However, pain’s contributing effects because of session-to-session transitions from low-to-high pain, high-to-low pain, or a constant pain level across sessions are inconclusive because participants’ session-to-session self-reported pain values did not consistently follow a single pattern. These factors may also account for the differences between baseline and PS/PO data between opioid administrations. The accuracy demonstrated in the classification of PS/PO windows of high pain subjects and all subjects as baseline may be because the machine learning model was trained with presurgery windows as baseline. These findings suggest that the PS/PO windows may be a different class from the baseline and the opioid use windows. Training a three-classification model would require more data and perhaps additional features. Namely, baseline represents a period without pain or opioid use; PS/PO data represent a period soon after surgery with pain and without opioid; and the opioid use period represents a period with opioid with or without pain.

A training set and a testing set of data were used to examine machine learning models to optimize sensitivity and specificity of detection. The values meet criteria for “high” estimates of both sensitivity and specificity reported in the medical literature,^14,26 but as noted by Trevethan,⁴⁰ care should be taken to evaluate each of these parameters separately, given the condition being addressed and the consequences to the individual of false-positive and false-negative identifications. In the current case, false positives would suggest that the individual ingested an opioid when they did not, and a false negative suggests that they did not ingest an opioid when they in fact did. Remotely monitoring the use of addictive medications may lead practitioners to err on the side of high sensitivity to best identify misuse of medication given its dangers. However, the use of monitoring with high sensitivity in a different scenario (eg, monitoring abstinence in OUD patients) may alienate the population if a false positive is issued.

The average onset of action from oxycodone and hydromorphone is within 30 minutes, and the peak serum level occurs for both opioids at 60 minutes when administered orally.^1,10,20,28 For this study, the machine learning algorithms were more successful at predicting opioid use when making predictions on the data in the 14- to 21-minute range after opioid administration. This falls within the average onset range for both opioids and allows for opioid detection before reaching the peak serum level at 60 minutes after administration.

4.1. Implications and impact

Using wearable devices to remotely access behavior presents benefits and issues regarding participant autonomy. In this validation study, we did not use active remote transmission of data, an available feature of wearable sensors. In opioid use research, we recognize that location access could infringe upon participants’ confidentiality and privacy. Thus, we recommend that researchers carefully consider the impact that such remote monitoring could have on individuals misusing opioids.

Some benefits of using wearable sensors in opioid prevention and treatment include detecting opioid intake and activating intervention materials sent through text. For patients prescribed opioids, text reminders could include the time of the next allowed dose, instructions to use non-narcotic pain relief, and a warning against concomitant alcohol or sedative use. For those in OUD recovery, text-based treatment could include the nearest location of Narcotics Anonymous or other community support, a reminder of the individual’s reasons for abstinence, and remote overdose monitoring. Because of the refractory nature of OUD, such moment-to-moment treatment could significantly reduce the negative events that typically accompany relapse and may even reduce mortality risk.

4.2. Limitations and future directions

There are limitations to the current work. We used self-report to track opioid administration. Although self-report limits data accuracy and validity, it is a practical way to track outpatient medication administration that moderately correlates with electronic monitoring.³⁷ The use of text reminders, technology-assisted data entry, and social desirability measures would minimize self-report problems in future studies. Also, the cost of wearable sensors renders them impractical for wide dissemination. Given the clear importance of accelerometer data to our models, more affordable activity trackers could perform a similar function, although considerable testing would be required.

Regarding the importance of the accelerometer data, the E4 would detect increased activity for a person who gestures frequently compared with someone who does not gesture at all. For this study, information regarding the tendency for a given participant to use hand gestures was not collected and could not be used to filter out the effects of hand gestures on the prediction accuracy of machine learning. Consequently, electrodermal activity and heart rate were used to train the machine learning models to offset the effects of hand movements unrelated to a subject’s opioid use.

The effects of alcohol and other sedatives on the measurements recorded were not explored. To the best of our knowledge, there is no literature on wearable sensor-based detection for sedative-hypnotic use, which physiologically has some similarities to opioid use. With regards to alcohol use, most literature on sensor-based alcohol detection involved biochemical detection (ie, sweat ethanol concentrations) as opposed to wrist-mounted physiologic measurements. These avenues provide opportunities for future research.

Because the PCA used aggregated data for the baseline, caution should be taken when interpreting the comparison between the baseline and opioid use principal component patterns. Specifically, the PCA data restructure resulted in the baseline components characterizing the average of the 30-minute data intervals while the participants were not under the effects of opioid medication, and the restructure did not account for the influence from any previous cumulative medication effects. Session-to-session cumulative effects, pain, and other factors may affect classifier performance and cause periods between opioid use to differ significantly from baseline. Future research could investigate the effect of these factors and include a large sample to predict 3 classes (eg, baseline, PS/PO, and opioid use) and stratify the features of opioid use period (eg, with and without pain, doses) to classify the PS/PO environment between opioid use events.

Because the machine learning models were trained on overlapping data windows, if there were any underlying patterns present between 2 adjacent windows, there is a chance that the trained models can overfit the data set by learning these underlying patterns in individual patients’ data instead of the differences between opioid and baseline cases. This would result in higher testing accuracies than would be apparent on out-of-sample test cases. However, 10-fold cross-validation over a randomization of the data set was used to mitigate this possibility. Finally, our sample size made it difficult to explore between-group differences (eg, sex and race) that may affect model development. More research is needed to establish the reliability of the current findings; future studies should include larger samples and across multiple clinical settings (eg, emergency department, inpatient, and outpatient).

5. Conclusion

This study explored the use of a wearable device developed for research to detect orally administered opioid medication in outpatients of dental extractions recruited from a public dental clinic. The physiologic signal analysis revealed significant differences between periods immediately after opioid intake and periods before dental extractions. Results from a training and testing model of machine learning indicated that opioid self-administration could be identified with reasonable accuracy. However, more research is needed to clarify whether machine learning could accurately detect opioid use compared with baseline physiology under different conditions and periods. Future work in this area could translate to prevention and treatment strategies for OUD.