Abstract
Background
A common challenge with all opioid use disorder treatment paths is withdrawal management. When withdrawal symptoms are not effectively monitored and managed, they lead to relapse which often leads to deadly overdose. A prerequisite for effective opioid withdrawal management is early identification and assessment of withdrawal symptoms.
Objective
The objective of this research was to describe the type and content of opioid withdrawal monitoring methods, including surveys, scales and technology, to identify gaps in research and practice that could inform the design and development of novel withdrawal management technologies.
Methods
A scoping review of literature was conducted. PubMed, EMBASE and Google Scholar were searched using a combination of search terms.
Results
Withdrawal scales are the main method of assessing and quantifying opioid withdrawal intensity. The search yielded 18 different opioid withdrawal scales used within the last 80 years. While traditional opioid withdrawal scales for patient monitoring are commonly used, most scales rely heavily on patients’ self-report and frequent observations, and generally suffer from lack of consensus on the criteria used for evaluation, mode of administration, type of reporting (e.g., scales used), frequency of administration, and assessment window.
Conclusions
It is timely to investigate how opioid withdrawal scales can be complemented or replaced with reliable monitoring technologies. Use of noninvasive wearable sensors to continuously monitor physiologic changes associated with opioid withdrawal represents a potential to extend monitoring outside clinical setting.
Keywords: Opioid use disorder, Surveys, Scales, Questionnaires, Wearable sensor, Physiological monitoring
Background
Opioids are natural (e.g., morphine), semi-synthetic (e.g., oxycodone) or synthetic (e.g., tramadol and methadone) narcotics primarily used to treat acute and chronic pain [1]. They are often used recreationally due to their euphoric, tranquilizing, and sedative qualities [2]. Misuse and abuse of opioids and associated overdose, referred to as opioid use disorder (OUD) [3], is a serious public health issue [4] and has been declared a public health emergency in United States [5]. Deaths due to opioid overdose continue to rise. For example, the number of opioid overdose deaths was 5 times higher in 2016 than in 1999, accounting for about 115 deaths per day [6].
OUD has significant societal impacts, such as escalating direct and indirect healthcare costs [7–9]. Florence et al. [8] estimated the total economic burden of prescription opioid overdose, abuse, and dependence for the year 2013 to be $78.5 billion. They found that more than a third of this amount ($28.9 billion) resulted from increased healthcare and substance abuse treatment costs. Indirect costs include lost workplace productivity [7], and criminal justice costs [7, 9]. In addition, the observed decline in men’s labor force participation from 1999 to 2015 has been associated with increased opioid prescriptions [10]. The opioid crisis is taking a toll on families, especially children, forcing an unprecedented number to enter foster care due to parental substance use [11].
Several treatment paths exist for OUD including opioid maintenance treatment, detoxification (detox) [2, 12], and medication-assisted treatment [13, 14]. Opioid maintenance treatment, a well-established first line approach to handle OUD [15], assesses suitability of opioid users who wish to start treatment, and provides legal substitutes. The detox process involves administration of medication in a controlled and medically supervised environment to reduce the severity of withdrawal symptoms that occur when people stop using opioids [16]. For opioid users who have expressed an informed choice, detox may be completed within a community-based program in up to 12 weeks or as inpatient in up to 28 days [16, 17]. Agonists, substances that fully bind to and stimulate opioid receptors (e.g., methadone, levomethadyl acetate) and partial agonists (e.g., Buprenorphine) medications are frequently used for both maintenance and detox purposes [12]. Medication-assisted treatment is an evidence-based treatment for OUD that combines behavioral therapy and approved medications including methadone, naltrexone, and buprenorphine [13, 14]. There is evidence suggesting that medication-assisted treatment is more effective than opioid maintenance treatment, and detox [13].
A common challenge with all OUD treatment paths is withdrawal management. Opioid users experience challenging and often severe withdrawal symptoms when they abruptly discontinue or reduce opioid intake. Intensity of opioid withdrawal symptoms depends on, among others, the type of opioid used, duration of usage, underlying medical conditions, and family history [18, 19]. Time frame for opioid withdrawal symptoms may be classified into early, peak, and late phases. The early phase is characterized by acute withdrawal symptoms such as lacrimation, yawning, and rhinorrhea. Symptoms including gastrointestinal symptoms, gooseflesh and aching reach their maximum intensity in the peak phase [19], and tail off in the late phase [18]. Detox that is not followed up with evidence-based treatment leads to a state of dysphoria which may expose patients to a high risk of relapse, and, even more tragically, a high risk of opioid overdose and death [20].
A prerequisite for effective opioid withdrawal management is early identification and assessment of withdrawal symptoms. One common assessment method for inpatient facilities (e.g., rehabilitation centers) is patient observation. This method involves frequent patient observations (in some cases every 1–2 h; [21]) and depending on the legal substitute used for treatment, clinicians may be required to take vital signs every 1.5 to 2 h with patient both sitting and standing [22]. Over the years, several withdrawal scales have been developed and used by clinicians to complement patient observation and to aid in assessing opioid withdrawal in outpatient cases. Most scales require patients to self-report their immediate or recent (e.g., in the past 24 h) symptoms experienced. Many signs and symptoms may be lost during the hours of the day when clinicians are not observing patients or when patients are not self-reporting. While efforts have aimed at improving the sensitivity and specificity of opioid withdrawal detection [23], the withdrawal scales have been associated with several limitations such as recall bias, distortion, and imprecision as well as the burden, both cognitive and psychological, due to recurring self-reporting (by patients) or observations (by caregivers) [24].
While opioid overdose deaths have increased tremendously over the past two decades such trend has been much more stable for chronic diseases, such as heart disease (Fig. 1). In fact, the downward trends in fatalities of other chronic diseases might be associated, in part, with advances in remote monitoring tools and techniques which may suggests that opioid withdrawal and relapse have lagged behind. For example, conditions such as diabetes, obesity, hypertension,
and chronic heart diseases have leveraged and benefited from advancements in smartphone and wearable sensor technologies (e.g., Jakicic et al. [25] monitored obesity using a multi-sensor device worn on the upper arm; Pedone, Chiurco, Scarlata, and Incalzi [26] monitored chronic obstructive pulmonary disease with a multi-sensor). In comparison, technologies to address and combat the opioid epidemic are few and are limited to diversion or monitoring overuse/misuse of prescription opioids (e.g., Electronic Prescribing for Controlled Substances, and CancelRx) with choice of services that would benefit patients [27]. Furthermore, a relatively small number of studies [28–30] have explored the use of biosensors to detect physiological changes associated with opioid intake in real-time.
Our initial steps to address this gap was to investigate available opioid withdrawal assessment methods to uncover opportunities for innovative technologies for withdrawal management. To the best of our knowledge, no study has critically reviewed and synthesized the commonly used opioid withdrawal scales. Additionally, there has not been any review of technological methods for opioid withdrawal management. The objective of this research was to review and document different opioid withdrawal monitoring methods, including surveys, scales and technology, to identify gaps in research and practice that could inform design and development of novel withdrawal management technologies.
Methods
A scoping review of literature was conducted. We searched PubMed, Embase, and Google Scholar for articles published until April 19, 2019 using a combination of search terms (Table 1). We included studies if they (1) were written in English, (2) took place in an outpatient or inpatient setting, medication-assisted or opioid treatment program, rehabilitation center or were experiments (3) employed opioid withdrawal scale(s), wearable sensors, mHealth, or other technologies as predictor of opioid withdrawal severity (4) were peer-reviewed. In addition, we adopted the SPICE (setting, perspective, intervention, comparison, evaluation) framework, listed in Table 2. We excluded studies published in a language other than English, animal studies, alcohol or non-opiate studies, literature reviews, and non-peer-reviewed publications.
Table 1.
Term | Search terms combination |
---|---|
Predictor | (“opi* withdrawal scale*” OR “biosensor*” OR “wearable sensor*” OR “opioid use disorder” OR “substance withdrawal syndrome” OR “opi* withdrawal syndrome” OR “withdrawal symptom*” OR “prescribed opioid use” OR “mHealth” OR “smart sensing tech*”) |
Population | AND (“clinician*” OR “opioid patient*” OR “opioid addict*” OR “opioid user*” OR “caregiver*” OR “physician*” OR “doctor*” OR “surgeon*” OR “healthcare professional” OR “observer” OR “experimenter”) |
Setting | AND (“inpatient” OR “outpatient” OR “opioid treatment cent*” OR “opioid treatment program*” OR “opiate treatment program*” OR “treatment cent*” OR “rehab* cent*” OR “medication-assisted treatment” OR “medication assisted treatment” OR “medication assisted treatment program*” OR “medication-assisted treatment program*” OR “medication-assisted treatment therapy” OR “medication assisted treatment therapy” OR “drug addiction cent”, OR “experiment”) |
Evaluation | AND (“point score” OR “score” OR “severity of opi* withdrawal” OR “opi* withdrawal severity” OR “severity score” OR “severity of withdrawal” OR “withdrawal severity” OR “severity score” OR “score for severity” OR “withdrawal severity score” OR “withdrawal score for severity” OR “opi* withdrawal severity score” OR “opi* withdrawal score for severity”) |
Exclusionary | NOT “animal studies” NOT “rat” NOT “mice” NOT “alcohol” NOT “non-opiate” NOT “literature review” |
Table 2.
Parameter | Inclusion criteria |
---|---|
Setting | Outpatient, inpatient, opioid treatment program, opioid treatment center, rehabilitation center, medication-assisted treatment program, medication-assisted treatment center, medication-assisted therapy, drug addiction centers, experiment |
Population | Opioid patients, opioid users, opioid addicts, opioid dependents, caregivers, clinicians, doctors, surgeons, physicians, healthcare professionals, observer |
Intervention | Opiate withdrawal scales, opioid withdrawal scales, biosensors, wearable sensors, monitoring, physiological monitoring, mHealth |
Comparison | N/A |
Evaluation | Determine severity of withdrawal |
Selection and triage
Article selection was carried out in two stages. In the first stage, two reviewers independently reviewed titles and abstracts against the inclusion and exclusion criteria using a web-based tool for systematic and scoping reviews called Rayyan [31]. The decision to fully review an article was made when both reviewers agreed to include the abstract. The reviewers resolved disagreements regarding article eligibility by discussing with a third reviewer.
In the second stage, the full-text articles were reviewed to determine eligibility. Furthermore, backward and forward reference search were conducted on all full-text articles that met the study selection criteria. Our initial search yielded 312 unique articles from which 104 articles were included after screening the titles and abstracts. Out of these 104 articles, 21 met the inclusion criteria and were included in the final review (see Fig. 2).
Data extraction and analysis
Two reviewers independently read the full text of each article identified for inclusion in the review to extract pertinent data using a data extraction form. The SPICE framework was used to choose elements in the data extraction form. From each article, reviewers independently abstracted the following: setting (e.g., rehabilitation center), population (e.g., opioid user), intervention (e.g., scales, technology), and evaluation (e.g., severity of withdrawal). Reviewers transferred abstracted data to a detailed Excel spreadsheet. Authors met and organized the information extracted from articles into type of withdrawal assessment method, symptoms monitored, and temporal window covered. For the survey-based methods, authors also extracted information on scale name, mode of scale/survey administration and rating criteria employed. Date of scale development was inferred from when the study was first published.
Results
Type of assessments
Of the 21 articles that were included in the final review, 18 (86%) articles assessed opioid withdrawal using scales and surveys. Our search did not yield any empirical studies employing wearable sensors or mobile health (mHealth) apps to monitor opioid withdrawal symptoms. The remaining 3 (14%) articles employed technology to detect physiological changes during opioid intake.
Scales and surveys to monitor opioid withdrawal
The review revealed that opioid withdrawal scales are the main method of assessing and quantifying opioid withdrawal intensity. These scales employ a combination of observable behaviors (signs), patients’ self-reports (symptoms) and/or physiological measures. Overall, this search identified 18 different scales used over the past 80 years (see Fig. 3 for a visual timeline), dating back to Kolb and Himmelsbach’s [32] scale and the most recent being Clinical Opiate Withdrawal Scale (COWS) [19]. Table 3 provides a summary of the scales.
Table 3.
Scale name | Mode of administration | Scale items | Rating criteria | Temporal window covered | Citation |
---|---|---|---|---|---|
Kolb and Himmelsbach Scale | Clinician-administered | 14 | Points-based | 24 h | [33] |
Himmelsbach Scale | Clinician-administered | 14 | Points-based | Hourly or 24 h | [34] |
SDQ1 or Fraser Scale |
Clinician-administered Patient-reported |
6 (clinician) 4 (patient) |
Weighted 5-point scale | 15, 30, 45, 60, 90, 120 min post-drug | [45] |
ARCI2 | Patient-reported | 550 | True or False | Immediate feeling | [46] |
OPW3 | Patient-reported | 29 | 5-point scale | Immediate feeling | [47] |
WOW4 | Clinician-administered | 84 | True/False | Immediate feeling | [35] |
SOW5 | Clinician-administered | 80 | True/False | Immediate feeling | [35] |
Short Form ARCI | Patient-reported | 49 | True/False | Immediate feeling | [48] |
Wang Scale | Clinician-administered | 10 | Points-based | 10, 20, 30 min post injection | [37] |
Judson Scale | Clinician-administered and Patient-reported |
10 (clinician) 7 (patient) |
4-point scale | Prior to zero time, and 10, 20, 30 min post injection | [38] |
OWS6 | Patient-reported | 32 | 4-point scale | 24 h | [39] |
SOWS7 | Patient-reported | 16 | 5-point scale | Immediate feeling | [40] |
OOWS8 | Clinician-administered | 13 | Present/Absent | 10-min observation period | [40] |
ARSW9 | Patient-reported | 16 | 10-point scale | 24 h | [42] |
CINA10 | Clinician-administered | 13 | Point-based | 5, 10, 15 mins post injection | [49] |
SOWS-Gossop | Patient-reported | 10 | 4 -point scale | 24 h | [43] |
SOWQ11 | Patient-reported | 20 | Anchored 100 mm analogue scale | 24 h | [44] |
COWS12 | Clinician-administered | 11 | Weighted scale | 2-min observation period | [19] |
1Single Dose Opiate Questionnaire
2Addiction Research Center Inventory
3Opiate Withdrawal Subjective Experience Scale
4Weak Opiate Withdrawal Scale
5Strong Opiate Withdrawal Scale
6Opiate Withdrawal Scale
7Subjective Opiate Withdrawal Scale
8Objective Opiate Withdrawal Scale
9Adjective Rating Scale for Withdrawal
10Clinical Institute Narcotic Assessment
11Subjective Opiate Withdrawal Questionnaire
12Clinical Opiate Withdrawal Scale
Most of the scales used today are modeled after the Himmelsbach’s [33] scale, which assesses withdrawal syndrome intensity based on changes in observable behaviors (yawning, lacrimation, rhinorrhea, perspiration, tremor, restlessness, emesis, gooseflesh) and physiological measures (hyperpnea, systolic blood pressure, rectal temperature, weight). Using this scale, the total opioid withdrawal syndrome intensity score is computed as the sum of the points scored by each item on the scale.
The scales developed in the 1960s – Single Dose Opiate Questionnaire (SDQ), Addiction Research Center Inventory (ARCI), Opiate Withdrawal Subjective Experience Scale (OPW), focused on subjective effects of opioid withdrawal. To further study these effects, Haertzen, Meketon, and Hooks [34] developed two clinician-administered questionnaires – the 84-item Weak Opiate Withdrawal Scale (WOW) and the 80-item Strong Opiate Withdrawal Scale (SOW) to measure less intense and more intense withdrawal respectively.
With the emergence of methadone maintenance programs in the 1970s, physicians had a harder time establishing diagnosis of opiate dependence. Some patients mastered the art of going from one methadone program to the next, mimicking symptomatology of acute withdrawal to get a large initial dose of methadone. This widespread behavior caused a need for an objective scale to determine late-stage acute opiate withdrawal and withdrawal severity [35]. Researchers (e.g., [36]) developed protocols to precipitate withdrawal in order to assess the level of physical dependence on opioids. Using these protocols, the Wang scale [37], Judson scale [36], Opiate Withdrawal Scale (OWS) [38], Subjective Opiate Withdrawal Scale (SOWS) [39], and Objective Opiate Withdrawal Scale (OOWS) [39] sought to control feigned responses and improve on sensitivity and specificity for detecting withdrawal.
A number of patient-reported outcome instruments have been developed to measure acute symptoms of opioid withdrawal symptoms. These instruments developed to measure withdrawal include OWS [38], SOWS [39], Adjective Rating Scale for Withdrawal (ARSW) [40], Short Opiate Withdrawal Scale (SOWS)-Gossop [41, 42], and Subjective Opiate Withdrawal Questionnaire (SOWQ) [43].
SOWS-Gossop, Clinical Institute Narcotic Assessment (CINA) and Clinical Opiate Withdrawal Scale (COWS) are the most widely used instruments to evaluate opiate withdrawal symptoms in the reviewed articles [19, 22]. SOWS-Gossop [42] is a 10-item, patient-reported scale on which each item is rated as 0 (none), 1 (mild), 2 (moderate), or 3 (severe). It is a short version of the 32-item OWS [38]. CINA is a validated 13-item clinician-administered scale that measures opioid withdrawal signs and symptoms [44]. COWS [19] is a widely used 11-item clinician-administered instrument mostly due to its time-efficiency (i.e., can be completed within 2 min).
Mode of administration
Nine out of 18 available scales are clinician-administered only, 8 are patient-reported only, and 2 are both clinician-administered and patient-reported. Clinician-administered questionnaires are completed using traditional paper and pencil while clinicians talk face-to-face to patients and observe for opioid withdrawal signs. Patient self-administered questionnaires (patient-reported) require patients to complete questionnaires by hand and return them to the clinician. Scales that use both clinician-administered and patient-reported questionnaires (e.g., SDQ) concurrently but independently require both patients and clinicians to use traditional paper and pencil to complete questionnaires. None of the reviewed literature reported on which mode of administration is preferred or provides utility.
Number of scale items
The scales vary in the number of scale items (i.e., signs and/or symptoms) included, ranging from as many as 550 scale items (e.g., ARCI) to as few as 10 items (e.g., SOWS-Gossop). For example, the 550 items on the ARCI subjectively measure drug-induced effects as well as effects associated with personality and psychiatric disorders. The short-form of the ARCI has 49 items that allows for quick administration, and repeated measurements. On the other hand, OWS [38] is a 32-item patient-reported checklist of opiate withdrawal symptoms, with SOWS-Gossop [42] being a 10-item winnowed version.
Scale items are made up of signs and/or symptoms. Whereas a symptom is a manifestation of a disease apparent to a patient alone, a sign is a manifestation of a disease that a clinician and/or patient can perceive [48]. Thus, while signs are usually observed by the clinician, symptoms are self-reported by patients. Lacrimation and yawning were the most frequent signs, appearing on 15 out of the 18 scales. Weight loss, shivering, heart rate, hypernea, temperature change and blood pressure were the least frequent signs, appearing on 2 out of 18 scales (see Fig. 4a). Joint and/or muscle aches was the most frequent symptom, appearing on 11 out of 18 scales (see Fig. 4b). The least frequent scales were pleasant sick and feverish. One of the most widely used scales, COWS, assesses both signs and symptoms. Items include anxiety or irritability, gastrointestinal upset, restlessness, bone or joint aches, sweating, rhinorrhea, tremor, gooseflesh, yawning, pupil size, and pulse rate.
Rating criteria
The scales differ in their use of rating criteria. For each item on a scale, various response categories are used to rate its clinical severity. Across all scales, ratings for items are summed up to create a total score, with a higher score indicating greater severity. Five (ARCI, WOW, SOW, Short Form ARCI, OOWS) of the scales used binary rating systems (“true/false” or “present/absent”). Three out of these 5 scales are clinician administered and the remaining 2 are patient-reported. Here, the total score for each scale is the sum of “true” or “present” items. Only 1 scale, SOWQ, employed a 20-item anchored analog scale. Patients rate each item on a 100 mm analogue scale anchored on both ends by pairs of pleasant and unpleasant feelings. The total score is the sum of analog scores for all 20 items [43].
While most scales use Likert scale ratings, there is no agreement on the type of scale with four (Judson Scale, OWS, SOWS-Gossop out of 18 scales use a 4-point rating, 2 (OPW, SOW) out of 18 use a 5-point rating, 3 (Kolb and Himmelsbach Scale, Himmelsbach Scale, Wang Scale) out of 18 use point-based rating, and 1 (ARSW) out of 18 scales uses a 10-point rating system. One scale, SDQ, uses a weighted 5-point rating system, for which physiological measures are given highest weights than behavioral signs, followed by patient reported symptoms. Many of the studies did not provide the rationale behind choice of points or weights for items on the scales.
The scales also differ in the value of total score. For example, whereas scores for the 11-item CINA range from 0 to 31 and higher scores are associated with more severe withdrawal syndrome, scores for the 11-item COWS range from 0 to 47 with specific ranges for the level of severity (e.g., scores from 5 to 12 are considered mild, scores from 13 to 24 are considered moderate, scores from 25 to 26 are considered moderately severe and scores more than 36 are considered severe withdrawal).
Temporal window covered
Temporal window covered differ among the scales depending, in part, on the nature of the legal substitute used (e.g., methadone or naloxone) to precipitate withdrawal, mode of drug administration (e.g., sublingual or intramuscular), and mode of scale administration. Five out of 18 scales, mostly patient-administered, have 24-h post-drug temporal window. Six scales require addicts to report their “immediate feeling”. The remaining scales varied markedly in temporal window covered, ranging from a 2-min observation period for COWS to 10, 20, 30 min post injection for the Wang scale.
Technologies to monitor opioid intake
Three studies employed wearable biosensors to detect physiological changes during opioid intake. In all three studies, the biosensors were attached to the wrist as a band and the common biomarkers that were monitored included electrodermal activity (EDA), skin temperature, and whole-body acceleration. The temporal windows covered were before, during, and after opioid administration.
The first of these studies, a preliminary observational one, employed a wristband biosensor (Q sensor™, Affectiva, Waltham, MA) to continuously measure electrodermal activity (EDA), skin temperature, and physical movements of 4 patients before, during, and after cocaine and opioid administration. They found that patients with extensive opioid use demonstrated little to no change in EDA while low-moderate users of opioid showed rise in EDA and decrease in skin temperature [28]. In a follow up study, Carreiro et al. [29] employed the same wristband biosensor to continuously measure EDA, skin temperature, and physical movements of 30 patients before, during, and after naloxone administration. They found that although EDA did not show significant difference post administration compared to baseline, participants’ average skin temperature was significantly higher after naloxone administration. Overall movement was found to have decreased significantly after drug administration [29]. In the most recent study, Mahmud et al. [30] collected EDA, skin temperature and accelerometer data from 30 participants who wore the same wristband to develop an automated real-time system that detects opioid intake with 99% accuracy.
Discussion
The scoping review of different opioid withdrawal monitoring methods revealed greater research efforts and emphasis on scales/surveys to monitor symptoms compared to technologies, which focused solely on symptoms associated with opioid uptake. While limited, advancements in non-invasive and wearable sensor technology could potentially serve as efficient complementary solutions in managing withdrawal symptoms either in inpatient or remote settings.
Challenges with existing monitoring methods
Our results show that opioid withdrawal scales are and continue to be the main instrument used to assess and quantify opioid withdrawal symptoms. Over the years, scales have been developed either to assess the degree of physical dependence (before buprenorphine or methadone induction) and/or to compare the efficacy of opioid withdrawal treatments [19]. However, it is very difficult to compare these scales because of their wide heterogeneity, especially, with respect to the number of scale items, rating criteria, and temporal window covered.
CINA, COWS, and SOWS-Gossop appear to be the most widely used in research and clinical settings [19, 22]. In particular, COWS, designed to be administered by a clinician, is known to be more practical and relatively easy to use [19, 22]. However, with respect to the other scales there is no evidence suggesting the usability, practicality or lack thereof. In addition, while there are overlap and similarities between the scales, evidence of cross-validation is largely absent. Future work is needed to investigate context-dependent factors contributing to adoption and efficacy of different scales in different settings.
Monitoring of opioid withdrawal with scales can be conceptualized as sampling of patients’ behavior or experience over time. However, not all signs and symptoms may be captured by questionnaires or retrospective reports of behavior [50]. Reliance on global assessments or retrospective reports prevents clinicians from accurately understanding and characterizing dynamic behavioral changes over time and across situations in both real-world and clinical settings [51].
Another issue is that opioid withdrawal scales that require subjective patients’ reports ask for recall or summary information over long periods of time. Patients’ contexts and mental states at the time of recall often bias their memory retrieval process [50]. This exposes these scales to recall bias [51] which may affect the quality and reliability of the data provided. Indeed, opioid withdrawal scales may be considered as patient-reported outcome measures (PROMs) [52]. A patient-reported outcome is any information on the outcome of healthcare directly reported by a patient without interpretation or modification by a clinician [53]. Administering PROMs that measure subjective symptoms, like opioid withdrawal scales, requires consideration of issues about data collection – data source (self vs. proxy), mode of administration (self vs. interviewer) and method of administration (paper-and-pencil vs. electronic), and scoring [54]. These issues have limitations which complicate scoring and analysis of opioid withdrawal scales response data [54]. Scales that require direct response may be limited by patients’ cognitive or communication abilities. Those that require a proxy to respond about a patient’s experiences may not accurately represent the patient’s subjective experiences [54]. Scales that require patients to self-administer and record responses have potential for missing data, whereas those that require an interviewer to read questions and record responses suffer from interviewer costs and interviewer bias [55]. Scales that are administered using paper-and-pencil are time consuming and are prone to data entry errors [56]. Patient self-administered PROMs using technology minimizes data entry errors, can provide immediate scoring and are amenable to incorporation within electronic health records [57]. Scores for most PROMs, including opioid withdrawal scales, are based on the classical test theory, and are computed as a linear combination of all items on a particular PROM [58, 59]. For a score on an opioid withdrawal scale to be considered valid, all items must be used, making it test-dependent [54, 58, 59].
The findings suggest that some opioid withdrawal scales may impose high length, complexity and frequency of administration to users. Lengthy questionnaires have been identified as a general obstacle in clinical practice [60] and may lead to increased respondent burden and response fatigue, leading to reduced completion, and reduced data quality [61]. While more recent scales are relatively short, they still require physical and cognitive efforts to understand the questions, recall information from memory, evaluate the connection between the retrieved information and the question, and communicate their responses [62]. Furthermore, patient-reported scales demand that patients do not have visual impairments and are able to read and write in the language of the questionnaire. Since some scales require several observations and measurements per day, there is a need to support clinical decision-making by facilitating the comparison between measurements (e.g., to determine trends over time). No evidence was found in the reviewed literature suggesting that this need is currently fulfilled.
Potential technological solutions
The current method of monitoring opioid withdrawal using scales is challenging outside of clinic or research environments. An opioid monitoring method that accurately monitors withdrawal symptoms as they occur in real time would provide several distinct advantages including the ability to obtain environmental and behavioral contexts surrounding withdrawal symptoms [28], additional source of information does not rely on self-reports and does not suffer from recall bias, and the potential for advanced data analytics, trend analysis, and decision support.
Using noninvasive wearable sensors, including temperature, accelerometer, electrodermal activity, and photo plethysmography sensors, to continuously monitor physiologic changes associated with opioid withdrawal represents a potential to extend monitoring outside clinical setting [63]. Sensors have shown promise to measure symptoms such as tremor, joint/muscle aches [64], gooseflesh [65], and anxiety [66]. Unlike opioid withdrawal scales, wearable technologies have the advantage of using biosensors to continuously measure and record physiologic changes in multiple contexts [67]. Continuous biosensing may provide a way to decrease opioid withdrawal observation time, and possibly allow for remote monitoring [68].
Even though remote monitoring technologies have shown promise in the management of chronic diseases (e.g., [60, 61]), application of such technologies in opioid withdrawal monitoring has been very limited. Short- and long-term physiological symptoms associated with opiate withdrawal timelines have been established [18]. This implies that wearable smart sensing technologies may be used to effectively and reliably identify, evaluate, and communicate physiological responses associated with opioid withdrawal and may inform clinical decision support tools and self-management technologies, with minimal burden to patients or care providers. Additionally, by retroactively examining patient-specific data on withdrawal symptoms and future doses, information can be provided back to clinical researchers to derive better intervention strategies.
While sensor-enabled remote monitoring tools to assess signs and symptoms of opioid withdrawal shows potential, several important challenges remain to be investigated. Most importantly, patient engagement and compliance have consistently shown to be problematic in other domains utilizing such tools and technologies [69, 70], particularly diabetes [71]. Work is needed to understand if such tools and technologies will be used on a sustainable manner and if not, investigate contextual factors contributing to such behaviors.
Limitations
There are some limitations in the study that warrant discussion. The scoping review utilized relatively fewer, albeit relevant, number of databases to identify potentially eligible studies. Despite this limitation, we found saturation in the types and descriptions of subjective methods for opioid withdrawal monitoring (e.g., surveys and scales). The limited findings pertaining to technology-based methods may likely be due to the lagging nature of technology development for opioid withdrawal and/or uptake monitoring. It is also possible that technology design processes are not effectively captured and disseminated using traditional peer-reviewed analysis approach. Indeed, to capture emerging trends in technologies for opioid withdrawal monitoring, non-empirical investigations like technology landscape analysis of commercial technologies (e.g., mHealth apps) may be more appropriate [72].
Conclusions
While traditional opioid withdrawal scales for patient monitoring are commonly used, most scales rely heavily on patients’ self-report and frequent observations, and generally suffer from lack of consensus on the criteria used for evaluation, mode of administration, type of reporting (e.g., scales used), frequency of administration, and assessment window. Therefore, it is timely to investigate how such scales can be complemented or replaced with reliable monitoring technologies. Smart sensing technologies that provide real-time information on the physiological and psychological status of patients have shown promise in addressing a similar need for other conditions such as diabetes, cardiovascular diseases, and mental health and may significantly, and proactively, improve treatment care of opioid patients by keeping clinicians aware of patient status.
Work is in progress to identify viable technologies to assess a wide range of withdrawal symptoms more objectively. This includes a comprehensive systematic review and analysis of past, present and future trends in wearable sensor technologies. Our long-term goal is to develop a technological framework using a human-centered design process that is non-invasive, reliable, and proactive to detect and manage opiate withdrawal symptoms by utilizing an array of sensors that can be employed to detect temporal and spectral patterns of physiological and psychological responses associated with short- and long-term withdrawal symptoms.
Acknowledgments
The open access publishing fees for this article have been covered by the Texas A&M University Open Access to Knowledge Fund (OAKFund), supported by the University Libraries and the Office of the Vice President for Research.
Abbreviations
- ARCI
Addiction research center inventory
- ARSW
Adjective rating scale for withdrawal
- CINA
Clinical institute narcotic assessment
- COWS
Clinical opiate withdrawal scale
- Detox
Detoxification
- EDA
Electrodermal activity
- OOWS
Objective opiate withdrawal scale
- OPW
Opiate withdrawal subjective experience scale
- OUD
Opioid use disorder
- OWS
Opiate withdrawal scale
- PROMs
Patient-reported outcome measures
- SDQ
Single dose opiate questionnaire
- SOW
Strong opiate withdrawal scale
- SOWQ
Subjective opiate withdrawal questionnaire
- SOWS
Subjective opiate withdrawal scale
- SOWS-Gossop
Short opiate withdrawal scale-Gossop
- SPICE
Setting, perspective, intervention, comparison, evaluation
- WOW
Weak opiate withdrawal scale
Authors’ contributions
JKN, FS, RKM, ME contributed to the conception, design, and conduct of the study. All authors contributed to the preparation, review, and approval of the final manuscript and the decision to submit the manuscript for publication.
Funding
This independent work was supported by the Texas A&M Triads for Transformation. The funding body was not involved in any aspects of the study design, analysis, or dissemination.
Availability of data and materials
Not applicable.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Joseph K. Nuamah, Email: jkgnuamah@tamu.edu
Farzan Sasangohar, Email: sasangohar@tamu.edu.
Madhav Erranguntla, Email: merraguntla@tamu.edu.
Ranjana K. Mehta, Phone: 979-458-2369, Email: rmehta@tamu.edu
References
- 1.Monwell B, Bülow P, Gerdner A. Type of opioid dependence among patients seeking opioid substitution treatment: are there differences in background and severity of problems? Subst Abuse Treat Prev Policy. 2016;11(1):23. doi: 10.1186/s13011-016-0066-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schuckit MA. Treatment of opioid-use disorders. N Engl J Med. 2016;375(4):357–368. doi: 10.1056/NEJMra1604339. [DOI] [PubMed] [Google Scholar]
- 3.American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (fifth edition). Washington, DC: American Psychiatric Association; 2013.
- 4.Campbell CI, Bahorik AL, VanVeldhuisen P, Weisner C, Rubinstein AL, Ray GT. Use of a prescription opioid registry to examine opioid misuse and overdose in an integrated health system. Prev Med. 2018;110:31–37. doi: 10.1016/j.ypmed.2018.01.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pravetoni M, Robinson C, Schmiel SE, Mueller DL. Alum adjuvant is more effective than MF59 in promoting early germinal center formation in response to peptide-protein conjugates and enhancing efficacy of candidate vaccines against opioid abuse in adult and old mice. J Immunol. 2018;200(1 Supplement):181.20. [Google Scholar]
- 6.Centers for Disease Control and Prevention, & National Center for Health Statistics. (2017). Underlying Cause of Death 1999–2015 on CDC WONDER Online Database, released 2017. Data are from the Multiple Cause of Death Files, 1999–2016, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Accessed at http://wonder.cdc.gov/ucd-icd10.html on August 1, 2018.
- 7.Birnbaum HG, White AG, Schiller M, Waldman T, Cleveland JM, Roland CL. Societal costs of prescription opioid abuse, dependence, and misuse in the United States. Pain Med. 2011;12(4):657–667. doi: 10.1111/j.1526-4637.2011.01075.x. [DOI] [PubMed] [Google Scholar]
- 8.Florence CS, Zhou C, Luo F, Xu L. The economic burden of prescription opioid overdose, abuse, and dependence in the United States, 2013. Med Care. 2016;54(10):901–906. doi: 10.1097/MLR.0000000000000625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.McAdam-Marx C, Roland CL, Cleveland J, Oderda GM. Costs of opioid abuse and misuse determined from a Medicaid database. J Pain Palliat Care Pharmacother. 2010;24(1):5–18. doi: 10.3109/15360280903544877. [DOI] [PubMed] [Google Scholar]
- 10.Krueger AB. Where have all the workers gone?: an inquiry into the decline of the U.S. labor force participation rate. Brook Pap Econ Act. 2017;2017(2):1–87. doi: 10.1353/eca.2017.0012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rivlin AM, Matthew DB. Unfinished business: bipartisan help for child victims of the opioid crisis. 2017. [Google Scholar]
- 12.Stotts AL, Dodrill CL, Kosten TR. Opioid dependence treatment: options in pharmacotherapy. Expert Opin Pharmacother. 2009;10(11):1727–1740. doi: 10.1517/14656560903037168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fullerton CA, Kim M, Thomas CP, Lyman DR, Montejano LB, Dougherty RH, et al. Medication-assisted treatment with methadone: assessing the evidence. Psychiatr Serv. 2014;65(2):146–157. doi: 10.1176/appi.ps.201300235. [DOI] [PubMed] [Google Scholar]
- 14.Maglione MA, Raaen L, Chen C, Azhar G, Shahidinia N, Shen M, et al. Effects of medication assisted treatment (MAT) for opioid use disorder on functional outcomes: a systematic review. J Subst Abus Treat. 2018;89:28–51. doi: 10.1016/j.jsat.2018.03.001. [DOI] [PubMed] [Google Scholar]
- 15.Soyka M, Strehle J, Rehm J, Bühringer G, Wittchen H-U. Six-year outcome of opioid maintenance treatment in heroin-dependent patients: results from a naturalistic study in a nationally representative sample. Eur Addict Res. 2017;23(2):97–105. doi: 10.1159/000468518. [DOI] [PubMed] [Google Scholar]
- 16.Diaper AM, Law FD, Melichar JK. Pharmacological strategies for detoxification. Br J Clin Pharmacol. 2014;77(2):302–314. doi: 10.1111/bcp.12245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pilling S, Strang J, Gerada C. Psychosocial interventions and opioid detoxification for drug misuse: summary of NICE guidance. BMJ. 2007;335(7612):203–205. doi: 10.1136/bmj.39265.639641.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Farrell M. Opiate withdrawal. Addiction. 1994;89(11):1471–1475. doi: 10.1111/j.1360-0443.1994.tb03745.x. [DOI] [PubMed] [Google Scholar]
- 19.Wesson DR, Ling W. The clinical opiate withdrawal scale (COWS) J Psychoactive Drugs. 2003;35(2):253–259. doi: 10.1080/02791072.2003.10400007. [DOI] [PubMed] [Google Scholar]
- 20.Nunes EV, Gordon M, Friedmann PD, Fishman MJ, Lee JD, Chen DT, et al. Relapse to opioid use disorder after inpatient treatment: protective effect of injection naltrexone. J Subst Abus Treat. 2018;85:49–55. doi: 10.1016/j.jsat.2017.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.American Society of Addiction Medicine: ASAM National Practice Guideline for the use of medications in the treatment of addiction involving opioid use. 2015. ASAM website. https://www.asam.org/docs/default-source/practice-support/guidelines-and-consensus-docs/asam-national-practice-guideline-supplement.pdf. Accessed 20 Sept 2018. [DOI] [PMC free article] [PubMed]
- 22.Sigmon SC, Bisaga A, Nunes EV, O'Connor PG, Kosten T, Woody G. Opioid detoxification and naltrexone induction strategies: recommendations for clinical practice. Am J Drug Alcohol Abuse. 2012;38(3):187–119. doi: 10.3109/00952990.2011.653426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Tompkins DA, Bigelow GE, Harrison JA, Johnson RE, Fudala PJ, Strain EC. Concurrent validation of the clinical opiate withdrawal scale (COWS) and single-item indices against the clinical institute narcotic assessment (CINA) opioid withdrawal instrument. Drug Alcohol Depend. 2009;105(1–2):154–159. doi: 10.1016/j.drugalcdep.2009.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Fishman SM, Wilsey B, Yang J, Reisfield GM, Bandman TB, Borsook D. Adherence monitoring and drug surveillance in chronic opioid therapy. J Pain Symptom Manag. 2000;20(4):293–307. doi: 10.1016/s0885-3924(00)00195-0. [DOI] [PubMed] [Google Scholar]
- 25.Jakicic JM, Davis KK, Rogers RJ, King WC, Marcus MD, Helsel D, et al. Effect of wearable technology combined with a lifestyle intervention on long-term weight loss: the IDEA randomized clinical trial. Jama. 2016;316(11):1161–1171. doi: 10.1001/jama.2016.12858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Pedone C, Chiurco D, Scarlata S, Incalzi RA. Efficacy of multiparametric telemonitoring on respiratory outcomes in elderly people with COPD: a randomized controlled trial. BMC Health Serv Res. 2013;13(1):82. doi: 10.1186/1472-6963-13-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Uhrig P. Changing the course of the opioid epidemic: the power and promise of proven technology. 2018. [Google Scholar]
- 28.Carreiro S, Smelson D, Ranney M, Horvath KJ, Picard RW, Boudreaux ED, et al. Real-time mobile detection of drug use with wearable biosensors: a pilot study. J Med Toxicol. 2015;11(1):73–79. doi: 10.1007/s13181-014-0439-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Carreiro S, Wittbold K, Indic P, Fang H, Zhang J, Boyer EW. Wearable biosensors to detect physiologic change during opioid use. J Med Toxicol. 2016;12(3):255–262. doi: 10.1007/s13181-016-0557-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Mahmud MS, Fang H, Wang H, Carreiro S, Boyer E. Automatic detection of opioid intake using wearable biosensor. In: 2018 international conference on computing, networking and communications (ICNC): IEEE; 2018. p. 784–8. [DOI] [PMC free article] [PubMed]
- 31.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. doi: 10.1186/s13643-016-0384-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kolb L, Himmelsbach CK. Clinical studies of drug addiction, III: a critical review of the withdrawal treatments with method of evaluating abstinence syndromes. Am J Psychiatr. 1938;94(4):759–799. [Google Scholar]
- 33.Himmelsbach CK. The morphine abstinence syndrome, its nature and treatment. Ann Intern Med. 1941;15(5):829–839. [Google Scholar]
- 34.Haertzen CA, Meketon MJ, Hooks NT. Subjective experiences produced by the withdrawal of opiates. Br J Addict. 1970;65:245–255. doi: 10.1111/j.1360-0443.1970.tb01159.x. [DOI] [PubMed] [Google Scholar]
- 35.Wang RI, Wiesen RL, Lamid S, Roh BL. Rating the presence and severity of opiate dependence. Clin Pharmacol Ther. 1974;16(4):653–658. doi: 10.1002/cpt1974164653. [DOI] [PubMed] [Google Scholar]
- 36.Judson BA, Himmelberger DU, Goldstein A. The naloxone test for opiate dependence. Clin Pharmacol Ther. 1980;27(4):492–501. doi: 10.1038/clpt.1980.69. [DOI] [PubMed] [Google Scholar]
- 37.Blachly PH. Naloxone for diagnosis in methadone programs. Jama. 1973;224(3):334–335. [PubMed] [Google Scholar]
- 38.Bradley BP, Gossop M, Phillips GT, Legarda JJ. The development of an opiate withdrawal scale (OWS) Br J Addict. 1987;82:1139–1142. doi: 10.1111/j.1360-0443.1987.tb03294.x. [DOI] [PubMed] [Google Scholar]
- 39.Handelsman L, Cochrane KJ, Aronson MJ, Ness R, Rubinstein KJ, Kanof PD. Two new rating scales for opiate withdrawal. Am J Drug Alcohol Abuse. 1987;13(3):293–308. doi: 10.3109/00952998709001515. [DOI] [PubMed] [Google Scholar]
- 40.Bickel WK, Stitzer ML, Bigelow GE, Liebson IA, Jasinski DR, Johnson RE. Buprenorphine: dose-related blockade of opioid challenge effects in opioid dependent humans. J Pharmacol Exp Ther. 1988;247(1):47–53. [PubMed] [Google Scholar]
- 41.Vernon MK, Reinders S, Mannix S, Gullo K, Gorodetzky CW, Clinch T. Psychometric evaluation of the 10-item short opiate withdrawal scale-Gossop (SOWS-Gossop) in patients undergoing opioid detoxification. Addict Behav. 2016;60:109–116. doi: 10.1016/j.addbeh.2016.03.028. [DOI] [PubMed] [Google Scholar]
- 42.Gossop M. The development of a short opiate withdrawal scale (SOWS) Addict Behav. 1990;15:487–490. doi: 10.1016/0306-4603(90)90036-w. [DOI] [PubMed] [Google Scholar]
- 43.Loimer N, Linzmayer L, Grünberger J. Comparison between observer assessment and self rating of withdrawal distress during opiate detoxification. Drug Alcohol Depend. 1991;28(3):265–268. doi: 10.1016/0376-8716(91)90059-8. [DOI] [PubMed] [Google Scholar]
- 44.Peachey JE, Lei H. Assessment of opioid dependence with naloxone. Br J Addict. 1988;83(2):193–201. doi: 10.1111/j.1360-0443.1988.tb03981.x. [DOI] [PubMed] [Google Scholar]
- 45.Haertzen CA, Harris EH, Belleville RE. Development of the addiction research center inventory (ARCI): selection of items that are sensitive to the effects of various drugs. Psychopharmacology. 1963;4(3):155–166. doi: 10.1007/BF02584088. [DOI] [PubMed] [Google Scholar]
- 46.Haertzen CA, Meketon MJ. Opiate withdrawal as measured by the addiction research center inventory (ARCI) Dis Nerv Syst. 1968;29(7):450–455. [PubMed] [Google Scholar]
- 47.Martin WR, Sloan JW, Sapira JD, Jasinski DR. Physiologic, subjective, and behavioral effects of amphetamine, methamphetamine, ephedrine, phenmetrazine, and methylphenidate in man. Clin Pharmacol Ther. 1971;12(2part1):245–258. doi: 10.1002/cpt1971122part1245. [DOI] [PubMed] [Google Scholar]
- 48.King LS. Signs and symptoms. JAMA. 1968;206(5):1063–1065. [PubMed] [Google Scholar]
- 49.Fraser HF, Van Horn GD, Martin WR, Wolbach AB, Isbell H. Methods for evaluating addiction liability. (A) “attitude of opiate addicts toward opiate-like drugs, (B) a short-term “direct” addiction test”. J Pharmacol Exp Ther. 1961;133:371–374. [PubMed] [Google Scholar]
- 50.Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415. [DOI] [PubMed] [Google Scholar]
- 51.Prince SA, Reid RD, Bernick J, Clarke AE, Reed JL. Single versus multi-item self-assessment of sedentary behaviour: a comparison with objectively measured sedentary time in nurses. J Sci Med Sport. 2018;21(9):925–929. doi: 10.1016/j.jsams.2018.01.018. [DOI] [PubMed] [Google Scholar]
- 52.Bansal D, Bhagat A, Schifano F, Gudala K. Role of patient-reported outcomes and other efficacy endpoints in the drug approval process in Europe (2008–2012) J Epidemiol Glob Health. 2015;5(4):385–395. doi: 10.1016/j.jegh.2015.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Black N. Patient reported outcome measures could help transform healthcare. Bmj. 2013;346:f167. doi: 10.1136/bmj.f167. [DOI] [PubMed] [Google Scholar]
- 54.Cella DF, Hahn EA, Jensen SE, Butt Z, Nowinski CJ, Rothrock N, Lohr KN. Patient-reported outcomes in performance measurement (Vol. 97) Research Triangle Park, NC: RTI Press; 2015. [PubMed] [Google Scholar]
- 55.Mercieca-Bebber R, Palmer MJ, Brundage M, Calvert M, Stockler MR, King MT. Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review. BMJ Open. 2016;6(6):e010938. doi: 10.1136/bmjopen-2015-010938. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.MacKenzie Scott B., Podsakoff Philip M. Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies. Journal of Retailing. 2012;88(4):542–555. [Google Scholar]
- 57.Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health. 2009;12(4):419–429. doi: 10.1111/j.1524-4733.2008.00470.x. [DOI] [PubMed] [Google Scholar]
- 58.DeVellis Robert F. Classical Test Theory. Medical Care. 2006;44(Suppl 3):S50–S59. doi: 10.1097/01.mlr.0000245426.10853.30. [DOI] [PubMed] [Google Scholar]
- 59.Martinez-Martin P. Composite rating scales. J Neurol Sci. 2010;289(1–2):7–11. doi: 10.1016/j.jns.2009.08.013. [DOI] [PubMed] [Google Scholar]
- 60.Mark TL, Johnson G, Fortner B, Ryan K. The benefits and challenges of using computer-assisted symptom assessments in oncology clinics: results of a qualitative assessment. Technol Cancer Res Treat. 2008;7(5):401–405. doi: 10.1177/153303460800700508. [DOI] [PubMed] [Google Scholar]
- 61.Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14(8):1101–1108. doi: 10.1016/j.jval.2011.06.003. [DOI] [PubMed] [Google Scholar]
- 62.Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health. 2005;27(3):281–291. doi: 10.1093/pubmed/fdi031. [DOI] [PubMed] [Google Scholar]
- 63.Chintha KK, Indic P, Chapman B, Boyer EW, Carreiro S. Wearable biosensors to evaluate recurrent opioid toxicity after naloxone administration: a Hilbert transform approach. Proc Annu Hawaii Int Conf Syst Sci. 2018;2018:3247–3252. [PMC free article] [PubMed] [Google Scholar]
- 64.Treister R, Kliger M, Zuckerman G, Aryeh IG, Eisenberg E. Differentiating between heat pain intensities: the combined effect of multiple autonomic parameters. PAIN®. 2012;153(9):1807–1814. doi: 10.1016/j.pain.2012.04.008. [DOI] [PubMed] [Google Scholar]
- 65.Benedek M, Kaernbach C. Physiological correlates and emotional specificity of human piloerection. Biol Psychol. 2011;86(3):320–329. doi: 10.1016/j.biopsycho.2010.12.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Rosebrock LE, Hoxha D, Norris C, Cacioppo JT, Gollan JK. Skin conductance and subjective arousal in anxiety, depression, and comorbidity. J Psychophysiol. 2016;31(4):145–57.
- 67.Krinsley JS, Chase JG, Gunst J, Martensson J, Schultz MJ, Raccone FS, Wernerman J, Bohe J, De Block C, Desaive T, Kalfon P, Preiser J. Continuous glucose monitoring in the ICU: clinical considerations and consensus. Crit Care. 2017;21:197. doi: 10.1186/s13054-017-1784-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Steinhubl SR, Waalen J, Edwards AM, Ariniello LM, Mehta RR, Ebner GS, et al. Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS randomized clinical trial. JAMA. 2018;320(2):146–155. doi: 10.1001/jama.2018.8102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Bertz JW, Epstein DH, Preston KL. Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research. Addict Behav. 2018;83:5–17. doi: 10.1016/j.addbeh.2017.11.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Schubart JR, Stuckey HL, Ganeshamoorthy A, Sciamanna CN. Chronic health conditions and internet behavioral interventions: a review of factors to enhance user engagement. Comput Inform Nurs. 2011;29(2):81–92. doi: 10.1097/NCN.0b013e3182065eed. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Nelson LA, Coston TD, Cherrington AL, Osborn CY. Patterns of user engagement with mobile-and web-delivered self-care interventions for adults with T2DM: a review of the literature. Curr Diab Rep. 2016;16(7):66. doi: 10.1007/s11892-016-0755-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Guderian Carsten C. Identifying Emerging Technologies with Smart Patent Indicators: The Example of Smart Houses. International Journal of Innovation and Technology Management. 2019;16(02):1950040. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.