Abstract
Improving clinical outcomes remains the gold standard in advancing healthcare. Focusing on outcomes holds the potential to unite all clinical stakeholders including payers, industry, providers, and patients. Yet, the dominant ways in which outcomes are captured, provider-collected or patient-reported, have significant limitations. The emerging field of biosensors and wearables, which aims to capture many types of health data, holds promise to specifically capture outcomes while complementing existing outcome collection methods. A digital outcome measure, unlike a traditional provider-collected or patient-reported outcome measure, depends less on active patient or provider participation. Thus, digital outcome measures may be more amenable to standardization as well as greater collection consistency, frequency, and accuracy.
Keywords: Outcome, Wearables, Sensors, Digital health
Introduction
The technology industry has created an enormous set of health technologies that create patient-generated health data, most of which are not developed to represent or capture clinical outcomes. Where current technology may collect blood pressure or heart rate in patients with coronary artery disease, a digital outcome-focused biosensor identifies ambulatory function and time spent at work [1, 2]. Currently, commercially available technologies largely focus on vital signs and physical activity, the latter of which may characterize ambulation, exercise, and sleep. Some of these technologies, including movement-related digital sensors, have the promise to measure outcomes. Digital outcome measures represent an im portant new focus and unrecognized opportunity for these emerging health technologies.
Why Do Clinical Outcomes Matter?
Clinical outcomes are measures by which all clinical stakeholders can align their goals and efforts. In healthcare today, although the primary patient goal is to maximize outcomes, payer and provider financial incentives are largely unaligned with outcomes [1]. Instead, they are aligned with care volume and with delivering high margin or high reimbursement services and products. Further, measures of care quality mainly depict care processes and only indirectly reflect true clinical outcomes.
In an idealized version of healthcare, providing the best possible outcome for each patient would be the primary goal of patients, providers, care systems, payers, government, and industry. Focusing on outcomes in this context provides each of the different stakeholders with common means to measure, track, and compare the care received and achieved. Further, outcomes would be the sole determinant of care costs and reimbursements.
The healthcare community is striving to create standard outcome measures across the array of specific clinical conditions. This movement would allow the varied stakeholders to manage care using the same measurement and reporting systems across an aligned goal. Leading voices of the outcome-focused and “value-based” healthcare approach created the International Consortium for Health Outcomes Measurement [2]. In 2017, this consortium had published 23 outcome measure sets for conditions they reported to represent 54% of the global disease burden [3].
Each measure set aims to capture the wide range of outcomes relevant to each condition, incorporating patient and provider voices. Currently, however, only a fraction of these measures is reliably obtained in practice. Outcome measurement is further challenged by collecting data across varied provider systems and care settings (home, emergency ward, inpatient ward). These gaps represent opportunities for digitally captured outcome measures.
Traditional Provider-Collected Outcome Measures
Some hospital systems, like the Cleveland Clinic [4], collect and publicly report clinical outcomes beyond what is mandated by regulation. A focus on clinical outcomes has become increasingly important as government and other payers continue to seek ways to improve value, specifically by driving down costs and improving quality. Payers are increasingly demanding a greater proportion of care to be delivered under outcome- or value-based models [5, 6, 7, 8, 9, 10] as opposed to the traditional fee-for-service models.
A Centers for Medicare & Medicaid Services (CMS) program for end-stage renal disease metric-based reimbursement requires some clinical outcome reporting. It mandates mea surement of hospital readmissions, mortality, and care complications, including adverse transfusion-related events and bloodstream infections [10, 11].
While outcomes-focused care is not yet standard, there are multiple additional efforts driving change. Accreditation programs represent some of these efforts. For example, the American Heart Association's and American Stroke Association's “Get With the Guidelines” [12] requires a host of clinical outcome metrics to obtain the desired Certified Comprehensive Stroke Center hospital designation. These include stroke-relevant composite functional scores at presentation (National Institutes of Health Stroke Scale) or 3 months after discharge (modified Rankin scale) [12]. In addition, some specialty societies have developed condition-specific measure sets, but the focus on outcomes remains minimal and integration into clinical medicine varies considerably. For example, the American Academy of Neurology's 2016 mea sure set for essential tremor contains six measures [13].
Patient-Reported Outcome Measures
Patients' perceptions of their symptoms, function, and quality of life are an increasingly recognized type of outcome measure, complementing traditional provider-collected outcome measurement in clinical practice and trials. The adoption of patient-reported outcome measures (PROMs) has been facilitated by advancements of design and validation of PROM methodology, demand for PROMs by multiple clinical stakeholders, and the rapid adoption of enabling technology (smartphones, tablets, internet access, and electronic health record integration) [14, 15]. PROMs monitoring of metastatic cancer, for example, may improve communication between patients and providers and even increase patient survival [16].
Limitations in Current Approaches to Capturing Outcomes
In the current execution, outcome measures are logistically difficult to collect and are resource-intensive to obtain. In part, this originates from the provider model of data collection and reporting. Clinical outcomes may be recorded in clinical notes, but notes are generally unstructured free text, unstandardized, and highly variable, making them untenable vehicles of outcome collection in their current form. Thus, outcomes cannot be reliably derived from typical encounter data. Existing means to measure outcomes predominantly piggyback research efforts or consume tremendous resources such as dedicated data collection teams separate from primary clinical teams. In addition, many clinical outcomes require long periods of time capture, which also limit their ability to be tracked reliably.
As a result, other non-outcome quality measurement types (e.g., process, structural) dominate healthcare measurement. Some serve as outcome proxies. One low back pain measure is the “medical documentation of a physician advising a patient against bed rest lasting 4 days or longer during the initial back pain visit” [17]. Another measure is the “percentage of low back pain patients who received magnetic resonance imaging” [18]. Examples of outcome measures, which more directly represent patients' interests [19], include “disability status” and “ability to work.” Most actual current outcome measurements in practice are not granular condition-specific metrics, but reflect blunt outcomes such as mortality. CMS has a host of such programs [10, 20] that aim to improve outcomes, but copy the model of blunt outcome measures plus additional measures that only indirectly address condition-specific outcomes [20].
Similarly, although three of the measures from American Academy of Neurology's 2016 measure set for essential tremor seek to characterize clinical signs or clinical outcomes (i.e., tremor severity, depression/anxiety, and quality of life), none are actually outcome metrics [13]. Instead, like the low back pain example, they focus on the presence or absence of a provider documenting a particular assessment. This nuance is important because a provider would be assessed by the documentation of recording an outcome instead of the actual outcome. The justification for creating measure sets that are not outcome-focused often centers on real or perceived limitations of existing health systems. Provider and hospital infrastructure, it is often believed, are unable to collect true outcomes. In other words, quality sets are created based on what can be done instead of what should be done. Identifying low-cost, scalable means that integrate with patients' lives and providers' practices would lower the threshold for broader use of outcome-based care and quality assessments.
While providers' and patients' perceptions of the patient clinical status are undoubtedly relevant to outcomes, the current environment precludes population-wide and consistent large-scale outcome collection. While providers and patients generate an extreme digital health data volume [21], the content of true patient-reported and provider-collected outcomes may be obscured by data that are irrelevant, absent, or not easily abstracted.
In addition, PROMs and provider-reported outcomes may suffer from subjective assess ment and survey dependence, both of which can be administered arbitrarily. For example, traditional clinical outcomes are largely collected at arbitrary time intervals such as the 6-month clinic follow-up visit and in artificial settings such as the clinic. PROMs may rely on memory and not necessarily convey an objective reality. This is particularly problematic for fields that depend on patient reporting, like headache and sleep, which can produce unreliable histories [22, 23]. Further, both provider-reported outcomes and PROMs require active, not passive, data entry. This limits the feasibility of collecting information in large amounts or frequently.
In addition, active data entry by the user necessarily entails data voids, periods in which the user should enter data, but does not. For example, a mood disorder mobile application may query the patient twice per day, perhaps at times misaligned with the availability or desire of the patient to comply. Such queries may also lead to “ping fatigue,” lessening the yield of data collection. The intrusive and costly nature of data collection limits these tools' ability to collect outcome data frequently, continuously, or in real-world settings.
Digital Outcome Measures
Digital health tools offer the possibility to overcome many of these problems and offer unique advantages like scale, cost, capture frequency, data volume, measurement objectivity, and setting captured. All clinical stakeholders can benefit. Early work has highlighted potential uses, such as low-resource remote monitoring of congestive heart failure patients obtaining weight, blood pressure, and heart rate [24]. Pharmaceutical companies have already begun integrating the use of technology to incorporate PROMs into clinical trials. Digital outcome measures could augment these efforts by providing a complementary source of patient-centered data [25, 26, 27].
The ultimate goal of outcome measurement is to align stakeholder goals and drive individual- and population-level care improvement. Changing reimbursement paradigms, in cluding a new CMS Current Procedural Terminology code for “remote monitoring,” could unleash remote monitoring and digital outcome measure programs [28, 29].
From a validated condition-specific outcome measure set [30], an array of technologies could be used to capture digital outcome measures, which would complement some existing provider-based and patient-reported methods for high-burden health conditions such as lung cancer [31, 32], diabetes mellitus [33, 34], ischemic heart disease [35], and mental health disorders [36] (Table 1).
Table 1.
Outcomes captured by providers, patients, and digital technologies for high-burden health conditions or risk factors [45, 46, 47, 48, 49]
| Condition | Provider-reported outcome | Patient-reported outcome | Digital outcome measurement | Validation example of digital outcome measure for target condition | Ref. |
|---|---|---|---|---|---|
| Lung cancer | global functional oncology-focused assessment of health through validated questionnaire (e.g., ECOG-PS) | validated lung cancer-focused questionnaire assessing functional status (e.g.,EORTCQLQ-LC13) | wearable activity monitor | wearable activity data correlated with performance status and patient-reported outcomes in cancer patients | 45 |
| Hyperglycemia | provider assessment of hyperglycemia through point of care glucose monitoring and hemoglobin A1c | validated diabetes-focused questionnaire assessingsymptoms (e.g., DSC-R) | disease control (hyperglycemic and hypoglycemic avoidance) through continuous glucose monitoring | continuous glucose monitoring associated with improved hemoglobin A1c and decreased episodes of hypoglycemia | 46 |
| Ischemic heart disease | admission for acute myocardial infarction | validated questionnaire assessing symptoms of angina (e.g., SAQ-7) | disease control through continuous electrocardiogram | developmental stage: biosensor that detects cardiac ischemia through continuous monitoring of cardiac electrical activity (e.g., ST segment monitoring on electrocardiogram) | 47, 48 |
| Mental health | recurrence of depression | validated questionnaire assessing symptoms of depression (e.g., PHQ-9) | multiple biosensors and wearable inputs (electrodermal activity, sleep-related activity, activity tracker-based motion, phone-based communication, phone usage patterns, and physical location) | disease activity correlated with various digital biomarkers (poor sleep, less movement and location diversity, less communication volume and diversity) | 49 |
A representative conditions-specific approach is depicted in Table 2, which demonstrates how digital outcome measures could be used to collect a comprehensive outcome measures set for inflammatory bowel disease [30]. Some outcomes, such as symptoms, activities, and energy level, are captured by existing technologies (mobile symptom diary, smart activity tracker). These are also applicable to other conditions. Other outcomes would require technologies more specific to the disease, such as a skin patch to noninvasively monitor disease-relevant analytes. An ingestible pill could detect intestinal biomarkers relevant to inflammatory relapses or the development of colorectal cancer [37]. Some of these technologies remain in development [37, 38], but represent the frontier of sensor technologies that will facilitate digital outcome measures.
Table 2.
Proposed and potential digital outcome measure tools for inflammatory bowel disease [24, 37, 42, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]
| Technology |
International Consortium for Health Outcomes Measurement measures by domain |
Ref. | |||||||||||||
| category | type | description | symptoms, function, and quality of life |
disutility of care |
healthcare utilization | survival and disease control | |||||||||
| bowel symptoms | discomfort | activities | energy | weight | steroid use | complications | hospital/ED visits | anemia | disease activity | CRC | survival | ||||
| Mobile app | mobile diary | collects patient-reported history including activity, conditions, and symptoms | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 50 | |
| Electronic health information | smart card | microprocessor with sensors and ability to store and transmit traditional information (e.g., medical history, medications, allergies) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 51 | ||||||
| smart badge | encodes and monitors personal health digital information (e.g., location, activity, sleep schedule) for use in health care delivery system or personal use | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 52 | ||||||
| smart ring | wearable that contains sensors, transfers information, transmits emergency signals, and makes payments | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 53 | ||||||
| Activity | activity tracker | monitors physical activity and movement | ✓ | 42 | |||||||||||
| smart watch | acts as biosensor and may connect with other digital devices to monitor diverse activities and health data types | ✓ | 54 | ||||||||||||
| smart footwear | sensors worn on the feet in the form of shoes or socks | ✓ | 55 | ||||||||||||
| Skin | epidermal electronics | electronics transferred to the skin with varied sensor and transmitting capabilities | ✓ | ✓ | ✓ | 56 | |||||||||
| galvanic skin sensors | measures electrical conductivity of skin to gauge physiological and psychological conditions | ✓ | ✓ | ✓ | 57 | ||||||||||
| skin bio patch | soft, flexible wearable with specific sensor and transmitting capabilities (e.g., skin temperature, skin conductance, and pulse wave) | ✓ | ✓ | ✓ | ✓ | 58 | |||||||||
| Device | smart toilet | collects biomarkers and analyzes for known indicators of disease from urine and feces within toilet | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 59 | ||||||
| luminal intestinal monitoring | ingestible devices that monitor chemical, biologic, and physiologic parameters, such as gas constituents | ✓ | ✓ | ✓ | ✓ | ✓ | 37 | ||||||||
| digital drug compliance monitoring | tracks adherence through remote and noninvasive means | ✓ | 60 | ||||||||||||
| mobile emergency response system | wearable notification systems to engage medical attention | ✓ | ✓ | 61 | |||||||||||
| mobile EEG | records/analyzes brain activity remotely | ✓ | ✓ | ✓ | ✓ | 62, 63 | |||||||||
| digital scale | measures weight and transmits data wirelessly | ✓ | 24 | ||||||||||||
| CRC, colorectal cancer. | |||||||||||||||
Limitations of Digital Outcome Measures
The steps required for digital outcome measure creation and implementation will be met with various challenges. First, agreement on and adoption of condition-specific outcomes is neither widespread or integrated into routine clinical care. This challenge applies to all types of outcome measurements. Thus, digital outcome measures cannot be useful unless outcome measures are deemed clinically appropriate and useful. Second, not all health data can be easily captured, validated, and translated into digital outcome measures. This challenge may be particularly relevant to conditions or specialties in which objective outcomes are difficult to measure, like depression and pain. Third, cost effectiveness assessments are needed to determine the value generated by digital outcome measures. Expensive, intrusive, or invasive biosensors will pose the greatest challenges. Fourth, the integration of any data stream, particularly new data streams, into clinical workflows or with other technologies poses special challenges. Current electronic health record systems, which are not designed to easily capture or report provider-recorded outcomes or PROMs, are also not well positioned to accept diverse digital outcome measures from diverse devices and applications. Further, some digital outcome measures may require more than on technology type, device, or sensor to reflect a particular outcome of interest. Although there are wide calls for interoperable devices and technologies across health systems, this remains largely unrealized [39]. The adoption of digital outcome measures and their related technologies can only occur when providers and patients demand them – not simply tolerate them. Thus, digital outcome measure technologies must make patient and provider lives better, easier, or both.
A Digital Outcome Measure Roadmap
The path to implementing digital outcome measures requires a systematic approach (Fig. 1) described in five steps as outlined below.
Fig. 1.
A digital outcome measure roadmap: example case with outcome standards for inflammatory bowel disease [30].
Step 1: Create Standards. Regardless of the outcome collection or reporting mechanism (provider-based, patient-reported, or digital-based), clinical specialists and patients must create and agree upon meaningful outcome measures and definitions. Although many of these metrics should be condition-specific, others can be general and cross-cutting (e.g., activities of daily living). Foundational conditions, like various cardiovascular diseases, in which quality- or value-based reimbursement programs already require outcome reporting are good starting points for digital outcome measure testing and deployment.
Step 2: Choose or Create the Tool(s). Once outcome measures are established for a par ticular condition, technologies can be deployed or created to capture these outcomes. Some conditions and related outcome measures may not be amenable to digital outcome measures, which must be determined in this step or when the digital measure cannot be validated (step 3). Outcome measures relevant to congestive heart failure [40] such as hospital visits, mobilization, normal breathing, and even psychological health are amenable to digital outcome capture. Enabling digital outcome measure applications could utilize smartphone geotagging, wearable or home sensor movement detection, wearable breathing monitors, and smartphone communication monitoring and analytics, respectively.
Step 3: Validate the Tool. Health technologies that capture clinical outcomes must reliably record the desired measurements in the relevant clinical population and setting. The first requirement is technical validation of the data stream. Does the device's step count mea surement reflect actual step count? Next, a digital outcome measure must be validated against a gold outcome clinical outcome (from step 1), which incorporates both traditional provider-captured and patient-reported elements. For example, after a movement tracker has been shown to accurately capture movement, the digital outcome measure relevant to ambulatory function must correlate to digital data with a standard clinical measure, score, or rating of ambulation. Many existing or emerging technologies are not clinically ready. For example, ambulatory function, which could be measured by various digital health tools (Fitbit wearables, Misfit wearables, Apple Watch), has not consistently demonstrated high-fidelity step counting [41, 42]. Further, all validation testing must occur not only in healthy individuals, but in the target population and relevant environment.
Step 4: Integrate the Tool. To be useful, digital outcome measure technologies must be unobtrusive, require minimal patient or provider action, and be integrated into routine activities of patient life, provider care, and the relevant clinical technologies required to analyze, view, or respond to the data. Such technologies should capture relevant data before, during, and after office visits and while patients are in hospital settings. Digital outcome measures would allow continuous or frequent monitoring in settings not routinely accessible to pro viders, like during medical transport, after discharge, and in between clinic visits. Such methods would complement PROM tools when digital outcome measures are not yet able to capture meaningful data (e.g., dyspnea, pain). Digital outcome measure applications must be interoperable with patient or provider data viewing platforms such as the digital patient portals, the electronic health record, or new visualization platforms not currently conceived.
Step 5: Use the Data. Once valid and reliable data have been collected, they should be synthesized into meaningful clinical summaries accessible to providers and patients. Digital outcome measures would then be routine discussion points between patients and providers, allowing providers to objectively understand their patients. They would also allow patients to reliably and effectively communicate with their providers. Since digital outcome measure data could be available before planned encounters (ambulatory and inpatient), they would allow providers to more effectively manage time during the encounter, schedule clinic visits, and time inpatient check-ins. A suite of digital outcome measure tools would complement congestive heart failure patient home checkups nurse-run programs [43]. Digital outcome measure data would arm a central triage command center with meaningful health data that, when combined with decision support tools, would result in better targeting of interventions during health checkups.
Conclusions
The only way to successfully manage care is to measure it. The measurements must reflect the goal of healthcare, which is to achieve the best possible outcomes – not the best possible process. The ongoing tremendous efforts to create and capture quality measures, although well intended, have created an incredibly voluminous library of metrics. These will likely continue to consume immense resources and be met with criticism, skepticism, and frustration by providers and hospitals that struggle to keep up with these evolving measure mandates [44]. Outcome measures are limited in number and do not change with evolving care practices. Further, they should be relevant to any care stakeholder. Outcome-based healthcare will continue to submit to fee-for-service care until robust means to capture outcomes are integrated into patient lives and provider workflows. Digital health technologies, now generating immense amounts of health data, could become legitimate means to capture outcomes more efficiently than current methods. If successful, digital outcome measures would complement current approaches to outcome capture, making these existing methods more meaningful. The simultaneous demand for outcomes and the advancement of technologies that could capture them will create better methods to measure, manage, and engage patients.
Statement of Ethics
The authors have no ethical conflicts to disclose.
Disclosure Statement
The authors have no conflicts of interest to declare.
Funding Sources
Both authors were funded by their institutions (Johns Hopkins University Applied Phys ics Lab, Johns Hopkins Medical Institutions). There were no additional sources of funding.
Author Contributions
Both authors take full responsibility for the data, analyses, and interpretations within the paper, and both contributed to its writing.
Acknowledgments
The authors would like to thank Susan Kennedy and Alan Ravitz of The Johns Hopkins University Applied Physics Lab for their review of this paper.
References
- 1.Berwick DM. Era 3 for medicine and health care. JAMA. 2016;315:1329–1330. doi: 10.1001/jama.2016.1509. [DOI] [PubMed] [Google Scholar]
- 2.Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374:504–506. doi: 10.1056/NEJMp1511701. [DOI] [PubMed] [Google Scholar]
- 3.International Consortium for Health Outcomes Measurement Our Standard Sets. http://www.ichom.org/medical-conditions (accessed April 22, 2018) [Google Scholar]
- 4.Cleveland Clinic Treatment Outcomes. 2016 https://my.clevelandclinic.org/departments/clinical-transformation/depts/quality-patient-safety/treatment-outcomes (accessed April 10, 2018) [Google Scholar]
- 5.Porter ME, Teisberg EO. Redefining Healthcare: Creating Value-Based Competition on Results. Boston, Har vard Business School Press. 2006 [Google Scholar]
- 6.Porter ME. What is value in health care? N Engl J Med. 2010;363:2477–2481. doi: 10.1056/NEJMp1011024. [DOI] [PubMed] [Google Scholar]
- 7.Porter ME, Lee TH. The Strategy That Will Fix Health Care. https://hbr.org/2013/10/the-strategy-that-will-fix-health-care (published October 1, 2013; accessed April 15, 2018) [Google Scholar]
- 8.Porter ME. A strategy for health care reform – toward a value-based system. N Engl J Med. 2009;361:109–112. doi: 10.1056/NEJMp0904131. [DOI] [PubMed] [Google Scholar]
- 9.Blue Cross and Blue Shield of North Carolina Hospital Quality Incentive Program. https://www.bluecrossnc.com/providers/quality-based-programs/hospital-quality-incentive-program (accessed February 10, 2018) [Google Scholar]
- 10.Centers for Medicare & Medicaid Services CMS Value-Based Programs. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/Value-Based-Programs.html (published November 9, 2017; accessed March 10, 2018) [Google Scholar]
- 11.Centers for Medicare & Medicaid Services Infection Monitoring: National Healthcare Safety Network (NHSN) Bloodstream Infection in Hemodialysis Patients Clinical Measure. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/ESRDQIP/Downloads/PY-2018-Technical-Measure-Specifications.pdf (accessed March 2, 2018) [Google Scholar]
- 12.American Heart., Association Stroke Fact Sheet. http://www.heart.org/idc/groups/heart-public/@wcm/@gwtg/documents/downloadable/ucm_451247.pdf (published/reviewed April 16, 2018; accessed July 1, 2018) [Google Scholar]
- 13.American Academy., of Neurology Essential Tremor. Quality Measurement Set. https://www.aan.com/siteassets/home-page/policy-and-guidelines/quality/quality-measures/16etmeasureset_pg.pdf (published January 3, 2017; accessed April 22, 2018) [Google Scholar]
- 14.Dueck AC, Mendoza TR, Mitchell SA, et al. Validity and reliability of the US National Cancer Institute's patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) JAMA Oncol. 2015;1:1051–1059. doi: 10.1001/jamaoncol.2015.2639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Jensen RE, Rothrock NE, DeWitt EM, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53:153–159. doi: 10.1097/MLR.0000000000000289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318:197–198. doi: 10.1001/jama.2017.7156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.American Medical., Association CPT Category II Codes. https://www.ama-assn.org/sites/default/files/media-browser/public/cpt/cpt-cat2-codes-alpha-listing-clinical-topics_0.pdf (published December 2, 2016; ac cessed April 20, 2018) [Google Scholar]
- 18.Office of., the National Coordinator for Health Information Technology Use of Imaging Studies for Low Back Pain. https://ecqi.healthit.gov/ecqm/measures/cms166v6 (published July 12, 2017; accessed April 21, 2018) [Google Scholar]
- 19.Clement RC, Welander A, Stowell C, et al. A proposed set of metrics for standardized outcome reporting in the management of low back pain. Acta Orthop. 2015;86:523–533. doi: 10.3109/17453674.2015.1036696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Chee TT, Ryan AM, Wasfy JH, Borden WB. Current state of value-based purchasing programs. Circulation. 2016;133:2197–2205. doi: 10.1161/CIRCULATIONAHA.115.010268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Inf Sci Syst. 2014;2:3. doi: 10.1186/2047-2501-2-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Harvey AG, Tang NKY. (Mis)perception of sleep in insomnia: a puzzle and a resolution. Psychol Bull. 2012;138:77–101. doi: 10.1037/a0025730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ustun B, Westover MB, Rudin C, Bianchi MT. Clinical prediction models for sleep apnea: the importance of medical history over symptoms. J Clin Sleep Med. 2016;12:161–168. doi: 10.5664/jcsm.5476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Zan S, Agboola S, Moore SA, Parks KA, Kvedar JC, Jethwani K. Patient engagement with a mobile web-based telemonitoring system for heart failure self-management: a pilot study. JMIR Mhealth Uhealth. 2015;3:e33. doi: 10.2196/mhealth.3789. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Gotay CC, Kawamoto CT, Bottomley A, Efficace F. The prognostic significance of patient-reported outcomes in cancer clinical trials. J Clin Oncol. 2008;26:1355–1363. doi: 10.1200/JCO.2007.13.3439. [DOI] [PubMed] [Google Scholar]
- 26.Calvert M, Blazeby J, Altman DG, et al. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. 2013;309:814–822. doi: 10.1001/jama.2013.879. [DOI] [PubMed] [Google Scholar]
- 27.Muoio D. In-Depth: The rise of the digital clinical trial. http://www.mobihealthnews.com/content/depth-rise-digital-clinical-trial (published December 1, 2017; accessed April 5, 2018) [Google Scholar]
- 28.Johnson T. What reimbursement for remote patient monitoring means for adoption. https://medcitynews.com/2017/12/reimbursement-for-remote-patient-monitoring/?rf=1 (published December 7, 2017; ac cessed March 3, 2018) [Google Scholar]
- 29.Centers for Medicare & Medicaid Services Final Policy, Payment, and Quality Provisions in the Medicare Physician Fee Schedule for Calendar Year. 2018 https://www.cms.gov/newsroom/fact-sheets/final- policy-payment-and-quality-provisions-medicare-physician-fee-schedule-calendar-year-2018 (published November 2, 2017; accessed July 1, 2018) [Google Scholar]
- 30.International Consortium for Health Outcomes Measurement Inflammatory Bowel Disease. http://www.ichom.org/medical-conditions/inflammatory-bowel-disease/ (published April 10, 2017; accessed April 4, 2018) [Google Scholar]
- 31.Bouazza YB, Chiairi I, Kharbouchi El O, et al. Patient-reported outcome measures (PROMs) in the management of lung cancer: a systematic review. Lung Cancer. 2017;113:140–151. doi: 10.1016/j.lungcan.2017.09.011. [DOI] [PubMed] [Google Scholar]
- 32.Mak KS, van Bommel ACM, Stowell C, et al. Defining a standard set of patient-centred outcomes for lung cancer. Eur Respir J. 2016;48:852–860. doi: 10.1183/13993003.02049-2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.O'Connor PJ, Bodkin NL, Fradkin J, et al. Diabetes performance measures: current status and future directions. Diabetes Care. 2011;34:1651–1659. doi: 10.2337/dc11-0735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Reaney M, Elash CA, Litcher-Kelly L. Patient reported outcomes (PROs) used in recent phase 3 trials for type 2 diabetes: a review of concepts assessed by these PROs and factors to consider when choosing a PRO for future trials. Diabetes Res Clin Pract. 2016;116:54–67. doi: 10.1016/j.diabres.2016.04.009. [DOI] [PubMed] [Google Scholar]
- 35.McNamara RL, Spatz ES, Kelley TA, et al. Standardized outcome measurement for patients with coronary artery disease: consensus from the International Consortium for Health Outcomes Measurement (ICHOM) J Am Heart Assoc. 2015;4:e001767. doi: 10.1161/JAHA.115.001767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Obbarius A, van Maasakkers L, Baer L, et al. Standardization of health outcomes assessment for depression and anxiety: recommendations from the ICHOM Depression and Anxiety Working Group. Qual Life Res. 2017;26:3211–3225. doi: 10.1007/s11136-017-1659-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kalantar-Zadeh K, Berean K, Ha N, et al. A human pilot trial of ingestible electronic capsules capable of sensing different gases in the gut. Nat Electron. 2018;1:79. [Google Scholar]
- 38.Yang Y, Gao W. Wearable and flexible electronics for continuous molecular monitoring. Chem Soc Rev. 2018 doi: 10.1039/c7cs00730b. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 39.Pronovost P, Palmer S, Ravitz A. What Hospitals Can Learn from Airlines about Buying Equipment. https://hbr.org/2017/06/hospitals-are-dramatically-overpaying-for-their-technology (published June 13, 2017; ac cessed April 22, 2018) [Google Scholar]
- 40.International Consortium for Health Outcomes Measurement Heart Failure. http://www.ichom.org/medical-conditions/heart-failure/ (published October 25, 2017; accessed February 20, 2018) [Google Scholar]
- 41.Beevi FHA, Miranda J, Pedersen CF, Wagner S. An evaluation of commercial pedometers for monitoring slow walking speed populations. Telemed J E Health. 2016;22:441–449. doi: 10.1089/tmj.2015.0120. [DOI] [PubMed] [Google Scholar]
- 42.Xie J, Wen D, Liang L, Jia Y, Gao L, Lei J. Evaluating the validity of current mainstream wearable devices in fitness tracking under various physical activities: comparative study. JMIR Mhealth Uhealth. 2018;6:e94. doi: 10.2196/mhealth.9754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Adib-Hajbaghery M, Maghaminejad F, Abbasi A. The role of continuous care in reducing readmission for patients with heart failure. J Caring Sci. 2013;2:255–267. doi: 10.5681/jcs.2013.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.MacLean CH, Kerr EA, Qaseem A. Time out – charting a path for improving performance measurement. N Engl J Med. 2018;378:1757–1761. doi: 10.1056/NEJMp1802595. [DOI] [PubMed] [Google Scholar]
- 45.Gresham G, Hendifar AE, Spiegel B, et al. Wearable activity monitors to assess performance status and predict clinical outcomes in advanced cancer patients. NPJ Digit Med. 2018;1:7. doi: 10.1038/s41746-018-0032-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Rodbard D. Continuous glucose monitoring: a review of recent studies demonstrating improved glycemic outcomes. Diabetes Technol Ther. 2017;19((suppl 3)):S25–S37. doi: 10.1089/dia.2017.0035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Lee SP, Ha G, Wright DE, Ma Y, Ghaffari R. Highly flexible, wearable, and disposable cardiac biosensors for remote and ambulatory monitoring. NPJ Digit Med. 2018;1:2. doi: 10.1038/s41746-017-0009-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Walsh JA, Topol EJ, Steinhubl SR. Novel wireless devices for cardiac monitoring. Circulation. 2014;130:573–581. doi: 10.1161/CIRCULATIONAHA.114.009024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Ghandeharioun A, Fedor S, Sangermano L, et al. Objective assessment of depressive symptoms with machine learning and wearable sensors data. 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), San Antonio. 2017 [Google Scholar]
- 50.Con D, De Cruz P. Mobile phone apps for inflammatory bowel disease self-management: a systematic as sessment of content and tools. JMIR Mhealth Uhealth. 2016;4:e13. doi: 10.2196/mhealth.4874. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Auber BA, Hamel G. Adoption of smart cards in the medical sector: the Canadian experience. Soc Sci Med. 2001;53:879–894. doi: 10.1016/s0277-9536(00)00388-9. [DOI] [PubMed] [Google Scholar]
- 52.Peixoto R, Ribeiro J, Pereira E, Nunes F, Pereira A. Designing the smart badge: a wearable device for hospital workers; in: Proceedings of PervasiveHealth Conference (PervasiveHealth'18) New York, ACM. 2018 article 4 http://clockworkproject.eu/wp-content/uploads/2018/06/designing_the_smart_badge_a_wearable_device_for_hospital_workers.pdf. [Google Scholar]
- 53.Partheniadis K, Stavrakis M. Springer: Cham; 2017. Nov 29. Designing a smart ring and a smartphone application to help monitor, manage and live better with the effects of Raynaud's phenomenon; in International Conference on Smart Objects and Technologies for Social Good; pp. pp 1–10. [Google Scholar]
- 54.Reeder B, David A. Health at hand: a systematic review of smart watch uses for health and wellness. J Biomed Inform. 2016;63:269–276. doi: 10.1016/j.jbi.2016.09.001. [DOI] [PubMed] [Google Scholar]
- 55.Rahemi H, Nguyen H, Lee H, Najafi B. Toward smart footwear to track frailty phenotypes using propulsion performance to determine frailty. Sensors (Basel) 2018;18:e1763. doi: 10.3390/s18061763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Kim J, Banks A, Cheng H, et al. Epidermal electronics with advanced capabilities in near-field communication. Small. 2015;11:906–912. doi: 10.1002/smll.201402495. [DOI] [PubMed] [Google Scholar]
- 57.Kim J, Kwon S, Seo S, Park K. Highly wearable galvanic skin response sensor using flexible and conductive polymer foam. Conf Proc IEEE Eng Med Biol Soc. 2014;2014:6631–6634. doi: 10.1109/EMBC.2014.6945148. [DOI] [PubMed] [Google Scholar]
- 58.Yoon S, Sim JK, Cho YH. A flexible and wearable human stress monitoring patch. Sci Rep. 2016;6:23468. doi: 10.1038/srep23468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Yamamoto N, Kawashima N, Kitazaki T, et al. Ultrasonic standing wave preparation of a liquid cell for glucose measurements in urine by midinfrared spectroscopy and potential application to smart toilets. J Biomed Opt. 2018;23:1–4. doi: 10.1117/1.JBO.23.5.050503. [DOI] [PubMed] [Google Scholar]
- 60.Hafezi H, Robertson TL, Moon GD, Au-Yeung KY, Zdeblick MJ, Savage GM. An ingestible sensor for measuring medication adherence. IEEE Trans Biomed Eng. 2015;62:99–109. doi: 10.1109/TBME.2014.2341272. [DOI] [PubMed] [Google Scholar]
- 61.Stokke R. The personal emergency response system as a technology innovation in primary health care services: an integrative review. J Med Internet Res. 2016;18:e187. doi: 10.2196/jmir.5727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.McKenzie ED, Lim ASP, Leung ECW, et al. Validation of a smartphone-based EEG among people with epilepsy: a prospective study. Sci Rep. 2017;7:45567. doi: 10.1038/srep45567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Ribeiro DMD, Colunas MFM, Marques FAF, Fernandes JM, Cunha JPS. A real time, wearable ECG and continuous blood pressure monitoring system for first responders. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:6894–6898. doi: 10.1109/IEMBS.2011.6091736. [DOI] [PubMed] [Google Scholar]