A Survey of Challenges and Opportunities in Sensing and Analytics for Risk Factors of Cardiovascular Disorders

Abstract

Cardiovascular disorders cause nearly one in three deaths in the United States. Short- and long-term care for these disorders is often determined in short-term settings. However, these decisions are made with minimal longitudinal and long-term data. To overcome this bias towards data from acute care settings, improved longitudinal monitoring for cardiovascular patients is needed. Longitudinal monitoring provides a more comprehensive picture of patient health, allowing for informed decision making. This work surveys sensing and machine learning in the field of remote health monitoring for cardiovascular disorders. We highlight three needs in the design of new smart health technologies: (1) need for sensing technologies that track longitudinal trends of the cardiovascular disorder despite infrequent, noisy, or missing data measurements; (2) need for new analytic techniques designed in a longitudinal, continual fashion to aid in the development of new risk prediction techniques and in tracking disease progression; and (3) need for personalized and interpretable machine learning techniques, allowing for advancements in clinical decision making. We highlight these needs based upon the current state of the art in smart health technologies and analytics. We then discuss opportunities in addressing these needs for development of smart health technologies for the field of cardiovascular disorders and care.

1. INTRODUCTION

Cardiovascular diseases are the worldwide leading cause of death [22]. In 2016, cardiovascular diseases accounted for nearly one in three deaths in the United States. While the range of cardiovascular diseases and treatments can be broad, the Framingham Heart Study teaches us that a number of the risk factors that lead to primary adverse events or secondary recurrent events are often the same or quite similar [98, 168, 173]. Real-time monitoring of these risk factors (i.e., the signs and symptoms associated with cardiovascular disorders) allows for care providers to track patient progress and to rapidly respond to any changes in patient condition. In the hospital, monitoring patients is part of routine clinical practice. Providers are able to monitor cardiac status and basic vitals from anywhere in the hospital at any time. Slight deterioration in health can be observed and interventions put into place before patients suffer worsening harm. However, length of stay in these acute care settings is often quite short [18, 199], representing only a small portion of a patient’s life despite the prolonged impact that the decision making in these settings has. Such monitoring is currently deficient in remote settings, where the ability to diagnose new conditions or monitor treatment effectiveness based upon measured changes in vitals and cardiac status that are known to be risk factors for primary adverse events or secondary recurrent events is important to prevent future admissions to acute care settings. Monitoring physiologic parameters and symptoms outside of the hospital in ambulatory and/or remote settings can enable better detection and response systems before a person becomes acutely ill and requires hospitalization or after hospitalization to prevent early readmission to the hospital; however, many of the devices today are targeted to healthy people. With the prevalence and ubiquitous nature of remote and wearable sensors, opportunities exist to broaden the applications of sensing and for adapting analytic techniques to enhance diagnosis, monitoring, and treatment of risk factors for primary and secondary prevention of cardiovascular disease. In particular, the ability to capture these measurements is only the first step. Indeed, end-to-end smart health systems are needed that couple the hardware development with advanced analytic techniques to provide both the patient and clinical provider the necessary confidence in data and risk prediction based upon the measured risk factors.

A challenge in monitoring patients with or at risk for cardiovascular disorders is designing the technology and algorithms to support a variety of conditions and signs/symptoms. While the treatment of cardiovascular disorders such as heart failure [98], coronary artery disease [56], and stroke [168] may differ (the latter, for example, moving from monitoring a potential cardiovascular disorder to neurological treatment), they share a common set of cardiovascular risk factors [111, 125, 138]. The selection of these three disorders highlights their global disease burden, but certainly the Framingham Heart Study teaches that the important risk factors that should be monitored are not limited to tracking only these disorders.

Patients at risk for cardiovascular disorders (or recurrent events due to diagnosed cardiovascular disorders) present a number of challenges for remote monitoring and diagnosis because of complexities within the diseases or trajectory leading to the initial diagnosis. Many of these diseases involve seemingly trivial symptoms that may suddenly change from a minor inconvenience to a debilitating lack of function. A patient with a given disease may feel well for multiple years and then suddenly decompensate and require emergent care. Ideally, remote monitoring along with advanced analytics on the captured ambulatory data should be able to track the slow, daily progression of a disease state and alert the patient and healthcare providers to worsening disease before decompensation and patient suffering. However, preliminary studies in remote monitoring have failed at preventing adverse events, such as in preventing repeated hospital admissions in patients diagnosed with heart failure (HF). For example, the Telemonitoring in Patients with Heart Failure trial (Tele-HF) used patient self-reports of daily changes in symptoms, weight, and a variety of other factors (e.g., medication changes, depression scores, etc.) to identify worsening symptoms in an effort to intervene prior to another acute event but did not find a statistically significant difference between control and intervention arms [31]. However, an analysis of participant subgroups did find that patient self-reported data could improve prediction of readmission likelihood, showing potential for more advanced analytic techniques to better identify participant risk and to improve estimates in this space [91]. The BEAT-HF trial was designed as a further exploration in automating the capture of the relevant biometric signals, including heart rate, blood pressure, and weight, using remote sensors rather than participant self-report of such data. This study, however, was similarly unable to find a statistically significant difference in control and intervention arms [142], suggesting that further exploration of additional biomedical signals is needed and that the advancements in improved remote and ambulatory monitoring of these key risk factors, alone, is not sufficient to address clinical need. Instead, improved remote and ambulatory sensing likely needs to be coupled with advancements in analytic techniques to process and interpret data generated by these sensors.

Remote sensing technologies have increased in prevalence and have made personalized health data collection feasible. In human activity recognition (HAR), wearable sensors and inertial measurement units embedded within smartphones and smartwatches have enabled the tracking of detailed motions [15, 95]. Coupled with wearable sensors that capture motion via video, these sensing systems allow for the tracking of motions of healthy participants [32] to tracking of disease state with custom-built sensors, such as smartshoes [46]. The data provided by these wearable and remote sensors has more recently enabled advanced machine learning techniques to identify more complex patterns of motions, better understanding personalized behavior [71, 195]. Eventually, these techniques have emerged to personalize models of activity recognition to individual users, and this personalized modeling provides the most robust interpretation of activities of daily living per user [167], enabling feedback and the measurement of clinical outcomes [42]. This progression from the development of new sensing modalities to the analytic techniques that detect patterns within the data and finally to personalization in tracking and disease progression modeling is an end-to-end pathway that is required for advanced clinical disorder monitoring for smart health technologies.

The development of new sensors to measure risk factors (e.g., symptoms) of cardiovascular disorders would ideally enable a similar progression for tracking of cardiovascular outcomes. These new sensors would be able to identify conditions that may not be apparent to patients or providers, such as different sounds from the heart, slowly decreasing patterns of activity, or a combination of vitals that may appear normal in isolation but may be indicative of risk given a combination of values and certain patient contexts. By identifying dangerous signs before symptoms manifest, earlier interventions can lead to improved health outcomes. A variety of technologies and machine learning techniques to this purpose exist in condition-specific settings [54, 130] to varied success [55, 68, 131]. Understanding the pathologies of the disorders is important in understanding the clinical needs and opportunities that exist in developing new wearable and remote sensors for diagnosis and treatment of a variety of cardiovascular conditions and using advanced analytic techniques that are enabled from the collection of new, comprehensive patient ambulatory risk factor data.

In this survey, we break down the needs and opportunities in monitoring risk factors for the prevention of primary or secondary recurrent adverse events of select cardiovascular disorders into key technological areas that couple remote sensing with analytic developments: (1) we discuss different sensing modalities that have been or that could be applied to tracking cardiac health in remote settings, (2) we consider the opportunities that advanced analytic strategies present with the acquisition of remote sensing data for continuous risk modeling, and (3) we discuss the needs and opportunities for advancements in clinical models using machine learning techniques, including advancements in longitudinal monitoring and interpretability made possible through newer deep learning techniques. As cardiac pathologies manifest, they can also be indirectly observed through physical changes in the body, potentially measured by sensors on or around the body. These changes can be utilized to track patient health, to plan interventions to maximize patient wellness, and to decrease the overall impacts of the disease. One of the oldest technologies used for assessing cardiac health is the stethoscope. In the digital era, the electronic stethoscope is a varied group of technologies that incorporate a microphone in order to automate acoustic diagnosis and facilitate remote monitoring [101]. Other technologies, such as photoplethysmography and sphygmomanometry, allow for remote measurement of the characteristics of a heart beat including heart rate and blood pressure [209]. Doppler radar can detect vital signs such as respiratory rate and heart rate [103]. Electrical techniques such as electrocardiography (ECG) or other conduction studies such as bioimpedance can give insights into the internal physiology of the heart [196].

Sensing systems provide for opportunities to proactively detect and alert patients and physicians to worsening health states. However, to allow for timely and effective interventions as well as to rapidly evaluate the impact of those interventions, development of advanced signal processing and machine learning techniques needs to keep pace with the development of raw sensor modalities. This article presents a survey of state-of-the-art sensing technologies and analytics with respect to monitoring key risk factors for cardiovascular disorders, in order to highlight successes and provide areas for additional growth. Two key ways in which analytics associated with sensing systems can provide support are to develop personalized models for longitudinal tracking of the risk factor measurements and to develop clinical risk prediction models that monitor disease state trajectories for identifying the onset of a new disease and to track the progression of preexisting disease to avoid recurrent adverse events. Tracking the progression of existing disease is the easier task: once an underlying disease state is known, appropriate monitoring can be put into place and utilized to follow the progression of the disease. Monitoring for the start of new disease is more difficult, as the focus is more general. In either case, sensing and clinical characteristics must be combined for decision support with the aid of machine learning approaches. In this article, we survey the current state of the art in patient monitoring and analytics for patient risk and care, highlighting needs and opportunities for advancements in the field of smart health with respect to monitoring signs, symptoms, and treatments in patients at risk for diagnosis and adverse events with respect to cardiovascular disorders. We highlight the need to view this technical challenge as an end-to-end smart health solution, requiring both advancements in sensing systems and advancements in analytic techniques to properly analyze and interpret data generated from these systems. The workflow described in this article towards developing new tools for remote clinical decision support is shown in Figure 1.

Fig. 1.

Overview of a workflow to develop personalized, remote clinical decision support tools for patients to monitor risk factors of cardiovascular disorders. Needs are shown in three categories: needs in sensor development and data handling, needs in continuous data collection and analysis, and needs in developing comprehensive and personalized analytical models. Addressing these three categories will allow for improved personalized remote clinical decision support for patients and the design of end-to-end smart health systems for clinical modeling.

The rest of this work is organized as follows. Section 2 introduces the cardiovascular disorders, their common risk factors, and needs in remote and ambulatory monitoring for these conditions. This section provides a focus for the clinical tasks and describes how the particular case studies generalize to common risk factors and outcomes. Then the article provides a description of the current state of technologies, remaining needs (technical gaps), and opportunities for technological advancements in end-to-end smart health systems designed for addressing the clinical needs by discussing sensing (Section 3), analytics on the sensing systems (Section 4), and clinical analytic models on the data generated for patient and provider use (Section 5). Finally, Section 6 provides a discussion and conclusion.

2. CASE STUDIES AND NEEDS

This work considers risk factors associated with primary adverse events and secondary recurrent adverse events associated with the diagnosis and treatment of cardiovascular disorders. A number of the chronic conditions listed may have disparate treatment patterns; however, the underlying risk factors that lead to the initial events have significant overlap. To highlight this, we consider several conditions, namely, HF, coronary artery disease (CAD), acute myocardial infarction (AMI), and stroke. In particular, we include stroke as a condition given that the primary risk factors are cardiovascular in nature, even if treatment afterwards may tend to be covered by neurologists. In this section we provide a brief overview of the conditions and their measurable factors and provide definitions and abbreviations used throughout the manuscript. Table 1 provides a list of the key terms and definitions for this section.

Table 1.

Abbreviations and Definitions of Key Clinical Terms

Abbreviation	Clinical Term	Definition
AFib	Atrial Fibrillation	Cardiac arrythmia that is highly associated with risk of stroke
AR	Aortic Regurgitation	Disease state where the aortic valve fails to completely close during diastole, allowing for backwards flow of blood into the heart
AS	Aortic Stenosis	Disease state where the aortic valve encounters resistance during opening, requiring additional force from the heart to drive blood forward
BP	Blood Pressure	The pressure present in the arterial circulatory system. Blood pressure os-cillates from a peak value (systolic blood pressure) to a trough value (diastolic blood pressure) as the heart beats
CAD	Coronary Artery Disease	Disease state characterized by impaired blood flow in the small arteries around the heart (the coronary arteries)
ECG	Electrocardiography/Electrocardiogram	Tracing that shows a measurement of cardiac electrical activity
HAR	Human Activity Recognition	Utilizing sensors to classify a patient’s physical activities
HF	Heart Failure	Disease state characterized by an impaired ability of the heart to drive blood forward
HFpEF	HF with Preserved Ejection Fraction	Heart failure state characterized by a normal ejection fraction, often related to defects in ventricular filling during diastole
HFrEF	HF with Reduced Ejection Fraction	Heart failure state characterized by a decreased ejection fraction, often related to impaired ventricular contractility or to pressure overload
HTN	Hypertension	A health condition characterized by chronically increased blood pressure that puts patients at risk of heart disease
(A)MI	(Acute) Myocardial Infarction	Disease state characterized by impaired coronary blood flow leading to some degree of cardiac muscle death. Commonly known as a “heart attack”
MR	Mitral Regurgitation	Disease state where the mitral valve fails to completely close during systole, allowing for backwards flow of blood within the heart
MRI	Magnetic Resonance Imaging	Imaging technique using powerful magnets to image internal structures
NSTEMI	Non-ST-Elevation MI	MI characterized by a lack of ST segment elevation on ECG
NYHA	New York Heart Association	Entity that issues guidelines for heart failure classification
PMI	Point of Maximal Impulse	Point on a patient’s chest where the movement of the patient’s heart can most strongly be felt
S3	Third Heart Sound	A heart sound occurring after the normal second heart sound. It is benign in younger patients, but indicative of pathology in older patients
S4	Fourth Heart Sound	A heart sound occurring prior to the normal first heart sound. It is indicative of pathology
SA	Stable Angina	Chest pain that occurs after a certain amount of exertion caused by restricted blood flow through the coronary arteries
STEMI	ST-elevation MI	MI characterized by an elevation of the ST segment on ECG
TEE	Transesophageal echocardiography	Imaging technique that uses ultrasonography and a probe in the esophagus to evaluate the function of the heart
TIA	Transient Ischemic Attack	Temporary restriction of blood flow to a part of the brain. Similar to a stroke but resolves within minutes
TTE	Transthoracic echocardiography	Imaging technique that uses ultrasonography and a probe on the chest to evaluate the function of the heart
UA	Unstable Angina	Chest pain that occurs at rest caused by restricted blood flow through the coronary arteries

HF is typically a chronic condition where the heart is unable to drive blood forward through the body sufficiently or can only do so under damagingly high pressures. HF is a debilitating disease that causes significant global disease burden. In 2016, HF was the most rapidly growing cardiovascular condition in the world [210]. CAD occurs when blood flow through the coronary arteries, the small arteries that provide blood to the heart, becomes impeded. This occurs both gradually as plaque builds up within the coronary arteries and suddenly when a plaque ruptures and clots. The former causes chest pain and exercise intolerance, while the latter, commonly known as a myocardial infarction (MI), can cause severe pain, loss of consciousness, and death. Each year around 800,000 Americans suffer an AMI, and rapid care following an AMI is a chief predictor for minimizing long-term morbidity and mortality [92, 121, 133]. Stroke is any disease impacting the blood vessels to the brain. In particular, acute stroke is a condition that occurs when either a blood vessel in the brain ruptures or when one of those blood vessels becomes blocked. Stroke manifests with the sudden onset of neurological deficits, some of which may be irreversible. Stroke is the fifth leading cause of death in the United States and is a leading cause of long-term disability [22].

This work considers three primary cardiovascular disorders for the review of gaps and opportunities, though by no means encompasses the entirety of technologies available for monitoring and treating these conditions nor the entirety of conditions to which these technologies could be applied. Instead, these conditions serve as meaningful examples in which technical solutions that monitor and model the known clinical risk factors would be clinically impactful and demonstrate the similarity in key risk factors despite the potentially divergent care required after the diagnosis of each condition. Additional discussion of normal cardiac physiology can be found in the Supplementary Appendix (Section A).

2.1. Clinical Conditions

HF occurs when one or both halves of the heart are unable to drive blood flow forward at the rate required by the body or can only do so under high pressures. This discussion of pathology will focus primarily on left-sided HF rather than right-sided HF, but the two are often closely associated and technologies for monitoring the two will have a large amount of overlap. The two will also often coexist. HF can result from ineffective heart contractions, from high pressure limiting the effect of heart contractions, or from difficulty in filling the heart. The first two causes lead to HF with reduced ejection fraction (HFrEF), and the last leads to HF with preserved ejection fraction (HFpEF). Ineffective heart contractions can result from muscle damage caused by CAD, by chronic volume overload as seen in mitral regurgitation (MR) or aortic regurgitation (AR), or by a family of cardiac muscle disorders known as cardiomyopathies. High pressure can lead to HF either from aortic stenosis (AS) or from uncontrolled hypertension. In either case, the pressure that the heart works against is so high that the pumping becomes ineffective. Difficulty in filling the heart can be caused by ventricular hypertrophy, cardiomyopathy, fibrosis, disease around the outside of the heart (the pericardium), or coronary artery disease (CAD).

CAD is a family of diseases where blood flow through the small arteries of the heart, the coronary arteries, is restricted. This restriction can be caused by deposits of fatty plaques within the arteries or by clotting caused by the rupture of one of these plaques. Depending on the extent of the blood flow restriction and the current oxygen demands of the heart, CAD may cause different symptoms. CAD is represented by a spectrum of conditions that are defined by specific clinical and physiological signs.

Stroke occurs when blood supply in and around the brain is acutely disrupted and results in acute neurologic defects. Ischemic stroke is a type of stroke where a blockage in cerebral arteries rapidly blocks off blood flow, leading to cell death. Hemorrhagic stroke is a type of stroke where a blood vessel in the brain ruptures, rapidly raising pressure inside the skull and causing cell death. Transient ischemic attacks (TIAs) are similar in cause and presentation to strokes but resolve spontaneously. They are often an indicator of underlying disease and put the patient at increased risk for future TIA or stroke. The neurological pathology goes beyond the scope of this work, but there are several notable cardiovascular impairments that may cause a stroke.

A common key risk factor to all the conditions above is hypertension (HTN). HTN is a condition where a patient’s blood pressure is persistently elevated and is often a condition that serves as a modifiable precursor to each of the three cardiovascular disorders discussed [141]. HTN is divided by cause into two categories: primary (or essential) HTN, which has no particular medical cause, and secondary HTN, which is caused by some other medical condition. Primary HTN accounts for roughly 90% of all HTN, while secondary HTN accounts for the remaining 10%. Causes of secondary HTN include renal disease and endocrine diseases that disrupt the body’s natural control of blood pressure [143]. Essential HTN is a diagnosis of exclusion and requires ruling out the possibility of any secondary causes. Risk factors for essential HTN include both hereditary and environmental factors [161]. There is a strong association between HTN, obesity, and insulin resistance. HTN is associated with poor diet, excessive alcohol intake, and age. By measuring blood pressure and identifying patients with HTN, we can consider HTN as a disease state, and potential progression to the other conditions listed in this work, while also similarly considering it a measurable risk factor for those conditions. Because an HTN diagnosis is a modifiable risk factor prevalent in numerous cardiovascular disorders, we highlight it here specifically as a clinical condition in its own right, but consider the measurement of blood pressure as a key sensing parameter for the rest of this work for both diagnosing HTN and using blood pressure directly as a risk factor for the other cardiovascular conditions.

2.2. Needs for Monitoring Signs and Symptoms for Cardiovascular Disorders

Figure 2 illustrates the three primary needs this survey will discuss: (1) need for sensing technologies that track longitudinal trends of the measures important in identifying risk of cardiovascular disorder despite infrequent, noisy, or missing data measurements; (2) need for new analytic techniques designed in a longitudinal, continual fashion to aid in the development of new risk prediction techniques and in tracking disease progression; and (3) need for personalized and interpretable machine learning techniques, allowing for advancements in clinical decision making. A number of varied signs and symptoms exist for HF, CAD, and stroke. The remainder of this section briefly introduces some common signs and symptoms. Here, a symptom is a change caused by disease that is noticed by and likely an irritant to the patient, while a sign is a change that the patient may not notice or that may not be concerning to the patient.

Fig. 2.

Progress from individual building blocks provided by new sensing opportunities to joint, multi-modal analytics to combined end-to-end modeling for clinical use (y axis) and how they generally relate to each of the three conditions (x axis).

In HF, the symptoms result both from insufficient blood flow and from excess fluid buildup. The three main symptoms that are associated with diagnosis of HF and quantification of its severity are dyspnea (shortness of breath) on exertion; sudden, choking dyspnea at night; and difficulty breathing while lying down. In left-sided HF, pulmonary vein pressure increases, causing buildup of fluid in the lungs (pulmonary edema) that worsens while lying down. In right-sided HF, systemic venous congestion results in fluid buildup in the periphery (peripheral edema) that worsens while upright, resulting in noticeable swelling in the wrists and ankles. HF is difficult to precisely define as it is a clinical syndrome resulting from many different heart conditions, and many variants exist. Therefore, attempts to understand HF and to monitor its progression must focus on identifying the symptoms and identifying cardiac dysfunctions. Symptoms that can be measured include peripheral edema (swelling of ankles, rapid weight gain), decreased activity, and changes in respiratory patterns when lying down versus remaining upright. Changes in blood flow to the kidneys result in decreased urine production during the day and increased urine production at night. Patients with HF will therefore often get up frequently in the night. These patients will also likely change posture in the night, with patients with advanced HF needing to sleep upright. One of the most used classification schemes for HF is the New York Heart Association (NYHA) Functional Classification [102]. In this classification scheme, classes are separated based on the physical activity that the patient is able to achieve and the discomfort that results from physical activity. Class I is when no symptoms are present, and in Class IV the patient is unable to perform any physical activity without discomfort and symptoms of heart failure are never alleviated. As can be seen, a variety of sensing modalities could be employed to track signs and symptoms of HF, including measurements of peripheral blood flow, respiration rate, exercise capacity, and posture while sleeping. This illustrates the need for new sensors that can measure each of these various symptoms. However, not every sensor may be worn at all times, due to excessive burden on the user. Therefore, there is a need for new sensing modalities that can track different patterns and trends in captured data, as well as transfer learning techniques that can be adapted to estimate values of sensors that may be malfunctioning or not worn.

If the right set of sensors are selected and are designed to be worn longitudinally, new patterns and trends in signs and symptoms might be detected. In CAD, for example, restrictions in blood flow of the coronary artery may result in a condition called stable angina (SA). The rate at which the restrictions in blood flow occur, however, might change as the disease progresses. At some point, the restriction responsible for SA may rapidly increase, producing a situation where the patient is in emergent need of medical care. The most common way for this progression to occur is for a fatty plaque to rupture, leading to the formation of a clot that blocks blood flow. The first disease after this point is unstable angina (UA). As the restriction increases to a partial occlusion, the patient will experience chest pain that worsens without activity or that is not relieved with rest. Both stable and unstable angina present similarly in a patient. Typically, the patient will have episodes of chest pain that last from 3 to 10 minutes, but potentially lasting up to 30 minutes. This pain may radiate to the jaw, neck, shoulder, or arm. The patient will likely feel short of breath and may also experience nausea. If the patient takes a medication called nitroglycerine, the pain should resolve within 1 to 3 minutes. In UA, damage is still reversible, but intervention is emergently necessary to ensure that the disease does not progress. If UA progresses, it will progress to a condition commonly known as a heart attack, or in medical terminology as a MI. There are two types of MI: non-ST-elevation MI (NSTEMI) and ST-elevation MI (STEMI). In NSTEMI, some muscle in the heart has begun to die, and therefore at least some of the damage caused is irreversible. In a STEMI, there is a complete blockage of blood flow at some point and a large amount of muscle in the heart has begun to die. NSTEMI and STEMI are distinguished by characteristic findings on ECG; in a STEMI, the ST segment will be elevated above the baseline in some leads, while this elevation is absent in NSTEMI. The leads showing this change reflect the area of the heart impacted by the MI. This demonstrates the second need, longitudinal monitoring of continuous signals that can identify disease progression, and machine learning techniques that can account for the personal progression and varied rates of this progression.

In order to prevent conditions such as stroke, which are treated by neurologists after the primary adverse event, interventions are necessary in known cardiovascular risk factors, such as HTN, which can lead to stroke in multiple ways. Very high blood pressure raises the risk of hemorrhagic stroke, as blood vessels in the brain may not be able to support higher pressures. Additionally, chronic HTN is the main risk factor associated with ischemic stroke. The diagnosis of HTN requires repeated blood pressure measurements (sustained HTN), as measured by ambulatory blood pressure measurements. Various reasons for blood pressure elevation must be identified, including white-coat HTN (when the blood pressure is elevated during a visit to a doctor but normal when measured in home settings), masked HTN (when blood pressure is regularly elevated but detected as normal during a visit to a doctor), and evaluation in changes of blood pressure when sleeping versus when awake (nocturnal nondipping HTN). HTN typically does not manifest with any symptoms, as the body is very good at masking the feeling of this pressure. Although high blood pressure has been colloquially associated with stress, headaches, or dizziness, these symptoms are typically not caused by chronic HTN. The primary sign (and part of the diagnostic criteria) of HTN is an elevated blood pressure. For diagnosis, at least two measurements on two different occasions of blood pressure above 120/80 mmHg are required. More recently, guidelines have suggested measuring blood pressure with an ambulatory blood pressure monitor over a 24-hour period, measuring blood pressure every 15 minutes during the day and every 30 minutes during sleep at night, and using the average values to have a better understanding of a patient’s blood pressure [61]. This sustained elevation may result in stiffer arteries, reducing arterial compliance. Additionally, over time, this chronic elevation may result in left ventricular hypertrophy seen on ECG or in changes in the retina. Most patients with HTN are largely asymptomatic, with the chief clinical sign being that of elevated blood pressure. When symptoms of HTN do manifest, they are largely caused by organ damage that results from chronically elevated blood pressures. Chronically elevated blood pressure can lead to heart damage, as the heart must work harder than normal to produce these elevated pressures. This can lead to HF as the heart gains mass and loses efficiency, or to CAD as the increased mass of the heart requires increased myocardial oxygen supply. Chronically elevated blood pressure can also lead to damage of the arteries. This can lead to atherosclerosis, where plaque buildups can compromise coronary arteries, leading to CAD, or cerebral arteries, leading to stroke. Weakening of arterial walls can lead to kidney disease or to retinal disease. Advanced HTN can cause changes to the eye that can be observed visually by a physician. The definition of high blood pressure has undergone changes in recent years, with the SPRINT trial indicating that aggressive treatment of blood pressure to <120/<80 mmHg is associated with decreased mortality [62]. The potential measurement of blood pressure from new sensing modalities can enable analytic techniques to identify cases of HTN and evaluate the effectiveness of medication on reducing blood pressure, such as in the SPRINT trial. This illustrates the third need, where machine learning techniques, trained on continual data captured from new sensing modalities (the prior two needs), must provide actionable, interpretable estimations of signs, symptoms, and disease progress, in order to help guide treatment decision making and evaluate treatment effectiveness both prior to a diagnosis of a cardiovascular disorder and in the treatment and evaluation of recovery from an adverse cardiovascular event.

Table 2 highlights the available commercial devices currently suited for tracking a number of the risk factors highlighted for the three cardiovascular disorders. Most devices use light-based sensing for tracking heart rate and pulse oxygenation, and a few have additional sensing capabilities. In the following sections, we explore the state of the art in technology associated with each of the clinical needs, highlighting research advancements beyond the currently available commercial solutions. This survey reviews the technology available and the gaps that remain in addressing the needs, and highlights opportunities for researchers within the smart health field to design solutions with impact to clinical decision-making problems.

Table 2.

Sample of Current Commercially Available Devices and Common Cardiovascular Parameter Monitoring

Product Class	Product	(Optical) Heart Rate	Blood Pressure	Other Measures
Smartwatch	Amazfit Verge [1]	✓	✕	ECG
	Apple Watch [182]	✓	✕	ECG
	Empatica Watch [45]	✓	✕	Galvanic Skin Response & Temperature
	Fitbit [64]	✓	✕
	Garmin Fenix Watch [58]	✓	✕	PulseOx & Temperature
	Samsung Galaxy Watch [10]	✓	✓
	Valencell-associated smartwatches [191]	✓	✕
	Withings (Move ECG, ScanWatch, Steel HR, Pulse HR) [198]	✓	✕	PulseOx, ECG
Smart Ring	Oura (Ring) [163]	✓	✕
Headphones	Samsung Galaxy Buds [10]	✓	✕
	Valencell Blood Pressure Kit [190]	✓	✓
	Valencell-associated earbuds [191]	✓	✕
Chest Strap	Garmin HRM-Series Chest Strap [57]	✕	✕	ECG
	Polar H-Series Chest Strap [189]	✕	✕	ECG
	QardioCore Chest Strap [160]	✕	✕	ECG
Medical Devices	AliveCor ECG [8]	✕	✕	ECG
	Caretaker [30]	✓	✓	PulseOx
	Finapres Nova [52]	✓	✓	PulseOx
	QardioArm [159]	✕	✓

3. NEW SENSORS, TRENDS IN LONGITUDINAL CAPTURE, MISSING DATA, AND SENSOR SELECTION

New sensing techniques that capture acute data as well as detect changes in sensed data over time are needed to measure the important signs and symptoms that are risk factors for HF, CAD, and stroke. Each condition has a set of similar risk factors as well as unique signs and symptoms that manifest through a variety of changes in the body. For HF, improper blood flow can result in fluid retention (edema) in the lungs or the periphery, as well as cause signs of heart remodeling. Heart remodeling can be evidenced by third and fourth heart sounds (S3 and S4), as well as by a laterally or inferiorly displaced point of maximal impulse (PMI) of the heart on physical exam; the place where the heartbeat can be felt most strongly will migrate down and to the left of the thorax. One way in which improper blood flow can be detected is that the extremities will be cooler than normal.

In CAD, stable and unstable angina will often result in physical pain felt by the patient in an episode that may last up to 30 minutes in the chest that may also radiate to the jaw, neck, and arm. The patient’s heart rate and blood pressure will initially be elevated, although these can potentially decrease in NSTEMI and STEMI as the heart fails to operate optimally. The patient will breathe more quickly and will put more effort into breathing. Additionally, abnormal sounds may be heard with a stethoscope. It is possible for rales, an abnormal lung sound, to be heard at the posterior base of each lung. During chest pain, an ECG will show ST-segment depression, but this will change and progress to ST-segment elevation in STEMIs.

For stroke, this work focuses on the signs and symptoms that might lead to a stroke. Atrial fibrillation (AFib) is a relatively common arrhythmia that increases risk of stroke. AFib results when the atria of the heart beat ineffectively and randomly, causing turbulence within the atria. This turbulent flow allows for clots to form within the atria. If these clots are dislodged, they may travel through the arteries and become lodged in the brain, causing an ischemic stroke. AFib is classically defined as an “irregularly irregular” beat—the beat is not a typical rhythm (irregular) and additionally has no pattern determining when beats occur (irregularly). This is most often seen as absent P waves on ECG with variably occurring QRS complexes over a noisy baseline. However, this pattern could be detected by many techniques that measure pulse. Chief risk factors that predispose patients to AFib are age, other heart disease, diabetes, and chronic lung disease. HTN can also lead to stroke in multiple ways. Very high blood pressure raises the risk of hemorrhagic stroke, as blood vessels in the brain may not be able to support higher pressures. Chronic HTN is the main risk factor associated with ischemic stroke.

These cardiac conditions present a range of sensing opportunities:

• Acoustic measurement: Capture of heart sounds to identify specific classes as well as respiratory effort is important in understanding acute conditions and changes in heart function over time. This also includes respiratory distress when lying down, causing patients diagnosed with HF to need to sleep in a more upright position. (See Section 3.1.1.)

• Electrical measurement: Remote ECG measurements can identify periods of atrial fibrillation and other arrhythmias or help identify progression of CAD during an acute event. (See Section 3.1.2.)

• Heart beat and associated characteristics: Understanding cardiac output, as well as measurement of blood pressure, is an important risk factor that needs periodic measurement. (See Sections 3.1.3 and 3.1.4.)

• Fluid retention/weight change: HF often results in lung and peripheral edema that results in swelling and can be measured by cooler temperatures in the periphery and changes in weight. (See Section 3.1.5.)

• Diet, exercise, and pain: In all cases, patient diet (for identifying glucose intolerance, obesity, etc.), patient self-reported pain, fatigue, and general physical activity may be surrogates for worsening conditions. Activity recognition can include posture detection to link with respiratory measurements and can impact monitoring of glucose intolerance, which can lead to diabetes. (See Sections 3.1.6 and 3.1.7.)

3.1. Existing Technologies and Applications

3.1.1. Acoustic Sensing/Vitals.

Vital sign monitoring has been explored through a variety of technologies. Each sensor type has been designed to address some of the sensing needs described in the previous section in an effort to replace or replicate tools available in acute care settings for remote environments. The stethoscope is one of the oldest such tools in medicine and is an implementation of acoustic sensing. By hearing and interpreting sounds from the patient, the physician can develop insights into the health of the patient and the functionality of the organs. Recently, digital stethoscopes have been utilized to better capture sounds. Digital stethoscopes provide benefit in allowing soft sounds to be more easily heard but also allow for recording of sounds for later manual or computational analysis. As physicians have grown more reliant on advanced imaging techniques such as ultrasound, physical exam skill, including skill at auscultation, has decreased [37].

Developing a digital stethoscope involves multiple components requiring heart sound capture, segmentation of the audio signal, and understanding of the cardiac cycle, best paired with an external signal such as ECG or pulse to determine the reference interval as described by Leng et al. [101]. A limitation here is that the time from electrical activity to sound production is not constant in all samples. Direct segmentation techniques involve utilizing Shannon energy to calculate an envelope and to find its peaks, and then use those peaks to reconstruct the cardiac cycle. Following sound segmentation, it is necessary to then classify these sounds. Leng et al. describe various machine learning techniques to classify these sounds, including support vector machines (SVMs), artificial neural networks (ANNs), hidden Markov models (HMMs), and Gaussian mixture models (GMMs), for identifying sounds and identifying the next likely sound given the state in the heart beat cycle currently detected. Leng et al. report that these techniques have accuracies near 90% for classifying signals as either normal or as having aortic or mitral valvular lesions [101]. In 2016 a collection of heart sounds was published [109] and this dataset has served as a standardized way to benchmark progress in identifying heart sounds. Work in this dataset was summarized by Clifford et al. in the 2016 PhysioNet Computing in Cardiology Challenge, who reported that several varied techniques reached high performance [36]. Notably, the top three models had completely different approaches but similar performances. Those three models consisted of AdaBoost and a convolutional neural network (CNN), an ensemble of SVMs, or a regularized neural network. Subsequent work has continued to improve on this task with performance improving with more sophisticated ensemble algorithms [35].

Work has also been done to develop low-cost devices that can act as a bridge between a traditional stethoscope and a cell phone [177]. Constructing a cavity with good resonance is necessary in collecting good-quality sound transmissions from the stethoscope. In particular, Sinharay et al. have evaluated using different kinds of sensors to capture sounds to be transmitted from and to smartphones for analysis.

In addition to detecting abnormal sounds in the cardiac cycle, there has been successful work in eliciting heart pathology from abnormalities within normal heart sounds. The normal cardiac cycle is composed of two sounds, S1 and S2. S2 in turn is caused by the superposition of two separate sounds occurring nearly simultaneously, one from the aortic valve closing and the other from the pulmonic valve closing. Both happen at nearly the same time, typically creating a single sound. However, some heart pathologies can impact the time between these. In a study of pediatric patients, high pressure in the pulmonary vasculature was found to be predicted by certain aortic and pulmonic valve relative intensities [43]. Although this work has not been applied to adult patients, it could theoretically help to elicit information about the pressures at different points within the heart.

In several cases, radar has been utilized instead of direct, on-body measurement for detecting vital signs. Radar is able to detect periodic changes caused by both breathing and the heart, allowing heart rate and respiratory rate to be detected. Vinci et al. described a remote sensor that uses a six-point radar to monitor respiration and heart beat [194]. It uses a continuous 24 GHz wave and a radiated power of less than 3 microwatts. It captures these values noninvasively in patients at rest. This is notable as it is a sensing modality that does not require attaching sensors to the human body. This is particularly valuable in infants, in adults in severe conditions that cannot have additional attachments placed on the body, and as a modality that improves patient quality of life by limiting on-body sensors. The sensor designed in this article does not have the limitations of other radar systems that require a wide-frequency band to achieve more accurate results. Because of the six-point receiver architecture, this sensor can accurately measure angle and displacement by only measuring phase difference in backscatter patterns. Models regarding the permittivity of the skin allow them to estimate that their signal has 1.52 mm penetration as well as estimates of blanket and clothing impact. As a result, they can estimate where the edge of the torso is to aid in monitoring breathing. This provides an opportunity to noninvasively measure respiration and heart rate. However, it requires known, fixed postures of the individuals. Additionally, it will only work for one patient at a time. While this modality provides activity, displacement, and vitals monitoring in controlled, clinical environments or within specific remote environments (such as in the bedroom while asleep), it does not provide flexibility while moving. There are needs to extend such sensing systems to a variety of environments.

Work by Li et al. explore the use of radar technology for vital sign monitoring [103]. Their system uses a hardware-controlled clutter cancellation system. This allows their radar technology to identify the difference between the person being monitored and background clutter that is likely present in rooms the person would be in. Authors propose taking ka-band radar systems that are meant for motion sensing and modifying them for vitals sensing. Authors discuss existing work, design considerations for advancements, then opportunity to extend this to infant monitoring. The advancements in radar usage have come through the detection of the right frequency band to use. Different frequencies were shown to be able to go through different rubble with and without metal mesh. Authors then discuss the chip-level decisions that need to be made to create CMOS Doppler-based motion detectors. This allows vital sign detection through obstacles that can be important for noninvasive monitoring and for detection of vitals in emergency disaster scenarios. The application, however, is not clear for advanced signal processing of multiple vitals.

3.1.2. Electrical Measurements.

Remote ECG monitoring has been utilized since the development of the Holter monitor in 1962 [117]. However, recent advances allow for not only recording of remote ECGs but also real-time analysis and for longer periods. One necessary advancement for increased length of monitoring was the long-term electrode. Traditional wet electrodes are poor choices for long-term monitoring due to their inconvenience [34]. Chi et al. surveyed a number of advancements in dry-contact and noncontact electrodes that have been developed [34]. Majumder et al. similarly survey numerous developments in dry electrodes that provide superior remote monitoring performance for long-duration ECG monitoring [116].

Remote ECG monitoring has been explored by a number of researchers, primarily to solve the challenges that arise in noisy measurement. One issue that arises in continuous ECG monitoring, as with wearable ECG implementations, is that signals are often hidden by the noise of activity. Li et al. presented an approach for qu0061ntifying this noise [104]. While earlier approaches focused on labeling ECGs as either clean or noisy, the approach presented by Li et al. introduced five classifications, each with different amounts of information available to be extracted from the ECG. They defined the noisiest strips as those where artifact obscures signals to the point that there can be no confidence in any interpretation of the ECG. Strips with severe noise were those where some interpretation could be made, but interpretations could be confused as to where the QRS complexes fell or to whether ventricular flutter rhythms were present. In strips with moderate noise the QRS complex and presence or absence of ventricular flutter rhythms could be assessed, but finer signals such as P or T waves could not be extracted. Minor noise was the label given to strips with some amount of noise but where P waves and T waves could be extracted. This level of noise allows for the analysis of atrial arrhythmias such as atrial flutter. Finally, clean ECGs were those where no noise was present. The authors produced training data by adding three types of noise to the original clean dataset: baseline wandering, electrode motion, and muscle artifact. They trained an SVM to classify strips based on the amount of noise present and validated this classification scheme on real noisy data. This validation showed good agreement between manually annotated labels and model output labels, with the greatest confusion present where samples had been manually annotated as having minor noise, but the model labeled the samples as having moderate noise. The authors note that a chief limitation of this work was that the model was not trained for or with an arrhythmia database, which substantially lowers its effectiveness on samples with arrhythmias. Additionally, they note that methods based on continuous features rather than discretely extracted features would be likely to show greater performance.

Once identified, several approaches have been implemented in order to account for and to correct motion artifacts. Sriram et al. addressed this problem by utilizing a triaxial accelerometer [180]. ECG signals are usable as a means of continuous biometric security. However, this continuous security is lost when the ECG signal is distorted with motion artifact. This approach shows that supplementing the raw ECG signal with features extracted from acceleration allows for accurate classification of ECG subject identity. They segmented signals to windows containing roughly four heartbeats, averaged those four beats together, and then corrected for baseline abnormalities with linear interpolation of q-minima and a high-pass filter in association with the accelerometer features. These features then served to correctly identify users using either k-nearest neighbors or a Bayesian network classifier.

Several wearable ECG devices have been developed recently. The BioStamp is a wireless wearable device that received FDA 510(k) clearance for medical use [82]. The BioStamp provides ECG signals that are comparable to a traditional ECG [80]. It also includes accelerometers and gyroscopes, and in a population of 30 healthy adults it was able to provide accurate measures of heart rate, heart rate variability, respiratory rate, activity, and sleep events [176]. Another FDA-approved device incorporating ECG monitoring is the iRhythm Zio^XT [181]. This device is applied to a patient as an adhesive patch and was found to be more sensitive than a traditional Holter monitor at detecting arrhythmias [21]. This device is able to be worn for up to 14 days.

Another issue that arises with automatic ECG monitoring is that many abnormalities might be troubling in one patient while normal in another. Chen et al. [33] described an approach to train ECG monitoring systems to discover patient-specific abnormalities. This work utilized an accelerometer to reduce the number of false alarms in monitoring systems. Over time, this system learns the normal for a given patient and uses knowledge of this normal in order to reduce false alarms.

3.1.3. Blood Pressure.

The American College of Cardiology and the American Heart Association (ACC/AHA) recently released guidelines that suggest that ambulatory blood pressure measurements, those taken at home in 15-minute intervals including during sleep, should be captured to better understand a patient’s blood pressure and potential cardiovascular risks associated with HTN [161]. The sphygmomanometric and oscillometric techniques are well established as the predominant means by which blood pressure is typically measured [137]. Both methods involve the inflation of a pressurized cuff, typically around the patient’s upper arm and maintained at the level of the heart. The pressure in the cuff is increased to above realistic values of the systolic blood pressure and then slowly decreased. In the auditory sphygomomanometric method, sounds called Korotkoff sounds can be heard just distal to the cuff as it deflates. The pressure at which these sounds are first heard is the systolic pressure, and the pressure at which these sounds are no longer heard is the diastolic pressure. In the oscillometric technique, minute variations in pressure as the heart beats against the pressurized cuff are measured and the systolic and diastolic blood pressures are extracted from these variations [17]. Most at-home blood pressure monitoring devices utilize the oscillometric technique, which is well validated to have performance similar in quality to the sphygmomanometric technique [70]. Recently, cuff-less blood pressure monitoring techniques have been explored in order to record blood pressure.

The most common cuff-less approach thus far is to use photoplethysmography (PPG) and ECG to capture pulse arrival time and pulse transit time (and pulse wave velocity) as surrogates for blood pressure, then use analytic techniques to estimate the systolic and diastolic blood pressure values [114, 186]. If the posture of an individual is known, these techniques are able to measure an estimate of the blood pressure, without disturbing the individual with frequent cuff inflations. However, the ECG and PPG combination can result in error in blood pressure estimation because it does not appropriately account for artifacts that exist between the ECG measurement of a pulse and the PPG capture of the pulse arrival time [24]. In particular, the ECG and PPG combination shortcomings are a direct result of the pre-ejection period of the heart. The pre-ejection period constitutes a time delay between the electrical stimulation of the heart and the actual mechanical expulsion of the blood for each heartbeat [152]. The pre-ejection period can vary under different conditions and is not easy to measure, leading to an unpredictable error in estimating blood pressure when using ECG. Vascular tone can additionally complicate this estimation. Vascular tone can change as patients age or take different medications, and these changes can increase this error [149]. To account for this, researchers have turned towards dual PPG capture [38, 187] over a small portion of the artery to account for pulse transit time, which is better able to locate the artery and avoid capturing blood profusion time into capillaries [74, 75]. Ballistocardiogram approaches look to capture pulse arrival time through the small changes in pressure sensed by the waves in each pulse, providing a method for capturing cuff-less blood pressure whenever participants are still [77, 85, 86]. These approaches all look to address cuff-less blood pressure when the participant is in a fixed, known position and provide the opportunity for more frequent ambulatory blood pressure measurement.

More recently, bioimpedance-based approaches have also been developed to measure blood pressure in a cuff-less manner [76]. The impedance signals allow the sensors to identify the location of the arteries within the wrist, eliminating errors in blood pressure estimation that are a direct result of the pre-ejection period or the misplacement of light sources that may capture both the pulse transit time in the artery and blood profusion through the capillaries. Estimation of blood pressure characteristics was then made by extracting characteristic features from the multiple bioimpedance channels. This is enhanced by adding other heart beat characteristics, including capturing the inter-beat intervals for heart rate and heart rate variability characteristics [16], as well as respiratory rate [174].

3.1.4. Blood Flow.

Blood flow is a complex system characterized by pulsatile flow in a dynamic system [93]. While measurements related to arterial blood pressure are often a good proxy for systemic blood flow, different physiologic or pathologic states can alter this relationship [28]. Most notably, isolated vasoconstriction or a thromboembolic event can cause flow along an artery to drop while systemic pressure is relatively unchanged, or atherosclerosis can cause chronically decreased flow to various organs [63]. Ultrasonography can be used to assess blood flow along an artery [60, 118] and can also be used to estimate degree of systemic atherosclerosis [41]. Magnetic resonance imaging (MRI) can also be used to measure blood flow [135].

3.1.5. Fluid Retention.

While prior studies, such as Tele-HF and Beat-HF, attempted to use weight scales as a surrogate for fluid retention in HF, the measurement of 3 pounds of weight change was not an alert that was able to reduce HF readmissions [31, 91, 142]. A number of attempts to measure peripheral edema and fluid retention have focused on the development of smart socks that look to measure fluid buildup in the ankles [48, 49]. A stretch sensor measures the expanding duration of the patient’s ankle both as edema increases throughout the day and as edema increases over time. The context awareness allows the device to discard ankle measurements when motion, muscle contractions, or an incorrect posture would interfere with the measurement. This sock was able to reliably determine the participant’s posture, and measurements of fluid retention were well correlated, but additional study is needed to determine if this measurement is accurate enough, and whether it can generate alerts early enough to intervene in HF patients. Yao et al. came to similar conclusions of needing further study of their sensor to classify edema [201], as this remains an open area of research.

3.1.6. Physical Activity and Posture.

Activity, posture, and pain are important measurements in understanding symptom and treatment effectiveness in patients diagnosed with cardiovascular disorders. Measurement of respiratory distress in HF patients requires a measurement of posture, and measurement of blood pressure through proxy measures such as pulse transit time requires a measurement of posture, as did the smart sock for fluid retention (Section 3.1.5). While each sensor can capture posture, smartphones excel at this [205], often coupled with other applications tracking activities of daily living [146, 154]. Recently, smartwatches have been shown to accurately detect postures and exercises [132, 175], which is important for patient monitoring, since smartphones are often in the proximity of the user but often not physically on the user, unlike smartwatches [193]. These can also provide important context to the measurements captured by the other modalities discussed in this section [130].

3.1.7. Diet Monitoring and Glucose Intolerance.

Thirty million Americans live with diabetes, and another 80 million have pre-diabetes, a condition that, left untreated, often leads to diabetes [53]. Diabetes occurs when blood sugar is too high due to poor nutrition (e.g., too many refined carbohydrates) and/or inadequate insulin regulation (i.e., insulin resistance). Sustained high levels of blood glucose can have disastrous long-term health consequences, including cardiovascular diseases. An essential component of clinical interventions for diabetes is monitoring dietary intake, as it can help individuals and health practitioners manage dietary habits and understand how dietary choices affect blood glucose. Various sensing techniques have been explored to capture dietary intake, such as wearable sensors (microphones, accelerometers) to detect eating behaviors such as hand gestures and chewing/swallowing [81], or computer vision techniques to recognize foods from photographs [50]. Using continuous glucose monitors has allowed researchers to develop models of estimated food intake [72] and, when coupled with other personal measures, such as gut microbiome data, can provide educational information towards treating glucose intolerance at a personalized level [206]. Not only is glucose intolerance, and a diagnosis of diabetes, a key factor that increases risk of cardiovascular disorders, but also other parameters, such as salt intake may impact blood pressure [44]. More recently, authors have shown that detecting glucose excursions, such as hyperglycemia or hypoglycemia, is possible from ECG signals [157]. This provides a potentially noninvasive way to track glucose variability while primarily developing sensors for tracking risk factors of a primarily cardiovascular nature.

3.2. Gaps

Table 3 summarizes the key developments in sensing including remaining gaps in the technologies. As these technology gaps are addressed, richness of the available data will increase. As richness of data increases across the variety of sensors, the potential for noise and missingness increases as well. It is difficult to understand the context in which measurements are captured. Accuracy of posture detection and presence of other noisy attributes impact the potential success of different sensing modalities. It is also unlikely a patient will wear all sensors all the time, as this will provide excessive burden. While a measurement performed on occasion is likely to be a high-quality measurement, continuous and automated measurements introduce a greater deal of variability in the quality of measurements. For instance, a once-a-day measurement is likely to be a measurement where the patient will intentionally position themselves appropriately and remain motionless during the measurement. A patient monitoring their blood pressure will likely sit upright with their legs uncrossed, or a technician performing an ECG will ensure that the printed ECG is taken at a point where the patient is motionless and no artifacts are present. Conversely, more frequent or continuous monitoring must account for noise introduced by motion artifacts as well as from noise introduced from other sub-optimal measuring conditions. As such, a number of challenges remain in capturing the necessary signals:

Table 3.

Summary of Sensing Types, Analytic Possibilities, and the Advantages and Disadvantages of the Technologies

Technology	Sensor	Analytics	Advantages	Disadvantages
Digital stethoscope [37, 177]	Microphone and appropriate casing	Automated segmentation could be applied to segment beats and analyzes characteristics of heart sounds	Relatively cheap and accessible technologies	Commercially available models require physician evaluation
Radar vital sign measuring [103, 194]	Radar receiver to monitor patient physiologies	Analyze movement to extract heart rate and respiratory rate	No contact required	Requires knowing patient posture, limited to a single patient at a time
Electrocardiography [21, 80, 82]	Measurement of cardiac electrical impulses	Automated segmentation required for analysis, pattern recognition required for detection of changes such as arrythmias or ST elevations	Gold standard in cardiac monitoring	Requires lead placement, susceptible to noise, difficult to interpret
Cuff-based BP monitor [137]	Pressure impulse or sound during cuff deflation	Automated analysis of cuff pressure allows for extraction of systolic and diastolic BPs	Performs well at providing accurate measures of patient BP, Easy to use	Unable to provide continuous blood pressure, obtrusive
PPG-based BP monitor [114, 186]	PPG signal in coordination with ECG signal	Extracts blood pressure by relationship of electrical signal and pulse signal arrivals.	Could be applied to continuous monitoring	Requires multiple signals, poor accuracy due to variable pre-ejection period timing
Bioimpedance-based BP monitor [76]	Bioimpedance signal	Extracts blood pressure by relationship of bioimpedance along arteries	Could be applied to continuous monitoring	Not commercially available
Ultrasonography [60, 118]	Ultrasound generator sends pulses into patient and analyzes rebounding signals to construct an image	Image analysis required for flow assessment and vessel characterization	Allows direct visualization of blood vessel, allows for real time visualization	Not available in wearable form, noisy image, quality dependent on skill with device, requires expert evaluation
Smart Sock [48, 49]	Accelerometer and stretch sensor	Estimates leg edema using learned mapping of inputs	Allows for remote monitoring of extremity edema	Not commercially available
Continuous Glucose monitor [72]	Subdermal electrode to sample interstitial fluid	Allows for estimation of food intake and for personalized treatment of glucose intolerance	Captures a difficult signal that is closely related to long-term health issues	Invasive, unlikely to reach universal adaptation, needs to be replaced after 14 days

• Acoustic measurement: Nonwearable sensors are limited by the challenge of identifying a particular patient when multiple people are present. Wearable sensors must account for noise across a variety of motions, environments, and potential sensor misplacement.

• Electrical measurement: Continuous ECG requires multiple leads to be worn at the same time. Devices such as the Apple Watch provide potential for requesting ECG periodically when other sensing modalities dictate when it is necessary [196], but the correlation between these modalities and necessary ECG readings has not been well studied outside of AFib.

• Blood pressure: Pre-ejection period and vascular tonal changes can impact estimation, resulting in pulse transit time calculations capturing both the arterial pulse and profusion into the capillaries. Additionally, misplacement of sensors may alter the accuracy of the readings, impacting performance of analytic models used to estimate blood pressure from data captured by these sensors. Cuff-less blood pressure monitoring must extend to continuous, beat-to-beat measurements without constantly restraining users to fixed, known postures.

• Fluid retention/weight change: Edema measurements have not been clinically validated to show the degree of fluid retention that must generate alerts that can clinically improve outcomes.

• Physical activity and pain: Remote measurement of acute and chronic pain remains an open challenge.

• Glucose intolerance: Tracking of diet, nutrition, and the direct link to cardiovascular care remains an open-ended problem without the use of invasive glucose monitoring technologies.

3.3. Opportunities

An additional source of noise can be introduced by the redundancy of signals that can exist. Different physical phenomena can be measured by different modalities, many of which will produce slightly different readings. Heart rate can be derived from multiple sources: auditorily by stethoscope, electrically by ECG, optically by PPG, and electromagnetically by radar. It stands to reason that these redundant values could be exchanged for each other, but that exchange may not completely be a one-to-one relationship. Transfer learning is an ongoing field of study that seeks to apply existing models to data that was not used in training or was only used minimally in training [165, 166, 200]. Transfer learning could be applied to this problem as a way to apply a single model to patients with disparate data collection modalities.

Missingness in data also increases as richness increases. While binary parameters used in many risk models (e.g., history of HF, current diabetic status, etc.) are easy to collect and even possible to impute, continuous monitoring opens the possibility of more complicated missingness. A battery may fail on a sensor, leading to a variable period of missingness. Wearable sensors may introduce missingness secondary to poor compliance or poor utilization. The missingness introduced by gaps in continuous monitoring is more difficult to impute and presents a challenge in building comprehensive models [107, 110]. Deep learning techniques to address missing data have shown promising results; however, simple imputation of time-series signals is currently the best approach [108], leaving the door open to further work to address this at the sensor and anlytic level. A number of opportunities emerge for immediate and impactful research on sensing signs and symptoms of cardiovascular disorders, illustrated in Figure 3, and listed below:

Fig. 3.

Overview of selected sensor categories proceeding to selected signs and symptoms measured and their potential progression to adverse events and diagnoses. The number of crossing connections illustrates the commonality in risk factors that can be sensed in progression to primary adverse events and secondary recurrent adverse events for a variety of cardiovascular conditions. The colors are only illustrative of different pathways in each level and are not meant to be illustrative between subsequent levels.

• Integration of multiple sensing modalities into a single platform, reducing the number sensors needed to be worn. High-impact areas appear to be the wrist (smartwatch) and chest (heart and lung sounds). Analytics that leverage this integration will be discussed further in Section 4.

• Using analytic techniques to estimate parameters traditionally captured invasively with noninvasive surrogates (e.g., glucose and hypoglycemia using wearable ECG).

• Integration of machine learning techniques to help identify when longitudinal data capture is necessary, similarly to ECG requests to verify periods of arrhythmias associated with AFib detection with the Apple Watch [196].

• Transfer learning, when coupled with uncertainty quantification techniques, enables improvement of model performance through personalization (see Section 4.1.5). However, when accounting for varying sensor types of the different domains, techniques are needed to quantify what domains of data and what quantity of those data are needed to transfer learn. Additionally, knowing which portions of models to retrain in a transfer learning mechanism should be further explored.

4. CONTINUOUS DATA COLLECTION AND ANALYTIC MODELS

Beyond the acute sensing and detection of symptoms related to HF, CAD, and stroke, analytic opportunities arise in the processing of this data longitudinally and continuously. As discussed, the progression of CAD from stable and unstable angina to NSTEMI and STEMI represent longitudinal changes that may have periods of rapid change interspersed. Similarly, untreated HTN can lead to stroke if untreated. Changes in heart remolding in HF may be represented by changes in heart sounds as captured by acoustic sensing. Patients living with HF may experience long-term changes in the amount of physical exertion required to perform activities of daily living. These changes may be gradual and unnoticeable to the patient, but may represent worsening condition or recovery.

These cardiac conditions present a range of analytic techniques necessary to capture longitudinal changes in continuously sensed data:

• Continuous capture of acoustic sensing: Understanding how sounds change over time may allow for the identification of new signals that represent earlier identifiers of worsening conditions or treatment effectiveness. (See Section 4.1.1.)

• Continuous capture of electrical signals: While the detection of arrhythmias may be present in surrogate measures such as heart rate, detection of changes in ST segments of an ECG may allow for early alerts and acute care. (See Section 4.1.2.)

• Continuous capture of vitals signals: Understanding the changes in the variety of vitals signals captured and how they may relate to each other can provide an understanding of improving or worsening risk factors relevant to HF, CAD, and stroke. (See Section 4.1.3.)

• Continuous capture of physical activity: Physical activity and sleep are important functional measures of recovery, and accurate, longitudinal understanding of functional change can be correlated with improved mortality and prevention of adverse events. (See Section 4.1.4.)

• Deep learning techniques for data analysis and modeling: A variety of deep learning techniques have the ability to develop personalized models using continuous, longitudinal data. While long short-term memory networks (LSTMs) and general Recurrent Neural Networks (RNNs) provide a standardized framework for signals, this section explores modification of existing techniques to work with a wider array of data discussed in this section. (See Section 4.1.5.)

4.1. Existing Technologies and Applications

4.1.1. Continuous Capture of Acoustic Sensing.

A primary application of acoustic sensing is for the assessment of cardiac murmurs [37]. Most auscultative techniques have not been developed for continuous monitoring but are rather focused on individual discrete observations. However, continuous-wave Doppler monitoring can be used in fetal monitoring [185] and continuous fetal monitoring has been shown to have superior outcomes relative to intermittent monitoring [7]. There has been some work in extending this technology to continuous adult cardiac auscultation [120]. Mc Loughlin and Mc Loughlin found that continuous auscultation was able to detect impaired ventricle relaxation and lesions of the aortic and mitral valves with higher sensitivity than was available with traditional auscultation alone [120]. However, there is a pronounced absence of further work in continuous cardiac auscultation.

Electronic auscultation is useful for deriving characteristics of other parts of the cardiovascular system than sounds generated specifically by the heart. A carotid bruit is a sound created by turbulent blood in a carotid artery, often caused by narrowing that in turn is produced by atherosclerotic plaques. Knapp et al. looked at the effectiveness of carotid bruit detection by electronic auscultation [88]. Out of 1,371 patients in this study, 84 were found to have carotid bruits by electronic auscultation. These patients were matched with controls who did not have bruits, and both patients from each pair were assessed with duplex ultrasound to determine the extent of carotid stenosis. Bruit detection with electronic auscultation and manual annotation was found to have a sensitivity of 88% for stenosis ≥ 50% and a specificity of 58%, with duplex ultrasound providing the ground truth.

Work by Palaniappan et al. surveyed machine learning techniques to further analyze lung sounds [147]. They evaluated 59 papers that used signal processing and machine learning techniques on a variety of lung sound problems including normal breath sounds, abnormal breath sounds, and a series of sounds called adventitious lung sounds. This survey highlights an important need by evaluating short-term sounds and long-term sounds and identifying normal and abnormal sounds across the different time periods. Most works in this survey focused on specific frequencies (between 150 and 2,000 Hz, though they found that most work typically worked at 150 Hz) and evaluated machine learning techniques such as k-nearest neighbor, ANNs, HMMs, GMMs, genetic algorithms, SVMs, and fuzzy logic to classify a variety of lung sounds. They found that by using piezoelectric microphones, contact microphones, electric microphones, and one commercially available lung sound instrument, they could design electronic stethoscopes that filtered out heart sounds to capture necessary lung sounds. Similarly, one could use the same techniques to filter out the lung sound to capture the heart sounds. Using standard time-domain and frequency-domain signal processing features, algorithms were able to classify lung sounds with between 83% and 93% accuracy. Rocha et al. published a database of lung sounds that were used in the 2017 ICBHI Scientific Challenge as a challenge for lung sound classification [164]. These sounds consisted of wheezes, crackles, wheezes and crackles, or normal breath sounds. Several groups have achieved good performance on this dataset by applying CNNs to this dataset [89, 150]. There is continuing work in applying RNN and LSTM architecture to this task as well [90, 151]. Work in this domain is largely limited by the large variation in pathological sounds and by a lack of additional publicly available datasets. Given the traditionally subjective nature of sound interpretation, there has also been some disagreement in lung sound nomenclature [148]. In addition to wheezes and crackles, there are many other sounds that should be included in training, including rhonchi, pleural rubs, diminished breath sounds, and differentiation of crackles into either fine or coarse crackles. Another step that could be accomplished in this domain is the replacement of particular sound identification with the identification of the underlying pathology. As additional data is collected and annotated, further developments should be made possible.

4.1.2. Continuous Capture of Electrical Signals.

In clinical settings, most ECGs are performed as 12-lead ECGs. In these ECGs, there are 10 electrodes attached to the patient and 12 different measurements taken from these electrodes. Each provides a one-dimensional view of the magnitude of the vectors of all electrical impulses in the heart relative to a given axis. Different axes allow for information to be obtained about the functionality about different parts of the heart. Depending on the goals of remote monitoring, remote ECGs will typically only include a subset of these typical views. As a result, methods that can accurately detect essential signals from minimal lead ECGs are necessary.

Work by Jambukia et al. surveyed machine learning techniques to analyze and classify ECG signals [79]. They evaluated 31 papers that used signal processing and machine learning techniques in order to extract clinically significant features from raw ECG signals. Most of the papers evaluated used the MIT-BIH arrhythmia dataset [127] for both training and testing purposes. Two aspects of ECG classification considered were ECG beat classification for individual, isolated beats and ECG signal classification for interpretation of a longer signal. Some approaches evaluated involve signal feature extraction followed by threshold-based algorithms such as the Pan-Tompkins algorithm. Other approaches utilized various neural network architectures, with the authors finding that of the architectures studied, multilayer perceptron neural networks provided the best performance. Recurrent neural networks, such as the LSTM architecture, were not evaluated in this survey. Deep learning techniques have also been utilized for ECG evaluation. Yildirim showed that a bidirectional LSTM architecture can reliably classify five different rhythms from the MIT-BHI arrhythmia database [202]. This bidirectional LSTM model achieved accuracies greater than other techniques. Additional deep learning techniques that combine CNN and LSTM have been used to detect AFib without explicit feature extraction (such as R peak extraction) [11]. Further deep learning techniques have looked at a variety of processing individual beat anomalies and sequence anomalies [19], though time series presented to CNN models often needs fixed windows of time to be pre-determined for evaluation. Additionally, some work uses a single lead [119] for detecting arrhythmia, though it is likely at least two leads are currently necessary for other ECG feature extraction.

There is evidence to suggest that patients at risk of cardiac pathology benefit from more continuous remote ECG monitoring. The mHealth Screening to Prevent Strokes (mSToPS) randomized clinical trial is an ongoing trial of 2,659 patients investigating the benefit of continuous monitoring for AFib [181]. As reported by Steinhubl et al., the initial phase of the trial discovered that for individuals at risk of AFib, home ECG monitoring was superior to routine care for discovering new incidence of AFib. In the actively monitored group, there was a 3.9% diagnosis of new-onset AFib, vs. 0.9% in the control. This resulted in earlier initiation of anticoagulative therapy (a preventative measure for stroke) in these patients. However, this has also resulted in higher healthcare utilization among these actively monitored patients. This trial is still ongoing—the ultimate clinical impact is still unknown. Clinical outcomes are due to be published in a 3-year follow-up.

4.1.3. Continuous Capture of Vitals Sensing.

Ultrasonography is a technique that uses ultrasonic sound waves to produce images of tissues beneath the skin. Ultrasonography is valuable for visualizing structures that are unreachable noninvasively. In hospital settings, point-of-care ultrasound has increasingly grown in utilization as mobile ultrasound systems become cheaper and comparable in quality to larger ultrasound systems [128]. Point-of-care ultrasonography is useful as a tool that physicians can bring to the bedside for aid in diagnosis, much like a stethoscope, but deep learning techniques are necessary to evaluate the ultrasound images and classify changes in conditions.

Ultrasonography can also be used to evaluate the fluid status of the lungs. As described in Assaad et al., lung ultrasound is a valuable tool for quickly assessing the health of a patient’s lungs [14]. Certain visual findings, such as “B-lines,” are highly associated with edema and various pulmonary pathologies. These visual findings also change very rapidly, reflecting the present disease state more accurately in some cases than measures such as blood oxygen saturation. Lung ultrasonography is also useful in differentiating between cardiogenic and non-cardiogenic pulmonary edema; cardiogenic pulmonary edema typically shows more uniform findings and plural effusion (fluid buildup in the tissue surrounding the lungs). Lung ultrasonography is an underutilized technique in medicine and lacks standardization in training and implementation.

Work by Bhuyan et al. explores an exciting possibility of wearable ultrasound for the monitoring of internal function noninvasively [25]. In order to create a small form factor that could be used to measure organ function with wearable, remote ultrasound, they created a small, flexible probe through a flexible PCB integrated circuit. They also used a system that has only one transmit and one receive channel to avoid excess signal degradation. This system has a bandwidth of 10 MHz, has power consumption of 6.72 mW per channel, and uses 16 such channels to measure a 5.6 mm × 1.6 mm area. They used classical image processing with ultrasound for their validation. Their system, however, used an attached cable to measure. There is an opportunity to create a remote, continuous version of such a system if a flexible PCB-based wearable ultrasound with necessary battery and wireless transmission capabilities were added, but computer vision techniques are needed to enhance the analytic component of the wearable ultrasound.

Echocardiography is the practice of using ultrasound in order to visualize the structures of the heart. Echocardiography can take place either as an invasive transesophageal echocardiography (TEE) or as a noninvasive transthoracic echocardiography (TTE). Many different aspects of the heart can be described and quantified via echocardiography [94], including size, function, and mass of various structures of the heart. Measurement of these parameters aids in the diagnosis of HF. For instance, left ventricular mass or poor emptying are markers of HF. Valvular disfunction, such as stenoses or regurgitations, can be directly observed. These measurements also aid in assessing cardiac function in CAD, particularly following MI; injured portions of the cardiac wall will often move less than they normally would.

Various groups have found preliminary success in applying deep learning computer vision techniques to the analysis of ultrasonographic images. The first step in automatic analysis of ultrasonographic images is to recognize the view in question. Østvik et al. describe the use of a CNN to classify TTE images according to the view being presented [145]. This method showed classification high accuracy in distinguishing among seven different TTE views. Additionally, the authors described a technique for extracting 2D slices from 3D images and achieved a mean error of 4°.

Techniques for measuring edema include cuffs that track ankle circumference and measurement of electrical impedance. Weight monitoring is sometimes used as a proxy for tracking edema, as edema co-presents with fluid retention. There has been success in implantable impedance monitors to measure pulmonary edema. Yu et al. found that intrathoracic impedance serves as a predictor for imminent hospitalization due to fluid overload [203]. In a population of 33 patients with HF, a device consisting of a pacemaker and defibrillator was implanted. The device measured the impedance between those two leads. This study found that there was a significant decrease in impedance prior to hospitalization with fluid overload. This decrease began on average 2 weeks prior to hospitalization and continued through the date of hospitalization.

Impedance monitoring has also been implemented in noninvasive and ambulatory monitoring systems. Weyer et al. describe a system that incorporates both ECG and noninvasive impedance cardiography [197]. This device includes Bluetooth connectivity and a battery that lasts for up to 21 hours. This system could be implemented for long-term monitoring in patients with HF to monitor pulmonary congestion and to potentially allow remote interventions before hospitalization is necessary.

The internal and external jugular veins provide drainage from the head into the heart. The right jugular veins are positioned almost directly above the right atrium, and therefore the pressure within them is very closely tied to the pressure of the right atrium. The external jugular vein’s filling level indicates the pressure within the right atrium and will be distended in cases of right heart failure. Pulsations can be observed with great difficulty in the internal jugular vein. These pulsations provide evidence as to the relative timing and forces involved in right atrial contraction, atrial relaxation, right ventricular contraction, venous filling after the closing of the tricuspid valve, and emptying of the atrium after opening of the tricuspid valve.

As venous pressures are so much lower than arterial pressures, measurement of the jugular venous pulse is much more difficult than the measurement of arterial pulses. However, Amelard et al. were recently able to utilize a technique called PPG Imaging (PPGI) as a viable technique to correctly measure the jugular venous pressure [9]. This technique uses a system located approximately 1.5 meters away from a supine patient. A light shines on the patient and the reflected light is analyzed to identify pulsations. The arterial pulsation from the carotid artery is easier to detect, and the jugular pulsation can be identified as a corresponding inverted pulsation at a location near but lateral to the arterial pulsation. In this study, the ground-truth arterial waveform was verified with a PPG measuring device. Pertinent clinical features were consistently able to be extracted from the venous waveform, including the c, v, x, and y waves (corresponding to the contraction of the right ventricle, systolic filling of the right atrium, relaxation of the right atrium, and beginning of the filling of the right ventricle). In about half of subjects, the a wave was also observed (corresponding to the contraction of the right atrium). The ability to regularly monitor and quantify these waveforms could allow for new techniques in monitoring right heart function.

Signals that capture continuous blood pressure, described in Section 3, may also be extended to capture a variety of heart rate, heart rate variability, blood pressure, respiratory rate, and changes in these values [26, 39, 174]. Obstructive sleep apnea, a condition in which airwaves are restricted, causing the body to wake up from sleep to begin breathing again, increases heart rate, respiratory rate, and blood pressure, keeping patients from falling asleep. This has a direct relationship with blood pressure and nocturnal nondipping HTN, and treatments for apnea have shown to be correlated to improvements in blood pressure [172]. This means approaches for measuring cuff-less blood pressure cannot be restricted to periodic, ambulatory measurements but must transition to continuous beat-to-beat measurement and interpretation.

Finally, telemonitoring trials for HF readmission have tracked longitudinal measurements of symptoms, vitals, and patient qualitative reports [31, 91, 142]. In the Tele-HF study, a number of vitals signals and patient-reported outcomes generated alerts for interventions if the values were below a specified threshold or represented a significant drop from the prior day’s values. However, the study was unable to find a statistically significant reduction in readmissions in the intervention arm. Ong et al. in the Beat-HF study tried to use some machine learning techniques to further identify risks of adverse events, and while the study was unable to reduce readmissions [142], the techniques did show some promise in stratifying patients [158], as did further statistical techniques applied to the Tele-HF data [91, 131]. With the addition of more signals captured and techniques that can better account for varied time-domain aspects of analytics, it is possible that better just-in-time alerts can be generated for preventing future recurrent HF events.

4.1.4. Continuous Capture of Physical Activity.

For cardiovascular disorders, the detection of activities and postures is important in understanding the other biomarkers captured, providing context for their readings. For example, at nighttime, knowing the posture of the user provides context for dyspnea measurements and heart sound recordings for HF patients. In addition to providing context for the other vitals measurements related to cardiovascular disorders, the change in physical activity performance can show increasing effects of HF symptoms and pain as a result of CAD, and acts as a surrogate for the general well-being of these patients.

Many research-oriented activity recognition platforms focus on the detection of activities of daily living [23, 129, 130, 134] and understanding daily exercise intensities. These sensors are capable of tracking sports movements in the healthy and measure sedentary time in the elderly, and come in many forms of smartwatches, smartphones, and smartwatch-sized sensors [129, 132], embedded within shoes, and most recently within eTextiles [2, 208].

HF participants have had improved outcomes in mortality and readmission when involved in cardiac rehabilitation programs that encourage continuous physical exercise [67]. This physical exercise routine has shown that measurements in improved peak exercise capacity correlate with improved cardiovascular outcomes. Home-based cardiac rehabilitation systems centered on physical activity detection in order to quantify a home-based exercise routine [115]. However, such systems do not yet quantify improvement directly from physical activity measurements. This is necessary since adherence in cardiac rehabilitation programs is often quite low [83].

4.1.5. Deep Learning for Personalized and Multi-Modal Models.

Deep neural network techniques have enabled the analysis and modeling of the data gathered from these sensing systems for a variety of event detection techniques. RNN and LSTM are deep learning models that are particularly well suited for developing models for event detection on time-series data, such as segments of ECG signals [170], blood glucose [124], and sleep [207] and in general are good for medical diagnostics using time-series data [106]. Often, the key to these techniques is the ability to generate its own features. For example, improvement in processing of ECG signals with deep neural networks rather than other machine learning techniques allows for the automatic identification of arrhythmias [140]. CNNs, when combined with LSTMs, allow for robust, automatic feature engineering that improves the classification of signals extracted from wearable sensors.

These models are further improved through two techniques: personalization and integrated multiple sensing modalities together in one model. While these deep neural networks are able to extract features that represent important classification properties, person-to-person differences may impact model performance. Therefore, techniques that can train on a user’s own data and then perform functions on later-captured data can show improved performance [157]. Personalized modeling techniques have been used to detect and warn of cardiac arrhythmias [87] and risk of recurrent events in HF patients by tracking changes to cardiac biomarkers [192], which presents a modeling opportunity if sensors can be developed to track those biomarkers in remote and ambulatory settings.

While having enough properly labeled data is a challenge, uncertainty quantification techniques can identify when labeled data is necessary to personalize models for improved performance. Deep neural networks are particularly well suited to this task because of the ability to implement uncertainty quantification techniques on the probabilistic output generated by the models as well as rapidly re-weigh the network through transfer learning or domain adaptation techniques. In HAR tasks, for example, uncertainty quantification techniques that look at the maximum entropy measure from the generated predictions to determine if the model is performing well on existing activities or identifying new users or activity types [13, 73]. Once these periods of uncertainty are found, new data can be captured and transfer learning techniques identify what part of the deep neural network must be modified to account for the new user, new activity, or new sensor type [6, 166, 167].

A number of tools have been developed for assistance in annotating subject-dependent data [51]. An initial challenge in the annotation of data is event detection and segmentation. Adams et al. described a model for event detection and activity segmentation in wearable sensor data streams [3]. They validated their model on several datasets, including one in which events were instances when a user took a puff from a cigarette and activity segmentation was determining whether a user was smoking or not at a given time. This allowed for a system by which smoking event analysis was able to proactively provide feedback to assist in smoking cessation [171]. Labeled data is often collected in artificially constrained environments. However, several groups have focused on developing approaches for collecting annotations in natural environments. Akbari et al. described one such algorithm [5]. This algorithm was demonstrated on a smartwatch and requested the user to annotate activities whenever uncertainty exceeded a threshold, but limited these requests to prevent overwhelming the user with request fatigue. Similarly, Fallahzadeh et al. describe an algorithm that uses context in order to determine the optimal time to request annotations from the user [47]. Each of these techniques allows for collection of personalized annotations, which could then be used in training of more sophisticated models.

While requiring a large amount of data, deep neural networks are well suited to improving classification tasks and analyzing the data from wearable sensors of various sources. Multiple-sensor fusion approaches have improved a number of modeling tasks, including estimating of blood pressure from ECG and PPG signals [123] and HAR through the use of inertial measurement units that integrate accelerometers, gyroscopes, and magnetometers with use of multiple sensors across the body [144]. Additionally, multi-modal deep learning has been used to estimate certain biomedical signals with devices primarily meant to capture other signals, such as smartphones for heart rate estimation [126], and in recognition tasks outside of the cardiovascular domain, including stress monitoring [99], highlighting a key gap and opportunity to improve sensing in cardiovascular care.

In a multi-modal setting, data synchronization, sensor selection, and power optimization are important, in both settings with multiple sensors and settings with sensors integrated into a single platform device. When distinct events are to be detected and classified, that event detection can be used to synchronize the data streams to ensure all are capturing the same events at the same time [20]. Additionally, sensor selection techniques can identify the key features for specific recognition tasks and reduce the number of sensors needed, optimizing the power [13]. Synchronizing data and optimizing usage of sensors and power is of extreme importance in longitudinal use of these sensors but are themselves a subject that requires deep analytic technique reviews out of the scope of this cardiovascular disorder survey, given the current lack of integrated sensing platforms for important risk factor monitoring.

4.2. Gaps

Deep learning techniques have made the exploration of time-series data more fruitful with the development of automatic features that represent longitudinal risk or outcomes. Techniques such as attention-based LSTMs have shown promise in exploring continuous time-series data to predict clinical mortality, decompensation, and length of stay, which outperform hand-crafted feature extraction, and other deep learning techniques that do not find focus on specific periods of time, in intensive care unit data [179]. However, these techniques have not been applied to this remote data yet because the integration of these sensing techniques has not yet occurred.

A number of the analytic techniques tied to the use of these new sensing paradigms have focused on the diagnosis of specific anomalies or classification of specific types of sounds or signals captured. What is still needed is the following:

• Integrated sensors that can capture signals frequently, or continuously, over entire study periods.

• Machine learning techniques that can explore multiple windows of time over multiple combinations of available signals in order to quantify trajectories in signals, identifying longitudinal patterns and changes in signal that may be indicative or worsening conditions or treatment effectiveness and recovery, extending beyond anomaly detection and signal classification.

• Data synchronization in multi-modal platforms: identifying how many sensors are needed for an application and minimizing the user burden in wearing them and needing to re-charge them is an important problem that will need to be addressed as these systems are developed for longitudinal use.

• Deep learning techniques listed demonstrate how personalization can improve model performance. However, personalization requires the labeling of data samples for supervised learning techniques. The longitudinal capture of these labels may result in undue burden on the users of the system. Finding a balance between the types and quantity of data needed and the passive collection is of utmost importance for user adherence.

4.3. Opportunities

One way in which advanced work in analytics could be incorporated is in better personalized monitoring for risk of CAD. As previously discussed, CAD is a condition often characterized by gradual worsening of chest pain that culminates in a heart attack. Ideally, monitoring systems would be able to follow gradual changes prior to the rupture of a plaque that causes a heart attack. In the early stages of CAD, activity monitoring could be used to assess wellness. By tracking a certain threshold of activity that the patient does not (or cannot) exceed, a monitoring system can estimate the severity of angina. As that threshold begins to decrease, the patient’s angina is likely increasing and greater intervention may be warranted. The monitoring system for a patient at risk of CAD should also include an ECG system to watch for changes associated with a heart attack. If any electrical changes concerning for a heart attack begin to appear in the patient’s ECG, then emergency services would be required. Earlier interventions are associated with better outcomes, and a monitoring system like this coupled with improved analytics could potentially allow for earlier treatment, leading to less overall damage and better patient outcomes.

Improved analytics could also be implemented to better treat valvular diseases. Unlike the other pathologies discussed here, there are few risk models for predicting future valvular heart disease. However, advanced analytics could be implemented to allow for earlier detection of valvular disease. As discussed above, these abnormalities change the way in which blood flows through the chambers of the heart, producing turbulence that can be detected as sound. The most straightforward evaluation for valvular disease in remote wearable settings would involve electronic stethoscopes continuously monitoring the patient’s heart sounds. By learning the normal sound profile of a patient, new changes and murmurs could be quickly identified. After identifying a particular valvular disease and its associated murmur, long-term monitoring with electronic stethoscopes could be used to characterize the severity of the valvular insult; most murmurs initially increase in intensity but in later disease stages decrease in intensity. Rather than risking false-negative screening in physical examinations, a longitudinal monitoring system could detect the changes along this trajectory to allow for more informed decision making. As a more advanced option in monitoring valvular disease, miniaturized ultrasound probes could be incorporated into a wearable system. These could be used for imaging and analyzing the valvular parameters such as cross section and flow. Additional work into computer vision interpretation of ultrasound images would be necessary in order to automatically process these signals. Vital monitoring can also directly feed into an understanding of valvular disease. In particular, blood pressure can reflect aortic valve lesions. Finally, systems to monitor valvular disease could monitor symptomatic disease progression. As many types of valvular disease may ultimately lead to HF, the opportunities presented above for HF apply here as well. Foremost among them would be activity recognition, where late-stage valvular disease can manifest with a loss of stamina in day-to-day activities.

Opportunities:

• Improved machine learning processing of existing sensor modalities: Development of machine learning techniques that can extract meaningful data from non-numerical sources, expanding on the computer vision work done in automatically processing and interpreting ultrasound images.

• Time-series machine learning models: Development of machine learning techniques that can process longitudinal data and account for multiple channels of data, sampled at different frequencies, and with different segment lengths of importance, is required to develop new risk prediction techniques and alerts based upon continuously captured data.

• Applying attention mechanisms to deep learning techniques to interpret what features are being extracted and better understand the interdependence of the multi-modal learning techniques will enable more rapid selection of key sensors for longitudinal tracking and event detection.

5. CLINICAL INTERPRETABILITY, ANALYTIC MODELS, AND TREATMENT PARADIGMS

Clinical risk prediction models and those that predict adverse events have helped guide medical treatments and improve patient care. These techniques, with machine learning modeling, have the potential to improve clinical care in both the acute care settings [169] and remote care settings. This includes understanding the diagnosis and progression of diseases and the personalized patterns and signals that can be captured by advances made in the categories listed in Sections 3 and 4. Some preliminary work has been conducted in clinical trials on HF patients, understanding distinct patient phenotypes within the disease. In one such HF trial, clinically distinct clusters of patients were found to have different time-to-event predictions and outcome rates [4]. Another relevant clinical trial in HF patients is the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist (TOPCAT) trial [40, 156]. The purpose of this trial was to determine if a treatment designed specifically for HF patients with preserved ejection fraction could improve outcomes, a patient population where such treatments have not been found to universally treat these patients. This trial was also unsuccessful in showing that HF patients treated with spironolactone had better outcomes [156]. However, due to some issues with data gathered in certain regions, investigators began taking a closer look at subsets of patients to determine if specific patients were actually helped by the treatment. The investigators found that regional variations led to different treatment effectiveness in cohorts of participants [40, 153]. This indicates that HF patients diagnosed with preserved ejection fraction may benefit from cluster analysis, looking at personalized differences in outcome rates where different treatments may be helpful for different subsets of patients. These provide for the basis of the following needs:

• Risk prediction models: As illustrated by the TOPCAT findings, these diseases are quite complex, and understanding the person-to-person variation allows for specific risk prediction based upon data collected, along with matching techniques that allow for comparison to patients most similar to the individual modeled. (See Section 5.1.1.)

• Dynamic adaptation: Models must account for the varied data types potentially collected, the varied rate at which they are collected, and how well to link them to data gathered in acute care settings, and be able to update as a disease progression worsens or treatment regimen proves effective, including providing confidence metrics that suggest the collection of additional data, if necessary. (See Section 5.1.2.)

• Time-to-event modeling: With longitudinal sensing, methods of survival analysis that adapt to time-varying would allow for updated risk estimates for adverse events both in terms of likelihood of event and in estimating the likely time to that event occurrence. (See Section 5.1.3.)

• Multi-task learning: Deep learning techniques are well suited to estimate risks of multiple, potentially varying adverse events, leveraging the commonality in risk factors associated with the primary adverse events or secondary recurrent events related to the different cardiovascular disorders. (See Section 5.1.4.)

• Interpretable machine learning: As the data size progresses, medical models must be able to explain the driving risk factors in a manner interpretable to clinicians in order to guide treatment decision making. (See Section 5.1.5.)

5.1. Existing Technologies and Applications

5.1.1. Risk Prediction Models.

Much of stroke risk prediction is tied to the risk associated with AFib. In particular, it may be appropriate for patients with AFib to undergo anticoagulation therapy in order to reduce their risk of stroke. Anticoagulation therapy is any therapy that works to reduce the rate at which blood clots form. This type of therapy can be beneficial by preventing thromboembolic stroke. Conversely, this type of therapy can be detrimental by promoting life-threatening bleeds, such as in hemorrhagic stroke. Therefore, implementation of any anticoagulation therapy must be implemented with great care. In addition to models that predict only stroke risk, the ACC/AHA Pooled Cohort Equations treat stroke and CAD together.

CHA2DS2-VASc is a model that predicts 12-month thromboembolic event rate (including stroke, pulmonary embolism, and peripheral thromboembolism) in patients with AFib who are not undergoing anticoagulation therapy [105]. Creation of this model drew upon the efforts of and improved upon multiple older models in order to apply more broadly and accurately to diverse patient populations. One chief exclusion in this model is that only patients with nonvalvular AFib are considered. The parameters considered in this model are presence of HF, HTN, age, diabetic status, history of stroke or other thromboembolic event, history of any vascular disease, and gender. This model aids clinicians in prescribing anticoagulants, which increase the risk of bleeds but decrease the risk of thromboembolic events (including stroke).

The HAS-BLED model was created to predict the risk of bleeding in anticoagulated patients with AFib [155]. The parameters included in this model are HTN, history of liver or kidney dysfunction, history of stroke, history of bleeding, difficulty calibrating oral anticoagulation therapy, use of alcohol, and use of certain drugs that may increase bleeding risk. Recommendations by groups such as the European Society of Cardiology [122] are that CHA2DS2-VASc and HAS-BLED be used in conjunction for informed decision making, and that HAS-BLED alone should not be a reason to withhold anticoagulant therapy.

Other models have been produced to predict general risk of stroke. The MyRisk_Stroke Calculator is a model to predict 10-year risk of stroke [136]. This estimator was built on a prospective dataset where collection began in 1992 and was validated with a second dataset with collection beginning in 1998. Follow-up was through the year 2007. In this cohort, the parameters found with an association to stroke risk were age, gender, education status, high blood pressure, smoking status, alcohol consumption, activity levels, anger, depression, and anxiety. Additionally, comorbidities such as renal disease, diabetic status, HF status, CAD, and peripheral arterial disease were included as features in this model. The model was created as a Cox proportional-hazards model and predicts 10-year risk of any type of stroke.

Another stroke risk model is the QStroke score [69]. QStroke was developed to be used for all patients without history of stroke but intended specifically to be used as a supplement or replacement for CHA2D2-VASc in predicting risk associated with AFib. The QStroke model features the following as parameters: age, gender, ethnicity, Townsend deprivation index (an index related to socioeconomic status), smoking status, body mass index, systolic blood pressure, blood lipid levels, and family history of CAD. HTN, diabetic status, AFib status, HF status, CAD, presence of rheumatoid arthritis, renal disease, and valvular disease were also included as pertinent comorbidities. The QStroke model was created as a Cox proportional-hazards model and predicts 10-year risk of any type of stroke.

Many attempts have been made to assess the risks of developing CAD. Among the most current of these are from the ACC/AHA Task Force on Practice Guidelines [61]. That work introduced a set of models termed the Pooled Cohort Equation to predict a primary CAD event within 10 years. The predicted risk in this model is based on age, gender, race, blood pressure (systolic and diastolic), diabetic status, smoking status, various cholesterol lab values, and certain current medications (HTN control, statins, or aspirin). This risk prediction tool was built to predict any type of “hard” atherosclerotic-based disease, and therefore in addition to predicting future CAD it also predicts future stroke. However, it does not distinguish between risks for these two different outcomes and treats them both as a positive outcome.

5.1.2. Remote and Dynamic Models.

Remote and telemonitoring studies that use telephones and call centers as the primary source of data have been used to track HF patients, in the hope of reducing heart failure admissions. These systems are intended to track patient symptoms, including impact of medication, weight gain (as a surrogate for edema), and depression, to identify early signs of decompensation aimed at providing interventions that prevent hospital readmissions in HF patients. In Tele-HF, Krumholz et al. found that a self-report telemonitoring system was not able to reduce readmissions in heart failure patients based upon daily reports of symptoms, medication usage, weight, and depression [91]. Ong et al., in the Beat-HF trial, looked to automate some of the data collection surrounding blood pressure and weight with machine learning risk models to drive interventions, but found similarly that HF readmissions were not reduced [142]. Anker et al. surveyed meta-analyses and prospective clinical trials that evaluated the efficacy of telemonitoring in patients with HF [12]. They found disagreement between the efficacy of telemonitoring for HF in different types of trials but stressed that the outcomes of telemedicine depend on personalization to the particular patient.

Models that predict risks within varied windows of time have, thus far, been restricted to medical settings. Henry et al. used a rolling model to predict the risk of sepsis in a hospital setting, selecting important features and identifying dynamic risks of sepsis within a single hospital admission [66]. Such dynamic models could be adapted to remote and longitudinal settings, but this has not been done yet.

Few models exist, however, that estimate clinical risks of adverse events using remote and sensible data. Cakmak et al. used a smartphone to estimate answers to the Kansas City Cardiomyopathy Questionnaire, which aims to rate health status and severity of symptoms of conditions such as HF [29]. These personalized models that use remote data to estimate clinically validated instruments used in current clinical models present a significant gap and opportunity for the systems discussed here.

5.1.3. Deep Time-to-Event.

Survival analysis is an important domain of clinical modeling where data provided to a model estimates the likelihood of an event occurring as a function of time and the measured risk factors. This provides both an estimate of the likelihood of an event occurring and when it will occur, providing better longitudinal analysis of risk. The primary method for this technique has been a Cox proportional hazards model, which estimates the likelihood of survival over time with an underlying logistic regression model, which is linear in nature. Recently, deep learning techniques have improved upon the fit of these estimates over time by allowing for complex, nonlinear interactions [84]. Similarly, work by Lee et al. estimate the direct survival using deep neural networks [97] and continued improving upon this work by allowing for a range of dynamic covariates prior to the point of estimation rather than just the last value available for each time series [96]. Ishwaran et al. proposed a new model for survival analysis based upon adaptive boosting, which allows for time-varying covariates that can change at different points in time, allowing for flexibility in captured time-series signals and the features used from them to estimate risk [100]. Ultimately, these models will need to be adapted to the use of remote sensing data to track long-term risk of events based upon daily captured data for individuals wearing remote sensing systems.

5.1.4. Multi-Task Learning and Attention.

The use of multiple sensing signals to track common risk factors over an array of differing cardiovascular disorders requires models that are robust to estimating multiple outcomes. Deep learning techniques are well suited for this multi-task learning, having already demonstrated model performance superiority in clinical settings, such as estimating outcomes of patients in an intensive care unit [183]. Similarly, multi-task learning has proven to have superior performance using clinical time-series data [108] and in estimating in-hospital mortality [204]. The primary principle behind the multi-task learning environment is modifying loss functions to account for how accurate the model is in estimating a set of outcomes rather than an individual prediction, with the assumption that the features being extracted from the data sources can estimate risk of each outcome, due to their dependent nature. This presents an opportunity to adapt these models to estimate risks from both in-hospital and remote, sensing data.

While these models are primarily built with CNNs, RNNs, and LSTMs, two key challenges arise when using these techniques: limiting the input space to signals of the same length sampled at the same duration and interpretation of their findings. While RNNs and LSTMs can handle varying-length sequences better than CNNs, with padding and masking techniques, they still sample time-series data across multiple channels at fixed time intervals. With larger sequences of varying types of data, models that can adapt to different lengths, such as those used in natural language processing techniques, are needed, namely, transformers. The transformer architecture presents opportunities to further enhance model performance of time-series signals by allowing for additional flexibility to point the deep learning model at which portions of signals to focus on [78] that may not be properly aligned [188]. This modification to the more standard CNN, RNN, and LSTM architecture allows for more accurate modeling of clinical time-series data by leveraging when signals change and when they are invariant [139] and by adapting time-warping techniques [112]. Transformers have an attention property that allows it to attune to specific regions of data. This attention allows for more accurate clinical risk prediction models using time-series data [179]. The attention mechanism also has a property of providing a level of interpretability to deep learning techniques by identifying portions of signals that are deemed more important for feature extraction and model training.

5.1.5. Interpretable Machine Learning.

A recent push in the machine learning field has been to explain predictions provided by deep learning methods that are generally considered black-box techniques. Ribeiro et al. developed a technique by which local logistic regression models are able to identify the reasons a particular prediction is made based upon the variables that generated the prediction of that element and similar model elements [162]. This work demonstrates model interpretability, which comes naturally in CNN deep learning models that can visualize data in intermediate models but becomes much more complicated in time-series-based models such as LSTMs. Work by Lundberg and Lee looked to develop personalized levels of interpretation that are model agnostic, demonstrating a feature distribution and visualization technique that shows how certain actors matter for each user in a model and how that impacts the overall model performance [113]. Additional interpretation of models for how personal factors impact estimations provides personalization of interpretation. Additional machine learning techniques look to automatically cluster patients and explain the phenotype discovery [65], while also learning to predict multiple outcomes at the same time across different patient types [184], but work on explaining the findings remains in preliminary stages [59].

Interpretability also indicates the confidence in estimations and understanding what data helps and hurts the predictive accuracy of techniques. In work aimed at improving real-time context and activity detection, Ardywibowo et al. evaluated selected sensors to improve HAR with constraints on the types of sensors and the power those sensors consume [13]. Work by Castilla et al. uses the idea of uncertainty quantification in order to direct users to gather more data in real-time, diet-logging settings [178]. Uncertainty quantification is an emerging field of interpretable machine learning that has the ability to guide confidence in predictions collected as well as suggest additional data that patients and clinicians should consider collecting.

5.2. Gaps

Existing models for predicting risk in cardiovascular conditions rely on sparse data that are measured on rare occasions. Many parameters are trivial to measure (age, gender), and many parameters are Boolean values relating to history. In comparison to the data-produced continuous monitoring systems, these data are sparse and likely overly simplistic. There are two chief ways in which the limitations posed by this sparsity of data can be overcome with richer data: existing models can be updated to include richer data sources, and richer data sources can be analyzed for anomaly detection and rare event detection.

The following gaps remain in developing personalized analytic models based upon the remote sensing data gathered:

• Integration of sensing data with acute care data and outcomes for robust risk prediction models.

• The clinical models, to date, do not use complex remote, ambulatory sensing data. The initial development of models that leverage this data is needed.

• Learning key features for predicting adverse events from longitudinal capture without the presence of ground-truth data remains a challenge. Understanding how to extract features that work in accurate risk prediction models using remote sensing data and providing confidence to clinicians on these findings will require collecting vast amounts of data on users to track for the potential of clinical events. This means integrating the sensing systems with electronic health record models that have the ground-truth diagnosis and treatment information for such events.

• Development of dynamic models that are flexible to the types of data collected, the windows of data collected, and the changes in patient condition throughout observation.

• Deep-learning-based time-to-event models either do not update when covariates change over time, fixing a longitudinal prediction with dataset at a certain point in time, or update model estimations at fixed time-grid intervals. A time-to-event model that is able to adapt to time-varying covariates as they are captured, such as those from the sensing systems described in this work, is needed to update risk estimation.

• Interpretable machine learning to explain the predictions of these complex models and help guide clinical decision making, including identifying similar patients and explaining potentially new phenotypes that might be discovered.

5.3. Opportunities

Existing models to quantify disease state and future disease risk could be improved through the implementation of richer data sources into the mode. The NYHA Functional Classification of HF relies in part on physical activity levels. The levels are subjective, with definitions in part of “no limitation of physical activity” (class I), “slight limitation of physical activity” (class II), “marked limitation of physical activity” (class III), and “unable to carry on any physical activity” (class IV). These classes are inherently subjective, and therefore susceptible to variability between patients with the same underlying disease state. Augmenting this classification with patterns detected from signals such as HAR and effort involved in activity (such as via heart rate monitoring) would allow for objective measurements from beyond the limited scope of direct patient-physician contact. The increase of objective measurements would likely lead to updates to existing models and better information to aid in making clinical decisions.

Existing models could also be improved by the detection of rare or uncommon events. For instance, when a patient presents with AFib, the duration of the AFib is typically unknown. As discussed above, the CHA2DS2-VASc score can aid physicians in predicting stroke and the appropriateness of implementing oral anticoagulation therapy. However, the parameters that contribute to the CHA2DS2-VASc score are simplistically sparse. Age and gender are (for cardiac risk purposes) nonmodifiable risk factors. Each of the other parameters are positive if the patient has ever had the given event once in their life: HF, HTN, stroke/TIA/thromboembolism, vascular disease, or diabetes. It stands to reason that this model may be improved from richer data, such as the pattern or frequency with which the patient experiences episodes of AFib. Addition of this richer data to the model could potentially result in a model that is better able to discriminate between those at risk of stroke and those at lower risk of stroke, allowing for more appropriate and judicious use of oral anticoagulants.

The emergence of new sensing and internet of things (IoT) technologies creates a need for new models to incorporate new data for better prediction and understanding of disease states. The drastic increase in technology such as smartphones and smartwatches allows for new rich data sources, and also creates a need for the utilization of these data sources. Recently, smartwatches have been adapted to detect conditions such as AFib [27]. Further work should look at implementing these new modalities into longitudinal risk models. For instance, HAR recognition could be implemented as a parameter in monitoring activity tolerance in patients with HF. This could supplement the existing subjective measures of heart failure with newer objective measures.

Ultimately, data from new rich data sources is only valuable so far as it contributes to improving the quality of patient healthcare. In order for this contribution to take place, models must generate actionable feedback that can be used for informed clinical decision making. Rather than presenting modeling through a black-box approach where data is supplied to the model and an answer is returned, it is desirable that the reasoning behind the risk score is understandable. If a model is interpretable, then the factors leading to a given score can be understood and interventions made to address the risk and to improve patient outcomes. Additionally, the greater the interpretability of a model, the more information that the physician and patient are able to have about the overall disease state. As this information is understood by the physician and the patient, it can be used to better inform and guide care. As a result, the following opportunities exist for immediate and impactful machine learning research:

• Machine learning models with cross-sectional and time-series data: Integration of sensing data with acute care data and outcomes for robust risk prediction models.

• Development of dynamic models that are flexible to the types of data collected, the windows of data collected, and the changes in patient condition throughout observation.

• Interpretable machine learning to explain the predictions of these complex models and help guide clinical decision making, including identifying similar patients and explaining potentially new phenotypes that might be discovered.

• Transfer learning: Transfer learning techniques will be able to take developed models and adapt to a variety of signals captured, a variety of patients modeled, or a combination therein, improving the flexibility of any analytic techniques developed to advance the prior three opportunities.

• Deep learning: Adaptation of deep learning techniques that have proven successful in natural language processing tasks and computer vision tasks to time-series modeling based upon the remote sensors discussed in this work provide an opportunity to develop new transformer and attention-based models that are adaptable to various signals of different domains and lengths.

• Adaptive time-to-event models: As deep time-to-event models improve the estimation of risk longitudinally, developing dynamic models that adapt the model structure to data that is newly available through different sensors is needed that not only account for changes in the values of the modeled covariates but also can adapt to new time-series signals as they become available.

6. DISCUSSION AND CONCLUSION

We surveyed the field of sensing technologies and machine learning analytics that exist in the field of remote monitoring for the tracking of risk factors that lead to primary adverse events and secondary recurrent events associated with cardiovascular disorders. Through the evaluation of these sensing modalities and machine learning techniques, we highlighted the potential for addressing three critical areas of need for care in patients monitoring risk factors associated with heart failure, coronary artery disease (and myocardial infarction), and stroke: (1) need for sensing technologies that track longitudinal trends of the cardiovascular disorder despite infrequent, noisy, or missing data measurements; (2) need for new analytic techniques designed in a longitudinal, continual fashion to aid in the development of new risk prediction techniques and in tracking disease progression; and(3) need for personalized and interpretable machine learning techniques, allowing for advancements in clinical decision making. We highlight these needs based upon the current state of the art in smart health technologies and analytics and discuss the ample opportunities that exist in addressing all three needs in the development of smart health technologies and machine learning (primarily deep learning) approaches applied to the field of cardiovascular disorders and care. Whereas the progression of smart health technologies in these needs has demonstrated success in fields such as HAR and physical disorder monitoring, the opportunities for addressing cardiovascular care are many.

These cardiovascular disorders are often very complex conditions characterized by multiple changes in a patient, many of which are slow and difficult to notice. However, systems could be built to take into account and monitor many different changes in order to track risk factor progress for disease state monitoring and to allow clinical decisions to be made before rapid decompensations. As a disease progresses, regular monitoring of heart sounds could be used in order to track heart remodeling. Instead of noticing these sounds in an acute care visit, computer-aided auscultation though wearable electronic stethoscopes could allow for earlier detection. Quantitative edema tracking would allow for monitoring functional changes within the heart. As pulmonary edema increases, clinicians are able to tell that left heart function is decreasing. Changes in ECG signals may indicate progression of CAD disorders that may result in additional patient pain, prior to leading to heart attack. As a patient’s condition worsens, they may gradually lose the stamina to walk certain distances or to perform a certain amount of activity, demonstrating changes in physical activity capacity, respiratory rate, or sleep quality. These changes may be so gradual that patients may not notice them. Instead, new analytics for progression could instead build activity recognition into the modeling to understand slow changes in baseline function. In this, departures from a patient’s baseline level of activity would be significant and could be useful information for guiding clinical care. Similarly, alterations in blood flow may lead to changes in urination habits. As blood flow to the kidneys might be restricted during the day and increased at night due to postural changes, the kidneys will produce more urine at night. Tracking frequency of nocturnal urination could provide more clues as to overall health. Note that this will be sensitive but not specific for heart failure. Additional measurement of hemodynamic characteristics, such as HTN, may show treatment effectiveness and better guide the improvement of factors that would lead to conditions such as stroke. In total, a comprehensive system to track the progression of cardiovascular disorders should incorporate a body of integrated sensors, capture this data over longitudinal periods of time, and, as a result, enable new advancements in machine learning techniques that can make best use of this data to help guide patient and clinician alike in improving patient care through personalized, dynamic time-to-event modeling.

This survey highlighted the needs in developing smart health applications to treat HF, CAD, and stroke, and the risk factors associated with them. It reviewed the existing technologies, highlighting the current gaps in solutions presented for those needs. Finally, it presented a series of opportunities, including advanced analytic techniques to be developed once new sensing solutions are available, that can guide impactful changes in the way patients with cardiovascular disorders are cared for.

CCS Concepts:

• Applied computing → Health informatics; Health care information systems; • Computer systems organization → Embedded systems; • Information systems → Data analytics; • Computing methodologies → Machine learning;