Epidermal mechano-acoustic sensing electronics for cardiovascular diagnostics and human-machine interfaces

Researchers report advances in materials and designs for skin-integrated devices capable of measuring acoustic signatures.

Abstract

Physiological mechano-acoustic signals, often with frequencies and intensities that are beyond those associated with the audible range, provide information of great clinical utility. Stethoscopes and digital accelerometers in conventional packages can capture some relevant data, but neither is suitable for use in a continuous, wearable mode, and both have shortcomings associated with mechanical transduction of signals through the skin. We report a soft, conformal class of device configured specifically for mechano-acoustic recording from the skin, capable of being used on nearly any part of the body, in forms that maximize detectable signals and allow for multimodal operation, such as electrophysiological recording. Experimental and computational studies highlight the key roles of low effective modulus and low areal mass density for effective operation in this type of measurement mode on the skin. Demonstrations involving seismocardiography and heart murmur detection in a series of cardiac patients illustrate utility in advanced clinical diagnostics. Monitoring of pump thrombosis in ventricular assist devices provides an example in characterization of mechanical implants. Speech recognition and human-machine interfaces represent additional demonstrated applications. These and other possibilities suggest broad-ranging uses for soft, skin-integrated digital technologies that can capture human body acoustics.

INTRODUCTION

Unusual classes of electronics enabled by recent advances (1–6) in materials science and mechanics principles can be designed with physical properties that match the soft, mechanical compliance of the skin, thereby allowing long-term (up to ~2 weeks) integration with nearly any external surface of the body, with form factors that resemble those of a temporary tattoo. These systems, referred to as epidermal electronics, qualitatively expand the range of physiological measurements that are possible in wearable device platforms (7–13). Many of these operational modes rely critically on an intimate, physical interface to the skin. Examples include precision measurement of temperature and thermal transport characteristics (14, 15), recording of electrophysiological processes and variations in electrical impedance (16–18), characterization of skin stiffness (5, 19), and monitoring of quasi-static or dynamic dimensional changes, such as those associated with swelling/deswelling or pulsatile blood flow (20, 21). The critical enabling properties of the devices and their interfaces with the skin include low thermal and electrical contact resistances, small thermal masses, and soft, compliant mechanics. Another (previously underused yet important) feature is that the devices can be constructed with exceptionally low mass densities, approaching those of the epidermis itself. An unexplored consequence of this characteristic is that mechano-acoustic coupling of the device to the body through the skin can be highly efficient. The associated opportunity examined here is in precision measurements of acoustic and vibratory signatures of body processes and of mechanically active implants.

Mechano-acoustic signals are known to contain essential information for clinical diagnosis and healthcare applications (22, 23). Specifically, mechanical waves that propagate through the tissues and fluids of the body as a result of natural physiological activity reveal characteristic signatures of individual events, such as the closure of heart valves, the contraction of skeletal muscles, the vibration of the vocal folds, and movement in the gastrointestinal tract. Frequencies of these signals could range from a fraction of 1 Hz [for example, respiratory rate (23)] to 1000 Hz [for example, speech (24, 25)], often with low amplitudes beyond hearing threshold (26, 27). Physiological auscultation typically occurs with analog or digital stethoscopes, in individual procedures conducted during clinical examinations. An alternative approach relies on accelerometers in conventional, rigid electronic packages, typically strapped physically to the body to provide the necessary mechanical coupling. Research demonstrations include recording of phonocardiography (PCG; sounds from the heart) (28), seismocardiography (SCG; vibrations of the chest induced by the beating of the heart) (29–34), ballistocardiography (BCG; recoil motions associated with reactions to cardiovascular pressure) (28, 30, 35, 36), and sounds associated with respiration (22, 23). In the context of cardiovascular health, these measurements yield important insights that complement those inferred from electrocardiography (ECG). For example, structural defects in heart valves manifest as mechano-acoustic responses and do not appear directly in ECG traces. Previously reported digital measurement methods are useful for laboratory and clinical studies but (i) their form factors (rigid designs and large size, for example, 150 mm × 70 mm × 25 mm) limit the choices in mounting locations and prohibit their practical utility as wearable; (ii) their bulk construction involves physical masses that suppress, through inertial effects, subtle motions associated with important physiological events; (iii) their mass densities and moduli are dissimilar from those of the skin, thereby leading to acoustic impedance mismatches with the skin; and (iv) they offer only a single mode of operation, without the ability, for example, to simultaneously capture ECG and PCG/SCG/BCG signals.

Here, we present a different type of mechano-acoustic–electrophysiological sensing platform that exploits the most advanced concepts in stretchable electronics to allow soft, conformal integration with the skin. The result allows precision recordings of vital physiological signals in ways that bypass many of the limitations of conventional technologies. The mechano-acoustic modality leverages miniaturized, low-power accelerometers with bandwidths tuned to essential body processes (0.5 to 550 Hz) and associated conditioning electronics. Soft, strain-isolating core/shell packaging assemblies, together with electronics for electrophysiological recording from dry, capacitive electrodes, represent the other essential features of these stretchable systems. The resulting devices have a mass of 213.6 mg, a thickness of 2 mm, effective moduli of 31.8 kPa (in the x direction) and 31.1 kPa (in the y direction), and bending stiffnesses of 1.02 μN m (in the x direction) and 0.94 μN m (in the y direction), which correspond to values that are orders of magnitude lower than those previously reported. The outcomes include qualitative improvements in measurement capabilities and wearability, in formats that can interface with nearly any region of the body, including curvilinear parts of the neck to capture signals associated with respiration, swallowing, and vocal utterances. The following describes the key properties of these devices and demonstrates their utility in wide-ranging examples, from human studies on patients with cardiovascular disease to human-machine interfaces for video game control. Specific data show simultaneous recording of arterial pressure variations on the neck, electrophysiology (EP) signals and SCG from the chest for systole and diastole cardiac cycles, and four auscultation sites (aortic, pulmonary, tricuspid, and mitral) for heart murmurs. Vibrational acoustics of ventricular assist devices (VADs) (that is, devices used to augment failing myocardial function, though often complicated by intradevice thrombus formation) can be captured and used to detect pump thrombosis or drive malfunction. Beyond cardiology, applications exist in speech recognition and classification for human-machine interfaces, in modes that capture vibrations of the larynx without interference from noise in the ambient environment. Baseline studies on the biocompatibility of the skin interface and on the mechanical properties and fundamental aspects of the interface coupling provide additional insights into the operation.

RESULTS

Device design and circuit considerations

The mechano-acoustic–electrophysiological sensing platform introduced here incorporates filamentary serpentine copper traces [3 μm, placed at the neutral plane between layers of polyimide (PI) encapsulation (1.2 μm)] as circuit interconnects between commercial, small-scale chip components, all encapsulated above and below by an ultralow-modulus elastomeric core [Silbione RT Gel 4717 A/B, Bluestar Silicone; Young’s modulus E = 5 kPa (2, 37)]. A thin layer of low-modulus silicone (Ecoflex, Smooth-On) [E = 60 kPa (9)] serves as a shell (Fig. 1A and fig. S1A). This core/shell structure minimizes physical constrains on motions of the interconnects to improve stretchability (2, 7–9, 11), and it mechanically isolates the constituent device components to reduce stresses (and associated discomfort) at the skin interface, as described previously in detail (37). Openings in this structure provide access to contact pads to attach a pair of electrophysiological measurement electrodes (Au, 200 nm) and a thin cable connection [100 μm; anisotropic conductive film (ACF), Elform] to an external data acquisition system (Fig. 1B). The result is a soft, skin-compatible device platform (Fig. 1C and fig. S2, C and D) that can accommodate significant levels of deformation without altering the operation (Fig. 1D and fig. S2, E and F). The direct mechanical interface to the skin, the robustness of adhesion that follows from the low-modulus construction, the low total mass, and the multifunctional operation represent key distinguishing features over previously reported wearable accelerometers. Each of these attributes is critical to the operational modes described in the following.

Fig. 1. Schematic illustration of an epidermal mechano-acoustic–electrophysiological measurement device.

(A) Exploded view diagram of the overall design structure of the system. (B) Illustration of the assembled device and its interface with soft EP measurement electrodes and flexible cable for power supply and data acquisition. A cross-sectional view appears in the upper inset. (C) Device held by tweezers in a twisted configuration. (D) Device mounted on skin while compressed by pinching. (E) Fluorescence micrographs of cells cultured on the surface of a device to illustrate its biocompatibility. Green and red regions correspond to live and dead cells, respectively. The white arrowheads highlight the latter. Scale bars, 200 μm. (F) Overlay of optical image and finite element simulation results for a device under biaxial stretching to a strain of 25%. (G) Magnified view of modeling results for the part of the interconnect structures that experiences the highest strain. (H) Vibrational response summarized in a plot of spectral power measured while mounted on a layer of chicken breast, to simulate tissue, on a vibration source.

The sensing circuit (fig. S1, B to D) consists of a mechano-acoustic sensor (ADXL335, Analog Devices; fig. S2, A and B), low-pass and high-pass filters, a preamplifier (TSV991A, STMicroelectronics), and removable and reusable capacitive electrodes for EP recording (fig. S3) (8). The sensor has a frequency bandwidth (0.5 to 550 Hz) that lies between the range of targeted cardiovascular sounds and speech. For healthy adults, the first sound (S₁) and the second sound (S₂) of the heart have acoustic frequencies of 10 to 180 Hz and 50 to 250 Hz, respectively (38). Vibration frequencies of vocal folds in humans range from 90 to 2000 Hz (39), with an average fundamental frequency of ~116 Hz (male; mean age, 19.5), ~217 Hz (female; mean age, 19.5), and ~226 Hz (child, ages 8 to 11) during conversation (40). To enable sensing of cardiac operation and speech, the cutoff frequency of the low-pass filter is 500 Hz. The high-pass filter (cutoff frequency, 15 Hz) removes motion artifacts.

Device characterization

The experimental and simulation results in Fig. 1 and figs. S4 to S11 summarize key characteristics of the materials and structures that lead to the type of soft mechanics, water-permeable, and adhesive, biocompatible surfaces needed for comfortable, robust, long-lived integration on the skin. Results in fig. S4A show that an additional base layer of Silbione on the bottom shell surface can provide adequate adhesive force (1.16 kPa) for nondestructive and reversible attachment to skin. Measurements of the water vapor permeability (fig. S5) of Silbione, in combination with previously reported results of Ecoflex (2), demonstrate that the core/shell encapsulation layer has a water vapor transmission loss rate that is similar to that of widely used medical dressings (Tegaderm, 3M Medical).

Cytotoxicity tests that involve culturing mouse embryonic fibroblasts (MEFs) on the surfaces of the device for 5 days demonstrate biocompatibility. Specifically, cells spread uniformly over the samples and remain attached for the duration of the assay, with no observable signs of apoptosis or necrosis (Fig. 1E). Visualizing cells at 1-, 3-, and 5-day time points by staining with calcein AM and ethidium homodimer-1 indicates >95% viability after 5 days (fig. S6).

Experimental studies and three-dimensional finite element analysis (3D-FEA) of the system under a biaxial strain of 25% allow for the examination of the mechanics at levels of deformation that exceed those likely to be encountered on the skin. Optical images and corresponding simulation results in Fig. 1F show good agreement. The strain contour in the upper layer of Fig. 1F indicates that the maximum principal strains in most locations are below 1%. Large strains (~2.5%), still below the fracture threshold of the PI/Cu/PI system, occur only in certain regions of the interconnects, highlighted by the red dashed box. These strains can be reduced by increasing the interconnect lengths or the thickness of the core encapsulation material. Figure 1G shows a magnified view of this region, where the influence of two adjacent components leads to a local region of strain concentration. The calculated strains are lower than the fracture strain of copper (~5%), indicating a total biaxial stretchability of the device that is larger than 25%. Consistent with previous studies, stretching is mainly absorbed by deformations of the serpentine interconnects (5). Assuming a yield strain of ~0.3% in the copper, the elastic stretchability in both directions is ~4.6%. Results of 3D-FEA for an otherwise identical system, but without any of the device components, appear in fig. S7B. The deformation patterns also show good agreement with experiment when biaxially stretched by 25%, with a similar strain concentration effect observed in the same region. Stress-strain measurements along the device length (fig. S8) reveal effective moduli of ~32.1 kPa (with chips) and ~8.68 kPa (without chips), which are much smaller than those of the epidermis (~100 to 200 kPa), and confirm the stretchability of up to 25% strain. The layouts can be adjusted to meet application requirements.

The mechano-acoustic response captured without analog filters using a vibration simulator (3B Scientific) shows the expected frequency bandwidth (fig. S9). For use on the body, the depth of the source varies according to the location and the associated organ. As examples, the larynx is ~5 mm below the surface of the neck, and the valves of the heart are ~30 mm away from the surface of the chest. In vitro experiments use fresh pieces of chicken breast, with thicknesses between 1 and 30 mm, placed between the sensor and the vibration simulator to simulate the effects of viscoelastic losses. Results indicate that the spectral power of the measured response exhibits a power law behavior with respect to signal frequency and an asymptotic decay with respect to tissue thickness (Fig. 1H), as expected from the acoustic attenuation (41, 42) by absorption and scattering in viscoelastic materials and at the materials interfaces (43–45). The average decrease in spectral power between frequencies in the measurement range is 51% on 1-mm-thick tissue and 83% on 30-mm-thick tissue.

Partly because of this attenuation and partly because of the small amplitudes at the biological source, mechano-acoustic signals at the surface of the skin are relatively weak, and increasingly so with increasing frequency (42). Therefore, measurements must account for effects in mechanical loading and mechanical impedance matching between the devices and the skin. The mass of the sensor system is an important characteristic in this regard. Increasing the device mass increases the mechanical loading at the skin interface, thereby decreasing the mechano-acoustic motions. In vitro experiments to demonstrate these effects involve experiments such as those described above but with the sensor placed in an acrylic box (19 mm × 42.5 mm × 55 mm, 9.36 g) with different added test masses (Fig. 2). Results in Fig. 2 (A to C) and fig. S10 show a general trend of decreasing spectral power with tissue thickness and mass for all frequencies within the accelerometer bandwidth. The additional mass in this case has negligible effect. A simple mechanical model consisting of a mass, a spring, and a damping source (note S1) can capture the overall behaviors (Fig. 2D). The computed results at three different frequencies (50, 100, and 200 Hz) indicate that the response decreases with increasing mass, tissue thickness, and frequency (fig. S10). In vivo studies of speech recognition confirm that increasing mass leads to decreasing signal (Fig. 2E).

Fig. 2. Summary of the experimental and computational studies of the effects of device mass, modulus, tissue thickness, and signal frequency on measured mechano-acoustic responses.

Experimentally measured spectral power and simulation results associated with the mechano-acoustic response of a device mounted in an acrylic box placed on a tissue sample on a vibrational source at frequencies of 50 Hz (A), 100 Hz (B), and 200 Hz (C). (D) Comparison of measured (experiment) and computed (analytical) dependence of spectral power on mass. (E) Measured maximum signal amplitude recorded with a device mounted on the neck as the subject said the word “left,” as a function of the mass of the device. (F) Amplitude measured using a device in a rigid box and on a thin substrate of Ecoflex, as a function of spatial location of the added mass.

In addition to overall device mass, the distribution of this mass and the overall mechanics of the structure are important. In particular, in a soft, low-modulus device platform, only the mass of the mechano-acoustic sensor chip is important, whereas in a rigid platform, the overall mass limits the performance. Results in Fig. 2F verify that in a low-modulus device platform, added mass is only significant when located at the position of the sensor, and that added mass at different locations has similar loading effects for the case of a rigid platform. FEA of a similar system (note S2) is consistent with the experimental data (fig. S11). These findings suggest that low-mass and low-modulus characteristics are critically important. An additional implication is that, in the physical forms reported here, batteries, radios, and other components of interest for future embodiments can be included in the platform without adversely affecting the measurement sensitivity.

Seismocardiography measurement

Seismocardiography (SCG) captures the thoracic vibrations from atroventricular contractions and blood ejection into the vascular tree on the skin of the sternum (22, 29, 46). Each beat cycle produces a characteristic SCG complex as a quasi-periodic waveform with frequency components that reflect contraction of the heart muscle and associated ejection of blood. Figure 3A shows the mechano-acoustic device and its pair of conformal capacitive electrodes laminated onto the sternum for simultaneuous measurements of SCG and ECG.

Fig. 3. Application of mechano-acoustic–electrophysiological sensing with an epidermal device in diagnosing cardiovascular health status.

(A) Image of an epidermal device mounted on the chest. (B) Schematic diagram of cardiac cycle: (left) artrial and ventricular diastole, (middle) artrial systole and ventricular diastole, and (right) ventricular systole and atrial diastole. (C) Plot of ECG (top) and heart sound (bottom) signals measured simultaneously. A.U., arbitrary units. (D) Magnified view of ECG (top) and heart sound (bottom) signals measured in (C). MC, mitral valve closure; AO, aortic valve opening; RE, rapid ventricular ejection; AC, aortic valve closure; MO, mitral valve opening; RF, rapid ventricular filling. (E) Comparison of heart sound signals measured using a commercial electronic stethoscope and the reported device. (F) Schematic illustration of the measurement site: A, aortic; P, pulmonary; T, tricuspid; M, mitral. Representative measurement from a 78-year-old female patient with diagnosed mild pulmonary and tricuspid regurgitation at the aortic (G), tricuspid (H), pulmonary (I), and mitral (J) sites.

A single cardiac cycle includes systole (contraction of heart muscle) and diastole (relaxation of heart muscle) motions of the atria and the ventricles, as illustrated in Fig. 3B. These motions involve electrical signals followed by mechanical coupling and a sequence of mechano-acoustic signatures as the heart chambers contract and the valves close. These electrophysiological and mechanical data form the basis of ECGs and cardiac auscultations, respectively. Figure 3C shows ECG and SCG signals measured simultaneously from a healthy male subject (age, 22). Magnified views of a single cardiac cycle (Fig. 3D) highlight all the key features of these two waveforms. This information is useful in the assessment of systolic and diastolic ventricular function. For example, the electromechanical activation time (the time interval from the onset of the QRS to the point of peak intensity of S₁) corresponds to the time required for the left ventricle (LV) to achieve sufficient pressure to force the mitral valve to close. Its prolongation indicates systolic heart failure. Reductions in the interval between S₁ and S₂ (termed left ventricular systolic time) are a sign of LV dysfunction. Overall, the data from the epidermal mechano-acoustic sensors reported here have a quality comparable to that of the data obtained using a commercial electronic stethoscope (JABES Electronic Stethoscope, GS Technology Co.), where S₁ and S₂ are delineated (Fig. 2E). This device can also measure pressure pulse waves associated with arterial blood flow. A sensor placed on the carotid artery at the neck (fig. S12) can capture these data, along with ECG signals.

For subjects with cardiovascular pathologies, murmurs are often present in addition to signatures associated with S₁ and S₂. The holosystolic murmurs of the mitral and tricuspid valve regurgitation that occur during systole have acoustic signatures of characteristic constant intensity and high frequency. In constrast, diastolic murmurs are often detected in patients with aortic or pulmonic valve regurgitation (43, 44). Clinical validation of the device operation in this context involves recording cardiac mechano-acoustic responses with ECG signals from eight patient volunteers diagnosed with cardiac valvular stenosis or regurgitation. Figure 3G shows the auscultation mounting sites that yield optimal results for the aortic, pulmonary, tricuspid, and mitral heart valves. An elderly female patient (age, 78) with diagnosed mild tricuspid and pulmonary regurgitation via echocardiography (fig. S13) manifests a short, constant intensity murmur at tricuspid and pulmonary sites in systole and diastole, respectively, as indicated by the arrows in Fig. 3 (G to J). Measurement from the aortic site shows no signs of stenosis or regurgitation. Signal from the mitral site is weak, likely because of nonoptimal sensor placement. Figure S14 shows a female patient (age, 82) with severe regurgitation of the tricuspid and mitral valve and an irregular beat rate. Studies on other related patients yield similar data.

Acoustic analysis of VAD

Additional biomedical applications include monitoring of mechanical circulatory support devices, such as those that augment dysfunctional ventricular pump function and serve as important temporary or permanent alternatives to heart transplantation (47). The latest continuous-flow left ventricular assist devices (LVADs) offer improved durability and hemodynamic restoration, though with the limitation of adverse events, including a loss of pump function due to pump thrombosis or other mechanical failure (48). Previous work in the context of the first failure mode shows that the formation of blood clots on the rotor leads to changes in the sounds of the pump (47–51). These changes can be difficult or impossible to discern using stethoscopes or unaided human hearing, particularly for early-stage thrombosis. The mechano-acoustic sensors reported here enable a surface-mounted mode to monitor changes in vibration signatures in the LVAD pump. Studies reported here focus on an in vitro model with a commercial LVAD (HeartMate II, Thoratec Corporation) and continuous flow to detect changes in acoustic signal correlating to variation in pump speed, circulating fluid, and thrombus embolization.

Figure S15A shows a system consisting of a circulatory closed loop that involves medical grade tubing (Tygon) connected to HeartMate II, with valves to assist in the removal of air bubbles and to allow the introduction of blood clots. The device laminates conformally onto the metal housing of the pump impeller and brushless dc motor to provide direct measurements of vibration (Fig. 4A). The spectral power of the signal collected for a short time (30 s) during operation of HeartMate II at 8400 rpm appears in Fig. 4B. The bottom panel in Fig. 4B shows characteristic signatures at 139.7 Hz (peak A) and 166 Hz (peak B) and its second harmonic at 332 Hz. Increasing the pump speed from 8400 to 9400 rpm leads to decreases in the frequency of peak A from 139.7 to 156.2 Hz (Fig. 4C), whereas peak B remains unchanged (Fig. 4D). These data suggest that peak A can serve as a reliable indication of the pump speed. Replacing water with glycerol, a fluid medium with a viscosity similar to that of blood serum but higher than that of water, leads to no significant change in the acoustic signature (Fig. 4E). This result suggests that the pump rotation dominates collected acoustic signatures, and that they are insensitive to changes in circulating fluid viscosity.

Fig. 4. Application of mechano-acoustic sensing with an epidermal device in diagnosing VAD operation.

(A) Image of the experimental circulation loop with the device mounted on the VAD (HeartMate II). (B) Fast Fourier transform (FFT) of the vibration response (top) and spectrogram (bottom) associated with the operation of the VAD at 8400 rpm in a water circulation loop. (C) FFT spectral power of the vibration response for operating frequencies between 8400 and 9400 rpm. Distinctive changes with VAD speed occur only on the peak around 150 Hz. (D) Comparison of vibrational responses in a circulation loop with water and with glycerol at 8400 rpm (top) and 9400 rpm (bottom). (E) Demonstration of changes in acoustic signature associated with circulation of a blood clot (500 μl) in the glycerol loop during stages of initial injection of the blood clot, first few circulation passes without decomposition, subsequent complete decomposition, and circulation of tiny blood clots.

Introducing a blood clot (500 μl) (fig. S15B) prepared from bovine whole blood through the air valve at the inflow of the HeartMate II during glycerol operation at 9400 rpm serves to simulate thrombosis and embolization. Immediately after injection, the blood clot travels through the LVAD and exits the outflow tubing with minimal distortion. The associated widening of peak A suggests that clot interaction with the pump impeller produces additional frequencies (Fig. 4F, top panel). While the clot travels through the remainder of the circulation loop, the pump remains undisturbed, and the vibration signature returns to its initial, that is, two-peak state, but with peak A at a higher amplitude than peak B (Fig. 4F, second panel), possibly as a result of tiny blood clots attached to the pump impeller. After several passages, the clot dissipates completely into microscopic fragments invisible to the unaided eye. This process creates another strong group of frequencies around peak A (Fig. 4F, third panel). Finally, the vibration signature restores to the circulation state, with peak A again at higher amplitude, confirming previous observation (Fig. 4F, bottom panel). These results serve as a reference that validates the possible use of an accelerometer to capture acoustic signatures in the LVAD for pump thrombosis detection and monitoring.

Speech recognition and human-machine interface

Two features of epidermal mechano-acoustic devices (Fig. 5A), specifically their use of multiple sensors in a single device platform and their compatibility with direct placement on curvilinear regions of the skin, enable their application in speech capture and recognition. Implications range from improved communication capabilities for individuals with speech impairments (52) to the design of voice-activated human-machine interfaces (53–57).

Fig. 5. Application of mechano-acoustic sensing with an epidermal device for speech recognition.

(A) Image of an epidermal device mounted on the vocal cords. (B) Plot of EMG (top) and vocal vibrational (bottom) signals measured simultaneously from the neck. (C) Comparison of speech recorded with the reported device (top) and with an external microphone (bottom). The left and right columns represent recordings made under quiet and noisy conditions, respectively. (D) Confusion matrix that describes the performance of the speech classification. (E) Demonstration of speech recognition and classification in a Pac-Man game with left, right, up, and down instruction.

First, with appropriate placement, epidermal mechano-acoustic devices can simultaneously capture both electromyogram (EMG) signals from articulator muscle groups and acoustic vibrations from the vocal cords. Figure 5B shows EMG signals (top) and mechano-acoustic vibrations (bottom) recorded while speaking “left,” “right,” “up,” and “down.” The spectrogram (Fig. 5C, top left) highlights the unique time-frequency characteristics of each of the four words. The low-frequency components of the nasal consonant in “down” are particularly prominent. Previous research suggests that the fusion of multiple sensors can improve speech recognition (58–60). A specific suggestion is that throat EMG can enhance traditional speech recognition techniques (57, 61), although simultaneous recording of EMG and acoustics in a single device has not been demonstrated. An earlier study (52) showed that fusion of acoustic data with EMG signals measured using separate devices improved word recognition accuracy in a small group of patients with dysarthria.

Second, the intimate contact between the sensors and the skin renders their operation almost unaffected by ambient acoustic noise. Figure 5C compares spectrograms of speech (“left,” “right,” “up,” and “down”) recorded by an epidermal sensor and by a standard microphone (iPhone, Apple Inc.; see fig. S16 for time domain data), both attached to a subject’s throat. The noise source (radio speakers) is 2.5 m away from the subject. In a quiet environment [~30 dB (62)], both the epidermal sensor and the microphone show similar responses. On the other hand, a noisy environment [~60 dB (62)] significantly degrades the quality of recording from the microphone but does not affect the epidermal sensor. This feature could allow the epidermal acoustic sensor to be used for communication in loud environments by first responders (63), ground controllers, or security agents.

A simple isolated word detection system, used in real time to play a Pac-Man game, demonstrates the potential of the epidermal acoustic sensor for human-machine interfaces. Figure S17 shows the signal flow for a control system for the isolated word detection system. Implementation begins with a training phase based on four commands: “left,” “right,” “up,” and “down,” Preprocessing involves implementation of noise reduction techniques shown in fig. S18, which does not alter classification accuracy. Classification occurs in real time using linear discriminant analysis (LDA). A confusion matrix (Fig. 5D) summarizes the accuracy of this classifier, in which the columns represent the predicted word and the rows represent the targeted word. In this example, the recognition accuracy is 90%. Further improvements are possible through additional training, different classification methods (64), and a wider passband on the sensor. A video of a user playing a Pac-Man game appears in video S3; these same speech recognition strategies can be applied to almost any type of human-machine interfaces, such as drone and prosthesis control (9, 11, 12). Possibilities in digital authentication appear in fig. S19.

DISCUSSION

The class of device reported here exploits a thin, lightweight, low-modulus, and skin-compatible architecture to enable mechano-acoustic sensing. These physical attributes, although important for wearability and comfort in previous types of “epidermal” technologies, represent critical enabling features for such systems because they allow high-fidelity mechanical coupling across the skin/device interface. The results create many opportunities in precision recording of sounds and vibratory signatures not only of natural body processes but also of the operation of mechanical implants, such as LVADs. Bench studies and simulation results highlight the fundamental physics associated with this type of sensing. A range of uses with human subjects—in contexts spanning the characterization of heart murmurs in patients known to have either regurgitation or stenosis at defined valvular listening areas (for example, tricuspid or aortic) to machine interfaces in real-time control of computer gaming systems—foreshadows some of the broad opportunities of these concepts. Other potential clinical applications include heart rate variability analysis, beat-to-beat assessment of the pre-ejection period, and left ventricular ejection time. Body sounds, such as snoring, respiration, and gastrointestinal tract movement, are also of some interest. In many cases, fully wireless capabilities in data transfer, on-board data storage/processing, and integrated power supply will be necessary, particularly for applications that require continuous, untethered operation. Preliminary data (fig. S20) indicate that the most advanced commercial skin-mounted devices with these features (BioStampRC, MC10 Inc.) offer areal mass densities and low-modulus designs that are sufficient to allow similar levels of mechano-acoustic sensing, as well as multifunctional operation in EP recording. Further optimization of the mechanics and mass distributions associated with this platform, using the design rules outlined here, and further exploration of its use in clinical applications to establish a catalog of pathological functions and conditions represent promising directions for future research.

MATERIALS AND METHODS

Fabrication of epidermal mechano-acoustic device

The fabrication process involves three parts: (i) patterning of the circuit interconnects; (ii) transfer-printing and chip-bonding onto a soft, core/shell substrate; and (iii) covering the top surface with a similar soft core/shell structure. Fabrication of the interconnects began with a commercial laminate (MicroThin, Oak-Mitsui Inc.) that contains a copper carrier film (17.5 μm) and a thin copper foil (3 μm) separated by a release layer. Spin-coating and thermal curing formed a film of PI (1.2 μm; PI 2545, HD MicroSystems) on the side with the thin copper foil (3 μm). Peeling this PI-coated layer from the thick copper layer allowed its attachment onto a glass slide coated with poly(dimethylsiloxane) (sylgard 184, Dow Corning). The following describes the fabrication process in more detail: (i) Photolithography and metal etching defined a pattern of interconnects in the copper. Another spin-coating and curing process yielded a uniform layer of PI on the resulting pattern. Photolithography and reactive ion etching (RIE, Nordson MARCH) defined the top and bottom layers of PI in geometries matching those of the interconnects. (ii) A piece of water-soluble tape (Aquasol) enabled the transfer of these encapsulated interconnects onto a trilayer film supported by a silicon wafer, prepared by spin-coating (4000 rpm) and curing a thin layer of an ultrasoft silicone (Silbione, RT Gel 4717 A/B, Bluestar Silicones), followed by a layer of slightly stiffer silicone (Ecoflex, 00-30, Smooth-On) at 1000 rpm and, finally, another layer of ultrasoft silicone at 1000 rpm. This trilayer defined the skin-adhesive interface and the core/shell substrate. Removal of the tape by immersion in water exposed the interconnects to allow bonding of the device components onto designated pads using solder paste (Indalloy 290, Indium Corporation) and a heat gun at ~165°C. (iii) Encapsulation began with manual placement of cured, individual pieces of silicone onto the pads that connect to the ECG electrodes and to those that interface to the ACF cable. Spin-coating (1000 rpm) and curing a layer of Silbione followed by a layer of Ecoflex at 1000 rpm defined the core/shell superstrate. Removal of the silicone pieces completed the fabrication process. Attachment of the ACF cable and ECG electrodes occurred just before mounting the device on the skin.

Device characterization

Adhesion strength tests.

Standard vertical peel measurements defined the adhesion strength between test samples and the skin on the flexor muscle. Each sample (2.5 cm × 2.5 cm, 1 mm thick) was prepared by mixing monomer and curing agent components for Silbione and Ecoflex and then thermally curing the materials. The bilayer structure consisted of a 500-μm-thick layer of Ecoflex on a glass substrate, with a 500-μm-thick layer of Silbione on top. The test substrate was placed on the skin, and a corner was attached to the hook of a force gauge at 90° (Mark-10). The reported strength of adhesion corresponds to the measured force divided by the substrate area.

Water vapor transmission loss test.

Measurements of water vapor loss follow the standards in ASTM E96-95 (www.astm.org/Standards/E96). Films of Silbione were prepared by spin-coating at 250, 500, 1000, and 2000 rpm on the wafer substrate. Flasks (125 ml) were filled with dry cobalt chloride (Drierite) at equal weight and sealed with the Silbione/Ecoflex films using plastic bands (fig. S6). Changes in weight of each flask were recorded daily for 6 days at room temperature (23°C) and 50% humidity. The water vapor transmission rates are based on these measurements.

Cell viability assay.

MEFs were obtained from K. Kilian’s laboratory. MEFs were isolated from embryos 13 days after coitus with 0.05% Trypsin (Gibco). Cells were cultured in high-glucose Dulbecco’s Modified Eagle’s Medium (DMEM) (4.5 g/ml) supplemented with 10% fetal bovine serum (Sigma) and 1% penicillin/streptomycin. The medium was changed every 3 days and passaged at 80% confluency.

Device samples were sterilized by autoclaving the samples at 121°C for 60 min, followed by exposing them to ultraviolet irradiation for 30 min, and finally by washing them in phosphate-buffered saline (PBS). The device surface was exposed to laminin (25 μg/ml; Sigma L2020) in PBS for 30 min and then transferred to a six-well plate. MEFs were seeded on samples at an initial concentration of 20,000 cells/ml and cultured for 5 days. After 1, 3, and 5 days in culture, devices were incubated with Hoechst 33342 (1 μg/ml), calcein AM (2 μM), and ethidium homodimer-1 (4 μM) in PBS solution for 20 min. Samples were mounted onto glass slides and imaged with an IN Cell Analyzer 2000 (GE). Immunofluorescent images were analyzed using ImageJ software. Measurements of cell viability correspond to the proportion of live cells (green) over all cells (green + red). Cells grown on tissue culture plastic in standard DMEM and in DMEM with 10% dimethyl sulfoxide served as positive and negative controls, respectively.

Vibration response.

Tests involved attaching the devices, without analog low- and high-pass filters, to a flat aluminum stand mounted on a vibration generator (3B Scientific). The vibration was generated by a 1-cm pole connected to the diaphragm of a loudspeaker (50 W, 100 mm, 8 ohm; SR 1010, Somogyi) fitted inside a plastic housing. The square wave output of a function generator (FG100, 3B Scientific) provided a 3-V output to the loudspeaker at discrete frequencies of 1, 5, 10, 50, 100, 250, and 500 Hz. A commercial system (PowerLab, ADInstruments) enabled data acquisition, without filters, at a sampling rate of 1 kHz.

Measurements of the influence of tissue thickness used fresh chicken breast (Miller Amish Poultry) sliced into 2 cm × 2 cm pieces at thicknesses of 1, 5, 10, and 30 mm. When inserted between the sensor and the vibration stand (4 cm × 4 cm), the moist surfaces of the tissue ensured sufficient adhesion to prevent relative movement during vibration, using square waves with an amplitude of 3.7 V and frequencies of 50, 100, 200, 300, 400, and 480 Hz.

The effect of mass and tissue thickness was determined using the same experimental setups described above. The sensor was taped firmly to the bottom center of an acrylic box (19 mm × 42.5 mm × 55 mm, 9.36 g). Medical tape (silicone tape, 3M Medical) wrapped onto the vibration stand stabilized the box on the chicken tissue. Screw nuts (3/8 inch, 1.38 g) were used as elements for added mass, fixed firmly to the top cover of the acrylic box by a double-sided adhesive.

Speech sensing was evaluated using a sensor placed in the acrylic box as described above. Measurements involved acoustic vibrations associated with a subject saying “left,” with different added masses. The acrylic box was attached to the subject’s throat via a double-sided adhesive between the skin and the box interface and medical tape on top of the box. To study the effect of mass location, a set of four mass elements was connected in a column and attached to the middle, upper right, upper left, lower right, and lower left locations of the bottom of the box.

Mechanical modeling and FEA

3D-FEA simulations based on commercial software packages (Abaqus 6.14, Dassault Systemes) guided the optimization of the mechanics of the system. The elastomers were modeled by eight-node, 3D hexahedron elements (C3D8R). The electronic chips, serpentine interconnects, and PI layers were modeled by four-node shell elements (S4R). Displacement boundary conditions applied to the substrate allowed the system to be stretched. The Young’s modulus (E) and Poisson’s ratio (ν) of the materials were as follows: for Silbione, E_Silbione = 5 kPa and ν_Silbione = 0.48; for Ecoflex, E_Ecoflex = 60 kPa and ν_Ecoflex = 0.48; for PI, E_PI = 2.5 GPa and ν_PI = 0.34; and for copper, E_Cu = 119 GPa and ν_Cu = 0.35.

FEA using Abaqus also determined the effects of frequency, mass, and tissue thickness on the mechano-acoustic signal. Here, C3D8R were used to model the tissue, the mass objects, and the accelerator, all under a sinusoidal force input. The tissue was modeled as a viscoelastic solid, with a Young’s modulus of 0.18 MPa (65) and a Prony series function with constants g_i = k_i = 0.91001 s and τ_i = 0.9899 s. After frequency analysis of the whole system, modal dynamic was chosen as the analysis method to simulate system vibration.

Demonstrations of seismocardiography

Clinical tests at Camp Lowell Cardiology involved eight elderly patients as volunteers, all providing informed consent. Optimal sensor placement sites at traditional aortic, pulmonary, tricuspid, and mitral locations were determined by ultrasound probes, with verification of heart murmurs by echocardiogram (GE Healthcare). A three-lead setup enabled simultaneous recording of ECG using the same device platform. PowerLab system (8/35, ADInstruments) with BioAmp modules served as the hardware for data acquisition and analysis. During measurement, the subject was asked to “stop breathing” for 3 s and then to “breathe normally” after a verbal countdown to eliminate the respiratory effect on the baseline and amplitude of the SCG data. Passing the output of the accelerometer through a 20-Hz low-pass digital filter followed by an analog-to-digital converter in the PowerLab system yielded processed data at a sampling rate of 1 kHz. A band-pass digital filter with a low cutoff and a high cutoff frequency of 1 and 30 Hz, respectively, was used with the ECG signal. All vibration signals were converted from output voltage to “mechano-acoustic response (arbitrary units).”

Measurements from LVADs

The test platform consisted of a closed loop created by connecting a commercial LVAD (HeartMate II, Thoratec Inc.) and its respective driver by ~1 m of medical grade tubing (Tygon) at the inlet and outlet, with syringe ports at each location for the introduction of water, without air bubbles. Tape secured the device to the housing of the LVAD. Baseline studies involved measurements of vibration during the operation of the LVAD at various speeds between 8400 and 9400 rpm, with 200-rpm increments. Additional similar experiments used 30% (v/v) glycerol in water.

Studies on the effects of VAD thrombosis used fresh blood clots formed via addition of calcium chloride added to 10% (v/v) acid citrate dextrose in fresh bovine whole blood, with the aim of reaching a concentration of 25 mM. Blood clots formed spontaneously during storage overnight at room temperature. Clots with weights of ~250 mg were introduced into the closed loop before activating the LVAD. The sensor response was recorded during circulation of a single clot while operating the LVAD at 9400 rpm. Additional similar experiments used 30% (v/v) glycerol in water.

Algorithms for the classification of data related to speech

Real-time classification of speech signals relied on a simple four-class (left, right, up, and down) isolated word recognition system with a “null” state. Before classification, the data were preprocessed to reduce ambient noise using spectral subtraction (66) and then digitally filtered, using an eighth-order Butterworth filter, from 30 to 1000 Hz. The resulting data were defined as null unless the root mean square value surpassed a threshold. Analyzing the energy of the signal in a sliding 50-ms window enabled determination of the exact onset and offset of the word. Fourier transformation with a 100-ms time window and a 70-ms overlap defined the time-frequency estimate of the data during the duration of the word. The results were averaged and reduced in dimensionality using principal components analysis to form a feature vector. This feature vector was finally classified using LDA. Training involved 20 trials from each class, with 90% accuracy (Fig. 5D). The resulting classifier enabled real-time operation in a simple video game (https://pypi.python.org/pypi/pacman-game/).

Experiments on human subjects

All experiments on human skin were conducted under approval from the Institutional Review Board of the University of Illinois at Urbana-Champaign (protocol number 13229), and volunteer subjects gave informed consent.

Statistical and data analysis

Spectra shown in Figs. 2, 4, and 5 resulted from an FFT algorithm with 1024 window size, Hann (cosine bell) window type, and 50% overlap. For data displayed in spectral power mode, the signal corresponded to an average of three FFTs in frequency domain. All data processing was performed using LabChart 8 (ADInstruments) and OriginLab 2016 (Origin).

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/2/11/e1601185/DC1

note S1. Analytical model for mass effect on acceleration.

note S2. Effect of low-modulus device substrate.

fig. S1. Device design and circuit layouts.

fig. S2. Computed x-ray tomography images of the internal structures of the accelerometer chip.

fig. S3. Schematic illustration of capacitive ECG electrodes and demonstrations of their reusability.

fig. S4. Adhesion strength of Silbione to the skin and dependence of its thickness on spin speed.

fig. S5. Measurements of water vapor transmission loss.

fig. S6. Cell viability assay and cytotoxicity test.

fig. S7. Mechanical simulation of the circuit interconnects during biaxial stretching.

fig. S8. Stress-strain response of the device.

fig. S9. Vibration response of the accelerometer chip without analog filters.

fig. S10. Comparison of experimental and simulation results on the effect of mass, tissue thickness, and signal frequency on measurement response.

fig. S11. Schematic illustration and measurement results of the vibration model to capture the effects of device modulus.

fig. S12. Application of an epidermal mechano-acoustic–electrophysiological device on the neck.

fig. S13. Echocardiogram characterization results on a patient with tricuspid and pulmonary regurgitation.

fig. S14. Acoustic signals from aortic, pulmonary, tricuspid, and mitral sites of a patient with irregular heartbeat.

fig. S15. Experiment on LVAD pump thrombosis.

fig. S16. Data captured using a reported device and a commercial microphone in quiet and noisy environments.

fig. S16. Data captured using a reported device and a commercial microphone in quiet and noisy environments.

fig. S17. Process loop for a speech-based human-machine interface.

fig. S18. Demonstration of noise reduction in time domain speech data.

fig. S19. Authentication application.

fig. S20. Wireless sensing of BioStamp.

movie S1. Movie of speech recording in a quiet environment.

movie S2. Movie of speech recording in a noisy environment.

movie S3. Movie of speech recognition and voice control of a Pac-Man game with real-time machine learning and signal classification.