Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 15.
Published in final edited form as: Biosens Bioelectron. 2017 Jan 25;91:796–803. doi: 10.1016/j.bios.2017.01.044

Soft, Conformal Bioelectronics for a Wireless Human-Wheelchair Interface

Saswat Mishra a, James J S Norton b, Yongkuk Lee a, Dong Sup Lee a, Nicolas Agee a, Yanfei Chen c, Youngjae Chun c,d, Woon-Hong Yeo a,e,*
PMCID: PMC5323068  NIHMSID: NIHMS847684  PMID: 28152485

Abstract

There are more than 3 million people in the world whose mobility relies on wheelchairs. Recent advancement on engineering technology enables more intuitive, easy-to-use rehabilitation systems. A human-machine interface that uses non-invasive, electrophysiological signals can allow a systematic interaction between human and devices; for example, eye movement-based wheelchair control. However, the existing machine-interface platforms are obtrusive, uncomfortable, and often cause skin irritations as they require a metal electrode affixed to the skin with a gel and acrylic pad. Here, we introduce a bioelectronic system that makes dry, conformal contact to the skin. The mechanically comfortable sensor records high-fidelity electrooculograms, comparable to the conventional gel electrode. Quantitative signal analysis and infrared thermographs show the advantages of the soft biosensor for an ergonomic human-machine interface. A classification algorithm with an optimized set of features shows the accuracy of 94% with five eye movements. A Bluetooth-enabled system incorporating the soft bioelectronics demonstrates a precise, hands-free control of a robotic wheelchair via electrooculograms.

Keywords: Soft electrode, stretchable electronics, conformal contact, fractal structure, electrooculograms, human-wheelchair interface

1. Introduction

A recent study shows that more than 10% of US population has some type of motor disability (Brault 2012); 3.6 million people rely on wheelchairs and 6.7 million people have difficulty grasping low weight objects. For those with frail grip strength, an alternative wheelchair interface is needed, which can be helped by human-machine interfaces (HMI). Multiple studies have demonstrated HMI applications (Aziz et al. 2014; Barea et al. 2002a; Belkacem et al. 2015; Keegan et al. 2009; Wu et al. 2013) and activity tracking (Bulling et al. 2009; Mala and Latha 2014) via non-invasive electrooculograms (EOG). EOG signals are derived from the potential that is created by the eye acting as a dipole through the positively charged cornea and negatively charged retina. Typical amplitudes of signals range from 50 to 3500 µV and are caused by the amount light incident on the retina (Barea et al. 2002a; Barea et al. 2002b).

The most important factor for successful HMI is to have high accuracy in signal classifications that are utilized for a machine control. Algorithms to enable multi-class differentiation use various features for maximized accuracy of subject-dependent signals. The continuous wavelet transform, Haar wavelet, is one of the most widely used features for EOG recognition (Aziz et al. 2014; Belkacem et al. 2015; Bulling et al. 2009; Mala and Latha 2014). For example, this feature was included in a classification method (Bulling et al. 2011) for multiple task recognitions such as copy, read, write, video, browse, and null.

A recent use of EOG (Belkacem et al. 2015) shows 90% accuracy of eye movements using both EOG and electroencephalograms for computer interfacing and 80% accuracy for manipulation of a robotic arm (Hortal et al. 2015). Nevertheless, the critical issue comes from the use of gels and sticky tapes, which cause skin irritation and allergic reactions. To address the issues, our prior works introduced a “skin-like” electronic system, which offered a non-invasive, high quality recording of biopotentials on the skin (Jeong et al. 2013; Jeong et al. 2014; Lee and Yeo 2015; Xu et al. 2015; Yeo et al. 2013a; Yeo et al. 2013b).

In this work, we focus on soft bioelectronics and wireless wheelchair interface (Fig. 1). This ergonomic HMI system incorporates conformal biosensors and a Bluetooth unit interfacing the bioelectrode with a robotic wheelchair. The “skin-like” bioelectrode enables a high-fidelity recording of EOG, supported by the direct comparison with conventional rigid electrodes. This soft bioelectronic system demonstrates mechanical compliance in both stretchability (30%) and bendability (up to 180°). A newly designed classification algorithm with an optimized set of features shows the accuracy of 94% with five different eye movements, which is used to demonstrate a hands-free control of a robotic wheelchair via EOG.

Figure 1. Soft bioelectronics for a tether-free wheelchair control via electrooculograms (EOG).

Figure 1

Open in a new tab

(a) Illustration of a subject wearing skin-mounted, fractal electrodes to control the robotic wheelchair via EOG signals (inset graph) and a tablet-operated wireless interface. (b) Photo of the exploded view of the soft electrode mounted around the eye. (c) The magnified view of (b) representing a conformal contact of the electrode on the skin.

2. Materials and Methods

2.1. Finite element analysis

Three-dimensional finite element analysis using a software (ABAQUS, Dassault Systems, Waltham, MA) was conducted to investigate mechanical behaviors of an electrode upon skin deformations: bending and stretching (Chen et al. 2016; Ma et al. 2016). The soft electrode used in EOG measurement was composed of three layers including 300 nm-thick Au, 1 µm-thick polyimide (PI; HD MicroSystems, Parlin, NJ), and 5 µm-thick silicone elastomer (Smooth-On, Macungie, PA; details in Supplementary Figure S1). Young’s modulus (Ε) and Poisson ratio (ν) of materials used in the modeling are Εsilicone = 69 kPa (Yeo et al. 2013b) and νsilicone = 0.49 (Zhang et al. 2014) for a silicone elastomer; ΕAu = 78 GPa and νAu = 0.44 for Au (Zhang et al. 2014); ΕPI = 2.5 GPa and νPI = 0.34 for polyimide (Zhang et al. 2014). The maximum principal strain on Au was mainly investigated to determine the mechanical stability upon multi-modal deformations. Note that Au membrane was used as an electrode, PI was a supporting layer to hold the membrane, and silicone elastomer was a substrate to mount the electrode.

2.2. Fabrication of a soft bioelectrode

Fabrication of a soft electrode was based on the combination of a conventional photolithography (details in Supplementary Figure S2) with material transfer printing (details in Supplementary Note 1 and Supplementary Figure S3). A glass slide (Ted Pella, Redding, CA) was prepared by a cleaning process using acetone, IPA, and rinse with DI water. Afterwards, a primer (MicroChem Corp., Westborough, MA) was coated on the substrate for enhanced adhesion. Multi-layer coating and curing of PMMA and PI was followed. The electrode patterning started with Au deposition on the PI and photolithography defined patterns. A second layer of PI was encapsulated while exposing the skin contacted patterns and wire connection pads. The electrode was retrieved by a water soluble tape after dissolving the sacrificial PMMA in solvent. Afterwards, it was transfer printed on an ultrathin elastomeric membrane (5 µm in thickness) on a polyvinyl alcohol film. Lastly, a flexible film was attached onto contact pads for connection with a wireless system (BioRadio; Great Lakes NeuroTechnologies, Cleveland, OH).

2.3. Measurement of mechanical behavior

Biaxial stretching of the fabricated electrode was conducted on a programmable stretcher (details in Supplementary Figure S4). The stretcher held a sample with four clamps. Precise control of the travel distance by an Arduino and stepper motors determined the applied strains. Ultrathin copper wires (100 µm in diameter) were connected to the electrode for recording of resistance change by a digital multi-meter (Model 2100, Keithley, Cleveland, OH). In a bending test, a sample was placed on a flexible plastic film, which allowed bending from 0 to 180° with the radius of curvature of 500 µm. Similar to the stretching test, electrical resistance was monitored according to the degree of bending. We utilized a portable microscope (Dino-Lite, Torrance, CA) to inspect any localized, microscale fracture.

2.4. In vivo measurement of EOG

Three healthy males, aged from 20 to 25, participated in EOG measurement based on the approved protocol (HM20001454) at Virginia Commonwealth University. All subjects with 20/20 vision were given an explanation for the study and signed a consent form. For EOG recording, two sets of electrodes were placed on the skin around the eyes. For horizontal eye movements, each electrode was positioned 1 cm away from the outer canthus of each eye. For vertical eye movement, left eye was selected; one was mounted 1 cm away from the eyebrow and the other was positioned 1 cm away from lower eyelid. A common ground was positioned on the forehead.

2.5. Data acquisition

Fractal bioelectrodes were configured to work with a commercial wireless unit. A two channel setup was configured with a sampling rate of 500 Hz. We made a quantitative signal comparison between commercial Ag/AgCl electrodes (MVAP II, Newbury Park, CA) and fractal electrodes. The measured EOG signals were wirelessly transmitted to a receiver connected to a laptop or tablet interface with a data acquisition interface.

2.6. Human-wheelchair interface

A subject was given auditory commands to start training by following the direction of the customized interface. A total of 40 trials were obtained for four classes (up, down, left, and right). Afterwards, each user tested the accuracy of trained data. Again, auditory commands were given to initiate the classification. Twenty commands for five classes were provided in the classification interface. Users repeated this process three times for conventional and fractal electrodes. The overall accuracy was acquired from the sum of all three trials.

3. Results

3.1. Design and characterization of a soft electrode

Skin-wearable, soft electrodes were designed to have mechanical stretchability for intimate contact to the dimpled skin and avoiding any failure caused by dynamic skin deformations. Finite element analysis (FEA) played a key role to design a highly deformable structure. We explored a self-similar, open-mesh structured ‘fractal’ patterns (Hattori et al. 2014a; Hattori et al. 2014b). We used an array of circular disks, interconnected by the fractals, which offered an improved areal coverage for an enhanced signal-to-noise ratio. The FEA result in Figs. 2a and 2c captures the stretching behavior of the electrode upon tensile strains and bending; the fractal interconnects in the design absorb the applied stress to isolate it from the disks. The maximum principal strain, applied to Au, is less than 2% under the biaxial strain (30%) (Fig. 2a), which is well under the fracture strain (5%). A microscopic observation and quantitative mechanical test follows to validate the mechanics analysis (Figs. 2b and 2d). A microscopic investigation monitors visual defects on the structure, caused by mechanical fracture during cyclic stretching (Fig. 2b) and bending (Fig. 2d). Two-point resistance measurement upon repetitive loading and unloading quantifies the structural safety since any abrupt spike in the plot captures unexpected fracture. A cyclic mechanical test includes an increment of 5% tensile strains and 10° folding for the stretching and bending test, respectively. In this mechanical study, we apply additional safety factors by considering a maximum strain upto 30% and bending upto 180° (with 500 µm of radius of curvature), which still shows negligible resistance change on the electrode (Figs. 2e and 2f). Note that normal human external tissues stretch upto ~20% without tissue damage (Yeo et al. 2013b) and typical skin deformation does not exceed the selected bending curvature (Hattori et al. 2014a). Overall, the FEA and experimental results ensure that the fractal structure makes a suitable electrode even if a user has unusually developed dimples and habits to create exaggerated facial expressions (Bao et al. 2007).

Figure 2. Computational modeling and experimental validation of a fractal structured electrode upon mechanical deformation.

Figure 2

Open in a new tab

(a, b, c, d) Comparison between the finite element analysis (FEA) results and experimental observations with (a, b) biaxial tensile strains upto 30% and (c, d) bending upto 180 degrees with the radius of curvature (R) of 500 µm. (e, f) Quantification of electrical resistance according to the (e) applied strains and (f) bending; (e) cyclic mechanical test with repetition of loading and unloading and (f) cyclic mechanical bending test. The scale bars in the FEA data indicate the maximum principal strain applied on the electrode.

3.2. Measurement of EOG signals

A set of the fractal-structured electrodes was used to record target EOG signals on the skin (details in Materials and Methods). For a long-term, persistent HMI demonstration, we conducted electrophysiological measurement, without the use of any conductive gels and acrylic adhesives. The fractal electrode measured four kinds of EOG signals from different eye movements involving up, down, left, and right (Fig. 3). The same experimental procedure was applied for both conventional and fractal electrodes for side-to-side comparison (photo of both electrodes in Supplementary Figure S5a). The EOG data recorded by skin mounted electrodes were wirelessly transmitted to a data acquisition unit for signal processing (photo of the setup in Supplementary Figure S5b). Raw EOG signals in Fig. 3a, measured by vertically mounted electrodes, compare ‘up’ and ‘down’ movements. Both conventional and fractal settings show very similar signal amplitudes along with the baseline noise. Horizontal electrodes (Fig. 3b) capture higher EOG signals (600 – 750 µV) than the vertical motions (~400 µV) where the signal strength is dependent on the amount of angle change of the eyes. The direct signal comparison via derivative filtering (Figs. 3c and 3d) makes clearly distinguishable features of target signals. This filtering step decides the correct threshold for data processing in a classification algorithm. In the given data, threshold values can be chosen in the range of 3 – 5 µV/s, dependent on a user’s training data.

Figure 3. A representative set of EOG signals involving four kinds of eye movements.

Figure 3

Open in a new tab

(a, b, c, d) Direct comparison of raw EOG signals measured from fractal and conventional gel electrodes: signals from (a) eyes ‘up’ and ‘down’ motions and (b) ‘left’ and ‘right’ motions. Bandpass (0.1 – 20 Hz) filtered EOG signals from (c) ‘up’ and ‘down’ motions and (d) ‘left’ and ‘right’ motions.

Changes of EOG amplitudes according to eye rotation angles (Figs. 4a and 4b) quantify the sensitivity of two types of electrodes (fractal and conventional). A test subject wears two differential electrodes, positioned 1 cm away from the outer canthus of each eye for concurrent comparison (Fig. 4a). The sensitivity measurement asks the subject to trace a series of circular targets (diameter = 1 cm), located 60 cm away from the eyes. Each target corresponds to 5 degree changes of eye rotation. Filtered EOG signals in Fig. 4b presents amplitude variations according to eye motions from 0 to 40°. The calculated resolutions are 13.3±0.6 and 11.7±0.9 µV/° for fractal and conventional electrode, respectively. The higher sensitivity from the fractals is explained by considering the signal-to-noise (SNR) related to the skin contact area. The skin-like electrode covers ~3.1 cm2 to maximize the available area on the skin, while the rigid electrode (MVAP II, Newbury Park, CA) including a gel occupies only ~0.8 cm2, limited by the required adhesive pad to affix on the skin. The direct contribution of the contact area to the signal strength is measured by changing the size of fractals, while the rigid electrode stays intact (Figs. 4c – 4e). A user wearing both horizontal electrodes focuses on three target locations corresponding to −40, 0, and +40° in the horizontal direction (details in Supplementary Figure S6). The result shows that SNR values decrease as the size of fractals shrinks from full to half and quarter (Fig. 4e; averaged values based on 8 trials); the quarter-sized electrode (contact area: 0.78 cm2) shows comparable SNR values to the rigid one. A slight DC baseline drift in Figs. 4c and 4d would come from the low frequency signals with respect to skin potential changes18.

Figure 4. Comparison of electrode sensitivity and assessment of the side effect from gel electrodes.

Figure 4

Open in a new tab

(a) An experimental setup targeting a series of circles to measure the angledependent EOG sensitivity. (b) Normalized EOG signal amplitudes according to the angle change of an eye where black and yellow bars represent conventional and fractal electrode, respectively. Error bars show the standard deviation (n=3). (c, d) Photos of fractal electrodes mounted on the skin (conventional electrode is located on the contralateral site) and the corresponding EOG signals comparing the signal strength between fractal and conventional electrodes. The signal strength is dependent of the electrode size; (c) full size and (d) quarter size electrode. (e) Averaged SNR values comparing the size effect of fractal electrodes with a fixed rigid electrode (error bars from n=8 with the standard deviation). (f, g) Comparison of infrared thermographs measured with (f) conventional and (g) fractal electrodes to reveal the side effect of the gel contacted to the skin (forearm) for more than 5 hours. (h) Photo and thermograph of a gel electrode mounted on the forehead, causing skin rash.

Skin assessments by infrared thermography and contact microscopic observation (Figs. 4f – 4h) capture the advantage of the fractal bioelectrode over the gel-covered rigid electrode. Electrolyte gels on the metal electrode causes an adverse effect (erythema) on the skin by increasing the localized temperature (Fig. 4f; 5 hours of skin contact), while the fractal electrode shows a negligible effect even with lamination over 10 hours (Fig. 4g). In addition, an adhesive pad, used to mount the rigid electrode, exfoliates the epidermis, which causes pains and skin rashes (Fig. 4h). Collectively, the comparative assessments demonstrate the novelty of the soft biosensor for a comfortable, long-term usable human-machine interface.

3.3. Classification algorithm

A signal processing algorithm, designed by Matlab (Mathworks, Natick, MA), captures targeted EOG from eye movements of “blink” and “down” (raw data; Fig. 5a). The next step is to process the signals via a 3rd order bandpass filter (Butterworth) at the range of 0.1 – 20 Hz to remove high frequency noise (Fig. 5b). Afterwards, a series of peak detection methods are implemented (Fig. 5c). The selected start and end positions are pertinent to the signal classification by increasing the detection accuracy. Examples of two sliding windows in Fig. 5d (left: “blink” and right: “down” movement) align the signals in the center since a misalignment results in incorrect classification (details of sliding window classification in Supplementary Figures S7, S8, and S9). Lastly, the sliding window data are transferred into a linear discriminant analysis (LDA) classifier where a test dataset is compared to the training dataset. The LDA classification plot in Fig. 5e shows a set of representative training data including both correct (o) and incorrect (x) classes.

Figure 5. Signal processing sequence with a LDA classifier.

Figure 5

Open in a new tab

(a) Raw EOG signals measured from eye movements of “blink” and “down”. (b) Preprocessed signals via filtering. (c) Absolute derivative signals to pinpoint target peak signals and apply a threshold boundary. (d) Sliding window algorithm using the detected peaks to parse the dataset into centered signals. (e) Machine learning algorithm using five distinct features from the centered signals and a LDA classifier to distinguish 4 classes of eye movements (up, down, left, and right).

The classification algorithm includes five features: definite integral, amplitude, velocity, signals mean, and wavelet energy. A common factor in the features is the sliding window indices, identified by t1 and t2. The area under the curve of the filtered signal (eq. 1) is acquired using the trapezoidal method. The signal amplitude (eq. 2) is determined from the derivative filtered signal, in which and are two consecutive peaks surpassing a threshold. A difference between the two peaks explains the order of the positive and negative peaks for each channel. The velocity (eq. 3) is simply the amplitude difference divided by the time difference and signal mean (eq. 4) is the average of the filtered signals. The Haar wavelet energy transform (eq. 5) outputs a set of scaled coefficients (Ca,b). The absolute multiplication of the coefficients creates the scalogram matrix, which is summed to resolve the desirable wavelet energy feature.

Difinite integral=∫t1t2f(t)dt (1)
Amplitude=At1−At2,where A=max(d(f(t))dt) (2)
Velocity=At1−At2t1−t2 (3)
Signal mean=1N∑t1t2f(t), where N=number of samples (4)
Wavelet energy=∑|Ca,b*Ca,b|, where Ca,b=∫t1t2f(t)φa,b(t)dt (5)

A simulated program was used to make an accuracy assessment of the developed algorithm. Confusion matrices in Figs. 6a and 6b compare the overall accuracy of two types of electrodes based on four classes of eye movements. The averaged accuracy from three subjects shows 91.9 and 94.1% for the conventional (Fig. 6a) and fractal (Fig. 6b) electrodes, respectively (all confusion matrices for 3 different users in Supplementary Figures S10 and S11). The result presents an enhanced classification accuracy of the fractal electrode, while offering user-comfortable, intimate contact to the skin. The accuracy discrepancy between those electrodes is caused by the limited contact area of the rigid electrode (Fig. 4e). Such an occurrence results in inconsistent signal recording of EOG. In addition, users wearing the rigid electrode expressed slight motion constraints caused by the rigidity of the electrode and adhesive pad, which might limit angular rotation of eyes.

Figure 6. Demonstration of a human-wheelchair interface via skin-wearable soft electrodes and EOG signals.

Figure 6

Open in a new tab

(a, b) Comparison of confusion matrix from (a) conventional rigid electrodes and (b) soft fractal electrodes; The averaged classification accuracy from three subjects shows 91.9% and 94.1% for conventional and fractal electrodes, respectively. (c) Top view of entire pathways of a robotic wheelchair; classified EOG signals from eye movements are wirelessly transmitted from the fractal electrode/DAQ to the wheelchair control interface via Bluetooth. (d) A series of photos capturing a real-time, wireless control of the wheelchair via eye movements. There are four control commands generated by different eye movements including “up”, “right”, “left”, and “down”. Insets show the raw EOG signals in correspondence with the eye movement from a subject.

Only the fractal electrodes demonstrate accuracy higher than 98% (average from three users) for ‘up’ and ‘down’ motions in the real-time EOG measurement. The maximum accuracy was achieved by optimization of five types of classification features in the LDA classifier (Supplementary Figure S12), which was used to analyze all EOG signals for confusion matrices in this work. Comprehensive assessments (true positive, false positive, false negative, and true negative) of the classification, summarized in Table S1, supports the enhanced device performance of the fractal electrodes, compared to the conventional setting. We validate our confusion matrix accuracy of each class with a secondary indicator by using the following calculation (eq. 6), where TP, TN, FP, and FN refer to true positive, true negative, false positive, and false negative, respectively.

Accuracy=TP+TNTP+TN+FP+FN (6)

The true positive is determined from the value of each class on the diagonal of the confusion matrix, whilst the false positive is the value of the incorrect classifications of that individual class. The other two measurements resemble the data from other classes not from the observed class. As observed in the comparative accuracy test, the fractal device shows higher successful rates in classification than the conventional electrode. Table S1 shows that one of subjects using conventional electrodes has a high baseline noise that gives low SNR, which attribute to a large number of false positives and false negatives in the “down” and “left” class. The SNR is calculated for each user individually using the first second of the trial as background noise and each sequential motion as the signal. The root mean square (RMS; eq. 7) of each window (Supplementary Figure S13) is used to calculate the SNR and then averaged for each channel of each user to quantify the cause of classification errors (summarized results from 3 users in Supplementary Table S2). Collectively, this quantitative comparison clearly captures the novelty of the soft electrode for persistent, ergonomic HMI applications.

RMS=1N∑n=1N|Xn|2 (7)

3.4. Demonstration of a human-wheelchair interface

EOG signals acquired from the skin-like electrodes control a commercial wheelchair (Invacare™ Pronto, Elyria, OH) that includes the data acquisition system with real-time classification capability. A microcontroller (Arduino, Adafruit, New York City, NY) and digital-to-analogue converters replace the wheelchair joystick; voltage signals are smoothly transferred to the wheelchair’s DC motors and a designed Arduino code adjusts the speed settings to limit motion artifacts from the wires connected to the Bluetooth data acquisition unit (details of the wheelchair setup in Supplementary Note 2 and Supplementary Figure S14). Four types of eye movements (up, down, left, and right), detected by the soft, fractal electrode around the eyes, control the robotic wheelchair: ‘up’ to go forward, ‘down’ to stop movement, ‘left’ to turn counter-clockwise, and ‘right’ to turn clockwise (Figs. 6c and 6d). Six consecutive commands operate the wheelchair in real-time to navigate the designed pathway from the position 1 to position 6 (Fig. 6c). Each eye movement is recorded by two sets of vertical and horizontal fractal electrodes. A representative video clip (Supplementary Video 1) captures the entire process of the real-time, human-wheelchair interface. A smart appliance (Android-based tablet; Microsoft Surface Pro 4) provides the concurrent signal display and data saving options to the user.

4. Discussion

The soft bioelectronic system enables successful measurement of EOG without any use of conductive gels. The computational modeling reveals the excellent mechanical compliance of the skin-like electrode and the experiment agrees well with the computational study. A series of in vivo measurement of EOG on volunteers demonstrates the feasibility of the soft bioelectronics for an ergonomic control of a robotic wheelchair. The experimental results present that the fractal electrode shows better performance than the rigid electrode, mounted on the dimpled surfaces of the skin. Overall, the soft bioelectronics shows the potential as a next-generation device for a comfortable, easy-to-use, and wireless control of rehabilitation devices. Areas for further development include an integration of wireless telemetry and powering units in the system. A single device platform will allow high-quality recording of electrophysiological signals for persistent human-machine interfaces.

5. Conclusion

We have demonstrated the feasibility of imperceptible, soft electrodes for a tether-free control of a wheelchair via EOG. The skin-friendly system enables a high-quality recording of EOG from the contoured, dimpled skin, proven by the direct comparison with the conventional electrode. The quantitative thermography and assessment of skin irritations shows additional advantage of the fractal structured electrode. A classification algorithm utilizing multiple features and a LDA classifier distinguishes four classes of eye activities with an accuracy of 94.1% that is higher than the typical rigid system (91.9%). The bioelectronics system incorporating conformal electrodes, real-time data classification, and wireless wheelchair controller shows a great potential for ergonomic, long-term wearable, and clinically applicable HMI.

Supplementary Material

1
Download video file (25.1MB, wmv)
2

Highlights.

  • This article describes materials and methods to design soft biosensors and bioelectronics for a wireless human-wheelchair interface.

  • The “skin-like” bioelectrode enables an ergonomic, high-fidelity recording of electrooculograms, supported by the direct comparison with conventional rigid electrodes.

  • This soft bioelectronic system demonstrates excellent mechanical compliance in both stretchability (30%) and bendability (up to 180°).

  • A newly designed classification algorithm with an optimized set of features shows the accuracy of 94% with five different eye movements, which is used to demonstrate a precise, hands-free control of a robotic wheelchair via electrooculograms.

Acknowledgments

W.-H.Y. acknowledges a grant support from CooperVision, Inc. and startup funding from the School of Engineering at Virginia Commonwealth University.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aziz F, Arof H, Mokhtar N, Mubin M. HMM based automated wheelchair navigation using EOG traces in EEG. Journal of neural engineering. 2014;11(5):056018. doi: 10.1088/1741-2560/11/5/056018. [DOI] [PubMed] [Google Scholar]
  2. Bao S, Zhou C, Li S, Zhao M. A New Simple Technique for Making Facial Dimples. Aesthetic Plastic Surgery. 2007;31(4):380–383. doi: 10.1007/s00266-006-0191-8. [DOI] [PubMed] [Google Scholar]
  3. Barea R, Boquete L, Mazo M, Lopez E. System for assisted mobility using eye movements based on electrooculography. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2002a;10(4):209–218. doi: 10.1109/TNSRE.2002.806829. [DOI] [PubMed] [Google Scholar]
  4. Barea R, Boquete L, Mazo M, López E. Wheelchair Guidance Strategies Using EOG. Journal of Intelligent and Robotic Systems. 2002b;34(3):279–299. [Google Scholar]
  5. Belkacem AN, Shin D, Kambara H, Yoshimura N, Koike Y. Online classification algorithm for eye-movement-based communication systems using two temporal EEG sensors. Biomedical Signal Processing and Control. 2015;16:40–47. [Google Scholar]
  6. Brault MW. Americans With Disabilities: 2010. U.S. CENSUS BUREAU. 2012:70–131. [Google Scholar]
  7. Bulling A, Ward JA, Gellersen H. Proceedings of the 11th international conference on Ubiquitous computing. Orlando, Florida, USA: ACM; 2009. Eye movement analysis for activity recognition; pp. 41–50. [Google Scholar]
  8. Bulling A, Ward JA, Gellersen H, Troster G. Eye movement analysis for activity recognition using electrooculography. IEEE transactions on pattern analysis and machine intelligence. 2011;33(4):741–753. doi: 10.1109/TPAMI.2010.86. [DOI] [PubMed] [Google Scholar]
  9. Chen Y, Howe C, Lee Y, Cheon S, Yeo W-H, Chun Y. Microstructured Thin Film Nitinol for a Neurovascular Flow-Diverter. Scientific reports. 2016:6. doi: 10.1038/srep23698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hattori Y, Falgout L, Lee W, Jung SY, Poon E, Lee JW, Na I, Geisler A, Sadhwani D, Zhang Y. Multifunctional Skin-Like Electronics for Quantitative, Clinical Monitoring of Cutaneous Wound Healing. Advanced healthcare materials. 2014a;3(10):1597–1607. doi: 10.1002/adhm.201400073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hattori Y, Falgout L, Lee W, Jung SY, Poon E, Lee JW, Na I, Geisler A, Sadhwani D, Zhang Y, Su Y, Wang X, Liu Z, Xia J, Cheng H, Webb RC, Bonifas AP, Won P, Jeong JW, Jang KI, Song YM, Nardone B, Nodzenski M, Fan JA, Huang Y, West DP, Paller AS, Alam M, Yeo WH, Rogers JA. Multifunctional Skin-Like Electronics for Quantitative, Clinical Monitoring of Cutaneous Wound Healing. Adv Healthc Mater. 2014b doi: 10.1002/adhm.201400073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hortal E, Iáñez E, Úbeda A, Perez-Vidal C, Azorín JM. Combining a Brain–Machine Interface and an Electrooculography Interface to perform pick and place tasks with a robotic arm. Robotics and Autonomous Systems. 2015;72:181–188. [Google Scholar]
  13. Jeong J-W, Yeo W-H, Akhtar A, Norton JJS, Kwack Y-J, Li S, Jung S-Y, Su Y, Lee W, Xia J, Cheng H, Huang Y, Choi W-S, Bretl T, Rogers JA. Materials and Optimized Designs for Human-Machine Interfaces Via Epidermal Electronics. Advanced Materials. 2013;25(47):6839–6846. doi: 10.1002/adma.201301921. [DOI] [PubMed] [Google Scholar]
  14. Jeong JW, Kim MK, Cheng H, Yeo WH, Huang X, Liu Y, Zhang Y, Huang Y, Rogers JA. Capacitive epidermal electronics for electrically safe, long-term electrophysiological measurements. Advanced healthcare materials. 2014;3(5):642–648. doi: 10.1002/adhm.201300334. [DOI] [PubMed] [Google Scholar]
  15. Keegan J, Burke E, Condron J. An electrooculogram-based binary saccade sequence classification (BSSC) technique for augmentative communication and control; 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2009. pp. 2604–2607. [DOI] [PubMed] [Google Scholar]
  16. Lee Y, Yeo W-H. Skin-Like Electronics for a Persistent Brain-Computer Interface. Journal of Nature and Science. 2015;1(7):e132. [Google Scholar]
  17. Ma Y, Jang KI, Wang L, Jung HN, Kwak JW, Xue Y, Chen H, Yang Y, Shi D, Feng X. Design of Strain-Limiting Substrate Materials for Stretchable and Flexible Electronics. Advanced Functional Materials. 2016 doi: 10.1002/adfm.201600713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Mala S, Latha K. Feature selection in classification of eye movements using electrooculography for activity recognition. Computational and mathematical methods in medicine. 2014;2014:713818. doi: 10.1155/2014/713818. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Wu SL, Liao LD, Lu SW, Jiang WL, Chen SA, Lin CT. Controlling a Human-Computer Interface System With a Novel Classification Method that Uses Electrooculography Signals. IEEE Transactions on Biomedical Engineering. 2013;60(8):2133–2141. doi: 10.1109/TBME.2013.2248154. [DOI] [PubMed] [Google Scholar]
  20. Xu B, Akhtar A, Liu Y, Chen H, Yeo WH, Park SI, Boyce B, Kim H, Yu J, Lai HY. An Epidermal Stimulation and Sensing Platform for Sensorimotor Prosthetic Control, Management of Lower Back Exertion, and Electrical Muscle Activation. Advanced Materials. 2015 doi: 10.1002/adma.201504155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Yeo W-H, Webb RC, Lee W, Jung S, Rogers JA. SPIE Defense, Security, and Sensing. International Society for Optics and Photonics; 2013a. Bio-integrated electronics and sensor systems; p. 87251I-87251I-87257. [Google Scholar]
  22. Yeo WH, Kim YS, Lee J, Ameen A, Shi L, Li M, Wang S, Ma R, Jin SH, Kang Z, Huang Y, Rogers Ja. Multifunctional epidermal electronics printed directly onto the skin. Advanced Materials. 2013b;25(20):2773–2778. doi: 10.1002/adma.201204426. [DOI] [PubMed] [Google Scholar]
  23. Zhang Y, Wang S, Li X, Fan JA, Xu S, Song YM, Choi KJ, Yeo WH, Lee W, Nazaar SN. Experimental and theoretical studies of serpentine microstructures bonded to prestrained elastomers for stretchable electronics. Advanced Functional Materials. 2014;24(14):2028–2037. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
Download video file (25.1MB, wmv)
2
NIHMS847684-supplement-2.pdf (1.4MB, pdf)

RESOURCES