Photoplethysmography revisited: from contact to noncontact, from point to imaging

Abstract

Photoplethysmography (PPG) is a non-invasive optical technique for detecting microvascular blood volume changes in tissues. Its ease of use, low cost and convenience make it an attractive area of research in the biomedical and clinical communities. Nevertheless, its single spot monitoring and the need to apply a PPG sensor directly to the skin limit its practicality in situations such as perfusion mapping and healing assessments or when free movement is required. The introduction of fast digital cameras into clinical imaging monitoring and diagnosis systems, the desire to reduce the physical restrictions, and the possible new insights that might come from perfusion imaging and mapping inspired the evolution of conventional PPG technology to imaging PPG (IPPG). IPPG is a noncontact method that can detect heart-generated pulse waves by means of peripheral blood perfusion measurements. Since its inception, IPPG has attracted significant public interest and provided opportunities to improve personal healthcare. This study presents an overview of the wide range of IPPG systems currently being introduced along with examples of their application in various physiological assessments. We believe that the widespread acceptance of IPPG is happening, and it will dramatically accelerate the promotion of this healthcare model in the near future.

I. Introduction

Remote sensing imaging offers additional value regardless of the parameters being used because it is appealing to the general public and facilitates insights that would otherwise be difficult or even impossible to obtain. Due to advances in the areas of integrated circuits, microelectronics and image/signal processing techniques, the past two decades have witnessed enormous progress in the development of optical imaging technologies for various biomedical and clinical applications, such as laser Doppler imaging (LDI) [1], laser speckle imaging (LSI) [2], photoacoustic tomography (PAT) [3], tissue viability imaging (TiVi) [4], and imaging photoplethysmography [(PPG), IPPG]. Many reviews and books have been written on LDI, LSI, and PAT (for reviews, see [5]– [8]), which will not be repeated in this paper. Compared to LDI and LSI, IPPG has a relatively short history [9], [10]. With the recent evolution of these optical blood perfusion imaging technologies, it was soon realized that PPG could be extended to imaging as well. The primary goal of this topical review is to introduce researchers who are unfamiliar with IPPG to their use, development, and applications. We hope that this paper will inform and inspire uninitiated readers to become consumers of IPPG technology.

All the PPG technologies discussed in this review can be roughly divided into four groups: single-point contact PPG, multi-point contact PPG, close/contact IPPG and noncontact IPPG. This paper will concentrate on the noncontact IPPG technique, which is a non-invasive optical method for sensing the blood volume pulse, and it has been proven to be superior in its ease of use, low cost, safety, convenience, and ability to offer multiple physiological assessments.

Four general topics will be covered in this paper. First, we will provide a brief review of the contact PPG technology and its limitations. An introduction to the fundamental biophysical concepts related to PPG could help to illustrate the principles of its extension, IPPG. Second, we will briefly introduce the current research on wearable and noncontact PPG technology, which is aimed at reducing the constraints of conventional PPG and improving comfort for long-term monitoring. Third, we will present approaches and precautions for IPPG signal extraction. Various applications of IPPG and the latest developments in the literature, as well as reviews of the contribution of IPPG techniques various physiological assessments, will be introduced. Finally, we will discuss the challenges in IPPG and suggest several future research directions.

II. Photoplethysmography

PPG is the core technology upon which IPPG is based. Therefore, an introduction of the fundamental concepts related to PPG could help to illustrate the principles of IPPG and show the development of the IPPG technique.

A. Background of PPG

As the pioneers who helped to establish the PPG technique, Hertzman and colleagues first introduced the term “photoplethysmography” in the 1930s and suggested that it represented the volumetric changes (“plethysmo” means “enlargement” in Greek) in the dermal vasculature [11], [12]. PPG is a simple and low-cost optical bio-monitoring technique used to non-invasively measure the blood volume changes that occur in the microvascular tissue bed beneath the skin, which are due to the pulsatile nature of the circulatory system [13], [14]. Despite its simplicity, our understanding of the origins of different components of the PPG signal are still rudimentary. It is generally accepted that these components could provide valuable information about the cardiovascular system [14]. Over the last twenty years, there has been a significant increase in the number of published articles on PPG (Fig. 1). The popularity of this topic can be attributed to the realization that PPG has important implications for a wide range of applications in cardiovascular system assessment, vital sign monitoring and blood oxygen detection. For example, in the early 1990s, pulse oximetry became a mandated international standard for monitoring during anesthesia [15].

Fig. 1.

Publication and citation report of PPG studies for the past two decades (1994 – 2014). Data was obtained in Web of Science™ using ‘photoplethysmography’ and ‘photoplethysmographic’ as topic (accessed on 25^th Mar. 2015

As an optical technique, PPG requires a light source and a photo detector to function. The light source illuminates the tissue, and the photo detector senses the small variations in reflected or transmitted light intensity associated with changes in perfusion in the catchment volume. The fundamental principle of PPG relies on the differences in sensitivity of different optical wavelengths for blood and other tissue components [16]. Given that the interaction of light with biological tissue can be complex (including scattering, absorption, reflection, etc.) [17], the choice of the light source wavelength is very important. The ideal wavelength(s) for PPG should have greater absorption for blood compared to other tissue components, as this would allow for accurate monitoring of blood volume changes in the microvascular tissue bed [18]. It is known that shorter wavelengths of light are strongly absorbed by melanin, while water absorbs light in the ultraviolet and longer infrared (IR) ranges. Therefore, red and near IR light are typically utilized as light sources in PPG sensors (for a review, see [18]). Recently, green-wavelength PPG devices are becoming increasingly popular for their large intensity variations in modulation that are observed during the cardiac cycle [19]–[21]. Compared to red/IR light, green LED has much greater absorptivity for both oxyhemoglobin and deoxyhaemoglobin [22], thus resulting in a better signal-to-noise ratio (SNR). Several studies have compared the performance of IR and green light PPG and found that the green light achieved higher accuracy of pulse rate detection than did the IR light [20], [21].

When the heart pumps blood to the body and the lungs during systole, the amount of blood that reaches the capillaries in the skin surface increases, resulting in more light absorption. The blood then travels back to the heart through the venous network, leading to a decrease of blood volume in the capillaries and less light absorption. The measured PPG waveform therefore comprises a pulsatile (often called “AC”) physiological waveform that reflects cardiac synchronous changes in the blood volume with each heartbeat, which is superimposed on a much larger slowly varying quasi-static (“DC”) baseline (Fig. 2). The DC component contains valuable information about respiration, venous flow, sympathetic nervous system activities and thermoregulation [13]. PPG has been widely used in the clinical community and has proven its usefulness not only for real-time monitoring of various physiological parameters (i.e., oxygen saturation, heart rate, blood pressure, respiration, and cardiac output), but also for vascular assessment (i.e., arterial diseases, stiffness and aging) and autonomic function assessment (i.e., heart rate variability (HRV)/pulse rate variability (PRV)). Recently, Allen presented a thorough review pertaining to the clinical applications of PPG [13], and this paper will not attempt to elaborate on that work.

Fig. 2.

Variation in light attenuation by tissue (modified from [23]).

There are two main PPG operational configurations, the transmission and reflection modes, that depend on the geometric arrangements of the light source and the photo detector. Specifically, in the transmission mode, a tissue sample is placed between the light source and the photo detector, while in the refection mode, the source and detector are placed side by side. In the transmission mode, the photo detector has to detect the light transmitted through the tissue, and, thus, the measurement site may be limited to tissues where transmitted light can be readily detected, e.g., fingertips and earlobes [33]. However, these measurement sites are more susceptible to environment extremes (i.e., low ambient temperature) and the PPG sensor may interfere with daily activities. Conceptually, in the reflection mode, the photo detector measures light that is back-scattered or reflected from tissue, bone and/or blood vessels. An opaque shield is usually positioned between the optoelectronics to prevent any direct illumination from the light source to the photo detector that has not first passed through the tissue. This configuration enables measurements from virtually any point on the skin surface, including those that may be hard to access with the transmission mode. Nevertheless, the sensor should be attached firmly to the skin for an accurate measurement, and therefore, the measurement site normally requires a flat skin surface. Webster has written an excellent review of optical sensor technology for PPG and its application in pulse oximetry monitoring [18].

B. Limitations of PPG

Despite PPG’s wide range of applications, there are several significant drawbacks that may limit the usefulness and evolution of conventional PPG: a) Spot measurement: A PPG sensor can only monitor the dynamic change of the blood volume at one site/point per probe. Multi-sensor PPG systems have been developed to monitor blood volume changes at different sites simultaneously [34]–[36]. With the emphasis on detecting the right-to-left side physiological differences in clinical applications, these multi-sensor systems typically require electronic and optical matching of each PPG sensor as well as high temporal synchronization and anatomically symmetric wearing locations (e.g., both earlobes, both index fingers, or both legs) [35]. However, multiple sensing locations significantly constrain the daily activities of the subject. Moreover, the use of multiple sensors will make the subject uncomfortable and constrain the feasibility for everyday use. An imaging PPG (IPPG) technology is therefore needed to offer detailed spatial information simultaneously from multiple sites of arbitrary sizes and locations, thus allowing the derivation and mapping of physiological parameters and, ultimately, facilitating insights that would otherwise be difficult or even impossible to obtain from single-point measurements [37]. b) Contact measurement: For accurate measurements, the conventional PPG sensor should be attached firmly to the skin, which constrains its practicality in situations such as wound diagnostics (burn/ulcer/trauma) and skin healing evaluation or when free movement is required. Recent studies have investigated the effect of contact force on the amplitude and timing of the PPG signal using spring-loaded conventional PPG probes and have shown that this force must be carefully controlled to obtain high-quality clinical data from the signals [38]–[41]. In addition, direct contact can easily cause discomfort for the subjects during continuous monitoring. To date, no general standards have been established for clinical or fundamental PPG measurements of contact pressure. The contact measurement configuration might lead to the deformation of the arterial wall at the detecting site or even block microcirculation in the capillaries. Therefore, from the perspective of patient comfort, the conventional PPG technique is not suitable for ambulatory monitoring. Recent “fixed in the environment” and “wearable” PPG systems have been introduced to address this issue and will be discussed in great details in the following section. Nonetheless, drawbacks still remain regarding the lack of perfusion information and the accuracy needed for clinical assessments. c) Motion artifact corruption: PPG is susceptible to motion-induced signal corruption. It has been clinically demonstrated that motion artifacts may cause errors in pulse oximeters response [42]. Moreover, its single-spot measurement modality makes motion artifact removal or attenuation one of the most challenging issues in PPG signal processing. This drawback partially limits the physiological monitoring capabilities of the technique in real world environments, such as homecare, ambulances, and sports performance assessment.

III. Wearable and noncontact PPG

To facilitate long-term recordings and to increase the comfort of subjects, a great effort has been made to develop wearable PPG systems [24]–[32], [43]–[51]. Attempts have also been made for noncontact PPG techniques to overcome the drawback of contact with the skin [52]–[55]. In Table I, we provide several representative studies of wearable and noncontact PPG techniques.

TABLE I.

Summary of wearable and noncontact PPG system

Reference	Light source	Wearable/ Noncontact	Reflection/ Transmission	Measures	Attachment	Remarks
Yang et al., [30]	Red & IR	Wearable	Transmission	HR & SpO2	Finger ring	Wireless + long-term monitoring + power saving
Rhee et al., [29]	Red & IR	Wearable	Transmission	HR	Finger ring	Wireless + power saving + pressure force assessment
Renevery et al., [43]	IR	Wearable	Reflection	HR	Wrist	Automatic noise cancellation
Mendelson et al., [31], [44]	Red &IR	Wearable	Reflection	HR	Tape(wrist & forehead)	Assess measurement site and PD size effect
Celka et al., [45]	IR	Wearable	Reflection	HR	Ear cup	PCA based motion cancellation
Wang et al., [46]	IR	Wearable	Reflection	HR	Tape(ear)	External wire recording circuitry + multichannel design
Wang et al., [25]	IR	Wearable	Reflection	HR	Ear hook	Automatic noise cancellation
Vogel et al., [24]	Red & IR	Wearable	Reflection	HR	Ear hook	External wire recording circuitry + passive motion cancellation
Shin et al., [26]	IR	Wearable	Transmission	HR	Clip	Wireless + automatic noise cancellation
Wartzek et al., [47]	Red & IR	Wearable	Reflection	HRV	Otoplastic insertion	Wireless + long-term monitoring
Poh et al., [27], [48]	IR	Wearable	Reflection	HR	Earring & earphone	Wireless + automatic noise cancellation
Kviesis-Kipge et al., [28], [49]	IR	Wearable	Reflection	FFI	Everyday clothes	Wireless + embedded to everyday clothes
Chen et al., [50]	Red & IR	Wearable	Reflection	SpO2	Tape(ear)	Vascular phantom development
Apple Watch [32]	IR & Green	Wearable	Reflection	HR	Wrist	Wireless + long-term monitoring + embedded in watch
Garbarino et al., [51]	Red & Green	Wearable	Reflection	HR	Wrist	Wireless + long-term monitoring + integrated with EDA + passive motion cancellation
Wong et al., [52]	IR	Noncontact	Reflection	HR & PPI	–	Embedded to mattress + long-term monitoring
Shi et al., [53], [54]	IR	Noncontact	Reflection	HR & HRV	–	Changeable angle of reflection light source + motionless
Cennini et al., [55]	Blue & IR	Noncontact	Reflection	HR	–	Measurement range from centimeters to several meters + Motion artefact cancellation

Note: EDA = electrodermal activity; FFI = foot-to-foot interval of PPG waveform; HR = heart rate; HRV = heart rate variability; IR = infrared; PCA = principle component analysis; PD = photodetector; PPI = peak-to-peak interval of PPG waveform; SpO2 = blood oxygen saturation.

Ideally, for long-term recording, the wearable PPG device should not interfere with daily activities and should be lightweight, robust, low cost, and comfortable to wear, while still maintaining the quality of the signal under various conditions. Similar to conventional PPG, the measurement sites for wearable PPG sensing also target the finger, forehead, wrist, and ear. Among these different measurement sites, the ear and finger remain of comparative interest because they have a rich arterial supply and are relatively easy to affix sensors to (e.g., finger-ring and earphone). Several types of wearable PPG sensors are shown in Fig. 3. Over the course of a variety of daily activities, the motion artifacts could be very complex. To address this issue, an accelerometer has typically been embedded in the system to provide motion reference for further artifact removal. Several research groups have assessed their wearable PPG systems in simulated and real conditions and found comparable capability in monitoring vital signs, such as heart rate. It is noteworthy mentioning that the newly launched Apple Watch also provides heart rate measurements [32]. Great detail about wearable PPG systems could be found in reviews of this topic [23], [56].

Fig. 3.

Several representative types of wearable PPG sensors: (a) micro-optic reflective sensor with individually tailored otoplastic housing [24]; (b) ear-hook design of reflective sensor [25]; (c) an ear clip transmissive sensor with accelerometer [26]; (d) wireless magnetic earing sensor with accelerometer [27]; (e) a golf-hat with integrated wireless reflective sensor [28]; (f) a wireless transmissive ring sensor with accelerometer [29], [30]; (g) a wireless attachable reflective sensor with receiver module [31]; and (h) Apple Watch [32].

Shi et al., first introduced a noncontact PPG prototype and demonstrated its feasibility for physiological monitoring [54]. In this setup, one single-channel IR LED was used as the illumination source and was positioned to illuminate the tissue area at a changeable angle. A high-speed silicon PIN photodiode was used to detect the reflected light intensity and was positioned perpendicular to the tissue surface at a distance of 5 cm. Later, Hu and colleagues introduced an optophysiological model based upon the Beer-Lambert law [57], [58] to show the relationship between the light incident on a biological tissue and the light received after transmission through that tissue [59], [60]. However, in these pioneering studies, motion artifacts were not carefully account for, and the systems operated in still conditions. Cennini and colleagues further expanded on this work, introduced a dual-channel (blue and IR) noncontact PPG prototype together with a motion artifact reduction technique and proved its feasibility for the assessment of vital signs from a distance of a few centimeters to several meters [55]. Nonetheless, success in addressing the issue of contact measurement was achieved with the sacrifice of limited motion range, i.e., tilt and translation of the hand at ±5 degrees around the normal position relative to the detector. It is generally known that remote sensing approaches are more vulnerable to motion artifacts than the corresponding sensors in ordinary contact PPG devices [55], [61]. Moreover, the valuable blood perfusion information is still missing. Therefore, a remote imaging technique that overcomes these drawbacks is needed. Indeed, driven by the desire of remote sensing, as well as the dramatic developments of imaging techniques and achievements in imaging instrumentation, there has been a resurgence of interest in PPG in recent years.

IV. Imaging Photoplethysmography

A. Background of IPPG

The investigation of the IPPG technique started in 1996 [10]. Conceptually, it is a camera-based remote measurement method for the visualization of dermal blood vessels and for detecting perfusion in different skin areas. Among the pioneering studies, Wu and colleagues presented a charge coupled device (CCD) based IPPG system and proved its feasibility in assessing local changes of dermal blood volume [9]. Since then, there has been rapid growth in the literature pertaining to IPPG techniques. In June 2015, using the keywords “Photoplethysmography/photoplethysmographic & camera”, we found 51 relevant studies in the Web of Science™ database. The main part of this topic review was based on 69 IPPG-related studies [9], [21], [37], [59], [60], [62]–[125]. A summary of several recent representative IPPG studies is presented in Table II.

TABLE II.

Overview of several recent IPPG studies

Reference	System setup			Measures						Remarks
	Camera	Light source (λ (nm))	fs (Hz)	HR	RR	SpO2	Perfusion	HRV	BP
Wu et al. [9]	CCD	LED(875)	25	✓	✓		✓			Ringlight design + venous & arterial perfusion
Hulsbusch et al. [90]	CCD	LED(800)	≥ 8	✓	✓		✓			TFR + wound skin perfusion
Wieringa et al. [89]	CMOS	LED(660, 810,) &940	6.7, 13.7 /λ	✓	✓	✓	✓			Light source design + image processing method
Humphreys et al. [88]	CMOS	LED(760, 880)	> 16/λ	✓		✓				SpO2 assessment + transmission mode
Hu et al. [59], [60]	CCD	LED(650, 870)	15/λ	✓						Opto-physiological model
	CMOS	LED(660, 870) &940	21, 30 /λ	✓			✓			Blood perfusion at different depth
Takano et al. [87]	CCD	Ambient light	5	✓	✓					1st study utilizing ambient light source
Verkruysse et al. [85]	DC	Ambient light	15/30	✓	✓		✓			Wave propagation + vascular skin lesions
Blazek et al. [92]	CCD	Green & IR	–				✓			Histamine induced skin reaction +
Poh et al. [75], [80]	Webcam	Ambient light	15	✓	✓			✓		ICA based motion cancellation + multiple subjects monitoring
Sun et al. [64], [73] [37]	CMOS	LED(870)	≥ 50	✓	✓			✓		Motion attenuation method + cycling
	Webcam	Ambient light	30	✓	✓					Ambient light intensity calibration
Jonathan et al. [77], [82]†	Cellphone	White LED	30	✓						Portable IPPG device
Anchan et al. [91]	Cellphone	Ambient light	25	✓						PWV assessment
Scully et al. [67]†	Cellphone	White LED	25	✓	✓	✓		✓		$\mathit{\text{SpO}}2=A-B\frac{\mathit{\text{AC}}(\mathit{\text{Red}})/\mathit{\text{DC}}(\mathit{\text{Red}})}{\mathit{\text{AC}}(\mathit{\text{Blue}})/\mathit{\text{DC}}(\mathit{\text{Blue}})}$, A, B could calibrate for each subject according to commercial oximeter
Huang et al. [106]†	Cellphone	White LED	–	✓					✓	Novel blood pressure transport model
McManus et al. [100]–[102]†	Cellphone	White LED	30	✓						Atrial fibrillation detection
Kamshilin et al. [76], [104]	CMOS	LED(530)	20				✓			High spatial resolution (1×1 pixel)
	CMOS	LED	30				✓			New opto-physiological model considering elastic deformations of capillary bed
Zhao et al. [94]	CMOS	Ambient light & LED(830nm)	15/80	✓	✓					YouTube video based vital sign monitoring + test on neonates & animals
Aarts et al. [93]	DC	Ambient light	15/30	✓						Clinical assessment in neonatal ICU
Stricker et al. [62]	CCD	Ambient light	30	✓						Integrated with a mobile service robot
Feng et al. [63]	Webcam	Fluorescent light	30	✓						Speeded-up robust features tracking + motion attenuation based on optical properties of skin
Estepp et al. [105], [107]	CCD	Bulbs	120	✓						Multiple cameras from different view angles for reducing motion artifacts + feasibility for PTT estimation
McDuff et al. [98] [115]	Multi-Band	Ambient light	30	✓	✓			✓		Five band cemera (RGBCO) + systolic and diastolic peaks detection
Kong et al. [96]	CCD	Ambient light	25	✓	✓	✓				Dual-camera IPPG + noncontact oxygen saturation
Rubinstein et al. [126], [127]	DC	Ambient light	>30	✓						Eulerian video magnification
de Haan et al. [99] [126]	CCD	Studio light	20	✓						Chrominance-based model Multiple cameras+skin classification
Shao et al. [116]	Webcam	Fluorescent lamp	–	✓	✓					Exhalation flow rate + PTT estimation
Hsu et al. [109]	VC	Ambient light	30	✓						Learning based HR detection

Note: BP = blood pressure; CCD = charge coupled device; CMOS = complimentary metal-oxide semiconductor; DC = digital camera; fs = sample frequency; HR = heart rate; HRV = heart rate variability; LED = light-emitting diode; PCA = principle component analysis; PTT = pulse transit time; PWV = pulse wave velocity; RR = respiration rate; SpO2 = blood oxygen saturation; TFR = time-frequency representation; VC = video camera

^†Running in a contact mode without pressing with additional force.

An IPPG system can provide an alternative functional imaging solution to clinical situations, which are currently addressed via magnetic imaging and LDI. This technique brings new insights by providing hemodynamic imaging and mapping capability. Indeed, IPPG has many similarities with conventional PPG because both operate on similar physiological systems and optical principles. For instance, IPPG also requires a light source and a photo detector to function, although the capability and the functionality of both are different. The basic concept of IPPG is to illuminate the specific tissue with a light source and then to measure the light leaving the tissue with an imaging sensor. A schematic setup of an IPPG system is shown in Fig. 4.

Fig. 4.

A schematic setup of an IPPG system which includes a light source and a camera. Physiological information (e.g., heart rate and perfusion) could be then extracted from the obtained images.

As a noninvasive imaging technique, IPPG enables remote large-area sensing and reduces the physical restrictions and cabling associated with patient monitoring. Specifically, IPPG avoids the deformation of the arterial wall, which is typically caused by the compression of spring-loaded clips in conventional contact PPG finger probes or by varying contact forces in handheld probes, and therefore provides more reliable signals for clinical applications. The use of an imaging sensor provides additional functionality with respect to a point sensor, particularly for motion artifact attenuation and region of interest (ROI) selection, both of which enable a robust and flexible noncontact PPG system.

B. IPPG instrumentation

The key part of an IPPG system is the camera, which collects the reflected or transmitted photons from the skin. The camera’s characteristics significantly influence the recorded images and, consequently, the physiological parameters. For instance, observations of measured intensity in conventional contact PPG sensors indicate that the AC component accounts for only a small proportion of the total intensity (approximately 1% of the DC level). Therefore, the camera should have a high sensitivity over the chosen light source spectral range and the flexibility to select variable readout speeds and exposure time for different applications. With the advances in semiconductor technology, we have observed a revolutionary development in IPPG instrumentation. Based upon the camera characteristics, IPPG systems can be categorized into three groups: high definition camera based IPPG systems, consumer level digital camera based IPPG systems and cellphone based IPPG systems.

1) High definition camera based IPPG system

In the pioneering exploratory studies, a CCD or complimentary metaloxide semiconductor (CMOS) chip based high-definition (HD) camera was utilized as the imaging sensor (Table II). A customized light source was typically coupled with the HD camera. Due to the historical emphasis of PPG on pulse oximetry monitoring and the need to sample the relatively deep microvascular tissue bed, a red/IR wavelength light source was repeatedly introduced in the HD camera based IPPG. To illuminate an area of skin tissue, the light intensity should be large and relatively uniform. Several arrangements have been proposed. Among them, two light source configurations have been widely used, namely, the ring-shape and the LED array. For instance, a dual wavelength array of LED light source has been adopted for pulse oximetry and heart rate measurements [88], [90], while in our previous study, a single-wavelength ringlight was introduced for heart rate monitoring [73]. Of note, the effect of motion artifact was not carefully addressed in these pioneering studies, and most of the IPPG systems still operated in a motionless condition.

2) Consumer level digital camera based IPPG system

Recently, several attempts have been made to obtain IPPG signals using simple consumer-level digital cameras with normal ambient light as the light source. Compared to the HD camera based IPPG system, these systems have the advantages of simplicity and low-cost. Takano and Ohta initially introduced this system and proved the feasibility of ambient light as the illumination source in an IPPG system [87]. Using a similar system configuration, Verkruysse and colleagues not only obtained plethysmographic signals but also found that, among the three recorded channels (red, green, &blue), the strongest plethysmographic signal was in the green channel, which is consistent with the fact that (oxy-) hemoglobin absorbs green light better than red [85]. In addition, they extended the IPPG application to characterize vascular skin lesions (e.g., port wine stains). It was soon realized that the a webcam was also capable of extracting IPPG signals. For instance, Poh and colleagues were the first to demonstrate the feasibility of a motion-tolerant webcam based IPPG system and show the ability of this approach to monitor the vital signs (i.e., heart rate) of multiple subjects simultaneously [80].

Regardless of how successful these systems has been in acquiring physiological parameters, e.g., heart/respiration rate and heart rate variability (HRV), a number of key questions still remain regarding IPPG systems which utilize ambient light as the illumination source, such as the practical issue of how variations in ambient light intensity could influence the performance of the system during physiological assessment. In one of our recent studies, we found an independent relationship between the ambient light intensity and the normalized plethysmographic signals [37]. This suggests that, under various external weather conditions, information contained in the AC component of the IPPG signal is not corrupted by the changes in ambient light intensity, thus providing solid support for the ambient light based IPPG technique as a method for practical, noncontact physiological assessment with clear applications in several domains, including telemedicine and homecare. To further verify the robustness of IPPG under various illumination conditions, further study is needed to assess its performance using artificial illumination in the evening condition.

3) Cellphone based IPPG system

Driven by the desire for simple personal healthcare evaluation techniques, research that moves away from the traditional, sophisticated IPPG system design and towards the everyday environment is becoming increasingly important. It is worth noting that the great effort on the cellphone based IPPG technique has witnessed fruitful achievements. For instance, Jonathan and Leahy successfully demonstrated the feasibility of a cellphone based IPPG technique in heart rate assessment [77], [82]. Specifically, in the cellphone based IPPG system, the embedded digital camera was configured in video imaging mode using the white LED (the flash light) as the light source (Fig. 5). The plethysmographic signal could then be extracted from a color movie of the volunteer’s finger covering both the LED and the camera. Given the huge population of smartphone users (the number of smartphone users worldwide reached 1 billion in 2012 [128]), such an application will provide potentially revolutionary approaches for personal healthcare. It is important to note that in these abovementioned studies, subjects were typically instructed to place their finger on the camera lens without pressing down with additional force [67]. More recently, cellphone based noncontact IPPG applications have been introduced and translated to commercial products (e.g., Philips Vital signs camera, Cardiio, What’s My Heart Rate, Cardio Buddy, etc.).

Fig. 5.

A cellphone based IPPG system with (a) showing position of the video unit comprising a white LED (WLED) and camera; (b) showing light PPG imaging of the index finger positioned to cover the cellphone video unit (modified from [77]).

C. Data preprocessing and IPPG signal extraction

Compared to the conventional contact PPG system, which presents the plethysmographic waveform in a straightforward way, the signals captured by IPPG are embedded in a sequence of raw image frames. To obtain the IPPG signals for further analysis, several (pre-)processing steps have to be carefully conducted, e.g., ROI selection and tracking, color channel selection, signal denoising, and physiological signal extraction. To the best of our knowledge, there is no standard processing procedure for all IPPG applications. We believe that an ideal procedure for IPPG signal processing should consider the application as well as maintain the balance between computational complexity and efficiency. In this section, we provide a brief review of the various methods for each step. Detailed information on formal procedures and introductions of the methods is available in the cited original research articles.

1) Spatial ROI selection and tracking

ROI selection and tracking, which aims to locate and/or adjust the ROI with the subject’s motion, is an indispensable first step to obtain reliable IPPG signals¹. In previous IPPG studies, the subject’s face was typically used as the ROI for pulse measurement. Therefore, it is very important to reliably select and track the facial region. Several approaches have been introduced for ROI selection and tracking. a) Manual selection: Once the videos of the human face are recorded, one can manually select the ROI, which typically has a fixed frame size for the whole video [37], [70], [73], [85], [96]. Although this is convenient for ROI selection, subjects are typically instructed to remain motionless during the recording [96]. b) Automatic detection based upon Viola-Jones (VJ) face detector: Poh and colleagues later introduced an automated ROI selection method based upon a face tracker (VJ face detector [129]) to detect faces within the video frames and localize the ROI for each video frame [75], [80]. Of note, this method has been trained mainly with frontal-face samples, which may lead to discontinuous face localization across subsequent video frames when the face position deviates from the frontal position. c) Automatic detection based upon speeded-up robust features (SURF): To further improve the accuracy, Feng et al., introduced an ROI selection method integrating the VJ face detector and a SURF detector [63]. Interestingly, instead of selecting the whole face, they selected two ROIs on the cheeks region (0.12×width by 0.15×height of the facial region detected by the VJ method) because of its higher signal-to-noise ratio compared to other skin regions on the face². d) Skin/non-skin pixel classification: More recently, Wang et al., developed a skin/non-skin pixel classification method [97], in which a “tracking-by-detection” with kernels (CSK) method [130] is utilized for face detection on various skin types (i.e., light, intermediate, and dark). The CSK method has been proven superior to others in tracking speed (hundreds of frames per second) and reliability in side-view face detection. Conceptually, every pixel of the recorded images can be considered as a discrete reflective PPG sensor. To extract parallel pulse signals from these spatial-redundant pixels and to concatenate the same skin pixels, they further introduced a skin/non-skin pixel classification method based upon the one class support vector machine technique.

2) Spatial information synthesis

According to Wang et al., [97], once the ROI has been successfully selected for each frame, there are two alternatives to use the spatially pruned IPPG pixel sensors. a) Spatial averaging: Verkruysse and colleagues introduced a spatial ROI averaging approach [85]. Although it compromises the spatial resolution, this most widely used approach has shown its ability to significantly improve the SNR in the presence of minor motion artifacts. Of note, when assessing blood perfusion information, there is a trade-off in the sub-window size and the spatial resolution of the perfusion. Based upon previous research results and our previous experience, a range of 8 – 16 pixels is practically suitable for maintaining the balance between the spatial resolution and SNR [73], [85], [89]. b) Spatial concatenation: To further reduce the effect of motion residual errors, Wang and colleagues introduced a spatial concatenation approach [97]. Briefly, independent pulse signals from the sensors within the ROI are extracted and sorted based upon their deviations from the mean pulse. Those sensors with similar rank orders (similar quality of pulse estimation) are concatenated. Therefore, the pulse signals and noise within the whole ROI could be further separated.

3) Color channel selection

Generally, a digital camera sensor has red, green, and blue (RGB) color channels. As mentioned previously, the optical absorption of hemoglobin varies across the light spectrum, and, therefore, different approaches have been proposed for color channel selection. a) Green channel: In 2008, Verkruysse et al., found that in an ambient light condition, the PPG signals have different relative strengths in the three color channels of an RGB camera, i.e., although the red and blue channels also contact the pulse signal, the green channel has stronger pulse amplitude and decreased noise [85]. Since then, several studies of pulse signal dynamics have employed the green channel [67], [77], [82], [121]. b) RGB channels: Based upon this finding, Poh et al., proposed a linear combination of RGB channels using independent component analysis (ICA). By maximizing the non-Gaussianity of ICA output components, the linear combination coefficients could be estimated [75], [80]. Because pulse signals in different color channels undergo a similar motion modulation, the inclusion of three channels and the ICA approach could further attenuate the motion artifact and improve the estimation accuracy of pulse signals. Lewandowska et al., varied this concept and defined three independent linear combinations of the color channels using principle component analysis (PCA) [70]. Both methods could be summarized as: S = c₁ R + c₂ G + c₃ B, where S is the pulse signal. However, with both blind source separation (BSS) techniques, the component that carries the pulse signal is a priori unknown. de Hann and Jeanne later introduced a chrominance-based method and showed that it is superior in SNR and motion robustness [99]. Briefly, they built two orthogonal chrominance signals from three color channels via linear combination: X and Y. The variations due to the blood volume changes in the skin will likely be different in each channel while the motion affects both chrominance signals identically. Thus ,the pulse signal could be estimated by S = X_s − αY_s, where α = σ(X_s)/σ(Y_s) (σ(X_s) is the standard deviation of X_s). c) RGBCO channels: For BSS methods, such as ICA and PCA, the number of source signals cannot exceed the number of observations. More recently, McDuff and colleagues moved a step forward through employing a novel digital single-lens reflex camera with five color channels (red, green, blue, cyan, and orange, (RGBCO)) [98]. Considering that there may be many sources of noise, such a color selection would allow for more observations, which would provide higher flexibility in the number of source signals.

4) Denoising

Prior to physiological signal extraction, a filtering step was typically employed to further reduce the noise effect. Essentially, the pulse rate of a healthy subject falls within the frequency range of 40 – 240 beats per minute (bpm). The parts of the signal that are not in this range can be safely blocked. Several filter designs have been introduced and successfully implemented for IPPG signal denoising, including a moving average filter [75], a band-pass filter [70], [85], an adaptive band-pass filter [97], and wavelet denoising [108]. Many books have been written on digital signal processing, and, therefore, a detailed description of the filter design will not be presented in this paper.

5) Physiological signal extraction

After signal denoising, there are two approaches for physiological signal extraction. a) Heuristic methods: McDuff et al., introduced a time-domain pulse peak detection method [98], [115] based upon local maximum detection in a moving window, e.g., within the moving window, if the signal maximum is greater than that in the previous window, the next window will be considered; if the maximum within the window is less than that in the previous window, then the previous maximum will be selected as a peak and the process will repeat [98]. However, this method only leverages the amplitude properties of pulse peaks and ignores the temporal regularity of pulse waveforms and therefore is more vulnerable to motion-induced residual noises. Another widely used heuristic method is transferring the pulse signal into frequency domain, which can straightforwardly provide fundamental physiological information, such as heart and respiration rates. Considering that a robust pulse signal is a periodic signal, a dominant frequency peak corresponding to the heart rate could be detected within the range of pulse frequency bands (40 – 240 bpm). To obtain a potentially more revealing picture of the temporal localization of a signal’s spectral components, one can use the joint time-frequency analysis [73], [90]. b) Learning-based methods: More recently, Hsu and colleagues introduced a support-vector regression technique for pulse rate detection [109]. Briefly, three frequency domain features were initially estimated, including the spectrum amplitude for the spatially averaged R, G, B channels, the spectrum amplitude for the ICA processed independent components, and the spectrum amplitude for the chrominance-based pulse signals. Through multiple feature fusion and support-vector regression, they achieved a relative higher estimation accuracy compared to the heuristic methods. Based upon an adaptive hidden Markov model, Fan et al. developed a probabilistic approach named BayesHeart³ for heart rate monitoring [110]. Through multiple comparisons with other state-of-the-art algorithms, they found that BayesHeart outperformed existing algorithms in both accuracy and processing speed for noisy, intermittent signals.

V. Recent advances in IPPG

A. Scope of recent IPPG activities

Although the research of remote IPPG techniques is relatively recent, it has rapidly spread from the laboratory to everyday use. In this section, we will discuss the scope of current IPPG research activities.

1) System development

In the past decade, a significant number of IPPG studies have been published, and different IPPG systems have been developed for various biomedical and clinical applications (Table II). In the pioneering exploratory setups, a major effort has focused on light source design as well as control circuit development for synchronization between the light source and the camera. Specifically, the wavelength of the light source should be suited to the quantum efficiency of the camera and should maintain good tissue optical properties. Uniformity is another feature that can verify the effectiveness of the light source design, and it will influence the evaluation and interpretation of the perfusion results. The complexity of the system design has, to some degree, constrained the evolution of IPPG techniques. Recently, several attempts have been made to develop ambient light based IPPG system as well as cellphone based IPPG systems. Such systems, typically coupled with a simple light source (ambient light or embedded flash light in cellphones), soon attract intensive research interest. Actually, we could observe a shift in research interest from sophisticated IPPG system to plain and portable IPPG setups.

It is also noteworthy that Estepp and colleagues introduced another possible solution to further mitigate the unavoidable motion artifacts. Briefly, they developed a multi-imager array, which includes nine synchronized digital cameras coupling with 10 full-spectrum bulbs [105], [107]. Through increasing the dimensionality of the imager channel space prior to the blind source separation, they achieved satisfactory HR readings compared to the motionless fingertip references, showing a promising trend for the development and transition of IPPG for everyday use.

2) Data processing

Along with system development, the IPPG signal processing technique has evolved dramatically. a) Motion artifact attenuation: Given its remote monitoring configuration, IPPG lacks resistance to motion-induced signal corruption. Researchers from different groups have proposed various motion attenuation methods, such as image registration [73], spatial averaging [85], a radiance model [63], ROI tracking [63], [75], [80], [97], and blind source separation [70], [73], [75], [80]. Through utilizing one or a combination would significantly enhance the accuracy of the physiological assessments. b) Signal processing: Fourier transformation is widely applied in conventional PPG/IPPG signal processing because it can provide fundamental physiological information, such as heart rate and respiration rate (the peak frequency at certain spectrum regions). However, it assumes that signals are in a steady state when physiological signals are transient in nature. To obtain a potentially more revealing picture of the temporal localization of a signal’s spectral components, several time-frequency analysis methods have been successfully adopted for IPPG signal processing [73], [85], [90]. In addition, Kamshilin and colleagues have introduced a patent pending method for monitoring blood perfusion [76]. This method shows a great advantage in the high resolution visualization of the blood pulsative flow using video recordings, which may provide supportive cardiovascular information to assist in the diagnosis of various diseases related to blood perfusion. More recently, several learning-based physiological extraction methods have been developed and proven to be superior in their processing speed and estimation accuracy [109], [110]. c) Application area extension: The various biomedical/clinical applications of IPPG have been extensively researched. As a substitute for conventional contact PPG techniques, IPPG will be able to assess almost all the physiological measurements that contact PPG currently provides. Among them, heart/respiration rate and blood perfusion were the first attempts to evaluate the feasibility of the IPPG technique. With recent developments in signal processing, current IPPG systems normally can provide multiple physiological parameters with a single recording. Several recent achievements and applications of IPPG techniques are recounted in the following section. d) Physiological signal visualization: Based upon Eulerian video magnification, an important study by Wu et al., introduced a novel visualization technique for real-time detection of physiological signals in videos of human subjects with ambient light [126]. Through detecting and magnifying the subtle color variations (0.5 intensity units in an 8-bit scale) of human skin due to blood flow, HR could be reliably extracted from a standard RGB video. Of particular note, this study inspired many new applications of IPPG, e.g., vital signs monitoring in the neonatal ICU. Details of this method⁴ and its implementation in detecting vital signs could be found in [127].

B. Progress of IPPG applications

In this section, some typical applications of IPPG are drawn from the selected studies (Table II). Of note, the applications listed here may not cover all aspects of current IPPG applications; however, they provide some insights into the progress of IPPG applications. More importantly, the applications of IPPG discussed in the current work are not intended to diagnose, monitor, alleviate, prevent or treat any medical/clinical conditions or diseases.

1) Heart/respiration rate

The heart and respiration rate are simple yet clinically important physiological measurements. They are routine assessments in a wide range of clinical settings, i.e., hospital-based and ambulatory patient monitoring. As seen in Table II, HR and RR were successfully assessed via different IPPG system setups. Particularly, the achievement of cellphone-based IPPG technique has already attracted significant attention. Considering that the current lifestyle is also a mobile lifestyle [131], it is possible to introduce mobile healthcare, which we believe will lead to a more healthy lifestyle in the near future. In addition, cellphone-based healthcare technology could easily extend beyond simply monitoring and measuring and could also be used to relay information to medical professionals. It is noteworthy that one smartphone app, “Instant Heart Rate” has attracted more than 25 million customers since 2010 [132]. In June 2015, using key words “heart rate monitor”, we found 173 relevant apps in the App store.

2) Blood oxygen saturation (SpO2)

The same principle, utilized in contact pulse oximetry sensors, i.e., that hemoglobin (Hb) and oxyhemoglobin (HbO2) have different absorption coefficients in the red and IR range, could be employed for remote blood oxygen saturation measurements using IPPG. In 2005, Wieringa and colleagues first introduced the idea of contactless imaging of arterial oxygen saturation distribution within tissue (SpO2 camera) [89]. In that feasibility study, heart rate and respiration rate were successfully extracted from plethysmographic signals simultaneously recorded at three wavelengths (λ = 660, 810, & 940 nm), therefore indicating the potential of IPPG for noncontact SpO2 assessment. Immediately after this study, Humphreys et al. presented a transmission IPPG system and proved the practicality of IPPG in appraising SpO2 [88], [133]. The ability to remotely assess SpO2 information could benefit research areas such as intensive care and sleep studies. Of note, a high definition camera coupled with a customized light source was employed in these pioneering studies. Scully et al. later showed that multiple physiological parameters, including SpO2, could be assessed using a cellphone with certain calibrations [67]. More recently, the embedded app “S Health” in the Sumsung Galaxy S6 also demonstrated the capacity to monitor SpO2 [134].

3) Pulse rate variability (PRV)

Fluctuations in the interval between heartbeats, attributed to continuous changes in the sympathetic-parasympathetic balance of the autonomic nervous system (ANS), are a sign of healthy cardiac function and a valuable tool for investigating the neural control of the heart. The quantification of these fluctuations to determine heart rate variability (HRV) has proven its effectiveness in various research and clinical studies (for a review, see [135], [136]). A number of recent studies have shown that the functional characteristics of PRV are comparable to those of HRV [137]– [139], where the former can be easily acquired from PPG. Recently, Poh and colleagues introduced a remote PRV assessment technique based on a webcam IPPG system [75]. This feasibility study was further supported by one of our recent studies, which verified the reliability of IPPG in PRV assessments [64]. Furthermore, recent advances in hardware development have provided a sufficient initial sample rate (including, but not limited to, the iPhone 6: 1080p HD at 60fps; the Samsung Galaxy S6: 1080p HD at 60fps; the LG G4: 1080p HD at 30fps; and the HTC One M9: 1080p at 60fps) for further investigation of the feasibility of smart-phone based PRV assessment. More importantly, the necessity for an interpolation approach when a low sample frequency IPPG system is employed for assessing PRV information has been addressed [64]. The last two decades have witnessed the recognition of a significant relationship between the ANS and cardiovascular mortality, including sudden cardiac death. One study of 763 elderly subjects reported on the relationship between HRV measurements and all-cause mortality during a four-year follow-up period [140]. However, the normal HRV standards for various age and gender subsets are still missing, and larger prospective population studies with longitudinal follow-up are needed [141]. Such studies may be significantly facilitated by IPPG technology.

4) Blood perfusion

Compared to conventional PPG techniques, IPPG holds a significant advantage in offering detailed spatial information from multiple sites simultaneously, thus allowing the derivation and mapping of physiological parameters. Verkruysse and colleagues demonstrated the capability of IPPG in the characterization of vascular skin lesions (e.g., port wine stains, a syndrome of cutaneous capillary malformations) [85]. Other areas that stand to benefit from blood perfusion monitoring are skin damage assessment and healing evaluation. For instance, high perfusion corresponds to superficial dermal damage that may require only clinical dressing and conservative management, while low perfusion areas might require surgical treatment [142]. Hulsbusch and Blazek demonstrated the practicability of IPPG for wound assessment [90] and the evaluation of allergic skin reactions and antihistamines inhibition [92]. More recently, Aarts and colleagues demonstrated, for the first time, the feasibility of IPPG in the neonatal intensive care unit for vital sign monitoring [93]. Interestingly, they found that PPG signal amplitudes in Staphylococcal Scalded Skin Syndrome affected skin were larger than in the unaffected skin area. In addition, Kamshilin and colleagues introduced a novel method for extracting high spatial resolution photoplethysmographic images [76]. Based on the lock-in amplification of the recorded video frames, the method could achieve a resolution of 1 × 1 pixel. A better understanding of hemodynamics is of great interest to improve the diagnosis of various vascular malformations and skin damage as well as to evaluate the therapeutic outcomes, which could be easily achieved via IPPG technique.

5) Pulse transit time/pulse wave velocity

The assessment of arterial stiffness is clinically important due to its strong connection with cardiovascular morbidity and mortality [143], [144]. An objective assessment of vascular aging could also provide valuable information about hypertension, which is a risk factor for stroke and heart disease [145]. From the available methods to assess arterial stiffness, pulse wave velocity (PWV) has emerged as the gold standard because of its relative ease in determination and perceived reliability. Pulse transit time (PTT), the time it takes a pulse wave to travel between arterial sites, is inversely related to the PWV [146] and is more convenient to obtain. Several feasibility studies have proven the effectiveness of conventional contact PPG techniques in assessing PTT information [147]–[149]. Recently, Anchan reported some preliminary results of PWV information extracted from a cellphone based IPPG system [91]. Shao et al., later presented a digital camera based PTT estimation method through measuring the time delay between the simultaneously recorded pulse signals from the mouth area and the palm area [116]. However, a number of key questions still remain in these interesting studies. For instance, as a time domain estimation, PTT measurements need a relatively fast sample frequency for accurate time-domain resolution (i.e., > 200 Hz). The sample rate for a mobile phone camera could only reach approximately 30 fps and less than 80 fps for the digital camera in [116], which makes the results less convincing. The obtained PWV measurements therefore required further verification against the standard readings.

6) Blood pressure

For people suffering from chronic cardiovascular diseases, blood pressure is one of the most important pieces of information for doctors to conduct a diagnosis. The traditional non-invasive digital blood pressure assessment system typically includes an inflatable cuff, a signal processing unit, a controller and a display. Although generally successful, several key drawbacks limit its use in certain clinical situations, e.g., continuous blood pressure measurement [150]. With recent advances in remote sensing, it has been realized that blood pressure could be assessed in a “cuff-less” manner [150]–[153]. Most recently, Huang and colleagues developed a blood pressure transport theory for noninvasively monitoring systolic and diastolic blood pressures (SBPs and DBPs). Through simple calibration, this method could be implemented with any IPPG systems (e.g., smartphone-based IPPG) for blood pressure assessment [106]. Compared to the commercial Omron sphygmomanometer, this method achieved satisfactory levels of accuracy in determining both SBPs and DBPs based upon the standards (mean < ±5 mmHg, S.D. <8 mmHg) set by the Association for the Advancement of Medical Instrumentation. We believe that this method move a step forward for blood pressure monitoring.

7) Atrial fibrillation

Atrial fibrillation (AF) is the most commonly sustained arrhythmia and affects approximately 5 million Americans [154]. Its strong association with an increased risk for heart failure and stroke makes it an important clinical indicator [155]. However, compared to the well-established treatment strategies, accurate AF detection and monitoring are still major challenges for clinicians and researchers due to its paroxysmal nature, particularly in the early stages [156]. The gold-standard diagnostic test for AF is electrocardiogram (ECG). Several recent studies have shown that more frequent monitoring can improve AF detection [157]. Given that ECG recording is burdensome and costly, a more effective paroxysmal AF detection technique that is readily accessible, inexpensive, and simple to operate is needed to increase the frequency of monitoring [158]. Most recently, McManus and colleagues introduced a cellphone-based real-time AF detection technique and assessed its performance in distinguishing an irregular pulse from AF using pulse waveforms obtained during a normal sinus rhythm [100]– [102]. Considering the prevalent nature of smartphones and acceptance by the general public for monitoring their health, we think that this technique could lead to better acceptance and more widespread use of out-of-hospital arrhythmia detection.

8) Systolic and diastolic peaks

The pulse shape characteristics can provide valuable diagnostic information about the cardiovascular system [159]. For instance, independent studies have shown that the point of inflection (the time delay between the systolic and diastolic peaks (see Fig. 2)) decreases in older individuals and in those with various cardiovascular diseases as a consequence of increased large artery stiffness and increased PWV of pressure waves in the aorta and large arteries [160]–[162]. Based upon a PPG waveform recorded from a finger, Millasseau and colleagues introduced an objective index (units m/s) to assess artery stiffness, which is calculated from the body height divided by the time delay between the systolic and diastolic peaks [160]. Of note, to achieve better estimation accuracy, a high sampling frequency is typically needed (i.e., >100Hz). More recently, McDuff and colleagues showed that the pulse waveform morphology (e.g., systolic and diastolic peaks) can be reliably and accurately recovered through integrating IPPG technique with advanced signal processing methods, thence providing new insights into arterial stiffness assessment [115].

VI. Prospective of IPPG research

A. Standardized system design for clinical applications

Eventually, the effectiveness and usefulness of IPPG technology will need to be verified with clinical applications. As a recently introduced technique, the IPPG system design lacks an established standard, which is an area that requires additional effort. Among those existing IPPG setups, some adopt ambient light as the light source. Though convenient and successful in some physiological assessments, ambient light is arguably not recommended for clinical applications. For instance, unstable light intensity during the recording would pose a significant engineering issue and even prevent successful measurements [93]. To become a clinically mature technique, several aspects of IPPG deserve attention: 1) The selection of optimum wavelengths is a matter of importance in the design of IPPGs for clinical applications, e.g., a minimum of two wavelengths together with a light source control circuit are necessary for the acquisition of the oxygen saturation information. 2) More sophisticated light source designs should be employed to achieve the high uniformity of the light source while maintaining light intensity. This approach will significantly improve the measurement accuracy of blood perfusion. 3) Integration with other imaging technologies could provide more comprehensive physiological assessments. In our recent study, we developed a multi-modal imaging technique that includes an IPPG system and a laser speckle contrast imaging system [163]. In an animal model, we demonstrated its usefulness in real-time monitoring of the cortical cerebral metabolic rate of oxygen.

B. Engineering model

The quantification of the optical propagation in tissue is a question of growing concern for biomedical and clinical applications. Optical bio-monitoring modalities can be described in general terms as systems that modify a specific light intensity, where the transfer function of the system is determined by a number of anatomical and physiological parameters. Compared to the major research activities in the application area extension, only a few studies have been published pertaining to an efficient opto-physiological model of IPPG technique [60], [164]. Considering the complexity of the interaction of light with biological tissues and the geometry differences of skin tissue at various anatomical regions, the slow development of the modelling is understandable. A proper opto-physiological model requires information on the light propagation in the skin related to the optical properties of the individual skin layers. This involves a solution for the radiation transfer function in a multilayer skin model that represents skin geometry, as well as experimental methods to determine the optical properties of each skin layer [165]. The Monte Carlo radiation transport techniques, which are based on the stochastic nature of radiation interactions, provide a practical solution for the radiation transfer function [164].

Another possible and complementary route to build an efficient opto-physiological model is through interpreting experimental results. Most recently, based upon observations of IPPG experiments, Kamshilin and colleagues introduced a new model in which pulse oscillations of the arterial transmural pressure deform the connective-tissue components of the dermis. These deformations may include changes in the orientation or structure of the connective tissue. Consequently, elastic deformations of the dermis will result in variations of the back-reflected light intensity due to both local changes in scattering and the absorption coefficient of the light [104].

We believe that with a better understanding of the existing engineering model for conventional contact PPG [166], an increasing availability and accuracy of tissue optical properties in the literature [167] and the use of numerical solutions of light propagation in human tissue, an optimal opto-physiological model for IPPG could be achieved. Such a model would also provide new insights for revealing the origins of different components of the PPG signal.

C. Personal health assistant and telemedicine

Telemedicine involves the delivery of healthcare and related information over long distances by combining biomedical signals with information technology and means of communication [168]. Due to rapid developments in information technology, the last decade has witnessed the advent of mobile telemedicine, in which physiological assessments are delivered anytime from anywhere via the latest generation of smart-phones. The measured physiological parameters can be sent to a remote healthcare server through wireless communication network for further clinical diagnosis. Recently introduced smartphone-based IPPG systems are revolutionizing the efficiency and convenience of personal healthcare with simple system requirements (without additional hardware setup) and the capability to simultaneously monitor multiple vital signs (Table II). Moreover, taking into consideration the increasing needs of the aging society for medical assistance, the emerging field of the low-cost IPPG systems seems to be a promising solution to enable elderly people to monitor their health conditions on a daily basis independently. With advancements in hardware, e.g., on-board computing capability, large memories, large touch screens and open operating systems, a multi-functionality and compact ubiquitous personal health monitoring system would be feasible and would dramatically accelerate the promotion of the preventive healthcare model.

VII. Summary

This review has introduced the nascent technique of imaging photoplethysmography (IPPG) and has demonstrated its great potential for use in a wide range of clinical assessments, such as homecare, telemedicine and personal healthcare. The emerging field of IPPG has offered some of the first quantitative insights into an effective and comprehensive interpretation of remote physiological assessments and has significantly contributed to the resurgence of research interest in PPG techniques. We started with a brief introduction of the conventional contact PPG technique, its principles of implementation and its limitations. From a biomedical engineering research perspective, we surveyed a large body of literature pertaining to IPPG, tried our best to cover those most representative IPPG studies, provided insights into the fundamental basis of IPPG techniques, and highlighted important considerations for its various applications. We hope that this paper will help those who are interested in using or developing IPPG techniques. While obstacles to developing a clinically useful IPPG device remain, the allure of remote physiological assessments and the desire for personal healthcare are strong. As a rapidly expanding field, the development of IPPG is continuously growing and evolving to embrace new technologies. We believe that the extensive acceptance of IPPG is irresistibly happening and will dramatically change our lifestyle in the near future.

Biographies

Yu Sun (M’12) received the B.Eng. degree in biomedical engineering from the Nanjing University of Aeronautics and Astronautics, Nanjing, China in 2005, and Ph.D. degree in electronic, electrical, and system engineering from Loughborough University, Leicestershire, UK in 2011 and joint-Ph.D. degree in biomedical engineering from Shanghai Jiao Tong University, Shanghai, China in 2012. Since 2012, he has been a Postdoctoral Research Fellow and now a senior research fellow in the Singapore Institute for Neurotechnology (SINAPSE) at the National University of Singapore. His current research interests are in the area of biomedical signal processing, imaging photoplethysmography, and functional/structural neuroimaging processing with particular focus on human connectome. Dr. Sun is a recipient of the Best Paper Award in the 2013 IEEE International Neurotechnology consortium Workshop and the Best Poster Award in the 2013 IEEE Life Sciences Grand Challenges Conference.

Nitish Thakor (S’78, M’81, SM’89, F’97) is a Professor of Biomedical Engineering, Electrical and Computer Engineering, and Neurology at Johns Hopkins University, Baltimore, MD, and directs the Laboratory for Neuroengineering. He has been appointed as the Provost Professor, National University of Singapore, and now leads the Singapore Institute for Neurotechnology (SINAPSE) Institute, focused on neurotechnology research and development. His technical expertise is in the areas of neural diagnostic instrumentation, neural microsystem, neural signal processing, optical imaging of the nervous system, rehabilitation, neural control of prosthesis and brain machine interface. He is the Director of a Neuroengineering Training program funded by the National Institute of Health. He has authored 250 refereed journal papers, generated 11 patents, co-founded four companies, and carries out research funded mainly by the NIH, NSF and DARPA. He was the Editor-in-Chief of IEEE Transactions On Neural And Rehabilitation Engineering (2005–2011) and is currently the Editor-in-Chief of Medical and Biological Engineering and Computing journal. Dr. Thakor is a recipient of a Research Career Development Award from the National Institutes of Health and a Presidential Young Investigator Award from the National Science Foundation. He is a Fellow of IEEE, the American Institute of Medical and Biological Engineering, International Federation of Medical and Biological Engineering, and Founding Fellow of the Biomedical Engineering Society, Technical Achievement Award from IEEE and Distinguished Alumnus award from Indian Institute of Technology, Bombay and University of Wisconsin, Madison.