Comparison of blood volume pulse and skin conductance responses to mental and affective stimuli at different anatomical sites

Abstract

Measurements of blood volume pulse (BVP) and skin conductance are commonly used as indications of psychological arousal in affective computing and human–machine interfaces. To date, palmar surfaces remain the primary site for these measurements. Placement of sensors on palmar surfaces, however, is undesirable when recordings are fraught with motion and pressure artifacts. These artifacts are frequent when the human participant has involuntary movements as in hyperkinetic cerebral palsy. This motivates the use of alternative measurement sites. The present study examined the correlation between measurements of blood volume pulse and skin conductance obtained from three different sites on the body (fingers, toes and ear for BVP; fingers, toes and arch of the foot for skin conductance) in response to cognitive and affective stimuli. The results of this pilot study indicated significant inter-site correlation among signal features derived from different sites, with the exception of BVP amplitude, the number of electrodermal reactions and the slope of the electrodermal activity response. We attribute these differences in part to inter-site discrepancies in local skin conditions, such as skin temperature. Despite these differences, significant changes from baseline were present in the responses to the cognitive and affective stimuli at non-palmar sites, suggesting that these sites may provide viable signal measurements for use in affective computing and human–machine interface applications.

1. Introduction

The autonomic nervous system (ANS) connects the brain stem and spinal cord with internal organs and regulates physiological parameters such as blood pressure, heart and breathing rates, perspiration level and body temperature. In general, activity of the ANS is involuntarily modulated by external stimuli and affective states. Given this emotional-physiological concordance, physiological measures of ANS activity are commonly used in the fields of psychophysiology and affective computing (Picard 2003) as indicators of psychological arousal. These indicators can also serve as an access pathway—an alternative means for the expression of functional intent in individuals who are unable to use physical movement or speech. For example, non-physical control of a binary switch can be accomplished by voluntary modulation of ANS activity through relaxation and mental activity (Blain et al 2008).

ANS-related physiological measures most commonly employed in the above applications include heart rate (HR) and electrodermal activity (EDA). Sympathetic and parasympathetic pathways of the ANS serve to increase and decrease HR, respectively. HR can be estimated using the electrocardiogram (ECG) or blood volume pulse (BVP). BVP is related to the changes in the volume of blood in vessels and is measured by a single, non-invasive optical sensor which senses changes in light absorption density of the skin and tissue when illuminated (photoplethysmography) (Allen 2007).

EDA is a measure of the changes in the skin’s ability to conduct electricity resulting from variations in sweat production and fluid concentration in sweat ducts. Since eccrine sweat glands have sympathetic cholinergic innervations (Dawson et al 2000), the amount of sweat in the skin and, consequently, skin conductivity change with ANS activity (Poh et al 2010). EDA can be measured as either the skin’s electrical resistance or as its electrical conductance. The latter is the preferred mode of measurement because the number of sweat glands and their secretion is linearly related to skin conductance (Dawson et al 2000).

Overwhelmingly, BVP and EDA measures are obtained from the finger or hand (Schmidt and Walach 2000, Kreibig et al 2007, Picard et al 2001, Min et al 2005, Blain et al 2008). However, physical factors such as skin abrasion and amputation may hinder the use of the hand as a viable site for obtaining physiological measurements. In addition, readings of physiological signals are highly sensitive to movement and pressure applied to the measurement site (Poh et al 2010). This complicates harnessing of physiological measurements in activities that involve the hands. These artifacts are of particular concern in access applications for individuals with disabilities where spastic and involuntary movements are frequent and motivate the use of alternative measurement sites in access technology applications.

A number of studies have previously investigated alternative measurement sites for BVP and EDA, mainly for clinical applications. Since the fingers, toes and the ear lobes have a high density of superficial vasculature, they are often used clinically as alternatives to the fingers as BVP measurement sites (Allen 2007). Recently, Fletcher et al (2010) found that BVP readings from the forearm were correlated with those obtained through electrocardiography in the absence of hand motion. However, differences in BVP waveforms among the hand, toe and ear sites have been reported in the literature (Allen and Murray 2003). The effect of these differences on signal features employed by access technologies has not been studied to date.

EDA reflects the activity of eccrine sweat glands. The distribution of these glands throughout the body is non-uniform with the highest density of these glands found on the forehead, palmar surface of the hand and plantar surface of the foot. Winterhalter et al (2008) found that while skin impedance (analogous to resistance) at both palmar and plantar sites increased significantly in response to various stimuli, the magnitude of the response was significantly greater at the palmar site, and the latency of the response was shorter for this site. Rickles Jr and Day (2007) qualitatively compared changes in skin resistance at 14 non-palmar body sites to those at a palmar (control) site. They found that skin resistance at the plantar and hypomalleolar sites most closely paralleled that of the palmar site during moderate psychological arousal. More recently, Poh et al (2010) showed that skin conductivity readings from the distal forearm were correlated with palmar readings under physical, cognitive and emotional stressors. However, their results indicated only a moderate correlation between the two sites during cognitive activity. Despite reports of correlation between various measurement sites, Toyokura (1999) found differences in waveform patterns of the sympathetic skin response measured from palmar and plantar sites. They suggested that although the sympathetic skin response is centrally controlled, the response measured at various sites is affected by local factors such as skin temperature and epidermal conditions.

Access technology applications often rely on ANS signal features to classify affective states or mental activity. The collective findings of the literature reviewed above indicate waveform differences between non-palmar and palmar sites for both BVP and EDA signals, but it is unclear whether these differences affect detection of psychological arousal. To address this gap, the present study addressed the following two questions:

1. Are significant changes in BVP and EDA signal characteristics observed in response to cognitive and mental stressors at non-palmar measurement sites?

2. Are signal characteristics relevant to access-technology applications correlated among palmar and non-palmar sites under cognitive and mental stressors?

2. Methods

2.1. Participants

A sample of ten participants with no known medical conditions (age 24.9 ± 2.3 years; 6 females) was recruited from Holland Bloorview Kids Rehabilitation Hospital in Toronto, Canada. The institutional research ethics board approved the study, and all participants provided informed written consent.

2.2. Measurement equipment

BVP and skin conductance signals were measured from three sites simultaneously using FDA-approved sensors and encoders (Flexcomp Infiniti, Thought Technologies Ltd). For BVP, three photoplethysmography sensors (Thought Technologies, model SA9308M) were secured, one each, to the palmar and plantar surfaces of the distal phalanges of the first digits of the non-dominant hand and foot, respectively, and to the superior portion of the medial surface of the auricle. Three skin conductance recordings were obtained using three pairs of 10 mm diameter Ag–AgCl electrodes which were secured, one pair each, to the plantar surface of the distal phalanx of the second and third digits of the non-dominant foot, to the palmar surface of the second phalanges of the second and third digits of the non-dominant hand and to the arch of the foot. To gauge local thermal conditions, we also measured skin temperature by fastening three thermistors, one each, to the palmar surface of the distal phalanx of the fourth digit of the non-dominant hand, to the plantar surface of the distal phalanx of the first digit of the non-dominant foot (proximal to the PPG sensor) and to the lateral surface of the lobule adjacent to the antitragus. All sensors were attached using velcro bands and secured with breathable tape.

BVP and skin conductance signals were sampled at frequencies of 2048 and 256 Hz, respectively, as required by the hardware. The signals were recorded to a laptop using custom software for subsequent off-line analysis.

2.3. Procedure

Participants took part in one data collection session consisting of three 5 min trials. For all trials, participants were positioned in a comfortable seated position in a dimly lit room and were asked to remain as still as possible. For the first trial (baseline), participants were instructed to relax and clear their mind. In the second and third trials, participants alternated between 1 min bouts of mental relaxation (baseline) and activity, as shown in figure 1.

Figure 1.

Structure of trials.

Mental activity trials were comprised of cognitive and affective tasks. For the cognitive task, participants were instructed to continuously subtract 7 from a machine-selected random initial value. After each minute of mental arithmetic, participants were prompted to stop subtracting, report the number they had reached and continue with mental relaxation. For the affective task, participants were asked to recite their favorite piece of music mentally. During all three trials, participants received audio feedback from the EDA measurement on their hand. This was done by directly using the EDA signal amplitude to generate an audio signal.

2.4. Data analysis

BVP signals, sampled at 2048 Hz, were down-sampled to 256 Hz prior to analysis. To avoid aliasing during the down-sampling process, signals were passed through a low-pass Butterworth filter, with cutoff frequency of 110 Hz. Since responses to stimuli occur with different latencies at different sites (Winterhalter et al 2008), we computed the lags among the three sites using the method of maximum cross-correlation and time-aligned both BVP and skin conductance signals.

Feature extraction and classification is a vital step in numerous practical applications of physiological signals, such as emotion identification (Picard et al 2001) and decoding of mental activity (Blain et al 2008). Thus, correlation between signal features from various bodily sites was calculated. For BVP, we extracted the following features that were previously shown to be correlated with affective states and mental activity (Picard et al 2001, Kreibig et al 2007).

• Mean BVP: this feature is computed as the average value of the BVP signal.

• Range BVP: this feature corresponds to the range of the BVP signal.

• HR: this feature is the rate of heart beats, computed as the inverse of the distance between successive peaks of the BVP signal in units of beats min⁻¹. This feature was computed using a rule-based method to detect the peaks of the BVP signal. In particular, each point in the BVP signal was compared to its neighboring values. A point was deemed a peak when its value was greater than that of neighboring points within 0.4 s (this corresponds to a maximum of 150 beats min⁻¹).

• Heart rate variation (HRV): this feature corresponds to the standard deviation of the HR.

Figure 2 illustrates examples of features extracted from BVP signals.

Figure 2.

Example features extracted from BVP signals.

The overall skin conductance response (SCR) typically exhibits a gradual decrease during relaxation and sharp increase in response to novel stimuli (also known as specific skin conductance response (SCR)) (Dawson et al 2000). This response consists of two components: (1) a tonic component which is the absolute level of conductance at a given level (skin conductance level (SCL)) and (2) a phasic component manifesting as increases in conductance (SCR). Based on this information and previous reports of features modulated with affective states and mental activity (Picard et al 2001, Blain et al 2008), we computed three features from the skin conductance signals.

• Mean amplitude: this feature corresponds to the sample mean of the raw skin conductance signal (measured in μS).

• Slope: this feature is related to the average rate of absolute change and is calculated as the mean of the absolute value of the first difference of the skin conductance signal (measured in μS s⁻¹).

• Number of electrodermal reactions (EDR): this feature relates to the number of SCRs in the skin conductance signal and is computed as the number of increases in the signal that are greater than 0.05 μS in amplitude over an interval of 5 s or less.

Figure 3 illustrates examples of features extracted from EDA signals.

Figure 3.

Example features extracted from EDA signals.

For all BVP and skin conductance signals, the above features were calculated over 10 s non-overlapping windows.

We employed analysis of repeated measures to examine the effect of mental arithmetic and music imagery on the features extracted from BVP and EDA signals. To evaluate the correlation among signal features extracted from the three sites, we used a hierarchical linear model (also known as random effect model) (Neter et al 1996). In this model, signal features from the hand and the alternative sites were used as the independent and dependent variables, respectively. We report the regression coefficient (slope) and the associated p-value as an indicator of the correlation (agreement) between the signal features extracted from the hand and those obtained from the other sites. A regression coefficient value of zero indicates no agreement between the sites, whereas a value of unity indicates perfect agreement.

3. Results

3.1. Signal analysis

Figure 4 shows examples of BVP and skin conductance signals from the three measurement sites.

Figure 4.

Examples of BVP and skin conductance signals from the three measurement sites.

The latencies between the signals obtained from the hand and those from the alternative sites were averaged over all participants and reported in table 1 for both the BVP and skin conductance signals. These latencies were all significantly different from zero (sign test, p < 0.001).

Table 1.

Average latency between the three measurement sites in seconds.

Signal	Activity	Toe and hand	Hand and arch/ear
BVP	Baseline	0.18 ± 0.23	0.21 ± 0.34
	Arithmetic	0.13 ± 0.09	0.09 ± 0.22
	Music	0.11 ± 0.08	0.15 ± 0.27
EDA	Baseline	1.32 ± 1.71	0.56 ± 0.61
	Arithmetic	2.37 ± 1.58	0.93 ± 0.32
	Music	1.94 ± 1.32	0.80 ± 1.02

Skin temperature averaged over all trials and participants was 31.4 ± 4.0 , 26.3± 3.4 and 31.7 ± 1.3 °C for the finger, toe and ear sites, respectively. The temperatures of the three sites were significantly different (Kruskal Wallis, p < 0.01). Pairwise comparisons revealed that the temperature of the toe was significantly lower than that of the other two sites (rank sum, p < 0.02).

3.2. Feature analysis

Average values of measured HR and skin conductance over participants and trials were consistent with those reported in the previous literature (HR: toe 77.4 ± 16.2, finger 77.5 ± 16.3, ear: 77.3 ± 16.0, EDA: finger 2.7 ± 1.46, toe 3.37 ± 2.05, arch 1.85 ± 0.97) (Dawson et al 2000, Poh et al 2010, Brownley et al 2000). The mean EDA signal over all trials and participants measured at the hand was significantly larger than that measured from the arch of the foot (estimated difference: 0.87 μS, p < 0.0001), but smaller than that of the toe (estimated difference: 0.64 μS, p < 0.0001). The mean BVP signal was significantly larger at the hand site compared to all other sites (estimated differences: 0.94 (hand and ear), 2.08 (hand and toe), p < 0.0001).

As shown in tables 2 and 3, various features of BVP and EDA changed significantly during mental arithmetic and music imagery compared to the baseline for all sites (BVP range, mean HR, mean EDA and EDA slope).

Figure 5 summarizes the hierarchical linear regression coefficients (slope) for the features extracted from BVP signals. Significant correlations were found between the features extracted from the three sites for all features but BVP mean amplitude (hand and ear). For mean HR and BVP range, the regression coefficient was close to unity in all cases, indicating excellent agreement among the sites, whereas the regression coefficient ranges between 0.67 and 0.87 for HR variation, indicating only moderate agreement among the sites.

Figure 5.

Regression coefficient (slope) indicating agreement among the sites with respect to features extracted from BVP signals. ^*Regression coefficient significantly different from zero (p < 0.0005).

Figure 6 summarizes the regression coefficient values for the three features extracted from EDA signals. The results indicate moderate agreement between the sites for mean EDA (range of values 0.63–1.14), but low agreement for slope and number of EDRs (range of values 0.34–0.64). In particular, the number of EDRs measured at the hand was significantly higher than that of the toe (estimated difference 0.13, p < 0.0001) and the arch (estimated difference 0.25, p < 0.0001).

Figure 6.

Regression coefficient (slope) indicating agreement among the sites with respect to features extracted from EDA signals. ^*Regression coefficient significantly different from zero (p < 0.05).

4. Discussion

We measured BVP and SCR to cognitive and affective stimuli from three different sites and evaluated the correlation among these measurements. Consistent with previous reports (Picard et al 2001, Blain et al 2008), BVP and EDA features measured during the stimulus periods were significantly different from those measured during baseline. This was not true in the case of HRV, a feature which has been previously reported to be relatively unaffected by affective stimuli (Picard et al 2001). Significant changes from baseline were observed at all three sites, suggesting that these alternative sites can provide viable measurements for use in access technology applications. While not directly investigated in this pilot study, it is important to further evaluate the role of potential confounding factors, such as sensor movement artifacts, and their relative contributions to variation in the measured signals in future studies.

The latency between the BVP signals obtained from the non-palmar sites (toes and ear) and those obtained from the hand was under a second, an acceptable delay for access technology applications. The BVP signal amplitudes were found to be significantly different among the three sites in this study. The amplitude of the BVP signal carries information regarding the placement of the photoplethysmograph sensor with respect to vasculature. For example, greater amplitudes are observed when the sensor is placed over an artery. Despite this, BVP features measured from hands, toes and ear sites were significantly correlated (with the exception of BVP amplitude). In particular, our results indicated high inter-site agreement in the mean HR and BVP range and moderate agreement in HRV measurements.

Consistent with previous findings (Winterhalter et al 2008), we also found significant delays in the SCR measured from the plantar sites as compared to the palmar site. These delays are largely explained by differences in conduction times through efferent nerves (Winterhalter et al 2008). Furthermore, the amplitude of the signal measured from the toe was significantly higher than that of the other two sites. With regard to features extracted from the EDA signals, we found that the mean signal exhibited moderate inter-site agreement. Furthermore, the toe site generated a significantly smaller number of EDRs as compared to the hand and arch sites. These results are consistent with findings of Toyokura (1999) that the waveform of the measured sympathetic skin response is affected by the measurement site. This is also consistent with our results indicating low agreement among the sites with respect to the slope feature.

Despite the fact that BVP and EDA signals are regulated by the ANS, measurements of these signals are obtained through the skin. Consequently, local skin conditions at the measurement site may be responsible for the observed differences in measurements from the three sites. For BVP, the photoplethysmograph measured from the skin surface is affected by various localized phenomena including metabolism, vascular dynamics, arterial blood pressure and flow, and surface temperature (Reisner et al 2008). Examples of conditions affecting EDA include the local skin temperature, skin thickness, water content, body posture and the density of sweat glands (Toyokura 1999). Skin temperature was of particular interest in our study since the permeability of skin to water decreases with decreasing skin temperature (Boucsein 1992). Therefore, the decreased EDRs may be attributed to lower skin temperatures at the toe site. Investigation of the effects of local skin conditions on signal features is an important and interesting direction for future research in this area.

Another contributing factor to differences in EDRs may be related to habituation to the stimuli used in this study. While differences in sensitivity to habituation effects have been observed among different palmar sites (Scerbo et al 1992), the effect of habituation on the plantar site is unknown. Finally, differences in coupling between the sensors and the skin among the three sites may have contributed to the observed differences in signals.

One limitation of the present study is that the identical feature extraction methods were used for all three sites. Considering the inter-site differences in signal characteristics (for example, amplitude), site-specific tuning of feature extraction techniques may potentially increase the agreement among the features obtained from the different sites.

5. Conclusion

Overall, the results of this pilot study suggest that non-palmar sites can provide viable measurements for detecting changes in BVP and EDA signal features in response to cognitive and affective stimuli.

Table 2.

Effect of mental arithmetic and music imagery on BVP signal features (increase from baseline).

Task	Feature	Hand	Toe	Ear
Arithmetic	Mean BVP (% pressure)	0.03	0.01	0.04a
	BVP range (% pressure)	3.96a	2.36b	2.31b
	Mean HR (beats min−1)	1.90a	2.42a	2.16a
	HR variation (beats min−1)	0.54	0.26	0.44
Music	Mean BVP (% pressure)	0.08	0.03	0.003
	BVP range (% pressure)	2.56a	2.33b	2.17b
	Mean HR (beats min−1)	1.55a	1.49a	1.45a
	HR variation (beats min−1)	0.42	0.05	0.05

^aSignificant difference (p < 0.001),

^bsignificant difference (p < 0.01).

Table 3.

Effect of mental arithmetic and music imagery on EDA signal features (increase from baseline).

Task	Feature	Hand	Toe	Arch
Arithmetic	Mean (μS)	0.17a	0.16a	0.08b
	Slope (μS s−1)	0.03a	0.02a	0.01b
	EDR	0.27a	0.13b	0.13
Music	Mean (μS)	0.08b	0.07b	0.03
	Slope (μS s−1)	0.01a	0.01b	0.01b
	EDR	0.18b	0.12b	0.13b

^aSignificant difference (p < 0.001),

^bsignificant difference (p < 0.05).