Event-related fast optical signal in a rapid object recognition task: improving detection by the Independent Component Analysis

Abstract

Noninvasive recording of fast optical signals presumably reflecting neuronal activity is a challenging task because of a relatively low signal-to-noise ratio. To improve detection of those signals in rapid object recognition tasks, we used the Independent Component Analysis (ICA) to reduce “global interference” (heartbeat and contribution of superficial layers). We recorded optical signals from the left prefrontal cortex in 10 right-handed participants with a continuous-wave instrument (DYNOT, NIRx, Brooklyn, NY). Visual stimuli were pictures of urban, landscape and seashore scenes with various vehicles as targets (target-to-non-target ratio 1:6) presented at ISI = 166 ms or 250 ms. Subjects mentally counted targets. Data were filtered at 2–30 Hz and artifactual components were identified visually (for heartbeat) and using the ICA weight matrix (for superficial layers). Optical signals were restored from the ICA components with artifactual components removed and then averaged over target and non-target epochs. After ICA processing, the event-related response was detected in 70–100% of subjects. The refined signal showed a significant decrease from baseline within 200–300 ms after targets and a slight increase after non-targets. The temporal profile of the optical signal corresponded well to the profile of a “differential ERP response”, the difference between targets and non-targets which peaks at 200 ms in similar object detection tasks. These results demonstrate that the detection of fast optical responses with continuous-wave instruments can be improved through the ICA method capable to remove noise, global interference and the activity of superficial layers. Fast optical signals may provide further information on brain processing during higher-order cognitive tasks such as rapid categorization of objects.

1. Introduction

There are many instances where the complexity of biological phenomenology can only be studied in intact organisms. Since the 1970’s, a growing number of methods have been developed that explore intact tissues at various spatial and temporal scales (Baert, 2008). One technique that has proven especially versatile is magnetic resonance (MR) imaging. While often used for its ability to define the anatomy in exquisite detail, the MR method has also been developed as an effective tool to explore neuronal activation. This is possible because the hemodynamic response associated with activation alters the T2* MR relaxation time. This has lead to a wide array of reports whose focus has been the spatial and temporal relationships between neuronal activation and complex behaviors involving sensory (Ogawa et al., 1992), motor (Richter et al., 1997; Wildgruber et al., 1997), learning (Vaina et al., 1998), memory (Reber et al., 2002) and other higher order functions.

The MR method, however, does have clear drawbacks. For one, the instrumentation and facility costs are high. Additionally, the need to restrict gross movement in the magnet prevents the examination of subjects in natural settings or undergoing complex tasks. Also, the relatively weak signal associated with the hemoglobin response limits the temporal resolution of the method to a few seconds.

An alternative technique that has been developed over the past twenty years to study brain function is near infrared spectroscopy (NIRS) (for review, see (Pereira et al., 2007; Taillefer and Denault, 2005; Wolf et al., 2007). The attraction of this technique is the considerable flexibility by which the instrumentation can be deployed and its fast temporal response. Also, its low cost and portability add to its utility. Further extending its utility is the wide array of physical phenomena that can be studied by optical methods. These include absorption (Aronson et al., 1991; Barbour et al., 1990), light scatter (Pogue et al., 2004), birefringence and optical activity (Baba et al., 2002; de Boer and Milner, 2002), fluorescence (Chang et al., 1995a; Chang et al., 1995b; Chang et al., 1996; Chang et al., 1997), bioluminescence (Welsh and Kay, 2005) and the Raman effect (Qu et al., 1999; Vinegoni et al., 2004). In recent years, a particular focus of the NIRS technique has been to employ array sensing methods to provide for topographic (Koizumi et al., 2003; Toronov et al., 2007; Wolf et al., 2007) and tomographic (Aronson et al., 1991; Barbour et al., 1990; Barbour et al., 1991; Barbour et al., 2001; Graber et al., 1993b) imaging studies. A key focus here has been examination of the event related hemodynamic response (Gratton et al., 1997; Gratton and Fabiani, 2003; Toronov et al., 2001) as a surrogate of neural activity. Compared to the MR method, the NIRS technique provides information about the complete hemoglobin response (i.e., oxygenation and volume changes). Also of interest is evidence of that the NIRS method can be employed to directly measure neuronal activation. This is believed to result from light scatter changes that are associated with ion currents across the neural membrane and occurs on a time scale considerably fast than the hemodynamic response. This dual sensing capability makes the NIRS technique the only imaging method that is sensitive to changes in both neuronal activity (fast signal) and hemodynamics.

During the last 10 years, there have been several attempts to record the evoked fast optical signal noninvasively through the scalp and skull in human subjects (Franceschini and Boas, 2004; Gratton et al., 1997; Gratton and Fabiani, 2003; Morren et al., 2004; Rinne et al., 1999; Steinbrink et al., 2000; Steinbrink et al., 2005; Syre et al., 2003; Wolf et al., 2002; Wolf et al., 2003). The results of these studies, however, have been controversial. In a series of reports, Gratton and colleagues have documented the fast signal as being associated with a variety of event related tasks involving the primary sensory (Gratton and Fabiani, 1998) and motor cortex (Gratton et al., 1995). The ‘event related optical signal’ (EROS) showed spatial agreement with fMRI and temporal agreement with a visual evoked potential (Gratton et al., 1997).

Reports by Steinbrink and colleagues have been less consistent (Steinbrink et al., 2000; Steinbrink et al., 2005; Syre et al., 2003). Steinbrink et al. (2000) were first to measure the fast optical signal during electrical median nerve stimulation using a continuous-wave NIRS system and intensity measurements (ΔI/I), rather than photon delay. The reported signal changes were, however, much smaller (~0.05%) than reports by Gratton’s group. In another study, the same group failed to reproduce the results obtained by Gratton’s group using an almost identical instrumentation (phase measurement with a frequency-domain system) and experimental protocol (Syre et al., 2003). A recent study by the same group yielded limited results as the authors detected a significant change in activity in only one subject during finger tapping task and a lack of signal in all 12 subjects during visual stimulation with a reversing checkerboard (Steinbrink et al., 2005).

Another research group recorded the fast optical signal using intensity measurements in 10 healthy volunteers during finger-tapping, tactile stimulation, and electrical median nerve stimulation (Franceschini and Boas, 2004). The fast signal was detected in 43% of the measurements during finger-tapping, 60% of those during tactile stimulation, and 23% of those during electrical median nerve stimulation. The relative changes in intensity associated with brain activation were ~0.07% with latencies ~100 ms.

There is evidence that the method used for signal analysis could be important. Morren et al.(2004) have employed an adaptive filter and Independent Component Analysis (ICA) for better separation of a signal component containing the fast signal. In 9 of 14 subjects, a significant fast neuronal signal related to the finger tapping was found in the intensity signals. In the phase signals,indications of the fast signal were found in only two subjects (Morren et al., 2004).

To summarize, it can be said that the feasibility of detecting fast optical signal noninvasively remains controversial because of the low signal-to-noise ratio, and the signal is not detected in all subjects. In these studies, the detected signal had a latency of 50–100 ms which corresponds well to electrophysiological correlates of early sensory processing.

It remains an open question, as to whether other types of event-related neural activity can be recorded using NIRS methods. One well-established example is the event-related potential (ERP) P300 recorded in response to infrequent stimuli in the conventional oddball task. We are aware of only one study where the fast optical signal was recorded in the dorsal frontal cortex during an auditory oddball task and its temporal dynamic was similar to the P300 potential (Low et al., 2006). This study used a frequency-domain instrument and phase-based measurements of the fast signal. Event-related cognitive potentials recorded by electrophysiological methods may have a wider distribution over the scalp, which include frontal-temporal, central and parietal locations depending on the task in question.

To extend the published data and to further explore the feasibility of detection of the fast optical signal from the human scalp, here we have explored our ability to detect the event-related fast optical signal using an intensity-based continuous-wave imager (Schmitz et al., 2002; Schmitz et al.,2005a; Schmitz et al., 2005b) while employing a visual oddball task protocol. Specifically, we have searched for optical analogs of cognitive potentials during a target detection task using Rapid Serial Visual Presentation (RSVP) of complex pictures, a variation of the object detection task initially introduced by Thorpe et al. (Thorpe et al., 1996). A key methodology employed has been use of Independent Component Analysis (ICA) to detect and remove technical and physiological artifacts from the signal including non-specific activity of the superficial layers (scalp and skull). The results show that the temporal dynamics of the optical signal matches the known dynamics of the ERP recorded in similar conditions, which allows us to relate the observed optical signal to neuronal activity.

2. Results

2.1. Independent Component Analysis

A representative 5-second segment of the raw optical data from subject #39 is shown in Fig 2A. All 15 channels (1 source × 15 detectors) are numbered starting from the leftmost top location (subject’s view, see Fig 1) and first going down (column-wise) and then to the right (row-wise). In this scheme, the ‘co-located’ channel has #2 (for description of the ‘co-located’ channel, see section 4.2. Optical Data Collection below). The most distinctive feature of the raw optical data is the presence of regular high amplitude waves with a period of approximately one per second, which represent a heartbeat artifact. Heartbeat artifact is the most common physiological artifact found within the optical signal. It represents periodic changes in blood oxygenation related to heartbeat and therefore belongs to the family of hemodynamic (slow) optical signals. Highpass filtering >2 Hz used in this study did not remove it completely and the main rhythm of heartbeat at ~1 Hz along with higher harmonics was usually present in the optical signal after filtering. This artifact is seen in all channels in Fig 2A but less evident in channel #1, which is closest to the source, and in the co-located channel #2.

Fig. 2.

(A) Fifteen channels of raw optical data for wavelength = 830 nm recorded from a representative subject (#39). Note the presence of regular waves with a period of slightly shorter then 1 s in almost all channels (marked by asterisks in channel #12). Those waves are caused by regular changes in blood oxygenation due to the heartbeat. (B) Independent components of the same data. Note that the heartbeat rhythm is present mainly in components #2 (marked by asterisks), #6, #10 and to a weaker extent in components #12 and #15. (C) First two seconds of the same record are shown with a superposition of the raw data (thin line) and the restored data with artifactual components removed (bold line). Note a significant reduction of heartbeat waves in the restored signal. Signals are baseline corrected and normalized to standard deviation.

Fig. 1.

Optical probe on the head of a subject (top) and schematic drawing of the probe position and geometry (bottom). Position of the light source is marked by asterisk and the area used to reconstruct the spatial distribution of the fast optical signal is depicted by rectangular. This area is defined as to over midpoint locations for all source-detector pairs and the activity recorded at each detector is assumed to be “located” at the midpoint of the corresponding source-detector distance.

The ICA components of the data presented in Fig 2A are shown in Fig 2B. It can be seen that the heartbeat artifact is present only in several components (#2, here the heartbeat waves are marked by asterisks; ## 6, 10, 12 and 15). This illustrates the ability of ICA to identify artifacts based on their statistical independence from other signals and its potential usefulness for signal de-noising.

The weight matrix A for the data of Fig 2A is shown in Fig 3A as a grayscale-coded plot. Matrix A was an important tool guiding our selection of components which would have to be removed from the raw signal. In this matrix, each cell with indexes (i,j) represents the weight relating the j-th component to the i-th data channel. Analysis of all available data revealed a certain structure in matrices A, which was common in all subjects (Fig 3). This structure is exemplified in Fig 3A by ovals “a” and “b” depicting two clusters of relatively large weights. Oval “a” represents the cluster related to the activity at detectors which are close to the source. The second cluster (oval “b”) was usually found at the most distant detectors separated from the source by 3–4 cm. In the first cluster in Fig 3A, components ## 1, 4, 7, 8, 11, 13 and 14 have large weights along the second row of matrix A and this means that these components represent most of the activity of the co-located channel #2 and,therefore, they should be removed from the signal. The distant cluster “b” is likely to represent the activity of deeper (presumably cortical) layers and it is in this cluster where the signal reflecting neuronal activity is expected to be found. A possible and undesirable influence of the superficial layers on the activity recorded at distant detectors can be also revealed using matrix A. For example,components ## 4, 5 and 8 in Fig 3A contribute to the co-located (superficial) channel #2 as well as distant channels ## 13–14. This illustrates the fact that, indeed, the brain activity recorded at distant detectors can be contaminated to various extents by the activity of superficial layers.

Fig. 3.

(A)The grayscale-coded plot of weight matrix A for data shown in Fig 2. (B–E) Matrices Aaveraged over all subjects for each of four experimental conditions: PR=6 Hz, wavelength=760 nm (B); PR=6 Hz, wavelength=830 nm (C); PR=4 Hz, wavelength=760 nm (D) and PR=4 Hz,wavelength=830 nm (E). Each cell in the plots represents coefficient a_ij of matrix A. Oval “a” represents one cluster of relatively large weights describing the contribution of several components into the activity of co-located channel (#2) and other channels located close to the source. The activity within this cluster is largely contaminated by the activity of superficial layers. Oval “b” represents the activity of channels distant from the source. These two clusters of activity were commonly found in all subjects and can be seen in group average matrices (B–E).

After finding components related to the activity of superficial layers, those components were also excluded from further analysis along with the heartbeat components found through visual inspection of data records as described above. At the next stage of data analysis, the signal for all channels was restored using the remaining components. For the data from subject #39, the restored signal is shown in Fig 2C where it is superimposed on the raw signal. The corresponding power spectra of the raw and restored data are shown in Fig 4. In the raw spectra, the heartbeat artifact is seen as two distinctive peaks at frequencies ~2.2 and 3.5 Hz which correspond to the 2^nd and 3^rd harmonics of the heartbeat signal. From Fig 2C and Fig 4, it can be seen that the ICA method was very effective in removing physiological artifact as well as noise generated in the superficial layers (note that the spectrum of the co-located channel # 2 in Fig 4 is flat which means that the signal in this channel contains mostly noise). The restored signal was then used to calculate event-related responses for target and non-target conditions.

Fig. 4.

Power spectra of the data presented in Fig 2 before (thin line) and after artifact removal (bold line). Heartbeat-related artifact present in the raw data is seen as two high amplitude peaks at low frequencies (2–3 Hz). Those peaks are the 2^nd and the 3^rd harmonics of the heartbeat rhythm. Note a marked reduction in power of both the heartbeat and broadband noise after ICA. Spectra for channels ## 10–12 are shown using logarithmic scale along y-axis to show the spectra after ICA in more detail.

2.2. Event-related optical signal

To demonstrate the effect of artifact removal on the event-related optical signal, the signal from the representative subject (# 37) is shown in Fig 5 before and after artifact removal. The top panel shows the EROS calculated for one channel using the raw data (before ICA) while the lower panel shows the same signal after the ICA procedure and artifact removal. The average raw signal in this subject appeared to be insignificantly different from baseline and therefore did not reveal any significant components related to the visual stimulus (Fig 5A). Contrary to that, the signal after artifact removal showed deviations from baseline within 100–300 ms after the stimulus. Most interestingly,the optical signal decreased from baseline in response to targets while showing an increase in response to non-targets (Fig 5B). This pattern was consistent in all participants.

Fig. 5.

Event-related signal (averaged over all target epochs (bold line) and the same number of randomly chosen non-target epochs (dotted line)) calculated using the raw data (A) and the ICA-processed data (B) in subject #37. Stimulus is presented at t = 0 (picture onset). Significant deviations from baseline (100 ms pre-stimulus) are marked by asterisks. Note that averaging of the raw data does not reveal any event-related signal while averaging of the ICA-processed data reveals significant deviations from baseline for both target and non-target stimuli. Signal amplitude scale is in units of ΔI/I₀ (%).

The group average EROS is shown in Fig 6. It appeared to be similar in all four experimental conditions. For each condition, a few subjects were excluded from the group average because the response was not significantly demonstrated in them. Therefore, for all four conditions presented in Fig 6, the numbers of subjects used to derive the group average were as follows: N_A = 8 (condition A:PR = 6 Hz, wavelength = 760 nm); N_B = 10 (condition B: PR = 6 Hz, wavelength = 830 nm); N_C = 7 (condition C: PR = 4 Hz, wavelength = 760 nm); N_D = 7 (condition D: PR = 4 Hz, wavelength = 830 nm). As in subject # 37, the major distinctive feature of the group average response is that the optical signal (measured by changes in light intensity) decreased significantly in response to targets and this decrease occurred within 200–300 ms after the stimulus. After 300 ms, the signal returned to baseline with some tendency for overshoot that is, developing a small positive wave at ~ 350 ms before the next negative deviation from baseline after 400 ms. The later components (> 300 ms), however, were weak and, as a rule, insignificant. The average response to non-targets was quite different showing an increase at approximately the same time of 200–300 ms after the stimulus (Fig 6). However, the non-target-related response was weaker and did not reach significance at wavelength = 760 nm and PR = 6 Hz (Fig 6A).

Fig. 6.

Group average event-related responses for presentation rate (PR) = 6 Hz and wavelength = 760 nm, N = 8 (A); PR = 6 Hz and wavelength = 830 nm, N = 10 (B); PR = 4 Hz and wavelength = 760 nm, N = 7 (C); PR = 4 Hz and wavelength = 830 nm, N = 7 (D). N is the number of subjects used in each of four conditions to derive the group average response. Only subjects showing a significant response were used in group averaging. Stimulus is presented at t = 0 (picture onset). Blue line – response to targets; green line – response to non-targets; dotted lines show standard errors for the corresponding signals at each time point; asterisks designate time bins with significant deviation of responses from baseline (t-test, p < 0.05). Signal amplitude scale is in units of ΔI/I₀ (%).

The differential response (targets minus non-targets) is shown in Fig 7. Its temporal pattern is,understandably, similar to the target-related response also showing the largest deviation from baseline around 200–300 ms. The differential response was somewhat more robust at wavelength 830 nm compared to wavelength 760 nm (showing more temporal bins where signal amplitude was significantly different from baseline; compare Fig 7B, D to Fig 7A, C).

Fig. 7.

Group average differential responses (target minus non-target) for the data shown in Fig 6.Dotted lines show standard errors for the corresponding signals at each time point; asterisks designate time bins with significant difference between targets and non-targets (t-test, p < 0.05).

2.3. Spatial distribution of the fast signal

To analyze the spatial distribution of event-related optical signal, we first identified an ICA component best representing the optical response. This component was identified as having the largest weight for a data channel (or a group of channels) where the response was maximal. We then linearly interpolated the weights of the response-related component over the area covered by the optical probe. Spatial maps for each participant were finally group-averaged for each of four experimental conditions. The group average spatial maps are shown in Fig 8. The cardinal points in these maps used for interpolation grids represent the midpoints for each source-detector pair and therefore the maps show spatial distribution over the left half of the area covered by the probe (the imaged area is depicted in the bottom panel of Fig 1). Spatial maps demonstrate that optical response was best recorded in the data channels distant (3–4 cm) from the light source. Removal of the ICA components contributing to the co-located channel resulted in that the activity of the channels near the source were also excluded (as illustrated in the spatial maps by blue color near the source) and therefore, the signal became free of the activity of superficial layers. Thus, the spatial maps also illustrate the effectiveness of ICA to exclude nonspecific activity of superficial layers from the signal of interest (which in this context is a signal related to neuronal activity).

Fig. 8.

Group average spatial maps of the fast optical signal. Panels A–B correspond to the panels in Fig 6 and Fig 7. Spatial maps were calculated for each subject using relative weights of the ICA component contributing to the observed event-related response with the largest weight. To calculate spatial distribution, those weights were interpolated over the area covering midpoint locations for all source-detector pairs (see Fig 1). S is location of the light source. Note that the observed event-related response is best seen at detectors distant from the source (3–4 cm source-detector separation) while the contribution into the response of superficial layers (activity of which is best seen at detectors close to the source) is minimized through the ICA.

3. Discussion

3.1. Artifacts present in the optical signal

The optical signal measured exclusively as changes in light intensity without concurrent measurements of phase delay, as is the case for all continuous-wave instruments, may be contaminated by noise or nonspecific changes from many different sources. Because of the positioning of light sources and detectors on the scalp, photons necessarily pass through superficial tissue layers (scalp and skull) before reaching the cortex. These superficial layers may provide noise as well as nonspecific hemodynamic variations which would unavoidably contaminate the measured signal. In addition, hemodynamic oscillations inside the brain (e.g., related to heartbeat and respiration) also provide interference signals to the optical measurements of brain activity. All these interferences either from superficial layers or from inside the brain are often referred to as “global interference” or “systemic physiological interference” (Zhang et al., 2007). The global interference is a common problem for intensity-based measures of both the hemodynamic measurements of oxy-/deoxyhemoglobin concentration changes and the fast (presumably neuronal) signal. Several studies have addressed this problem through development of methods to suppress global interference applying them mostly for hemodynamic measurements. Those methods include adaptive filtering, average wave form subtraction and others (Franceschini et al., 2003; Gratton and Corballis, 1995; Zhang et al., 2007). A recent study has successfully employed Independent Component Analysis to remove the skin blood flow artifact from functional near-infrared spectroscopic imaging data (Kohno et al., 2007). The ICA method has also been used to improve the signal in optical imaging of intrinsic signals (Chen et al., 2007; Schiessl et al., 2008) and we are aware of only one study where ICA was applied to detect fast optical signal in frequency-domain measurements (Morren et al., 2004). Because intensity-based measurements are more sensitive to noise from various sources such as superficial layers and changes in environmental light intensity compared to phase-delay measurements (Gratton et al., 2006), it is important to develop reliable methods for removal of global interference while attempting to detect the fast optical signal using continuous-wave instruments. The major finding of this study is that the feasibility to detect the fast optical signal in intensity-based measurements can be significantly improved through the application of Independent Component Analysis. As our data show, without removal of global interference, fast optical signal may be completely undetectable (Fig 5) and this may explain the relatively poor results of some previous studies demonstrating fast signal in only a relatively small percentage of subjects. Our data show that the superficial layers are a powerful source not only of nonspecific hemodynamic oscillations (such as heartbeat-related) but also of white noise as judged from the flat power spectra of the co-located channel (Fig 4). Independent Component Analysis was a significant part of data processing used for artifact removal from optical signal in the current study and with application of ICA we were able to detect fast signal in 70–100% of our subjects.

3.2. Independent Component Analysis as a de-noising tool

The ICA as a mathematical and computational method belongs to a broad class of linear representations of multivariate data. Principal component analysis (PCA), factor analysis and projection pursuit are several examples of linear transformation methods commonly used in various data processing techniques. Independent component analysis is a recently developed method belonging to the family of the blind source separation techniques (Bell and Sejnowski, 1995; Comon,1994; Jutten and Hérault, 1991; Nadal and Parga, 1994). The major goal of ICA is to find a linear representation of non-gaussian data so that the components are as statistically independent as possible given all the available data. It has been shown that such representation seems to capture the essential structure of the data in many applications including feature extraction and signal separation (see, e.g., (Hyvärinen et al., 2001)). While PCA considers signals as random variables with Gaussian distribution and minimizes their second-order statistics (i.e., decorrelates variables), ICA considers non-gaussian variables and minimizes both second-order and higher-order dependencies in these variables (i.e., maximizes their statistical independence). The assumption of non-gaussianity of input variables is essential for ICA and it is more general compared to the assumption of gaussianity used in PCA because non-gaussian distributions represent a broader class of variables. Moreover, PCA components are orthogonal while ICA components may be non-orthogonal, which is again a more general assumption because there is no reason to expect neurobiologically distinct sources of activity to be orthogonal (Jung et al., 1998). A direct comparison between regression methods, PCA and ICA in terms of their applicability for artifact removal from EEG while preserving spectral properties of the EEG signal has revealed a higher effectiveness of ICA (Jung et al., 2000). There have been suggested several algorithms realizing the Independent Component Analysis. These algorithms include minimization of mutual information, maximum likelihood estimation and the Infomax algorithm derived from the principles of the neural network theory (Bell and Sejnowski, 1995; Nadal and Parga, 1994). Being similar from the theoretical viewpoint, these algorithms differ in computational strategy and the assumptions on the nature of the data. One of the most computationally effective methods of ICA is the FastICA algorithm (Hyvärinen and Oja, 2000). The algorithm is based on maximization of non-gaussianity of the components as an approach to increase their statistical independence.

We used the FastICA algorithm to perform ICA of optical data and the following steps to identify those ICA components which were considered noise (and discarded) as well as those which were considered signal (and kept). First, the major artifacts related to cardiac and respiratory activity were identified from visual inspection of the records as illustrated in Fig 2. Second, we identified those ICA components which had a significant contribution to the co-located channel (#2) using the weight matrix A as illustrated in Fig 3. Those components were considered as representing the activity of the superficial layers and were also discarded. This was confirmed through the spectral analysis showing the flat spectra of those ICA components. After the removal of all artifactual components, we verified that the components with a significant contribution to the distant optodes (depicted by ovals “b” in Fig 3) were kept in, considering them as the best candidates for a neuronal signal.

3.3. Technical and physiological factors affecting the detectability of fast optical signals

A comparative analysis of the fast optical signal using a frequency-domain instrument and deriving the signal from three different metrics has been done in a recent study (Gratton et al., 2006). Those metrics were: 1) continuous measurement of intensity (DC intensity), which is similar to using a continuous-wave machine without frequency modulation; 2) measurement of modulated intensity (AC intensity), which is similar to the continuous-wave measurement with modulation of light intensity (low-frequency (a few kHz) modulation of light intensity is used in the majority of continuous-wave imagers, including the DYNOT instrument used in this study, to allow separation of different light sources through demodulation); 3) photon delay measurement, which is specific for frequency-domain instruments. The effect of wavelength (shorter and longer than the hemoglobin isosbestic point at 800 nm, the point at which the absorption spectra of oxy- and deoxy-hemoglobin cross over) and the effect of source-detector distance were also analyzed. It has been shown that the AC intensity and photon-delay measurement of the fast signal are more robust and sensitive compared to the DC intensity measurement. A possible explanation is that the DC intensity measurements are more sensitive to various sources of noise, e.g., contribution of superficial layers, while measures based on photon delay may be exquisitely sensitive to differential effects occurring deep into the tissue thus providing better control for external sources of noise (Gratton et al., 2006). Supportive of this view, the current study demonstrates a significant effect of superficial layers and global interference which may make the fast signal undetectable through intensity measurements. This result along with other published findings emphasizes the need to employ de-noising algorithms such as ICA or adaptive filtering, especially with intensity measurements, in order to reduce artifacts present in the optical signal. Given the fact that the continuous-wave instruments cost significantly less than the frequency-domain instruments, the demonstrated successful application of de-noising algorithms in the current study encourages further application of continuous-wave instruments for detection of the fast (presumably neuronal) optical signal.

The current results are also consistent with the previous data considering the effect of the source-detector distance and demonstrating the best fast optical effects for source-detector distances exceeding 22.5 mm (Gratton et al., 2006). In our study the event-related fast signal was also best recorded at the detectors distant from the source (3–4 cm source-detector separation), which provides further support for the brain origin of this signal. Also, the signal observed in the current study had the same sign for both wavelengths shorter (760 nm) and longer (830 nm) than the isosbestic point of hemoglobin, which is also consistent with the recent report (Gratton et al., 2006). The same-sign effect has been interpreted as pointing to light scattering as a more likely mechanism for the observed fast signal rather than rapid deoxygenated effects (Gratton et al., 2006). If the fast signal were due to rapid consumption of oxygen (and therefore concurrent decrease/increase in oxy- and deoxy-hemoglobin, respectively), then one should expect the opposite effects (e.g., decrease and increase) observed at wavelengths on opposite sides of the hemoglobin isosbestic point. If the fast optical signal is, instead,due to changes in light scattering, then the same effects should be observed at both wavelengths, as was the case in the current study, thus supporting the light scattering mechanism for the observed fast response.

3.4. Physiological correlates of the fast optical signal

The major signal observed in our study as a decrease in light intensity is consistent with a similar decrease in intensity described by other groups who also used intensity measurements (Franceschini and Boas, 2004; Steinbrink et al., 2000). We observed a decrease in light intensity in response to target stimuli only, which is in line with the results of Low et al (2003) who also recorded a decrease in light intensity in response to targets in a visual oddball paradigm. In our experiments, the response to non-targets (frequent stimuli) had a tendency to be opposite to the target-related response showing a slight increase in light intensity (Fig 5 and Fig 6). However, at relatively high stimulation frequencies used in this study (4 Hz and 6 Hz), some overlap between consecutive responses was unavoidable and a weak positive response to non-targets might merely represent a return to the baseline of a preceding target response, which occurred in approximately one fifth of all trials. At high presentation rates used in this study, only the contrast between target and non-target stimuli (i.e., a differential response) is really meaningful. A possible physiological interpretation of the observed differential optical effect between targets and non-targets is as follows. Attendance to a target stimulus requires mobilization of attentional resources; therefore it is reasonable to suggest that target detection is accompanied by increased neuronal activation in the brain areas engaged in the response generation. It is probably this neuronal activation what is recorded by an ERP (such as P300) during similar tasks switching on the attentional mechanisms. If we assume that the fast optical signal results from changes in light scattering, then target-related increased neuronal activation in our experiments was accompanied by increased light scattering with a parallel decrease in light intensity. This supposition is in line with the evidence obtained from optical measurements in cell culture that neuronal activity is associated with an increase in light scattering (Stepnoski et al., 1991). It is likely that optical response depends on the level of activation and/or engagement of the underlying neuronal substrates. How neuronal processes cause and shape those optical effects and what their electrophysiological correlates are (e.g., specific ERP components or spectral changes within physiologically relevant frequency bands) are open questions requiring further studies. For example, combined EEG and optical recordings may provide new insights on the largely unknown relationship between neural activity and fast optical signal.

In an object detection task initially introduced by Thorpe et al (Thorpe et al., 1996), the target stimuli were detected within complex natural scenes and they were defined by higher-order category information (presence of different animals within those scenes; the target/non-target ratio was 1:1).Several studies have explored this task in detail using the conventional ERP approach and described an early difference between targets and non-targets in the ERP starting at 150 ms after the stimulus and peaking at 200 ms, a response termed “differential ERP activity”. It was observed as negative potential over temporal-occipital and positive deflection over frontal regions (Delorme et al., 2000; Fabre-Thorpe et al., 2001; Thorpe et al., 1996). Using source localization techniques, a more recent study has found the sources of activity related to this object detection task within the posterior visual-associative brain regions and, although less pronounced, additional anterior sources in the prefrontal cortex (Codispoti et al., 2006). We utilized a modified version of this object detection paradigm using different stimuli (various types of vehicles within natural scenes rather than animals) and a lower target/non-target ratio (1:6). Despite the differences, we expected a similar response in the prefrontal cortex related to the early target detection and the optical correlate of this response was a focus of the current study. Although accurate localization of the optode positions with respect to cortical structures has not been performed in this study (which requires co-registration of optode positions with anatomical MRI and will be implemented in the future studies), our data demonstrating the differential optical response (targets minus non-targets) in the left prefrontal cortex at 200–300 ms after the stimulus are consistent with the object recognition-related “differential ERP activity” recorded electrophysiologically.

In conclusion, the current results demonstrate that fast optical response can be detected using continuous-wave instruments provided some advanced methods of artifact removal such as Independent Component Analysis are implemented. Similarly to the EEG data, when applied to the optical signal, the ICA method appears to be a powerful tool capable to reveal and remove components related to artifacts (such as global physiological interference), noise and nonspecific activity of superficial layers. The refined optical signal has a potential to contribute to the spatial and temporal characterization of various types of brain responses including those related to higher-order cognitive processes such as rapid object recognition.

4. Experimental procedures

4.1. Participants

Ten right-handed young adults (six females, age 18–36, mean age 26.6) participated in the study. All participants signed a consent form approved by the Georgetown University Institutional Review Board and reported as being in good health and without medications. All subjects had normal (or corrected to normal) vision and undertook a battery of behavioral tests which included measures of IQ (Weschler Abbreviated Scale of Intelligence; the average IQ score 118.3) and handedness before one experimental session lasting 2 hours during which they performed a target detection task with simultaneous optical recording of brain activity. All subjects were compensated for their participation in these experiments.

4.2. Optical Data Collection

Optical signals were recorded using a two wavelength (760, 830 nm) continuous-wave DYNOT (DYnamic Near-infrared Optical Tomography) imaging system from NIRx Medical Technologies, Brooklyn, NY. The system can be operated in two modes. Recording of the hemodynamic response is accomplished using a time multiplexed illumination scheme, adaptive gain control and frequency encoding techniques (Schmitz et al., 2002). The image framing rate achieved is roughly proportional to the number of illumination sites employed. In the case of the maximum number for the particular system, (i.e., 32 source locations, Model 264), the framing rate is approximately 2 Hz. The core technology is scalable, allowing for use of a greater number of illuminating wavelengths, source and detector locations. Another important element of the system is its optode design. In the current study, we used optical fiber cables designed to support “two-way” traffic of optical signals. Each cable contains micro-fibers transmitting light from the source to the tissue as well as micro-fibers transmitting light from the tissue to the detector. In this fashion, optical signals that are ‘co-located’ with the source (i.e., light reflected by a tissue volume in close proximity to the source) can also be captured. This has the advantage of isolating signals that have penetrated only the most superficial layers of tissue (< mm). In fact, we took advantage of this when applying the ICA method for data analysis and using the ‘co-located’ channel (i.e., the output optical signal spatially co-located with the source) as a channel representing mostly the activity of the superficial layers (channel #2 in the current design, which is co-located with the source marked by asterisk in Fig 1). Optodes positioned at greater distances capture signals that have propagated along a banana shaped path that has successively greater maximum depths (Graber et al., 1993a). It deserves emphasis that because this system functions as a tomographic imager, a greater sensing density can be achieved than that utilized with NIRS systems designed for surface topography studies (Koizumi et al., 2003; Taga et al., 2003). In the limit, data from every source and detector is collected (32 × 64, 2048 channels/wavelength/image frame). In practice, the maximum value achieved is reduced depending on the particular source-detector separations employed. Secure optode positioning is achieved using a helmet that employs an open scaffolding design (Fig 1). At maximum density, the optodes (3 mm diameter) have a center-to-center separation distance of 1 cm.

Recording of the fast optical signal requires a broader bandwidth which is achieved by limiting the optical switch to a single location thus allowing higher sampling rate of the signal. In this mode, the imager captures a time series at nearly 75 Hz. For the current study, probe geometry was designed so that to target the prefrontal cortex. A total of 15 optical fibers were placed on the left side of subject’s forehead and arranged as a grid with 3 horizontal rows and 5 optodes in each row separated by 1 cm (Fig 1). Here we use the term “data channel” to refer to the signal recorded by the corresponding detector. Because only one source was employed, 3-D image reconstruction was not done and instead was restricted to presentations of 2-D spatial maps based on the ICA weight matrices. During reconstruction, for simplicity, data channels were assumed to be “located” at spatial coordinates (x,y) defined at midpoints between the corresponding source and detector.

4.3. Experimental paradigms

To investigate the feasibility of utilizing optical methods to measure neuronal activity, we used a target detection task within a RSVP stream of pictures. The paradigm was based on an Animal-No Animal task modified from the original task introduced by Thorpe et al (Thorpe et al., 1996) to study fast object recognition using the ERP technique. In this paradigm, we used picture chips with size of 500 × 500 pixels cut from several broad view images of landscapes and some of those chips contained vehicles such as planes, helicopters, cars and boats. All these vehicles were designated as targets while picture chips without vehicles served as non-targets. All pictures were shown to the subjects at the center of a computer LCD monitor at a viewing distance of 75 cm in blocks each containing 13 chips. The presentation rate (PR) within each block was 6 or 4 pictures per second and there was a 1.5-second break between successive blocks. For each PR, 160 blocks were presented with 2080 pictures in total selected randomly from the same picture set. As both presentation rates were too fast to allow manual response on each trial, the task was to mentally count picture chips containing targets and then verbally report the number of targets immediately after each block during the inter-block breaks. Counting targets served to engage the subject’s attention and provided a measure of behavioral performance. Use of mental counting of targets instead of requiring a motor response (such as button pressing) was advantageous because this allowed us to achieve a relatively high PR while being able to assess the behavioral performance of subjects based on their verbal reports between blocks. Each picture was shown for about 60% of time between successive presentations and the baseline crosshair at the center of the screen was shown for the rest of time (40%). This translates into 100 ms for each chip exposure at PR = 6 Hz (inter-stimulus interval (ISI) = 166 ms) and 150 ms at PR = 4 Hz (ISI = 250 ms). Infrequent targets (total number = 280 for each PR) were distributed randomly between frequent non-targets (total number = 1800) and the target/non-target ratio was 15.5%. The overall structure of the experiments included two sessions (for PR = 6 Hz and PR = 4 Hz) with a couple of minutes break between sessions and the order of sessions was counterbalanced between participants. Each experiment also included a short practice followed by several blocks of chip presentation.

Data Synchronization

Synchronization of picture presentation and optical data acquisition was achieved using a TTL pulse generated by the imager and sent to the presentation software (E-Prime) through a hardware interface (DYNOT data synchronizer).

4.4. Data analysis

Optical data were recorded at 75 Hz sampling rate continuously during presentation of pictures and stored on acquisition PC computer for off-line analysis using original Matlab scripts. To remove the slow (hemodynamic) and DC components from the optical signal, the data were high-pass filtered at >2 Hz, normalized (the signal at each channel was divided by its standard deviation calculated over the whole record) and then subjected to the Independent Component Analysis.

4.4.1. Independent Component Analysis

The ICA decomposes signal into statistically independent components which are linearly related to the original data (Hyvärinen et al., 2001). If x is a vector of n observed variables (data channels) and s is a vector of n independent components, then a linear relationship between x and s can be written in matrix notation as:

$$\mathbf{x}=\mathbf{A}·\mathbf{s},$$ (1)

Here A is a transform matrix. The goal of ICA is to find vector s such as its components are statistically independent. This can be done if components of vector x have non-gaussian distributions and thus assumption of non-gaussianity is essential for the ICA method. After estimation of matrix A,its inverse W can be computed and therefore the independent components can be found:

$$\mathbf{s}=\mathbf{W}·\mathbf{x}$$ (2)

Note that matrix A is also a weight matrix because its elements a_ij show relative contributions of the j-th component to the i-th original variable (data channel). We used the FastICA algorithm available as a package of Matlab scripts at http://www.cis.hut.fi/projects/ica/fastica/. The algorithm is based on maximization of non-gaussianity of the components s as an algorithm to increase their statistical independence.

As it has been shown in many EEG stdies, each of the common artifacts present in the EEG signal, such as eye blink-related artifact, heartbeat and motion-related artifacts, is usually identified by the ICA method as one or several individual components. By definition, those artifactual components are statistically independent from other components and therefore all other components including those representing physiological activity of interest become (statistically) separated from the artifacts. At the next step, the original signals are linearly ‘restored’ back from the components and if all artifactual components are excluded during restoration, the restored signal becomes artifact-free. The method therefore is capable of removing artifacts from the original signal while preserving physiological signal in question (see, e.g., (Vigário, 1997) (Jung et al., 2001; Vigário et al., 2000)). Optical data were therefore processed using ICA with the goal to remove physiological artifacts (mostly related to heartbeat) and the contribution of superficial layers from original records.

4.4.2. Event-related averaging

After ICA procedure, the signal in each data channel was recalculated as relative change in light intensity (ΔI/I₀). The event-related optical signal was calculated for each source-detector pair in each subject as follows. Continuously recorded optical data were segmented into stimulus-related epochs lasting from 100 ms before (baseline) to 500 ms after the picture onset. Signals within each epoch were baseline-corrected and then averaged across all targets. Because the number of non-targets significantly exceeded the number of targets, in order to statistically balance target and non-target epochs, the non-target-related response was calculated over the same number of epochs (280) randomly selected from the total number of non-target epochs. Target (T)- and non-target (NT)-related responses as well as the T->NT difference were statistically evaluated within each subject using nonparametric Mann-Whitney test with the 5%-significance level. The test was applied for each time point within the trial epoch for all data channels, as suggested by Thorpe et al (1996) for the ERP analysis. Event-related signals were analyzed within each subject and channels were determined where the signal showed significant difference from baseline within the time window 0–500ms after the stimulus in at least two time bins. Those channels were then averaged within each subject giving an average signal representative of that subject. Those representative signals were finally averaged across subjects giving a group average signal, which was statistically assessed against baseline for each time point using t-test (p = 0.05). The group average responses were calculated separately for two presentation rates (6 Hz and 4 Hz) and two wavelengths (760 nm and 830 nm) giving four conditions in total. If in a given subject no channel showed significant deviation from baseline within 0–500 ms after the stimulus in a particular condition, this subject was excluded from the group average for that condition.

4.4.3. Spatial localization

To analyze spatial distribution of the fast optical signal, we used the weights of ICA components. Matrix A shows relative contributions of each component to every channel of the raw data. Namely, the n-th column of matrix A represents contributions of the n-th component to all data channels (Fig 3). If this component relates to the physiological response, its weights show how the response is spatially distributed over all data channels and therefore can be used to map the response. We represented the weights of the largest response-related ICA component as a two-dimensional function of surface coordinates (x, y) defined at mid-points of all source-detector pairs and did interpolation of that function over the area covered by the optical sensor grid.