ninjaNIRS: an open hardware solution for wearable whole-head high-density functional near-infrared spectroscopy

Abstract

Functional near-infrared spectroscopy (fNIRS) technology has been steadily advancing since the first measurements of human brain activity over 30 years ago. Initially, efforts were focused on increasing the channel count of fNIRS systems and then to moving from sparse to high density arrays of sources and detectors, enhancing spatial resolution through overlapping measurements. Over the last ten years, there have been rapid developments in wearable fNIRS systems that place the light sources and detectors on the head as opposed to the original approach of using fiber optics to deliver the light between the hardware and the head. The miniaturization of the electronics and increased computational power continues to permit impressive advances in wearable fNIRS systems. Here we detail our design for a wearable fNIRS system that covers the whole head of an adult human with a high-density array of 56 sources and up to 192 detectors. We provide characterization of the system showing that its performance is among the best in published systems. Additionally, we provide demonstrative images of brain activation during a ball squeezing task. We have released the hardware design to the public, with the hope that the community will build upon our foundational work and drive further advancements.

1. Introduction

fNIRS is a non-invasive neuroimaging technique that enables brain imaging under naturalistic settings. The ecological validity of traditional laboratory-based experimentation, and the extent of the representative value of such laboratory findings to their real-world counterparts, have been a concern for a long time [1]. fNIRS allows researchers to collect data in ecologically valid settings and thus has great potential for studying natural behaviors [2,3] and neurological populations [4,5]. fNIRS measures the hemodynamic response function (i.e., oxygenated (HbO), deoxygenated (HbR), and total hemoglobin (HbT)) from the cortical surface and is strongly correlated with the fMRI blood oxygen-level dependent (BOLD) signal [6]. fNIRS has been implemented to study healthy brain function in the context of language [7–10], cognition [11] and social functions [12], and to study pathologies such as autism spectrum disorders [13] in more natural environments.

High-density fNIRS (HD-fNIRS) technology expands upon typical sparse fNIRS source-detector separations (SDS) of ∼30 mm in adults by utilizing overlapping SDSs with varying distances (15-40 mm). The overlapping measurements improve imaging spatial resolution, contrast to noise ratio and enhance the specificity to the brain activation [14]. HD-fNIRS studies have led to new discoveries due to their higher spatial resolution allowing greater ability to discriminate brain activation between close cortical locations. Among them are decoding brain activity during movie watching [15], discriminating brain correlates of thumb and pinky finger [16], language processing of single words and sentences [17] and connectivity analysis comparable with fMRI [17,18].

Wearable fNIRS systems have been rapidly advancing over the last few years [19–21]. Wearable devices are much lighter and more flexible than conventional fNIRS devices with the use of lightweight electrical wires in place of heavier fiber optic bundles, allowing light emitters and receivers to be directly placed on the scalp. This enables studies of brain activity associated with natural behaviors in natural settings, and also reduces the cost of fNIRS systems. These two characteristics will significantly broaden the spectrum of fNIRS applications by allowing studies of brain function in more natural environments (social interactions, outdoor walking), and more efficient monitoring of patients with brain injury or neurodegenerative disease (stroke, Alzheimer’s, Parkinson’s, chronic traumatic encephalopathy) and of normal/abnormal brain development (language development, autism). Despite recent advances allowing multi-channel high dynamic range fNIRS acquisition [22–26], currently available wearable devices have limited numbers of channels or limited flexibility. The GowerLabs Lumo system [26,27] and the Kernel Flow system [28,29] are two commercial HD-fNIRS systems with impressively high channel counts. These systems are still being modified to optimize signal acquisition for different head sizes and hair characteristics. Larger numbers of channels are needed to simultaneously cover more brain regions and to enable higher resolution imaging. Simultaneously, it is essential to maintain flexibility in the spatial arrangement of the optodes to accommodate different head sizes and shapes, and to ensure effective signal acquisition from subjects with challenging hair characteristics.

In order to lower the barrier for new researchers to enter the fNIRS field and test their innovative hardware approaches, we decided to make our new system “an open-hardware” fNIRS system, following the lead of the first open-hardware fNIRS systems: the openfNIRS system developed by von Lühmann [30] which is a mobile, modular, multimodal bio-signal acquisition architecture that combined fNIRS with EEG and an accelerometer [31], and the HEGduino system, another small channel count open-hardware solution for combining fNIRS and EEG which is accessible through GitHub.

To build on the important contributions of these early open-hardware fNIRS efforts, we aimed to develop a HD-fNIRS system that covers the whole head and make it fully open so that other researchers can easily adapt it for their purposes and expand upon it with their own innovations. In the following sections, we start with a high-level description of the system architecture and then detail the circuitry for the sources, the detectors, and then the control board. We then elaborate further on several sub-components of the control board, and the multiplex strategy we employ. We then present results for the system characterization. Finally, we present demonstrative experimental results showing brain activation using our whole head HD-fNIRS system.

2. ninjaNIRS22 system architecture

The ninjaNIRS22 system was designed as an interconnected ecosystem of control electronics, source and detector modules, and auxiliary data collection interfacing with the human subject participant, and an external computer. The architecture, shown in Fig. 1, provides a high-level overview of the inter-connection of each of these components. The main component of the system is the control unit. It consists of the control board, which connects to up to four detector backplanes, and a single source backplane PCB. Each detector backplane can support up to six detector interface cards for a total of up to 24 cards. Alternatively, detector cards can be substituted for inertial measurement unit (IMU) interface cards. The source backplane can support up to seven source interface cards. Each interface card is connected to a bundle of source/detector/IMU optodes, which are mounted on the head using ninjaCap [32]. The whole system is controlled by a graphical user interface (GUI) software called ninjaGUI (available from GitHub) running on an Experiment PC connected to the control unit via USB.

Fig. 1.

(a) Schematic of the ninjaNIRS22 system showing the main components of the control board including the FPGA that communicates with the source and detector interface cards and the IMU/Accelerometer controller. The FPGA also communicates with an external computer either through an FTDI USB interface or through a Raspberry Pi Zero which in the future will permit wireless communication with an external PC through TCP/IP. Our acquisition software runs on the external PC allowing the user to control ninjaNIRS22 and view the acquired data in real-time. The interface cards in the control unit are connected through ribbon cables with the source and detector optodes as well as the IMUs placed on the subject. (b) A photograph of the control board with the source interface cards (blue) and the detector interface cards (red). (c) A photograph of ninjaNIRS22 with 200 optodes on the head of a subject.

This system has evolved since we first started building a wearable fNIRS system in 2016. Under NIH grant R24-NS104096 “Establishing an fNIRS Ecosystem for Open Software-Hardware Dissemination”, we built ninjaNIRS20 which could control 24 optodes, each of which combined a dual-wavelength LED with a silicon photodiode (SiPD). The original idea was to combine the source and detector together to permit short-separation measurements which are advantageous for signal processing [33]. This combined optode placed the source and detector within 8 mm of each other, but within an 18 mm diameter package that prevented extension of the system to HD-fNIRS. From this effort, we realized that (1) we could make each optode smaller and (2) that this would be advantageous to permit HD-fNIRS measurements. Under NIH grant U01-EB029856 “The Neuroscience of Everyday World- A novel wearable system for continuous measurement of brain function” we have been developing these smaller sources and detectors and the control unit for managing the signals for 56 dual wavelength sources and up to 192 SiPD detectors. We first developed ninjaNIRS21 with 8 sources and 12 detectors to test our system architecture. We iterated on that design to arrive at ninjaNIRS22 which we describe in detail here. Note that in our whole-head implementation described below, we utilized only 144 detector optodes along with the 56 sources, as this covered the whole-head with our high-density arrangement of optodes in a hexagonal pattern with an 18 mm nearest neighbor distance. All circuit designs, firmware, acquisition software, 3D printing files and list of materials can be found on the openfNIRS.org website.

2.1. Source circuitry

The source module is shown in Fig. 2. We designed our source optode dimensions and electrical connections to ensure compatibility with the commercially available NIRx NSP2 (NIRx, Germany) optode modules and their spring-loaded mounting solution for better scalp coupling. Our source optodes contain a simple printed circuit board (PCB) with a dual wavelength LED (735 nm & 850 nm, Marubeni SMT735D/850D) as shown in Fig. 2(e). This PCB is placed into a simple 3D printed housing printed with a Formlabs Form 3 SLA printer. The 3D printed housing is made with thin walls and recesses to align the LEDs on the PCB with a 3.0 mm diameter polished acrylic light-pipe (Edmund 53-833) that protrudes from the bottom of the housing. This light-pipe helps penetrate through the hair to increase light coupling efficiency into the head. The PCB is secured and protected inside the housing with non-conductive dielectric epoxy potting (Loctite E-60NC).

Fig. 2.

(a) A complete source bundle from the optodes on the left to the splitter board which interfaces the wire from each of eight optodes to a ribbon cable which extends further to connect with the source interface card in the control unit. (b) and (c) show the two sides of the source interface card and (d) shows the simple PCB of the splitter board for linking the optode wires to the ribbon cable. (e) shows the simple PCB for the source optode which simply contains the dual wavelength LED as all current supply circuits are contained on the interface card.

An ultra-flexible miniature cable with 3 conductors (e.g. Daburn 2713/2) connects each source optode to a splitter PCB (Fig. 2(d)). Eight source optodes are attached to each splitter box. The splitter box connects the wires from each of the eight sources to a 26 conductor 0.025” pitch ribbon cable to assist with cable management. The ribbon cable then connects with the source interface card (Fig. 2(b,c)). The source interface card contains 16 n-channel LED select transistors and 7 p-channel transistors that allow different current limiting resistors to be switched between the voltage supply and the common LED anode. With this circuitry on the interface card, the optodes can be operated at 7 approximately logarithmically spaced power levels ranging from 0.7 to 61 mW peak optical power. Note that since a given LED is never on more than 2% of the time, this is within light level safety limits for skin exposure established in ANSI Z136.1. Both the LED select and power level select transistors are controlled by the FPGA on the control board via three 3-to-8 line decoders. The ninjaNIRS22 system uses temporal multiplexing to distinguish individual source signals at the detectors and the source interface card is designed to drive a single LED per card/optode bundle at a time similar to a matrix display. This limitation was imposed to reduce the number of digital lines running to the main FPGA on the control board, and in recognition of the likelihood that most applications would never drive more than one LED from a group of eight dual-wavelength LEDs at a time in order to minimize crosstalk between sources during temporal multiplexing. Our temporal multiplexing strategy is discussed in more detail in Section 4. Note that this interface card circuitry is designed to be compatible with the commercially available splitter boxes and source optode bundles produced by NIRx for their NSP2 fNIRS device as well as our own open source optode bundles described above.

2.2. Detector circuitry

The detector module is shown in Fig. 3. We designed our detector optode to have the same dimensions as the commercially available NIRx NSP2 spring loaded mounting solution for better scalp coupling and so that they could be used interchangeably with commercial detectors from the NIRx NSP2 fNIRS device. The PCB for the ninjaNIRS22 detector is shown in Fig. 3(f). An SiPD in photovoltaic mode with integrated daylight filter (Vishay Semiconductors, VEMD5160X01) collects the NIR light through a 3 mm diameter polished acrylic light guide (same as in the source optode). The raw photocurrent is amplified by a transimpedance amplifier with 330 MOhm feedback resistance using an op-amp (Texas Instruments OPA810). To shield the sensitive circuit from ambient EMI, especially at the line frequency of 50/60 Hz, the detector PCB is encased in a brass housing and a conductive plastic film (TDK FleClear 10) is placed over the SiPD. As on the source PCB, the SiPD is aligned with a plastic light pipe that protrudes through the housing. The different voltages necessary for the photodiode and op-amp are generated and supplied to the optode by the splitter PCB. In situations where higher sensitivity performance is required, the commercially available NIRx NSP2 SiPD bundles can be used with the same interface cards.

Fig. 3.

(a) A complete detector bundle from the optodes on the left to the splitter board which interfaces the wire from each of eight optodes to a ribbon cable which extends further to connect with the detector interface card in the control unit. (b) and (c) show the two sides of the detector interface card with the four dual ADCs for digitizing data from the eight detectors. (d) and (e) show the two sides of the splitter board. (f) shows the two sides of our detector PCB.

An ultra-flexible miniature cable with 4 conductors (e.g. Daburn 2713/3) connects each detector optode to the splitter PCB (Fig. 3(d,e)). Eight detector optodes are attached to each splitter PCB. The splitter PCB connects the wires from each of the eight detectors to a 34 conductor 0.025” pitch ribbon cable to assist with cable management. The ribbon cable then connects with the detector interface card (Fig. 3(b,c)). The splitter PCB also contains a positive and negative linear voltage regulator for each optode to minimize crosstalk via the power supply rails.

The detector interface card contains the analog to digital converters and a microcontroller. Digitization of the data collected with the detector modules is performed by four dual channel ADCs (Analog Devices AD7380, 16bit, up to 4MSPS) driven by differential ADC drivers (Analog Devices ADA4940-2). The ADCs are set to 4x on-chip oversampling. The samples are then read by the microcontroller (Raspberry Pi RP2040) via SPI and summed a further 237 times for a total of 948x oversampling. The 237x oversampling results from the time available during the 1.25 ms sampling period described in Section 3.3. When requested by the FPGA on the control board, the microcontroller sends the final result as a 24 bit value for each of the eight channels (i.e. 3 bytes per channel) to the FPGA on the control card via UART.

2.3. Inertial measurement unit (IMU) interface card/IMU bundle

To facilitate motion artifact rejection and human pose estimation, we have designed small IMU units that can be attached in different locations around the body. Each IMU unit consists of a 6-axis IMU (STMicroelectronics ISM330DHCX) and a microcontroller (Microchip ATMega168 PA) on a 8 × 16 mm PCB. Up to 6 IMU units are connected to an IMU splitter box using 4 wire telephone cable. The splitter box contains the necessary voltage regulator and can also itself contain an IMU unit. The splitter box is then connected via a 0.025” pitch 26 conductor ribbon cable to the IMU interface card. The IMU interface card can be plugged in to a detector backplane instead of a detector interface card. The interface card itself can accommodate an additional IMU unit for a total of 8 IMU units. It also contains a microcontroller (Raspberry Pi RP2040) that receives the serial data streams from all IMU units using its programmable IO hardware, and then forwards the data to the main control board. Pictures, diagrams and parts lists can be found on the openfNIRS.org website.

2.4. Backplanes

The backplanes are simple connection PCBs distributing signals and power to the individual interface PCBs. The system uses a source and four detector backplanes. The source backplane has slots for seven source interface cards. Each detector backplane has slots for six detector or IMU interface cards.

We have also developed source and detector backplanes that only feature a single interface card each for a more compact version of our system. Finally, there are also special, larger versions of these single layer backplanes that provide easy access to all signals to help with system debugging.

2.5. Control board

The main component on the control board (Fig. 4) is a field programmable gate array (FPGA, Lattice Semiconductor Mach XO2). The FPGA serves two main functions. First, it controls the acquisition sequence, including LED illumination patterns and ADC sampling points with precise timing. Secondly, it aggregates the data from all detector interface cards and auxiliary data sources, and then sends it via the on-board Serial to USB converter (FTDI FT2232 H) to an attached laptop running our acquisition software ninjaGUI. Alternatively, the FPGA can send the data via SPI to a Raspberry Pi Zero 2 single board computer (SBC) mounted on the control card. The SBC can then store the data on its microSD card.

Fig. 4.

ninjaNIRS control board with key components labeled.

Additionally, the control board features two auxiliary analog inputs and two digital inputs/outputs directly coupled to the FPGA to synchronize with other recording instruments (e.g. EEG) or record physiology directly. The analog inputs each use a 12bit ADC (Texas Instruments ADS7886, up to 1MSPS) sampled at exactly the same time as the optode ADCs but without the ADC internal oversampling, resulting in a 237x oversampling ratio. The digital inputs are directly connected to an FPGA general purpose IO pin and are sampled once every state. All 4 inputs use a MMCX coaxial connector and have additional input protection.

Additional digital triggers or stimulus markers can be received using a wireless receiver module (Qiachip RX480-E4). Two transmitters are available, either a handheld remote controller with 4 buttons (Qiachip TX118S-4) or a ‘trigger box’ that accepts digital triggers on four coaxial BNC inputs and features a transmitter module (Qiachip TX118SA-4). Multiple transmitters can be paired to a single receiver module on the control board.

The control board features a microcontroller (Raspberry Pi RP2040) that monitors the battery voltage, system current consumption and system temperature. Two pins of the microcontroller are connected to MMCX connectors and can serve as additional analog inputs or digital inputs or outputs. Four Sparkfun QWIIC compatible connectors enable the connection of a large selection of sensors, such as GPS or environmental. Currently we use one of these ports to attach a single IMU when we are not using the IMU bundle. Output devices like small displays or LEDs can also be connected using QWIIC.

The last main function of the control board is to generate the different positive and negative power rails for the interface cards and plug in components from the single 12 V input. To limit noise and electrical crosstalk, separate voltage regulators supply the source and detector backplanes. Most voltage regulators can be disabled by the FPGA through firmware to conserve power when the system is idle. A shunt resistor and 12bit ADC (Texas Instruments ADS7886, up to 1MSPS) allow for rapid measurement of the current flowing to the source backplane. This ADC is sampled simultaneously to the auxiliary analog input ADC 237 times during each source state and thus allows for monitoring of the LED drive currents.

2.6. System power sources

The power necessary to run the ninjaNIRS22 system is delivered by a waist mounted battery box. This battery solution allows for two 12 V DeWalt tool batteries to be used simultaneously for up to 10 Ah advertised capacity. A PCB inside the battery box contains circuitry to allow for hot-swap functionality without causing any interruption of power. With the battery box worn on the waist strap of the harness, the entire package with 5 Ah batteries connected measures approximately 75 mm off the strap, 140 mm left to right, and 115 mm top to bottom with a weight of about 900 g. If mobility is not required for a measurement and the system can be kept stationary, a medical grade power supply (Mean Well GSM60A12-P1J) can be used in place of the battery box to provide clean 12 V power to the system.

2.7. Casing

The case for the ninjaNIRS22 system is made primarily with laser cut acrylic panels and 3D printed parts (see assembled case in Fig. 1(c)). The original plan for the enclosure was a 2-part 3D printed enclosure, but this was found to be unfeasible for two primary reasons. First, the footprint of all electronics to be located inside the control unit was too large for the SLS printers we had access to, which meant the case would have to be made by an external manufacturer. The other reason was having the parts manufactured externally was prohibitively expensive due to their large size and complex features. Using laser cut panels for much of the case and only using 3D printed parts where necessary allowed for the size of these parts to be reduced enough to fit on small form factor SLS printers. Printed panels were used to create a slot mounting solution for the interface cards allowing for all these cards to be easily installed or removed with much tighter spacing than would have been possible otherwise. All other panels of the enclosure were made with laser cut acrylic which greatly reduced the complexity of the case. The acrylic and 3D printed panels are joined together with acrylic epoxy which securely cements the pieces together. The lid and one of the side panels is secured using neodymium magnets rather than being epoxied in place to facilitate access to the expansion ports and microSD card when saving the data internally. The lid is made of 3 acrylic pieces reinforced with a structural rib and is also held in place with magnets but is augmented with Velcro straps to reduce the risk of it dislodging during a measurement. The main control card is screwed into the enclosure using standoffs and locking nylon insert nuts to ensure that no shorts can happen due to a loose fastener. The casing dimensions are approximately 110 mm by 185 mm by 190 mm.

Designing the electronics of the control unit to be packed so tightly in the box lead to a concern that passive cooling would be insufficient in hotter temperature environments, so an 80 mm computer fan (Noctua NF-R8 redux-1200) was also added to the case. This fan pushes cool air into the side of the case with the airflow split by the interface card backplanes with half of the air directed through the interface cards and the other half directed over the control card with all air exiting the opposite side panel. This fan has a dust filter installed as this system is intended to be used outside of the controlled environment of a lab. The dust filter with the positive internal pressure should minimize dust accumulation inside the case. Finally, the bottom of the enclosure has adhesive backed Velcro to allow the unit to be easily mounted onto the ninjaHarness which the subject uses to wear the ninjaNIRS system on their chest.

2.8. ninjaHarness

One of the most important aspects of the design for our ninjaNIRS systems is the wearability of the system during use. We decided to create a front-mounted harness (see Fig. 1(c)) rather than a backpack style holder for the ninjaNIRS as this would allow the subject to sit down on a seat with a back rest while wearing the ninjaNIRS system. With a backpack holder, the backpack would get in the way of the subject sitting comfortably on a chair. We have also found that this front-mounted harness, that we call ninjaHarness, makes it easier for the subject to put on and to remove the system to / from their head along with the 200 optodes, compared with a backpack design.

The harness was made by modifying an off the shelf baby carrier (Infantino Flip). Detailed information of how the harness was modified is available on the openfNIRS website [34]. The main features of the harness are as follows. We removed the outer flap of the baby harness and stitched in hook webbing that matches with the adhesive loop tape applied to the bottom of the control unit casing. We attached buckles with a strap to provide extra protection by securing the control unit casing against the loop tape and chest of the subject. Nylon straps were added to the shoulder straps to which we mounted the optode splitter boxes. This was done to ease cable management between the control unit casing on the chest and the optodes on the head. This also allowed for much of the weight of the cabling to be transferred to the padded shoulder straps rather than to the head of the subject. Reinforced stitching on the shoulder straps has given the harness great durability and the shoulder and waist straps distribute the weight of the unit. The waist straps also provide a convenient location to mount the battery. The harness and system are stored together in a padded pelican case and with this case a system can go from secure storage to experiment ready in a matter of minutes.

3. NinjaNIRS22 system characterization

3.1. Overall system

The whole-head system with 56 sources and 144 detectors used for the measurements in Section 7 has a weight of 4.7 kg without batteries. With two 5 Ah / 12 V batteries the weight increases to 5.6 kg. Especially important is the weight carried on the head. Our ninjaCap including all optodes weighs 910 grams, which is significantly less than a typical motorcycle helmet. The 900 grams for the two batteries and their holder is carried around the waist of the subject, and the remaining 3.8 kg for the control unit is carried on the chest of the subject, supported by the shoulder straps of the harness.

The total current draw of the system while running is approximately 1.9 A at 12 V. This current draw depends somewhat on the LED powers chosen by the calibration GUI, as well as the amount of system and ambient light reaching the detector optodes. Finally, the power draw can also be influenced by background processes running on the Raspberry Pi Zero 2 SBC. Most voltage regulators can be disabled in software, greatly reducing power consumption when the system is idle. We measured the usable capacity of one of our 5 Ah batteries at 3.7 Ah, as the battery voltage drops too low beyond that point. Using two 5 Ah batteries thus allows for a run time of approximately 3.8 hours. The batteries are hot-swappable such that one battery can be replaced at a time without the system shutting down.

3.2. Source characterization

The LEDs of our source optodes have a typical peak wavelength of 735 nm and 850 nm with a FWHM of 21 nm and 27 nm respectively. We measured a rise and fall time of less than 2 µs using our source interface card at the maximum power level. The power levels are spaced approximately by a factor of 2, with the lowest power level having a peak current of 2.2 mA and peak optical power of 0.5 mW at 735 nm and 2.6 mA and 0.7 mW at 850 nm. The highest power level has a peak current of 136 mA and a peak optical power of 53 mW at 735 nm and 184 mA and 61 mW at 850 nm. The optical powers were measured without a light-pipe installed.

3.3. Detector characterization and system dynamic performance

With a feedback resistance of 330 MOhm, the rise time of our detector optode where the signal settles to within 1% of final value is 274 µs (see Fig. 5(a)). The main bandwidth limitation is likely the parasitic capacitance of the feedback resistor. Given this settling time, we chose a holdoff time at the beginning of each state of 350 µs where no sampling happens. Additionally, there is a holdoff time at the end of each state of 10 µs. We thus chose a state length of 1.25 ms, resulting in 800 available states per second, with a sampling duty cycle of 71%, representing a compromise between number of states per second, and thus obtainable full image frames per second, and sampling duty cycle, and thus SNR.

Fig. 5.

(a) A plot showing the rise and fall time of the detection circuitry when an LED is turned on and off. The green dots also indicate when we digitize the signal during the LED on state and during the LED off state. The signal is normalized to emphasize the rise and fall times. (b) A plot of the digitized signal versus optical power reaching the detector using the 850 nm LED. Each dot represents an independent measurement where the signal is averaged over 1 second of LED on time, with 1 second of dark signal subtracted. The noise floor reveals a noise equivalent power (NEP) of 36 fW/√Hz. The signal saturates above 0.1 µW indicating a dynamic range (DR) of 131 dB.

Using the feedback gain resistance and hold off times indicated above (i.e. a 71% duty cycle), we measured a noise equivalent power (NEP) of 36.4 fW/√Hz at 850 nm, using a single source detector channel (see Fig. 5(b)). Given the active area of 7.5 mm², this results in a detectivity of 4.9 fW/mm²/√Hz. The detector saturates at a power of 116 nW, resulting in an instantaneous dynamic range of 131 dB (20·log₁₀). Given that the LED power can be adjusted between 0.7 mW and 61 mW, we obtain an additional dynamic range of 39 dB, and thus the total system effective dynamic range is 170 dB. There is no measurable electrical crosstalk between detector channels in a 1 Hz bandwidth.

The NEP and dynamic range measurement was made using a source attached to an integrating sphere through 0.5 inch tubes that permitted automatic switching of neutral density filters into the light path to sweep through several orders of magnitude of optical power, and the detector attached directly to another port of the integrating sphere. Since the instantaneous dynamic range of the instrument exceeds the dynamic range of neutral density filter combinations available to us, Fig. 5(b) was created by combining two measurement runs at different LED power settings. NEP was calculated as the intersection point of the noise floor and signal fit lines. The noise floor was estimated as the root mean square (rms) value of the signal at the lowest optical power setting of the test fixture. At this lowest optical power setting a total of 20 independent measurements were taken, whereas at all other optical power levels 4 independent measurements were acquired. The signal fit line was obtained by linearly fitting in logarithmic space all measurements at optical power levels approximately 100 times above NEP, excluding the highest 4 optical power levels where saturation effects were observed. The signal fit line slope was 1.0045, confirming system linearity. Note that the detector did not have a light pipe attached to it, which we’ve measured to reduce light transmission by about 50% (which would double the detector module NEP). Note that in an actual experiment the duty cycle, and thus the effective NEP and effective detectivity, for a given channel of data is reduced further by the specifics of the multiplexing scheme. As we describe below, our multiplexing scheme cycles through 89 states per frame of data and thus the duty cycle is further reduced by a factor of 89.

To characterize end to end system drift, we placed a source and a detector optode in a light tight box. Source power was set to the third highest power level, and light absorption between source and detector was adjusted so that the signal was about 3.5% of full scale at 735 nm and 10% of full scale at 850 nm. We then acquired data for 60 minutes. The acquired signal was down sampled to 1 Hz by averaging. We then calculated the change of signal in rolling one-minute windows, excluding a 5 minutes warm up period. The maximum observed signal change was 0.12% at 735 nm and 0.073% at 850 nm. The root mean square value of the changes was 0.030% and 0.019% at 735 nm and 850 nm respectively. Similarly, electrical crosstalk was characterized by placing a source optode and an ‘offender’ detector optode in a light tight box with varying optical absorption between them. ‘Victim’ detector optodes were placed in another close-by light tight box. We then measured the signal increase in the ‘victim’ optodes. This signal rise was always significantly less than the standard deviation (noise) of the signal.

4. Multiplexing strategy

The primary method to distinguish light from different source optodes is to turn them on sequentially (i.e. temporal multiplexing). Ambient light can be estimated by inserting dark states where no LED source is turned on. In addition, in our whole head system groupings of optodes are far enough apart that it is possible to turn on multiple LEDs at the same time without danger of channel crosstalk. We use this strategy (spatial multiplexing) to increase our frame rate. This spatial multiplexing does limit us to measuring signals from only the nearest (∼18 mm) and second nearest (∼33 mm) detectors as the third nearest detectors are generally the second nearest to another simultaneously illuminated LED. Finally, we illuminate each LED twice during each frame, once at high power to capture long separation channels, and once at low power to capture short separation channels that generally saturate under high LED power. Specifically, in the whole head configuration of ninjaNIRS22, we distinguish 7 areas on the head, corresponding to the 7 source bundles (see Fig. 6(a)). At the high-power setting, up to 3 areas can have one optode illuminated at the same time with minimal interference. At the low power setting, all 7 areas can have one optode illuminated simultaneously. During a single frame acquisition (as illustrated in Fig. 6(b)), we first cycle through all sources at low power simultaneously, then through groups A, C, E at high power, then B, D, F at high power, and finally group G at high power. At low power we insert only one dark state after cycling through the whole group. At high power we insert a dark state after multiplexing the two wavelengths for each optode. Thus, for recording the high LED power states, we also record 24 dark states compared with recording 1 dark state for the low LED power state.

Fig. 6.

(a) A flattened version of the whole head fNIRS arrangement of 56 sources (red numbers) and 144 detectors (blue dots). The sources are in groups (A through G) of eight. For spatial multiplexing, at any given time either all LEDs are OFF to measure the ambient light, or an LED is ON from multiplexing unit 1 (groups A, C, E), unit 2 (groups B, D, F) or unit 3 (group G). (b) illustrates the spatial multiplexing strategy as detailed in the text.

Multiplexing through an entire image frame of data thus takes a total of 89 states, 17 states for the low LED power measurements (i.e. 1 group at low LED power X 8 optodes per group X 2 LEDs per optode X 1 state per LED + 1 dark state) and 72 states for the high LED power measurements (i.e. 3 groups at high LED power X 8 optodes per group X 3 states per optode (i.e. two wavelengths and one dark state)). Given 800 states per second as described above, this results in a frame rate of 8.99 fps. Compare this to not using spatial multiplexing in which we have 168 states for the high LED power (56 optodes X 3 states per optode (i.e. two wavelengths and one dark state)) and the same 17 states for the low LED power for a total of 175 states and a frame rate of 4.57 fps. No spatial multiplexing allows us to get measurements at all available distances while spatial multiplexing constrains us to first and second nearest measurements only as longer distances will have crosstalk from other simultaneously illuminated LEDs.

The resulting data rate for this whole head configuration with 18 detector cards, auxiliary channels and one accelerometer is 426 kbytes/s, independent of the multiplexing scheme used as this only depends on the bytes per state and the number of states per second. For each state, we record 3 header bytes from the FPGA, 8 bytes of auxiliary channel data, 28 bytes (3 bytes for each of the 8 detectors plus 4 header bytes) from each of 18 detector boards, and 18 bytes from the single IMU attached to the QWIIC connector on the control board (2 bytes from each of the 6 axes plus 2 bytes for temperature and 4 header bytes). Thus, there is a total of 533 bytes per state. Given 800 states per second, we thus arrive at 426 kbytes / sec.

5. Firmware

The FPGA Firmware centers around a programmable state machine. Its behavior is defined by two memories of 1024 rows and up to 32 columns each. The rows represent individual states, whereas the columns represent signals sent to other parts of the system or FPGA internal logic.

The first memory “RAM B” controls system internal events such as ADC sampling and data routing. This sequence is the same for each sampling state described above in the Multiplexing section. In our typical configuration, each RAM B internal event state lasts 1.25 µs. However, this can be changed in the GUI software by changing a configuration register controlling a variable clock divider. In our typical configuration, 1000 RAM B states are active. When state 1000 is reached, the state machine returns to state 1 and sends and ‘advance’ signal to RAM A. The first several of these 1000 RAM B states are directing the RP2040 on each detector interface card to prepare the data sampled and summed during the previous multiplexing sampling state to be transmitted to the FPGA on the control board. The rest of the RAM B states are then transmitting the previously summed samples to the control board and directing the timing of the 237 ADC samples for the current state for each of the 144 detectors, accounting for the 350 + 10 µs holdoff time described in Section 3.3 above. Note that we arrive at 237 samples because we can sample for the state period duration of 1250 µs minus the hold off time of 360 µs equaling 890 µs, which provides for 712 intervals of 1.25 µs each. In one interval we give an ADC sample trigger which must be followed by two intervals of nothing to provide time for the ADC to complete the sample. Thus, in 890 µs, we are able to trigger 237 independent ADC samples.

The second memory “RAM A” controls which LED(s) are powered at which power level. That is, RAM A encodes our multiplexing sampling states. In our configuration, each RAM A state lasts 1.25 ms. When a stop bit is encountered, RAM A returns to state 1 when it receives the ‘advance’ signal from RAM B otherwise it advances by one state. Therefore, the number of A states is set by the GUI and depends on the probe. In our full head probe, there are 89 RAM A states as described in the Multiplexing section above. A full description of RAM A and RAM B is provided on the openfNIRS website [34].

A different part of the FPGA firmware receives the data from all detector cards as well as the auxiliary and accelerometer data and assembles the final data stream. The data is then sent via UART to the FTDI Serial to USB converter. This UART link features hardware flow control due to the USB bus being busy at times with other devices. A first-in first-out (FIFO) buffer using both FPGA internal block RAM as well as an 8 MByte pseudo static random access memory (PSRAM) attached to the FPGA stores data if it can’t be immediately transmitted.

6. Acquisition software

We extended the ninjaGUI software [35] developed in the Matlab environment to control ninjaNIRS22. This fork of the ninjaGUI software is available on GitHub. When launching the ninjaGUI software, it asks the user to select the configuration file for connecting with the ninjaNIRS22 system. It then asks for the probe geometry file that the user has designed with AtlasViewer [36] and which details the arrangement of sources and detectors on the head of the subject as well as all the pairs of sources and detectors that comprise the measurement channels. The software then connects with ninjaNIRS22 and displays a flattened version of the probe geometry.

Assuming the user has already placed the probe on the subject’s head, the user should then select the menu item for calibrating the LED power levels and providing real-time feedback for optimizing coupling through the hair of all the sources and detectors. In this calibration GUI, the user should first calibrate LED power levels. This procedure takes about 25 seconds collecting data from each LED at each of the 7 power levels plus a dark measurement and plotting the results. The GUI then determines the optimal HIGH and LOW power level for each LED to provide the best signal-to-noise ratio without saturating the detectors. The average dark level signal for each channel is plotted on the flattened probe geometry display so that the user can determine if too much ambient light (i.e. not from the ninjaNIRS22 system) is reaching the detectors (see Fig. 7(a)). We often use a dark shower cap to help block ambient light and sometimes one can see more ambient light leaking into the detectors around the edges of the shower cap. Generally, we observe that the dark signal is below 1e-4 AU (arbitrary units) where the maximum possible signal is normalized to 1 and the signal is generally considered saturated above 0.7 AU (as seen in Fig. 5). Note that the detector circuits have a bias that corresponds to about 1e-4 AU, and thus the reported dark signal generally does not go below 1e-4 AU. This bias is not an issue when measuring the LED signals since it is cancelled out when subtracting the ambient light signal measurement from the LED signal measurement. In addition to displaying the channel-wise dark signal, the power spectrum of the dark measurement averaged over all detectors is plotted so that the user can assess if room light modulation is adversely impacting the measurements (see Fig. 7(b,d)). Room lights today are generally using efficient white light LEDs that are pulse-wave modulated. The pulse-wave modulation frequency is not standardized, but is generally greater than several kHz, but sometimes can be much lower and closer to our temporal multiplexing frequency which can challenge separation of fNIRS detector signals from ambient light levels. If a peak above 0 Hz is seen in this power spectrum, it is an indication that modulated ambient light is being detected and could be compromising the measurement. We provide an example of this room light pulse wave modulation appearing in our dark signal in Fig. 7(d).

Fig. 7.

(a) shows the dark level (i.e. ambient light level) signal measured on all nearest and second nearest neighbor measurements with the cap of optodes on a subject sitting near a sunny window. (b) is the average power spectrum of the dark signal from all channels for the data in (a). (c) and (d) show the same when the window shade is closed to block sunlight and a dark shower cap is placed over the cap of optodes to further block ambient light. The dark level drops below 10⁻⁴. Peaks are evident in the power spectrum which arise from a pulse-wave modulated LED light source in the room leaking into the detectors. Generally, the experimenter should aim to reduce these peaks to below the noise floor by ensuring that sufficient care is taken to block any ambient light.

After the LED power levels have been calibrated, the calibration GUI then goes into a real-time signal quality check and every second updates a flattened display of the probe to indicate signals that either have poor signal or channels that are saturated (Fig. 8(a)). Conveniently, the GUI identifies a few of the worst sources and detectors in terms of how many channels with that source or detector have poor signal or are saturated (Fig. 8(b)). These sources and detectors are identified by optode number and the GUI indicates those optodes on the flattened probe display to guide the user in quickly adjusting those optodes to help them make better scalp contact through the hair of the subject. After adjusting several optodes, the user should then recalibrate the LED power levels to further optimize signal levels. The user repeats this procedure until they have optimized signal levels to their satisfaction. Depending on the subject’s hair characteristics, it is generally not possible to get good signal from all channels. We find that this optimization procedure will take from 5 to 20 min for our system with 56 source and 144 detector optodes. The resultant signal versus distance plot for all 567 channels for each of the two wavelengths for a typically good subject is shown in Fig. 9. Having completed signal level optimization, the user then returns to the main GUI window to commence experimental data collection.

Fig. 8.

A screen shot of the real-time signal quality adjustment GUI which displays saturated channels in red and channels with poor signal quality in blue and black, with black indicating very poor signal quality. The sources (red) and detectors (blue) producing poor signal quality on the most data channels are indicated which helps the experimenter to rapidly identify where to focus attention on adjusting optode coupling to the scalp through the hair.

Fig. 9.

The signal measured on a subject versus source-detector distance. This measurement is taken during the LED power calibration step when no spatial multiplexing is utilized and thus every detector can uniquely measure light from every source. Results are shown for the measurement at low LED power and high LED power and reveal that signals can be measured above the noise floor beyond 60 mm on this subject. The noise floor is indicated by the dashed black horizontal line at approximately the 4·10⁻⁶ level where the signal decrease with distance is no longer evident because of the constant electronic noise floor. To be clear, any signal level above this noise floor is considered to be above the noise level. Note that the noise floor signal is around 4e-6 AU which is 20x higher than the noise floor signal of 2e-7 AU seen in Fig. 5(b) because these data were averaged over ∼2.5 ms as opposed to 1.0 s.

7. Human brain function results

To demonstrate the imaging potential of this whole-head HD-fNIRS system with 56 sources and 144 detectors, we had n = 3 subjects perform a ball squeezing task with their right hand. The human subject protocol was approved by the Boston University Internal Review Board. The ball squeezing task was performed for 5 seconds followed by 15 seconds of rest, similar to what we did in [37]. This was repeated 14 times in a 5 min run and each subject performed 3 runs for a total of 42 task repeats each. Measurements were acquired for the nearest neighbor (19 mm) and second nearest neighbor (33 mm) detectors for each source resulting in 567 channels of data for each of the two wavelengths. For these measurements we used the commercially available NIRx NSP2 dual-tip source and detector optode bundles because we have not fabricated 200 of our own ninjaNIRS22 optodes described above. Our characterization of these NIRx optodes revealed that their performance was slightly better than that of our own optodes. Additionally, the NIRx optodes benefit from a dual tip design that is advantageous for reaching through hair. To process the data in channel space, we converted the signal to change in optical density, low pass filtered the data with a 0.5 Hz cut-off frequency and used the General Linear Model (GLM) implemented in Homer3 to estimate the hemodynamic response function (HRF) [38]. We used sequential gaussian functions from -2 to 16 seconds to model the HRF, the average of all 19 mm channels as the short separation regressor and the 3rd order polynomial as the drift regressor. The group average HRF from 5 to 10 seconds is displayed in the circular channel plot shown in Fig. 10(a) where color indicates the magnitude of the HbO response to the task. To get the image space result, we followed the procedures in [37] for reconstructing images in the scalp and brain. The group averaged image result for HbO is shown in Fig. 10(b) and the HRF time course averaged over the vertices in the image with an HbO value greater than 50% of the peak HbO result is shown in Fig. 10(c).

Fig. 10.

A group average from n = 3 subjects performing a hand ball squeezing task with their right hand. (a) shows the oxygenated hemoglobin response with red indicating an increase and blue indicating a decrease in concentration. (b) shows the image reconstruction, again with red indicating an increase in oxygenated hemoglobin and blue indicating a decrease. (c) The time course of the hemoglobin response from the region of interest in the brain with the peak of the oxygenated hemoglobin response. Standard error bars across the three subjects are indicated.

8. Discussion

In this paper, we have detailed the development of our wearable whole-head high-density fNIRS system, which we call ninjaNIRS22. This system has evolved significantly since our first prototype in 2016, which led to an NIH award (R24-NS104096) for “Establishing an fNIRS Ecosystem for Open Software-Hardware Dissemination.” This award enabled us to build ninjaNIRS20, which featured 24 optodes, each with a combined dual-wavelength LED and an SiPD. Subsequently, another NIH award (U01-EB029856) titled “The Neuroscience of Everyday World - A Novel Wearable System for Continuous Measurement of Brain Function” allowed us to evolve the system further. This led to the creation of ninjaNIRS21, which featured individual optodes for sources and detectors, and ultimately to the ninjaNIRS22 system presented here, which includes 56 source and 144 detector optodes. We have made the documentation for all the circuits used in these systems openly available through our website, openfnirs.org. We encourage discussions about these systems through the forum on our website, and we also invite anyone expanding on these hardware designs to share their progress with the community through that forum.

After presenting details of the hardware design, we then presented characterization of the system. We showed that SiPD amplifier rise-time resulted in us choosing a sampling hold off time of 350 µs after switching LED power prior to sampling the signal. We sample and average the signal 237 times over ∼900 µs to reduce noise. This results in a total state sampling time of 1.25 ms (considering the holdoff time for settling and the time for digitization), allowing us to multiplex through 800 states per second. We described our spatial and temporal multiplexing strategy for recording signals at two different LED power levels that provided an image frame rate of 8.99 fps compared with 4.57 fps if we only used temporal multiplexing without spatial multiplexing. We note that we could decrease the sampling period from ∼900 µs to 650 µs to record 1000 states per second. This would increase the frame rate 20% with minimal impact on the noise performance of the system as the duty cycle would drop from 71% to 65%. Another idea we have considered for increasing image frame rate is to collect data only at 850 nm which is the wavelength that is most optimally sensitive to brain activation [39], as this would increase the frame rate from 8.99 fps to 14.04 fps with spatial multiplexing and from 4.57 fps to 6.6 fps without spatial multiplexing.

The noise performance and system dynamic range was reported in Fig. 5(b) showing a noise equivalent power of 36 fW/√Hz and a dynamic range of 131 dB. This is more than 3x better than typically published wearable fNIRS systems with SiPDs [31,40] and comparable to the best published SiPD fNIRS system [41]. Recently published systems using newly available silicon photomultipliers (SiPM) [42,43] achieve even lower NEPs and we anticipate that new wearable systems in the future will be able to reach an NEP of less than 5 fW/√Hz with currently available commercial SiPMs. This is a direction we hope to develop in the future as it will increase the inclusion of subjects with more challenging hair characteristics (e.g. darker and coarser hair) in fNIRS studies. Our system design preserved the large instantaneous dynamic range of 131 dB that is afforded by SiPDs. As shown in Fig. 9, this dynamic range permits measurements over a range of more than 60 mm of separation between the source and detector.

We were surprised with how quickly we could optimize the signal across so many optodes. This was greatly facilitated by four different features of the ninjaNIRS system. First, LED powers could be optimized to one of seven different levels ranging from ∼0.7 mW to 70 mW peak power to account for differences in optode coupling to the scalp. Additionally, we also make measurements at two LED powers to ease constraints on measuring nearest neighbor channels and more distant channels. Third, the GUI provides real-time feedback (updated under 1.0 sec) that quickly directs the user to the sources and detectors which will benefit most from manual adjustment for improving coupling through hair, as shown in Fig. 8. Finally, our ninjaCap permits access to the hair around the optodes to facilitate movement of the hair with a hair pick. We have used systems with a fully closed cap that prevents access to hair from around the optodes and in our experience it takes much longer to optimize signals with those systems. None-the-less, we have found that moving the hair around the sides of the optodes is more challenging for a high-density array of optodes than a sparse array as the gaps between the optodes becomes much smaller. In particular, for subjects with long hair, we have found that moving the hair from one optode often times moves it to under the neighboring optode. Related to this and other constraints, we never achieve an “optimal” signal for all channels. At present, we define “optimal” by below the saturation level of ∼0.7 and above an arbitrary level of ∼0.01. We will investigate using a lower level determined by the scalp coupling index and pulse spectral power as recommended in [44].

We note that ninjaNIRS22 has been built with the idea of supporting several future improvements. We are also presently working on a straight-forward modification of the source interface card that will allow spatially multiplexed LEDs to each have their own optimal LED power level, whereas presently all LEDs on at the same time must be on at the same power. We expect that this will greatly facilitate obtaining optimal signal levels on all channels for many subjects. This modified source card will also support driving new LEDs that can reach peak optical powers greater than 1 W. Given that multiplexing already results in each LED being on less than 2% of the time, and they can be turned on for even less time if desired, we can pulse near-infrared LEDs at peak powers greater than 1 W and still be within tissue heating safety limits set in ANSI Z136.1. This greater than 10x increase in LED peak power will be beneficial for subjects with more challenging hair characteristics and darker skin pigmentation, helping to overcome the extra attenuation of signal experienced when coupling optodes to the scalp. We also plan to upgrade the detector module to support SiPMs which we anticipate will reduce our NEP to below 5 fW/√Hz. Finally, we are also presently working on utilizing the Raspberry Pi Zero 2 single board computer mounted on the control card to communicate with the experimental computer over Wi-Fi so that the subject being measured with ninjaNIRS22 will no longer be tethered via a USB cable with the data acquisition computer. We note that future software development on the Raspberry Pi Zero will permit data to be acquired and stored on an onboard microSD card such that measurements can proceed without any link to an external computer. Please follow future updates to this system by us and hopefully others at the forum at openfnirs.org.

Funding

National Institute of Biomedical Imaging and Bioengineering10.13039/100000070 (U01EB029856); National Institute of Neurological Disorders and Stroke10.13039/100000065 (R24NS104096).

Disclosures

AvL is currently consulting for NIRx Medizintechnik GmbH. The authors declare no other conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.