Design and Evaluation of a Cable-Driven fMRI-Compatible Haptic Interface to Investigate Precision Grip Control

Abstract

Our hands and fingers are involved in almost all activities of daily living and, as such, have a disproportionately large neural representation. Functional magnetic resonance imaging investigations into the neural control of the hand have revealed great advances, but the harsh MRI environment has proven to be a challenge to devices capable of delivering a large variety of stimuli necessary for well-controlled studies. This paper presents a fMRI-compatible haptic interface to investigate the neural mechanisms underlying precision grasp control. The interface, located at the scanner bore, is controlled remotely through a shielded electromagnetic actuation system positioned at the end of the scanner bed and then through a high stiffness, low inertia cable transmission. We present the system design, taking into account requirements defined by the biomechanics and dynamics of the human hand, as well as the fMRI environment. Performance evaluation revealed a structural stiffness of 3.3 N/mm, renderable forces up to 94 N, and a position control bandwidth of at least 19 Hz. MRI-compatibility tests showed no degradation in the operation of the haptic interface or the image quality. A preliminary fMRI experiment during a pilot study validated the usability of the haptic interface, illustrating the possibilities offered by this device.

1 Introduction

Our hands are involved in nearly every physical interaction with the environment, most prominently in the grasping and manipulation of objects. Object interactions are governed by a rich variety of mechanical properties, such as stiffness and viscosity. The ability to delineate objects based on these differences is a critical and complex challenge in motor control [1], [2].

Mechanisms of grasping have been the subject of research for decades. Napier (1956) developed the modern ontology of prehensile movements, including the definition of precision grip, i.e. grasping an object using one or more fingertips opposed by the thumb [3]. Precision grip, most highly developed in humans, allows for fine object manipulation fed by mechanoreceptors in the skin that detect pressure, strain and friction [4]. It is widely believed that it is at least partly controlled by direct, monosynaptic corticospinal projections of pyramidal cells from the primary motor cortex [5], and recent work using functional magnetic resonance imaging (fMRI) has shown that precision grip control involves far more areas of the brain than the motor cortex. A differential whole-brain representation of precision grip has been found compared to power grip [6], as well as correlated to force level [7]. Furthermore, neural correlates of grasping measured via fMRI have been used as an assessment in aging [8] and neurological injury [9]. While there is much to be explored, including the neural mechanisms underlying object identification and manipulation and grasp stability (e.g. [10], [11]), such research requires a testbed capable of rendering variable parameters. Robotic tools are an ideal support for these investigations, as they are capable of presenting well-controlled and reproducible stimuli, and can provide objective and sensitive behavioral assessments of positions and forces involved in the task.

FMRI has established itself over the past years as a standard tool for non-invasive advanced brain research and devices used to supplement imaging experiments are playing an increasingly important role. Providing whole-brain coverage of structure and function with good spatial and adequate temporal resolution, it can give insights into the neural correlates associated with finger-object interactions. Existing devices have been engineered for arm [12]–[15] and hand [16]–[18] motions. While it is possible to instrument them for finger experiments, a custom device developed purely to study finger motion has the potential to dramatically improve control bandwidth and transparency. The dexterity of the fingers together with the small range of motion and relatively small interaction forces during object manipulation (limiting possible movement artifacts) make the hand the ideal body part for motor control studies during fMRI. A simple 1-DOF prototype force feedback device based on piezoelectric actuation was proposed for virtual reality (VR) – fMRI in [19], while in [20] a MRI-compatible forefinger manipulator actuated via a hydrostatic transmission was used to examine brain changes induced by passive forefinger movements. However, due to their inherent design characteristics (low output force [19] and low force control bandwidth [20]), these devices cannot match the dynamic capabilities of the human fingers.

An MRI system presents a harsh environment for any mechatronic device due to the high magnetic flux density and associated spatial gradient, the powerful radio-frequency (RF) pulses and the high-sensitivity of the receivers [21]. This prevents the use of conventional robotic systems by limiting the employment of electronics and hardware best suited to render a variable dynamics environment, and creates critical tradeoffs in actuator performance (for a detailed overview of actuation methods in MRI robotics see [22]). Co-located actuation can be achieved using MRI-compatible actuation technologies based on electric fields (e.g. ultrasonic motors [23], electro-rheological fluid brakes [24], electrostatic actuators [25], electroactive polymers [26]) or by taking advantage of the large static magnetic field of the scanner (e.g. MRI-powered actuation [27]) to generate loads and movements, however they present many difficulties for force feedback along with potential electromagnetic (EM) interference. EM actuators located close to the scanner bore have also been used, typically transmitting force and motion via long, stiff and lightweight rods. Although successfully shown for hand/arm movements of larger ranges, manipulation of these considerably sized mechanical structures introduces high inertia, as well as high reflected motor inertia, leading to relatively low mechanical bandwidth [17], [18]. Moving further away from the bore, remote actuation via pneumatic and hydraulic transmissions provide greater flexibility in actuator placement, but can suffer from high inertia, limited bandwidth (due to air and oil compressibility), transmission friction, and in the case of hydraulics, non-backdrivability, presenting difficult control problems [13]–[15]. Thus, all of these aforementioned actuation and transmission paradigms have not been able to match the dynamic abilities of the human fingers.

Cable transmissions present a good weight-to-length ratio that limits moving inertia, especially over long distances, while preserving stiffness, providing the basis for high performance haptic interfaces (e.g. WAM Arm (Barrett Technology Inc., USA), Freedom 6S (MPB Technologies Inc., Canada), Phantom Desktop (Sensable Technologies Inc., USA), [28], [29]). Combining shielded EM actuation [18], [30] with a cable transmission can thus represent a promising means to achieve low friction and high bandwidth, with sufficiently low output impedance to allow remote actuation and sensing from beyond the 20 mT safety line in the scanner room. This paper presents NeuRoGrasp, a fMRI-compatible finger interface based on such actuation and transmission principles. Following a short overview of the haptic and MRI-compatibility requirements for such a system in Section II, the design and implementation of the haptic interface is described in Section III. An in-depth performance characterization of the device as well as its MRI-compatibility are presented in Section IV, while Section V provides a discussion of the achieved performance. This device will allow exploration of the neural basis of precision grip and object perception.

2 Requirements for an fMRI-compatible haptic finger pinching interface

2.1 Haptic Requirements

Our goal was to design and develop a haptic interface for thumb/index finger opposition movement capable of rendering a large range of mechanical impedances representing physical object properties. As such, the device needed to be close to transparent (the device should minimally resist the user's movement) for simulation of free movement, able to render viscoelastic force fields over a wide range, and become stiff enough to passively move the individual's relaxed fingers. This is particularly important when using such a device to passively open the fingers of stroke patients who may involuntarily resist the movement (depending on stroke severity and hypertonia level in the affected hand). As neurologically impaired patients often suffer from contracture [31], the ability to render large forces is necessary for various training regimens. Moreover, as the movements of the fingers involved in precision grip are independently controlled [32], two separate DOF were required. Position adjustment of the finger interface was another desired characteristic, since hand location in supine position can vary by up to 50 cm depending on the individuals height.

The capabilities of the human hand provide a basis for the design of the haptic interface. We defined a maximum aperture (opening) of 125 mm derived from [33], a maximum force of 60 N [34], a speed of 0.74 m/s [35], and a bandwidth of at least 10 Hz [36]. As the perceptual threshold of force exerted at the fingertip is very small (< 0.1 N), friction force felt at the end-effector was aimed to be as small as possible (ideally < 1 N). The second column of Table 3 reviews these requirements.

Table 3.

Summary of performance characteristics of NeuRoGrasp with a cable transmission length of 2.6 m.

PERFORMANCE MEASURE	DESIRED VALUE	ACHIEVED VALUE
Range of motion	125 mm [33]	125 mm
Position resolution (encoder) *	n.a.	0.00005 mm
Velocity resolution @ 1 kHz *	n.a.	0.05 mm/s
Static friction (inherent / minimum apparent) **	< 1 N	2.8 N / 0.2 N
Apparent mass (uncontrolled / feedforward controlled) **	n.a.	0.24 kg / 0.39 kg
Apparent damping (uncontrolled / feedforward controlled) **	n.a.	17.8 N/(m/s) / 4.4 N/(m/s)
Output force (peak) **	> 30 N (∼50% of maximum pinching force) [34]	94 N
Output force (continuous) **	> 10 N (∼15% of maximum pinching force) [44]	38 N
Maximum stiffness **	n.a.	3.3 N/mm
Maximum damping **	n.a.	85 N/(m/s)
Maximum velocity / acceleration *	> 0.74 m/s / n.a. [35]	0.6 m/s, 70 m/s 2
Position bandwidth (closed-loop) **	> 10 Hz [36]	> 19 Hz
Force bandwidth (open-loop) **	> 10 Hz [36]	15 Hz
Impedance range [Z-width] **	n.a.	29 dB / 41 dB (6 Hz / 0.5 Hz)

^*Theoretical at output; measured at input.

^**Measured at output.

2.2 fMRI-Compatibility Requirements

Several criteria must be taken into consideration for a robotic device with EM actuation to function properly in an fMRI environment:

2.2.1 Safety

The device must not present any safety hazard to the participant, operator, medical staff or the fMRI equipment. Due to the presence of a strong static magnetic field (may range from 1.5 T to 9.4 T) and the associated spatial gradient, large ferrous structures should be avoided as high attraction forces can act upon them (missile effect) if placed in the spatial gradient of the magnetic field in the vicinity of the scanner (small ferrous components may still be used if placed at a safe distance and properly anchored). Moreover, in order not to disturb the homogeneity of the magnetic field of the scanner, mechatronic components should be constructed from materials with low magnetic susceptibility (for a review see [37]). Non-metallic materials (e.g. polymers) are ideal for such device developments. However, for structural stability, nonmagnetic metals (e.g. aluminum, brass, titanium) can also be used if static. Limiting displacement of conducting materials is important to minimize the eddy currents induced by the magnetic field gradients and RF pulses of the scanner, which can result in thermal and mechanical effects or even in image degradation [38].

2.2.2 Bidirectional electromagnetic compatibility

Careful attention should be given to the choice and placement of electric and electronic components of the robotic interface, as they often emit RF signals or generate magnetic fields and thus can produce artifacts in the MR images. Proper shielding and grounding should be performed, so that normal operation of actuators and sensors does not disturb the fMRI acquisition. Conversely, the fMRI environment (switching gradients and RF pulses) should not affect the functionality and performance of the robotic device (EM interference) [39].

2.2.3 Limited workspace

Access to the scanner bore, a cylinder of typically about 60–70 cm in diameter occupied by the subject during imaging, is limited. For a haptic interface designated for finger pinching movements, the area of interest (i.e. the hand region) is located at the entrance of the scanner bore for most MRI systems, which gives more flexibility to the size, position and orientation of the end-effector, but still imposes design constraints.

2.2.4 Movement artifacts

Head movements (induced through motion of various body parts) during an fMRI experiment should be less than the size of a voxel, as they can lead to spin history effects and susceptibility artifacts that induce signal loss, intensity modulation and image distortion [40]. The smaller the forces and movement ranges that are involved when the subject interacts with the robotic device (e.g. during pinching tasks), the smaller the chances are of such artifacts.

3 Design and Implementation of NeuRoGrasp

Taking into consideration the aforementioned constraints and requirements, a solution based on conventional EM actuation coupled with a cable transmission for remote actuation was found to be optimal for the given application. EM actuators can achieve high accuracy and precision in position and torque control, and produce high quality force feedback while presenting low internal friction and low motor inertia. In combination with a light, stiff yet flexible cable transmission, this results in a transparent, high bandwidth and high Z-width haptic interface for dynamic interaction with the fingers. NeuRoGrasp consists of a shielded actuation system that delivers the necessary motion or force to an end-effector finger interface positioned at the entrance of the scanner bore through a cable transmission (Fig. 1). The main components of the system are detailed in the following.

Fig. 1.

NeuRoGrasp haptic interface placed inside the MRI room during a usability test.

3.1 Shielded Actuation System and Control

The actuation system (Fig. 2, Top) is comprised of two conventional EM motors (RE35, 150 W, Maxon Motor, Switzerland), each actuating one cable loop (e.g. for thumb and index finger). They are coupled with high-resolution rotary optical encoders (R137, 720k counts/rev; Gurley Precision Instruments, USA) to allow good velocity resolution (0.05 mm/s at a sampling rate of 1 KHz) and are controlled by two linear servo drive amplifiers (4-Q-DC Servo Control LSC 30/2; Maxon Motor, Switzerland). In order to isolate the actuators and all peripheral devices from the EM field generated by the scanner and vice versa, the electromagnetic actuators are placed within cylinder-shaped mu-metal magnetic shields, which together with the power and control electronics are enclosed within an aluminum Faraday cage. The latter prevents electromagnetic interference (EMI) in the high frequency range. The ferromagnetic shield is used to prevent any dynamically varying magnetic fields generated by the EM motors during operation from disturbing the imaging, as well as to avoid performance degradation of the haptic interface due to the local static field of the MRI magnet. For safety reasons and to facilitate transportation and installation of the system, the actuation box is rigidly anchored on a MRI-compatible cart placed at the end of the scanner bed in a region where the local magnetic field strength is low compared to that of the imaging region [30]. Power and signal cables to the actuation system can carry noise from the outside (control room) into the MRI room and disturb the imaging, respectively pick up noise from the scanner during imaging. To prevent EMI, the 24 V lines powering the actuation box are fed through low-pass filters (Electric filter 16 A / 100 VDC, max. 60 Hz; Imedco AG, Switzerland) integrated both in the actuation box and in the penetration panel between the scanner and control rooms. Moreover, communication with the host PC located in the control room is performed via a fiberoptic Ethernet converter (Roline RC-100FX/SC Fast Ethernet Converter; Rotronic, Switzerland), which electromagnetically decouples the real-time target in the actuation box from the host PC in the control room (Fig. 2, Bottom). Any opening in the Faraday cage – either for the optical fiber connection, or to guide cables from motor pulleys towards the finger interface – is made through waveguides (aluminum tubes with length about 5 times the diameter) that prevent EMI through radiation.

Fig. 2.

Top: Components of the shielded actuation box, containing the two EM motors, as well as the powerand control electronics. Dimensions of the box are 55×30×32 cm³. Bottom:Control monitoring is performed from a host computer in the control room through optical fibers entering the actuation box through waveguides. Power lines enter the actuation box through low-pass filters. Visual display is presented to the subject on an MRI console.

Control is performed via a multi-rate timed-loop structure implemented in LabVIEW 2012 (National Instruments Corp., USA) running on a host computer (Windows operating system) located in the control room and a real-time target situated in the shielded actuation system in the scanner room. This is a dedicated system (CompactRIO-9074 integrated system with real-time controller and reconfigurable FPGA chassis, National Instruments Corp., USA) running all the time critical tasks such as data acquisition and control. Communication between the host and the real-time target is carried out using LabVIEW Shared Variable Engine. During experiments where visual feedback is presented to the subject, this is done through a MRI console situated in the scanner room and linked to the host computer using a HDMI connection (Fig. 2, Bottom).

3.2 Cable Transmission

The cable transmission system is composed of two independent closed-loop drives that actuate the two DOF, one for each finger. It consists of aluminum profiles that hold redirection modules (Fig. 3A) as well as a length and tension adjustment mechanism through which the cables are passed (Fig. 3B). For structural stiffness, the custom made guiding pulleys (outer structure of polylactide (PLA) material, combined with low friction polymer/glass ball-bearings) are fixed on an aluminum bar link rigidly attached on the side of the scanner bed through threaded inserts. In order to prevent slippage, each cable is rigidly fixed at both ends to the capstan attached to the driving motor shafts (Fig. 3C).

Fig. 3.

A: Side view CAD rendering of the cable transmission consisting of two closed-loop drives fed through a length/tension adjustment mechanism. For both DOF the cable is guided from the actuation system to the end-effector finger interface by 6 sets of cable redirection modules. Each redirection module consists of two horizontal pulleys on the side that copes with height changes, as well as two vertical pulleys on the side that copes with lateral deviation of the cables. B: Schematic of the length/tension adjustment mechanism: an MRI-compatible spring between two polymer sliding modules generates a force R, adjusting the cable tension T in the transmission. C: Schematic portraying the cable fixation on the two sides of the capstan which is attached to the motor shaft.

Cable properties are critical as they influence the transmission performance. Since the total cable inertia is small compared to the load inertia, the main selection criterion is a high Young's modulus. We use a high-density polyethylene (Dyneema) cable with E = 107 GPa and a diameter of 0.6 mm, as it yields a stiffness similar to that of the cable transmission presented in [28], allowing a low strain value even during the maximum finger pinching forces. Cable wear (due to friction along the transmission) is taken into consideration and the cable can be easily exchanged in case of an unexpected breakdown during an fMRI study.

3.3 Cable Length/Tension Adjustment Mechanism

Transmission cable length can be regulated through a special mechanism to adapt the system to the user in order to achieve a comfortable posture. It consists of two modules sliding vertically over 30 cm (corresponds to a cable length adjustment of approximately 50 cm) on a rail rigidly fixed to an aluminum profile (Fig. 3B). As cable tension affects both slippage and friction in the transmission, assuring a constant cable tension is essential to achieve reproducible conditions despite moving around and adjusting the setup. Thus, a compression spring between the two sliding modules is used to set the tension in the cable:

$$T=\frac{R}{2\cdot cos(\alpha /2)}$$ (1)

where α is the angle between the incoming and outgoing cable and R is the spring force.

Once the tension is set, the two sliding modules are locked on the rail using thumb screws to eliminate compliance added to the transmission by the spring.

3.4 Finger Interface

The finger interface must comfortably and safely fit a variety of users performing precision grasp tasks while inducing a minimum amount of interference with the unused fingers. It is composed of two independent finger modules (made of PLA) interacting with the thumb and index finger, running on two parallel aluminum rails (Fig. 4A). These rail guides (which we have previously used in [41]) offer a very low cost off-the-shelf solution, are MR-compatible and don't require any lubrication (important for clinical environments such as the MRI scanner). The finger modules are clamped to the two cable loops using a simple pin mechanism (Fig. 4C), and allow for a finger aperture up to 125 mm (limited by the length of the aluminum rails). They can easily be exchanged to fit various finger sizes or to conduct different pinching tasks, including precision grasp and multi-finger grasping (e.g. thumb against index and middle finger). As finger movement during a pinching task is not linear, two extra DOF are introduced through adjustable mechanical fixation structures to allow free forefinger and thumb rotation during the linear grasping movement. Velcro straps are used to fasten the fingers to the finger modules to prevent a loss of contact during opening movements, as well as in patients who may have difficulties in maintaining contact with the modules. In order to measure pinching forces or to reduce the apparent dynamics of the finger interface through force feedback, a custom made MR-compatible force sensor is attached to one of the finger modules (Fig. 4B). This is a compact and integrated elastic probe for fiberoptics-based force sensing, developed using low-cost off-the-shelf 3D printing technology [42].

Fig. 4.

A: End-effector finger interface with a typical hand pinching position. The two finger modules are attached to each of the two cable loops. An adjustable supporting arm allows for positioning the finger interface in various orientations. B: Custom made MR-compatible force sensor [42] attached to one of the finger modules for pinching force detection. C: Exploded view CAD rendering of the finger module. The top part can be easily exchanged to accommodate for various users or tasks. The finger module is attached to the cable loop using a pin mechanism with the cable wrapped around it and then rigidly fixed with screws.

The finger interface is attached to a bridge-like aluminum structure through a custom-made 7-DOF adjustable supporting arm (one translation and six rotations) rigidly mounted to the scanner bed. It can accommodate both right and left hand, and can be oriented to various wrist angles to maximize comfort of use, especially for people with neurological disorders affecting the hand posture. No ferromagnetic materials are employed in the finger interface; it is made of aluminum, brass, titanium (for fixed structure), and polymers (for the translating finger modules). As the transmission and remotely actuated finger interface must be installed and removed within a reasonable time, both are rigidly fixed to the same aluminum structure. This structure attaches to both sides of a 3.0 T Philips Achieva scanner bed using brass thumb screws and threaded polymer inserts.

3.5 Remote Position and Force Measurement

The position of each of the two finger modules is measured via encoders attached to the two motor shafts. The differential direct kinematics are given by:

$$\left(\begin{array}{c}{\dot{q}}_1\\ {\dot{q}}_2\end{array}\right)=\left(\begin{array}{cc}r& 0\\ 0& r\end{array}\right)\cdot \left(\begin{array}{c}{\dot{\theta }}_1\\ {\dot{\theta }}_2\end{array}\right)$$ (2)

where q₁ and q₂ are the positions of the two finger modules in task space, θ₁ and θ₂ are the angular positions of the two motor shafts in joint space, and r is the radius of the capstans attached to the motor shafts. The Jacobian matrix transforming joint to task space coordinates has full rank and hence the system does not have singularities, the only mechanical limitation being the range limits of the output rails on which the finger modules move.

Besides being detected by the MR-compatible force sensor attached to the finger module, pinching forces can also be estimated from motor currents using the following transformation:

$$F=\frac{k_T\cdot i_M}{r}$$ (3)

where k_T represents the motor torque constant, i_M the current flowing through the motor, and r is the radius of the capstan attached to the motor shaft.

3.6 Safety Considerations

To prevent any harm or damage and to fulfill safety norms designated for interaction with human subjects, the following features were implemented on NeuRoGrasp:

• Safety software routines that allow the experimenter to set position, velocity, acceleration and force limits.

• In case of unexpected events an emergency button is available, which activates a safety relay (PNOZ s4; Pilz, Germany) cutting off the power to the linear amplifiers of the actuators. The leads of the motors are then short circuited, resulting in high physical damping in the transmission.

• Mechanical stops are implemented on both finger interface rails to limit the movement to the user grasping aperture.

• Finally, a MR-compatible cart was designed and constructed, on which the actuation system is rigidly anchored at the end of the scanner bed.

4 Performance Evaluation

Several haptic performance metrics [43] were used to characterize NeuRoGrasp in terms of achievable working range, friction in the actuation drive and the transmission, transmission stiffness, peak velocity, acceleration and force, along with the output impedance and Z-width identification, system bandwidth, and MRI-compatibility testing (Table 3).

4.1 Uncontrolled System Characterization

We describe here all the inherent performance characteristics of the haptic device (i.e. passive device dynamics, without taking into consideration any control of the system) such as transmission friction, output force, transmission stiffness, maximum velocity and acceleration.

Friction influences the dynamics of the device as well as the interaction quality. System friction depends on both the internal friction of the motors and the bearings holding the motor shaft and capstan, as well as the friction in the transmission, which varies with transmission length (i.e. number of redirection modules). In order to identify the influence that various components of the device have on the friction apparent at the output, we identified the static and viscous friction levels in 3 conditions: open-loop (no control) for a minimum length transmission (i.e. placing the end-effector as close as possible to the actuation system (at about 0.3 m), thus bypassing all the transmission components such as redirection modules and length/tension adjustment mechanism) and full-length (about 2.6 m) transmission, as well as closed-loop (force feedback) using the MR-compatible force sensor attached at the end-effector for the long transmission (Fig. 5). Static friction was recorded when movement of the end-effector was detected (using a velocity threshold), while dynamic friction was measured by moving the end-effector at constant velocities and recording the force at the end-effector.

Fig. 5.

Top: Friction identification portraying the static friction and dynamic friction curves for both positive and negative directions in 3 tested conditions: open-loop (no control) for a short (∼ 0.3 m) and long (∼ 2.6 m) transmission, and closed-loop (force feedback) for the long transmission. Bottom: Example of interaction force with and without force feedback during a sinusoidal movement (maximum velocity ∼ 100 mm/s) of the end-effector.

Cable tension highly influences system friction, and Table 1 provides a measurement of this effect by identifying the static friction for four different cable tensions. These were set using different spring lengths in the tension adjustment mechanism (Fig. 3B). We selected for the transmission a cable tension of 11 N, which represents a good tradeoff between the inherent static friction (uncontrolled device) felt at the output (2.8 N) and the necessary tension to prevent cable slack and slippage of the pulleys. It is based on previous findings that recommend only 15% of maximum force should be exerted in typical one hour finger pinching experiments with a work-rest ratio of 1:1 to avoid fatigue [44].

Table 1.

Influence of cable tension on static friction force.

CABLE TENSION [N]	STATIC FRICTION FORCE [N]
8.5	2.2
11	2.8
13.5	3.6
16	4.6

In order to calibrate the system and properly identify the maximum continuous and peak force, a commercial force sensor (LSM 200; Futek Advanced Sensor Technology Inc., USA) was attached at the output for performance characterization (Fig. 8A). By blocking the output, a maximum continuous force of 38 N and a peak force of 94 N were determined, respectively. Further, structural stiffness of the transmission was identified to be 3.3 N/mm (performed by fixing the output and continuously increasing motor forces while recording the motor encoder readings) and the maximum stably rendered viscosity was 85 N/(m/s) (achieved by adding increasing damping to the motor while performing a low frequency (∼ 0.5 Hz) sinusoidal movement with the output until instabilities occurred). Peak velocity and acceleration are two additional important metrics for haptic devices, which were measured by applying a step peak current to the motors for a duration of 10 ms, and were found to be 0.6 m/s and 70 m/s², respectively.

Fig. 8.

A commercial force sensor (LSM 200; Futek Advanced Sensor Technology Inc., USA) and a linear encoder (MLFK 10T; Baumer Electric AG, Switzerland) were attached at the output to measure interaction force and motion of the finger interface; (A) Output impedance and Z-width measurements were performed by perturbing the end-effector by hand with a sinusoidal movement. (B) Position bandwidth was determined based on measurements performed at the output using the linear encoder; (C-D) The output was grounded (locked in place) via the force sensor for force bandwidth measurements.

4.2 Controlled System Performance

The closed-loop position bandwidth illustrates over which frequency range the output can follow a desired input command, and is particularly informative for interaction studies where the fingers are passively moved. It was identified by following with the end-effector finger modules a sinusoidal input reference position with an amplitude of 5 mm and increasing frequencies (from 0 to 35 Hz, in steps of 0.5 Hz), using a PID position control (free output, i.e. no interaction with the fingers). Position was controlled at the input, and measured at the output (end-effector) using a commercial linear position sensor (MLFK 10T; Baumer Electric AG, Switzerland) (Fig. 8B). Investigation of the system frequency response revealed a resonance peak around 18 Hz and a bandwidth of at least 19 Hz (Fig. 6). Due to saturation limits of the actuation system in terms of peak velocity and acceleration, it was not possible to excite the system beyond this frequency.

Fig. 6.

Bode plot of the system in closed-loop PID position control. A sinusoidal position with increasing frequencies (from 0 to 35 Hz, in steps of 0.5 Hz) is commanded to the motor, while the output position is measured at the finger module.

Furthermore, an experimental identification procedure based on frequency analysis was applied to the system, in order to describe over which frequency range the output finger modules of NeuRoGrasp can follow a desired input force commanded to the motors (i.e. open-loop force control). The frequency response was measured by commanding to the motors a sinusoidal input reference force (frequency range from 0 to 35 Hz, in steps of 0.5 Hz) with an amplitude of 6 N, under a fixed-output condition. Force was measured at the output using the commercial force sensor (LSM 200; Futek Advanced Sensor Technology Inc., USA) attached between the fixation and the finger module (Fig. 8C-D). The Bode diagram corresponding to the transfer function from the input force to the output force at the end-effector is presented in Fig. 7. The resonance frequency is around 16 Hz, while the force bandwidth could be identified around 21.5 Hz (the frequency at which the gain drops -3 dB from the lowest frequency level). However, as the phase margin is negative (i.e. phase lag > 180 deg) at this frequency, for stability reasons the usable bandwidth is limited to 15 Hz – the maximum frequency at which the phase margin is positive (i.e. phase lag < 180 deg) while the gain margin is positive as well (i.e. gain < 0 dB).

Fig. 7.

Bode diagram of the transfer function in open-loop force control. A sinusoidal force with increasing frequencies (from 0 to 35 Hz, in steps of 0.5 Hz) is commanded to the motor, while the output force is measured when the finger module is constrained.

Output impedance and Z-width

The inherent dynamics of the uncontrolled haptic device is defined by the so-called output impedance of the system. This was measured at the output using the commercial force sensor (LSM 200; Futek Advanced Sensor Technology Inc., USA) by perturbing the end-effector by hand with a sinusoidal movement (Fig. 8A) and a continuously increasing frequency (up to ∼ 5-6 Hz limited by human capabilities). It is described by the relationship between force and velocity at the output and is given by the relation:

$$Z_{\mathit{\text{HapticDevice}}}(s)=\frac{f_{\mathit{\text{out put}}}(s)}{v_{\mathit{\text{out put}}}(s)}$$ (4)

In order to better characterize the dynamic behavior, the range of achievable impedances the system can render (i.e. the Z-width of the haptic interface [45], [46]) was investigated. It was identified by determining both the lower impedance boundary (transparency rendering) as well as the upper impedance boundary (maximum impedance rendering) during stable human-robot interaction (Fig. 9). The residual interaction force during transparency rendering should ideally be imperceptible to the user while interacting with the device. An impedance controller with friction feedforward (Eq. 5) was optimally tuned to overcome the inherent dynamics of the system in order to minimize the interaction forces during the movement. Feedforward compensation is implemented to compensate for the Coulomb and viscous friction and comes into effect only after static friction has been overcome, while at forces below static friction the MR-compatible force sensor attached to the finger module is used to detect the direction of applied force:

Fig. 9.

Output impedance of NeuRoGrasp (passive device dynamics) and its renderable impedance range (Z-width). An impedance controller with friction feedforward (Eq. 5) was implemented to obtain the lower impedance boundary, while a simple impedance (spring-damper) controller (Eq. 6) was used to determine the upper impedance boundary. A mass-spring-damper, a mass-damper, and a spring-damper mechanical model were fit to the raw measured output impedance (middle curve), lower impedance boundary (bottom curve), and upper impedance boundary (top curve). In the latter case, the fidelity of the haptic rendering was determined to be 93.5% between the commanded (k_w ∼ 7.3 N/mm) and measured fitted (k_w ∼ 7.8 N/mm) stiffness parameters, and 82.5% between the commanded (b_w ∼ 68.7 N/(m/s)) and measured fitted (b_w ∼ 83.2 N/(m/s)) damping parameters, respectively.

$$F_m=\{\begin{array}{cc}\mathit{\text{sign}}(v)\cdot F_{\mathit{\text{ST}}}+v\cdot k_v& ,\text{if}v\ne 0\\ \mathit{\text{sign}}(F_{\mathit{\text{FS}}})\cdot F_{\mathit{\text{ST}}}& ,\text{if}{F}_{\mathit{\text{FS}}}>F_{\mathit{\text{threshold}}}\\ 0& ,\text{otherwise}\end{array}$$ (5)

where F_m represents the corresponding force commanded to the motors, F_ST is the inherent static friction force (∼ 2.8 N), F_FS and sign(F_FS) are the magnitude and direction of the applied force detected by the MR-compatible force sensor, F_threshold is a very small force threshold (∼ 0.2 N) set to avoid instabilities when changing direction, while v, sign(v) and k_v (∼ 2.6 N/(m/s)) are the movement velocity, direction, and viscous compensation factor, respectively.

Not only transparency, but also virtual simulation of stiff objects is desired when using haptic interfaces, e.g. to lock the fingers in place, render virtual walls, or to guide the fingers passively. The upper impedance boundary of the system was rendered with a simple impedance controller (Eq. 6) employing a large spring-damper combination:

$$F_m=\{\begin{array}{cc}{k}_w\cdot (x-x_w)+b_w\cdot \dot{x}& ,\text{if}x<x_w\\ 0& ,\text{otherwise}\end{array}$$ (6)

where F_m represents the corresponding force commanded to the motors, k_w (∼ 7.3 N/mm) and b_w (∼ 68.7 N/(m/s)) are the virtual wall stiffness and damping parameters, while x and x_w are the current position and virtual wall position, respectively.

A summary of the performance characteristics of NeuRoGrasp is presented in Table 3.

4.3 FMRI-compatibility

The purpose of the following tests was to demonstrate EM compatibility with an MRI system by assessing the influence on image quality associated with the introduction and operation of the device in the scanner room, and vice versa. All imaging tests were performed on a 3.0 T MRI system (Achieva, Philips Medical Systems; Eindhoven, The Netherlands) equipped with an echo-planar imaging (EPI) capable gradient system and an eight-channel sensitivity encoding (SENSE) head coil. We employed T2*-weighted, single-shot, field echo, echo-planar sequences with the following parameters: TR = 2 s, TE = 30 ms, FOV = 240×240 mm², flip angle = 82 deg, 15 slices, voxel size = 3×3×3 mm³ and 0.7 mm slice gap, as they will be used during functional studies.

Before bringing NeuRoGrasp into the MRI scanner room, its components (finger interface, transmission, shielded actuation system) were tested outside with a neodymium magnet and no forces were observed. The device was then set up in the scanner room by proceeding as follows:

• first the transmission with the finger interface (as one subsystem with mechanical components rigidly connected) were tightly attached to the scanner bed through threaded inserts;

• subsequently the actuation system (secured on the MR-compatible cart) was wheeled to the desired position, beyond the 3 mT line (indicated by the manufacturer at the end of the scanner bed, and confirmed by us through measurement with a teslameter) of the scanner magnet;

• transmission cables were then securely fixed on the capstans attached to the driving motor shafts and cable tensioning was performed using the cable tension adjustment mechanism;

• finally, power and signal cables from the control room to the actuation system in the scanner room were connected through the penetration wall.

The system can be brought into the MR environment and fully installed within about 20 min, whereas adjusting it to a new subject takes about 5 min (in addition to the time required to instruct and install the subject in the scanner without any device).

A phantom (plastic sphere filled with an aqueous solution) was placed in the region of interest, serving as baseline. Eight runs of EPI scans were acquired under each of the following conditions (200 volumes aquired for each run): (1) NeuRoGrasp outside the MRI room (baseline), (2) NeuRoGrasp in the MRI room and unpowered, and (3) NeuRoGrasp in the MRI room and running (linear motions at constant speeds). During conditions (2) and (3) NeuRoGrasp was installed in the MRI room as for a normal fMRI experiment, with the actuation system rigidly fixed on the MRI-compatible cart placed at the end of the scanner bed, about 2.6 m from the edge of the scanner bore (Fig. 1). Throughout condition (3) the haptic interface performed linear movements (position control) with an amplitude of 5 cm at a frequency of 1 Hz, thus simulating a typical pinching movement.

We first investigated the effect of the running fMRI scanning on the haptic interface, and no performance degradation could be observed during operation. Then we analyzed the variation of the temporal signal-to-noise ratio (tSNR) induced by the robotic interface during imaging with respect to baseline. The tSNR was calculated for a 21×21 voxel region-of-interest (ROI) in the center of the middle slice of the phantom (slice 8), as well as for two other slices equally spaced in the phantom volume (slice 4 and slice 12), comprising a large part of the phantom area (in accordance with previous MRI-compatibility tests for robotic devices [47], [48]). For each voxel in the ROI, its tSNR represents the ratio of the mean of its time series over the standard deviation of its time series. The final tSNR is the mean of the single-voxel tSNRs over all selected ROI:

$$\mathit{\text{tSNR}}=\mathit{\text{mean}}\frac{\mathit{\text{mean of voxel time series}}}{\mathit{\text{standard deviation of voxel time series}}}$$ (7)

The results are presented in Table 2. High tSNR values were obtained in all 3 experimental conditions, and no significant differences were found between conditions (two-sample t-test, α=0.05; No Device – Device OFF: p=0.464, No Device – Device ON: p=0.375, Device OFF – Device ON: p=0.264). Introduction of NeuRoGrasp into the MRI environment therefore did not increase the noise level and there was no signal degradation when the device was powered and running. We also performed a visual inspection of the mean images of the phantom over the time series in all 3 conditions (Fig. 10). Subtractions of the baseline image from the images of the other two experimental conditions are shown in the lower panel. No deformations (linked to inhomogeneities in the magnetic field) or dark spots (linked to RF interferences) could be observed.

Table 2.

Mean tSNR and measurement variability (std) values within 8 runs of each condition during compatibility tests with a phantom (plastic sphere filled with an aqueous solution).

CONDITION	tSNR (std)
No Device	351.6 (3.4)
Device OFF	348.7 (10.3)
Device ON	353.2 (3.4)

Fig. 10.

Phantom scans without the haptic interface (baseline) (top left), with the interface present and at rest (top middle) and running (top right). The differences with respect to baseline (bottom images) show neither deformations nor dark spots.

4.4 Experimental validation with a human subject

A typical fMRI experiment NeuRoGrasp can be used for is to investigate the neural representation of mechanical object properties such as stiffness and damping. Although investigations of the neural correlates of finger forces [6] and displacements [49] have been previously attempted, NeuRoGrasp provides the ability to combine the two parameters and move one step further in characterizing brain responses to finger-object interactions. We carried out a pilot study with a subject, where the goal was to evaluate the functionality and usability of the device within a fMRI experiment, as well as to potentially identify brain areas encoding stiffness (force divided by displacement).

The task consisted in performing pinching movements under a visually-guided target matching paradigm, while the device rendered 5 different levels of stiffness (1.75 N/cm, 3.5 N/cm, 7 N/cm, 14 N/cm and 28 N/cm) – combinations of three forces (10%, 20% and 40% of the 49 N maximum voluntary force of the participant) and three displacement amplitudes (10%, 20% and 40% of the 7 cm maximum aperture of the participant) presented pseudo-randomly in a factorial design. The visual feedback consisted of a target square and a cursor circle, which moves horizontally as a function of force and displacement, scaled in order to reach the constant target. The subject was instructed to reach the target as accurately as possible within a time window of 2 sec. Once the subject remained in the target for 1 sec, the fingers were passively moved by the robot to the initial position to start the next trial. Each combination was presented a total of 20 times, with an interstimulus interval of 5±1 sec between trials. Representative trials of the pinching exercises during two consecutive trials rendering different stiffness levels are presented in Fig. 11, portraying the position and force profiles during the different trial sequences. The fMRI scanning parameters used were the same as for the fMRI-compatibility test. Functional images were preprocessed and analyzed using SPM8 (Welcome Department of Imaging Neuroscience, London, United Kingdom) and custom MATLAB (MathWorks, Inc., Natick, MA) programming. Acquired volumes were slice-time corrected, realigned (estimate and rewrite), normalized to template and then smoothed with a standard Gaussian kernel of 8 mm full-width at half-maximum. First level general linear model analysis employed a movement regressor followed by first order parametric regressors of force, displacement and stiffness. Head movement regressors were added to the design matrix along with a constant.

Fig. 11.

Pinching position and force data during two consecutive representative trials of the experimental validation. Numbers on the top represent trial phases: (1) & (5) – active pinching against virtual spring load; (2) & (6) – fingers' position locked; (3) & (7) – robot passively moves fingers back to initial position; (4) randomized time between trials.

The robot successfully rendered the haptic environment, resulting in expected correlations. For instance, we found a linear correlation with precision grip force in contralateral primary motor and somatosensory cortices (M1, S1), and bilaterally in secondary somatosensory cortex (S2) and supplementary and cingulate motor areas (SMA, CMA) (Fig. 12), consistent with previous work [7], [50], [51]. Additionally, a linear relationship was found between the BOLD response and changes of precision grip stiffness in contralateral inferior parietal cortex (IPC), intraparietal sulcus (IPS) and dorsal premotor cortex (PMd), as well as bilateral fontal gyrus (FG) (Fig. 13). These are in accordance with previous findings that revealed the involvement of these areas in mediating grasp execution [52], integration of sensory modalities [53], or manipulation of complex (different 3D shapes, size and surface characteristics) objects [54] and grip-lift tasks [55]. An analysis of the head movement regressors revealed that performing pinching tasks with NeuRoGrasp introduced only negligible movement artifacts (maximum, mean and standard deviation of the head movement were 0.33 mm, 0.14 mm and 0.07 mm, respectively). No discomfort was reported using the haptic interface.

Fig. 12.

Brain regions corresponding to linear increases in BOLD signal with increasing precision grip force in one participant (SMA – supplementary motor area, M1 – primary motor cortex, S1/S2 – primary and secondary somatosensory cortices)

Fig. 13.

Brain regions corresponding to linear increases in BOLD signal with increasing precision grip stiffness in one participant (FG – frontal gyrus, PMd – dorsal premotor cortex, IPS – intraparietal sulcus, IPC – inferior parietal cortex).

5 Discussion

The neural mechanisms underlying grasp control in healthy individuals and those following neurological injury are still poorly understood. We have designed and characterized NeuRoGrasp, a MRI-compatible robotic device allowing both well-controlled and reproducible physical interaction during precision grip tasks as well as the objective assessment of hand function in conjunction with non-invasive whole-brain neuroimaging using fMRI. The haptic interface is actuated by a shielded EM actuation system, and employs a relatively high stiffness and low inertia cable transmission to maintain the actuation at safe distance away from the edge of the scanner bore (∼ 2.6 m), while achieving low output impedance, making the system suitable for remote actuation and sensing.

A detailed performance characterization demonstrated the ability of the device to stably render object stiffness of up to 3.3 N/mm (for comparison, the stiffness of a tomato is around 2.3–2.8 N/mm [56]) and viscosities as high as 85 N/(m/s) (for comparison the viscosity of honey is between 2–10 N/(m/s)). The finger interface has a working range of 125 mm (pinching aperture), can generate peak forces of up to 94 N (38 N continuous), and reach velocities as high as 0.6 m/s. Although slightly lower than previously identified human forefinger velocity (0.74 m/s [35]), this value is sufficient even for experiments involving higher finger pinching frequency rates (> 10 Hz [36]). Moreover, if velocities become too high, movement artifacts and/or subject discomfort can occur. Inherent static friction of the system (uncontrolled device) was determined to be 2.8 N, however, this could be reduced to below 0.5 N when performing friction compensation using the custom made MR-compatible force sensor attached to one of the finger modules. Position control bandwidth was determined to be at least 19 Hz, limited by saturation effects in velocity and acceleration, as well as by elasticity in the transmission. This could be increased by selecting more powerful motors and encoders, as well as by increasing transmission stiffness. Nontheless, the bandwidth requirement was met [36], while also outperforming other devices interacting with human finger motion in the MRI scanner [19], [20]. Transmission elasticity also influences the structural stiffness of the system. The rather low value (3.3 N/mm) compared to other non-MRI-compatible finger pinching interfaces [57] is due in part to cable elongation, but mostly to elasticity and mechanical play in the plastic components such as cable guides and ball-bearings (transmission stiffness is about 3 times smaller than that of the cable stiffness only).

The output impedance (uncontrolled device) reveals the frequency-dependent dominating characteristics of the device. At frequencies below 2 Hz, an elasticity dominated behavior is observed, which can be attributed to the stick-slip phenomenon present on the two aluminum rails of the finger interface. This nonlinear effect does not have a direct representation in the frequency space, but can be modelled as elastic deformation [58]. At frequencies above 2 Hz, the interaction behavior is dominated by damping, which originates both from viscous friction in the DC actuation (rotor damping) and in the transmission. It can be additionally noted that the output impedance does not increase with frequency (within the 6 Hz range, limited by human capabilities), the main reason being the small apparent mass at the end-effector which does not contribute towards an inertial effect.

The ability of the controlled device to render high (virtual wall) as well as low (transparency) impedances was characterized through the Z-width. For the upper impedance boundary rendering, a stiffness dominated behavior (caused by the virtual stiffness of the impedance controller) can be observed for the entire fitted frequency range. The lower impedance boundary (obtained using an impedance controller with friction feed-forward) presents similar characteristics to the inherent output impedance, i.e. an elasticity dominated behavior at low frequencies and a damping related one for higher frequencies. Nevertheless, a reduction of approximately 10 dB can be observed in the range of 0.5–1.5 Hz from the uncontrolled to the controlled case. The relatively large Z-width identified for our device is comparable to the Z-width of other non-MR-compatible haptic devices [59] (very few research papers report Z-width measurements, and no references could be found for MR-compatible devices), and allows for future rendering of virtual objects with a wide range of mechanical properties, ranging from close to transparent (∼ 0.2 N friction force) to relatively stiff (3.3 N/mm) objects, and viscosities (viscous force fields) up to 85 N/(m/s). The static friction is lower (in particular due to cable transmission and friction compensation using the MR-compatible force sensor [42] attached at the output) than in the devices we previously developed [20] and also compared to other MRI-compatible haptic systems [60], which display higher output friction (∼ 3 N) even after performing friction compensation using force sensors at the output. No literature was found on MRI-compatible grasping devices that report static friction.

Nevertheless, several further developments are thought to help improve the performance of the system. Inherent static friction could be reduced by replacing the plastic ball-bearings in the entire system with ceramic ones (in particular the large ball-bearings guiding the capstan drive in the actuation system) and the aluminum sliding rails with other non-magnetic linear rails based on ball rollers or ceramic bearings. While the device performs to specifications, these changes could expand its utility.

NeuRoGrasp was assessed in terms of MRI safety and compatibility and no degradation of haptic performance or of image quality could be observed. Analysis of the fMRI images acquired during the presence and operation of the device inside the scanner room reveals undetectable or negligible influence on the imaging and consistently high SNR values, demonstrating the effectiveness of the proposed shielded actuation concept. Compatibility is ensured through both magnetic shielding of the motors and electric shielding of the control and power electronics, with only filtered power lines and fiberoptic data lines connecting the actuation box with the external environment through integrated waveguides. Safety is assured through the non-magnetic components of the actuation cart weighing 80 kg, which anchors the actuation box at the end of the scanner bed, beyond the 20 mT safety line. We thus conclude that NeuRoGrasp is fully compatible with the MRI environment of the used 3 T scanner.

A preliminary fMRI experiment during a stiffness identification pilot study validated the usability of the haptic interface and revealed promising data on the neural response to changes in environmental mechanics, illustrating the possibilities offered by this device. NeuRoGrasp can not only render a large range of mechanical properties, but also provides the possibility to instantaneously manipulate these properties, allowing for more advanced and better-controlled experimental designs. It may thus become a powerful tool in the field of neural grasp control and representation of mechanical object properties, examining different viscoelastic force fields [61], multisensory integration during stiffness perception (vision vs. somatosensation) [62], neuro-cognitive assistive therapy [63], passive movement therapy [64], or the effect of ageing on these processes [8]. These investigations require the rendering of viscoelastic force fields, passive movement guidance, as well as transparency rendering, all of which NeuRoGrasp is well suited for.

Future work will therefore focus on the development and implementation of experimental paradigms combining virtual reality and haptic interaction for various neuroscientific investigations of grip control and object manipulation, in healthy subjects as well as in patients with neurological disorders.

Biographies

Bogdan Vigaru (S'08) received the B.Sc. degree in Automation Engineering from the University of Craiova, Romania in 2006, and the M.Sc. degree in Mechanical Engineering from Johns Hopkins University, U.S.A. in 2008. He completed the Ph.D. degree in the Department of Health Sciences and Technology at ETH Zurich, Switzerland in 2015, where his main research interests were related to neuroscience robotics and sensorimotor learning.

James Sulzer received the B.Sc. degree from The Ohio State University in 2002, and the M.Sc. and Ph.D. degrees from Northwestern University and Rehabilitation Institute of Chicago in 2006 and 2009 respectively, all in Mechanical Engineering. Following postdoctoral work at ETH Zurich, he then joined the University of Texas at Austin in 2013 as an assistant professor of Mechanical Engineering. His main research interests are related to restoration of healthy gait in people after stroke, as well as the development of novel, neurally-based rehabilitation strategies.

Roger Gassert (S'02–M'06–SM'13) received the M.Sc. degree in microengineering and the Ph.D. degree in neuroscience robotics from the Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland, in 2002 and 2006, respectively. He joined ETH Zurich in 2008, where he is currently associate professor of Rehabilitation Engineering. He has made contributions to the field of neuroscience robotics to investigate sensorimotor control and related dysfunctions, as well as to robot-assisted assessment and therapy. His research interests are in physical human-machine interaction, rehabilitation robotics, assistive technology and the neural control of movement.