Neuroengineering and neuroprosthetics

Abstract

This article contains a directed overview of the field of neuroengineering and neuroprosthetics. The aim of the article is, however, not to go over introductory material covered elsewhere, but rather to look ahead at exciting areas for likely future development. The BrainGate implant is focussed on in terms of its use as an interface between the Internet and the human nervous system. Sensory prosthetics of different types and deep brain stimulation are considered. Different possibilities with deep brain stimulation are also discussed.

Introduction

Neuroprosthetics are devices that can either act as a substitute for a motor, sensory or cognitive modality that might perhaps have been damaged as a result of an injury or a disease, or they can add new modalities. Neuroengineering is the study of how this is actually realised in practice and covers the problems faced in doing so.

An important feature of the field is that a connection needs to be made directly between the human brain or nervous system and the technology involved, whether this involves inputting sensory signals to the brain or transmitting motor signals from the brain to a prosthesis.

In terms of sensory input, the most commonly encountered examples are cochlear implants. These substitute the functions performed by the ear while simulating the frequency analysis performed in the cochlea. A microphone on an external unit gathers the sound and processes it; the processed signal is then transferred to an implanted unit that stimulates the auditory nerve through a microelectrode array (MEA). The implant does not restore normal hearing. Instead, it can give a person who is deaf a useful representation of sounds in the environment and help them to understand speech. Over 300,000 such devices are presently in use worldwide (National Institute on Deafness and Other Communication Disorders, 2017).

In terms of motor prostheses, the picture is not so well developed because most replacement arms, hands and legs of today either exhibit no voluntary movement functions at all or operate myoelectrically, that is from a person’s voluntary contracted muscles, usually in the residual limb. It is under these circumstances not appropriate to term them as neuroprostheses.

The most commonly encountered situation in which central areas of the brain are stimulated is via the method of deep brain stimulation (DBS), which is employed primarily in the treatment of Parkinson’s disease (PD). This can have the effect in patients of dramatically reducing their tremors, rigidity and walking problems. Typically, the implant is positioned in the thalamus or the subthalamic nucleus (STN), which are tiny parts in the centre of the brain. An electrical pulse, usually at a frequency of 150–180 Hz is then applied via the implanted electrodes. However, the same approach can be applied to different regions of the brain in order to tackle other problems such as clinical depression (Mayberg et al., 1999, 2005) (with the electrodes positioned in the subgenual cingulate), epilepsy and Tourette’s syndrome.

In this chapter, the aim is to look more to the future in each of these areas, to see what has been achieved so far and what will be possible in the next 50 years. By no means is the aim here to give a comprehensive overview of the neuroengineering field as that would simply not be possible in the space permitted. Rather the focus has been to present some information on areas where dramatic changes are likely over the next 50 years.

Future DBS

PD is presently the second most common neurodegenerative disorder. However, it is expected to increase in occurrence surpassing Alzheimer’s disease by 2040 (Dorsey and Bloem, 2018; Lonneke and Breteler, 2006). It is known that it is caused by the degeneration of dopaminergic neurons in the substantia nigra compacta (SNC). The loss of neurons in this brain area produces an imbalance between the direct and indirect pathways, with the resultant prevalence of the indirect pathway. This disequilibrium is responsible for the symptoms of the disease, which include tremor of the limbs at rest, akinesia and bradykinesia.

For treatment by DBS, this consists of the surgical implantation of a neuro-stimulator, which uses a pulse generator to deliver electrical current through a set of electrodes to the surgical target, normally the STN, thereby restoring its normal functioning.

However, the deep brain electrodes can be connected bi-directionally with a computer such that as well as being used for stimulation, electrical activity in the brain can be monitored. In this way, an understanding can be obtained of the inherent problem itself. Ongoing research is therefore developing an ‘intelligent’ stimulator. In the case of PD stimulation, this uses artificial intelligence (AI) to produce warning signals before Parkinsonian tremors begin (Wu et al., 2010). So the stimulator only needs to generate signals occasionally rather than continuously, thus operating in a similar fashion to a heart pacemaker. The principles are the same for whichever of the neurological problems is being tackled.

Using AI techniques, by better understanding the nature of the disease it has been found that there are distinct types of PD based on the different nature of the electrical activity in the brain (Camara et al., 2015). It is also quite possible for the monitoring computer to be located remotely from the patient. Hence, signals within the brain can be tracked in real time and fed into a computer. The computer is able to analyse these signals and generate alternative signals that are fed directly back into the brain in order to ensure the person in question continues to function.

The actual type of AI employed is a matter of present day research. It may well result in the case that a different method is needed for each of the problems dealt with. Essentially measurement of the electrical activity in the region of the brain under investigation is fed as input into the network which then forms a nonlinear model of the brain’s process in that region. Many early results have been obtained with neural networks, particularly those using multi-layered perceptrons, which are simplified models of human neurons (Warwick, 2011). The complexity and number of the perceptrons and the number of layers to be employed are all part of ongoing research.

The key focus here is on using the advantages of the AI system to benefit the treatment. So accurate prediction of future activity is a desired result, such that treatment can be provided when, or even before, it is needed. The AI system can be used, in real-time fashion as an early warning device. In this way, when applied directly, the AI system can give an indication that the patient will subsequently experience the problematic symptoms. If appropriate stimulation is then applied, the hope is that the patient will then not actually experience the symptoms themselves.

Sensory prosthetics

Visual prosthetics

A visual prosthesis is made up of an imaging unit, which obtains and processes the video input. This can be external to the person or it might be implanted. The visual data can be transmitted to the implant wirelessly by the unit. The implant uses the received data to convert the digital information to an analogue signal, which is used to stimulate the nerves concerned via microelectrodes. The stimulation can also be done anywhere along the optic signal’s path, possibly at the retina, or the optic nerve or even the visual cortex. However, clinical tests have proven to be most successful with regard to retinal implants.

The first experimental work in this field was done by cortical stimulation using a grid of surface electrodes. In 1968, Giles Brindley implanted an 80-electrode device on the visual cortical surface of a 52-year-old-blind woman. As a result of the stimulation, the patient was able to see phosphenes in 40 different positions of the visual field (Brindley and Lewin, 1968). This experiment showed that an implanted electrical stimulator device could restore some degree of vision.

The first clinical trial in 2000 of an implanted retinal prosthesis used a device with a passive micro photodiode array consisting of 3500 elements (Chow et al., 2004). Meanwhile in 2002, a trial began with an epiretinal implant consisting of 16 electrodes. The six subjects involved all exhibited very poor light perception. They were able to demonstrate their ability to distinguish between three common objects (plate, cup and knife). In 2007, a trial began using a 60-electrode retinal implant, involving 30 subjects in 4 different countries. Based on the results of the study, the technique was approved for commercial use in Europe (Humayun et al., 2012).

Other sensory implants

One interesting experiment is due to Neil Harbisson who is colour blind. The technology developed involves a head-mounted sensor (a camera with associated electronics) that translates colour frequencies into sound frequencies (Ronchi, 2009), which are formed into vibrations of his skull via an actuator. Initially, Harbisson memorised the frequencies related to each colour, but subsequently he decided to permanently attach the set up to his head, effectively meaning a small camera faces forward from over his forehead and is connected to the back of his skull by a metal bar.

The project was developed further so that Harbisson was able to perceive colour saturation as well as colour hues. Software was then developed that enabled Harbisson to perceive up to 360 different hues through microtones and saturation through different volume levels (Harbisson, 2008). What is particularly interesting about Harbisson’s experience is that his discrimination between different colours has improved over time as his brain has adjusted to the different vibrations experienced. Clearly, the extent of brain adaptability is a pointer to what can be expected in general with regard to either extending the present range of sensory input or even inputting a complex range of new sensory input information into the human brain that until now has not been possible.

One other line of research worth mentioning here is the use of permanent magnet implants for sensory extension. The pads of the middle and ring fingers are the preferred sites for magnet implantation in the experiments that have been reported (Hameed et al., 2010). The mechanoreceptors in the fingertips are most sensitive to frequencies in the 200–300 Hz range. An interface containing a coil mounted on a wire frame and wrapped around each finger is used to generate magnetic fields to stimulate magnet movement within the finger. The output from an external sensor is used to control the current in the coil.

Experiments have been carried out in a number of application areas (Hameed et al., 2010). Ultrasonic range information, involves an ultrasonic sensor for navigation assistance. Distance information from the sensor is encoded as variations in the frequency of pulses. Effectively, the closer an external object is to the sensor so the frequency of the pulses increased. The recipient has an accurate indication of how far objects are from the sensor. Further tests have used infrared sensors, which give an indication of the temperature of any objects remotely detected (Harrison, 2014). So the recipient ‘feels’ the temperature of remote objects.

Interestingly most of the present-day ‘research’ in the area of magnet implants is being performed outside of the traditional academic community (Harrison et al., 2018). Groups generally referred to as ‘Biohackers’ are investigating what is possible mainly through self-experimentation. It is expected that in the years ahead mainstream academics will eventually get to grips with this exciting area.

Motor prostheses developments

The technology behind motor prostheses is still in its infancy. Here, however, some examples are given where success has already been achieved.

Where a spinal cord lesion leads to paraplegia, patients can have difficulty emptying their bladders and this can cause infection. Giles Brindley et al. (1982) developed a sacral anterior root stimulator, with successful human trials occurring from the early 1980s onwards. The device is implanted over the sacral anterior root ganglia of the spinal cord and is controlled by an external transmitter. It delivers intermittent stimulation, which improves bladder emptying. It also assists in defecation and enables male patients to have a sustained full erection.

Philip Kennedy developed an operable system, which allowed an individual with paralysis to spell words by modulating their brain activity. Kennedy’s device used two simple neurotrophic electrodes; the first was implanted in an intact motor cortical region (e.g. finger representation area) and was used to move a cursor among a group of letters; the second was implanted in a different motor region and was used to indicate that a selection had been made (Kennedy et al., 2002).

As the patient thought about moving their fingers, these signals were translated into signals to move and stop a computer cursor. The patient could actually see where the cursor was on a large computer screen. Hence, they could decide as to when to stop thinking about moving. In this way, words could be spelt out letter by letter, but also heating and lighting could be controlled quite simply (Kennedy et al., 2004).

Developments continue in replacing lost arms with cybernetic replacements. One example of this is the work of Todd Kuiken in which nerves normally connected to the pectoralis muscles are employed in a process termed targeted reinnervation. In this procedure, nerves originally connected to arm muscles are reconnected to the pectoralis muscles. As the individual thinks about moving their hand and arm, so the muscles on the top of their chest flex instead. External electrodes monitor these movements and send resultant signals to a prosthetic arm worn by the patient. Effectively, the person’s nervous system is rewired via the pectoralis muscles (Kuiken et al., 2009).

The first beneficiary of this technique was Jesse Sullivan, hailed in the media as the world’s first ‘Bionic Man’, who lost both of his arms as a result of an accident he sustained during his work as a high-power electrical lineman. His arms were replaced with robotic prosthetics that he was able to control merely by thinking about using his original arms in the normal way.

BrainGate for therapy

To bring about a neuroprosthesis, a basic requirement is a neurological implant and as a result, a variety of different procedures is possible. As we have seen, these can be more of the handmade variety (Kennedy et al., 2002) or can have different operating depths on one shaft (Spira and Hai, 2013). However, the technology which has thus far shown itself to be of the most practical use is the MEA known as the Utah Array, more popularly (and commercially) referred to nowadays as the BrainGate, shown in Figure 1.

Figure 1.

A 100-electrode, 4 × 4 mm microelectrode array (BrainGate), shown on a UK one pence piece for scale.

The array consists of 100 spikes, which are 1.5 mm long and taper to a tip diameter of less than 90 microns. The spikes, essentially silicon shafts are arranged in a 10 by 10 array on a 4 mm x 4 mm substructure and each has a platinum electrode on its tip. The electrodes are linked to platinum wires and in this way, the array can be employed bi-directionally to both directly monitor neural activity and also to apply stimulating currents.

A number of trials have been carried out that did not use humans as test subjects, these involving chickens or rats. However, it is human studies only that we are more interested in here and these are limited to two groups of studies at the moment. In these experiments, the array has been fired into either the human brain or nervous system. In the first set of these experiments to be considered, the array has been employed in a purely recording role for therapeutic results.

Electrical activity from a few neurons monitored by the array electrodes, positioned in the motor cortex has been decoded into a signal that enabled a severely paralysed individual to position a cursor on a computer screen using neural signals for control in combination with visual feedback. The same technique was later deployed to allow the individual recipient, who was paralysed, to operate a robot arm even to the extent of learning to feed themselves in a rudimentary fashion by maintaining sufficient control over the robot arm (Hochberg et al., 2006, 2012).

The key to this is the ability to both record and interpret motor cortical neural activity and then to use the output in order to control prosthetic devices appropriately. This work is largely based on understanding the neuronal coding of movement direction in the motor cortex (Georgopoulos et al., 1986). Although individual motor neurons are only broadly tuned to a particular direction, the intended direction of movement can be uniquely predicted by means of the appropriate population of motor cortical neurons. The method represents individual cells as vectors, which make weighted contributions. The resulting vector sum of all cell vectors, referred to as the population vector, is in the direction of intended movement.

By this means, the same BrainGate implant was subsequently employed to enable a paralysed individual to regain some control over his own arm (Bouton et al., 2016). In this case, signals from the individual’s motor cortex were employed to bring about stimulation of hand/wrist muscles via a cuff worn around the person’s arm. The effect of this was a sort of bi-pass of the non-functioning nervous system. As a result, the individual recipient could make isolated finger movements and perform six different wrist and hand motions.

Initially, functional magnetic resonance imaging (fMRI) scans were taken of the recipient’s brain while he tried to copy videos of hand movements. This identified an exact area of the motor cortex dealing with the movements exhibited. Surgery was then performed to implant the array to detect the pattern of electrical activity arising when the recipient thought about moving his hand. These patterns were then sent to a computer, which translated the signals into electrical messages, which were in turn transmitted to a flexible sleeve that wrapped around the forearm and stimulated the muscles.

BrainGate for human enhancement

As the BrainGate has been described thus far, its use has been focussed on therapeutic, motor prosthesis, procedures, to assist those with a problem, particularly a paralysis. However, the very first use of this implant in a human while having a therapeutic under current was mainly aimed at investigating human enhancement beyond the human norm. Furthermore, the BrainGate is inherently an interface with bi-directional capabilities, in other words, it can act in both a motor and sensory capacity at the same time.

Peripheral nerve interfaces can be categorised into two distinct types, extraneural or intraneural. Extraneural, or cuff electrodes, generally wrap tightly around the nerve trunk, and allow recording of the sum of the single fibre action potentials, (the Compound Action Potential) (Loeb and Peck, 1996) in a large region of the nerve trunk, or crudely selective neural stimulation (Slot et al., 1997).

However, an ideal nerve interface needs to allow for very selective recording and stimulation, which is more suited to intraneural electrodes (Kovacs et al., 1994). MEAs (Figure 1) contain multiple electrodes, which are distributed within the fascicle of the mixed peripheral nerve to provide direct access to axons from various sense organs, such as muscle spindles, cutaneous receptors or motor axons to specific motor units. The device therefore offers a multichannel nerve interface with which both efferent signals can be measured and afferent signals applied.

To this end, in 2002, the BrainGate multi-electrode array shown in Figure 1 was implanted into the median nerve fibres of this article’s author, a healthy human individual, in the course of 2 h of neurosurgery to test bidirectional functionality in a series of experiments. Stimulation current applied directly into the nervous system allowed information to be sent to the recipient, while control signals were decoded from neural activity in the region of the electrodes (Warwick et al., 2003).

Overall, a number of trials were undertaken successfully using this setup (Warwick et al., 2004). In particular,

1. Extra-sensory (ultrasonic) input was successfully implemented.

2. Extended control of a robotic hand across the Internet was achieved, with feedback from the robotic fingertips being sent back as neural stimulation for a sense of force being applied to an object (achieved between Columbia University, New York (United States) and Reading University, England).

3. A form of telegraphic communication directly between the nervous systems of two humans (the author’s wife) was performed.

4. A wheelchair was successfully driven around by means of neural signals.

5. The colour of jewellery was changed as a result of neural signals – also the behaviour of a collection of small robots.

In all these cases, the trial could also be described as useful for purely therapeutic reasons, for example, the ultrasonic sensory input might be of use to an individual who is blind, while telegraphic communication might be beneficial to people with certain forms of motor neurone disease. Each trial can, however, be seen as a potential form of enhancement beyond the human norm for an individual. There was no need to have the implant for medical reasons in order to overcome a problem; the experimentation was carried out for the purposes of scientific exploration.

Human enhancement with the aid of brain–computer interfaces introduces all sorts of new technological and intellectual opportunities, but it also throws up different ethical concerns (Warwick, 2003). While the vast majority of present day humans are perfectly happy for interfaces, such as the BrainGate, to be used in therapy, the picture is not so clear when it comes to enhancement.

From the trials, it is apparent that extra sensory input is one practical possibility that has been successfully trialled along with extending the human nervous system over the Internet. However, improving memory and communication by thought are other distinct potential, yet realistic, benefits with the latter of these also having been investigated to an extent. To be clear these things appear to be possible (from a technical viewpoint at least) for humans in general.

Conclusion

Many different introductory aspects of neuroengineering and neuroprosthetics are covered elsewhere, and it was not the intention here to go over the same material. Rather the aim with this article has been to fill the gaps but at the same time to look ahead to see what particular new developments and research results are likely to dramatically change the field over the next 50 years.

DBS is primarily used to combat the effects of PD; however, it is already also being employed to treat Epilepsy, Tourette’s syndrome and depression. While this list is very likely to expand dramatically, it is really the use of AI that is likely to make the biggest impact. Although as the technique is employed for a wider range of neurological disorders so in turn the nature of the AI employed is likely to vary significantly.

Among the neural implants available the BrainGate has been employed in a variety of successful experiments. Hence, this was discussed in greater detail. A particular feature of its use thus far is that it has found a home both in terms of therapy, particularly assisting those with a paralysis, and also for human enhancement. In terms of the possibilities of commercial success, it is the opportunity of being available to a broad market that is likely to ultimately result in the success of a particular approach.

Current and future challenges

At present, the most exciting aspect of present day research in this field is clearly the introduction of artificial intelligence. The adaptable, learning features are clearly important when it comes to modelling characteristics of the brain, but it means that general methods of stimulation can be tailored to individual needs. In the next decade, it is felt that this area will realise dramatic results.

It has been found that certain neurological problems have clearly been more receptive to electronic treatments. Partly this is due to the depth and nature of research already carried out, but partly it is also due to the specific nature of those ailments. It is expected, however, that as research is ramped up in other areas then treatments or at least assistance by means of electronic stimulation might prove beneficial. A good example of this is in the treatment of schizophrenia.

One final area worth mentioning is that of human enhancement which has been discussed, to a certain extent, in this article. Clearly, such treatment, as far as medical personnel are concerned, involves ethical and societal directing as well as purely medical development. However, with the neurological enhancements considered the end realisation is an exciting upgrading in the functioning of the human brain.