Intraoperative multispectral and hyperspectral label-free imaging: A systematic review of in vivo clinical studies

Abstract

Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to transform the way surgery is performed but it is not clear whether current systems are capable of delivering real-time tissue characterization and surgical guidance. We conducted a systematic review of surgical in vivo label-free multispectral and HSI systems that have been assessed intraoperatively in adult patients, published over a 10-year period to May 2018. We analysed 13 studies including 7 different HSI systems. Current in-vivo HSI systems generate an intraoperative tissue oxygenation map or enable tumour detection. Intraoperative tissue oxygenation measurements may help to predict those patients at risk of postoperative complications and in-vivo intraoperative tissue characterization may be performed with high specificity and sensitivity. All systems utilized a line-scanning or wavelength-scanning method but the spectral range and number of spectral bands employed varied significantly between studies and according to the system's clinical aim. The time to acquire a hyperspectral cube dataset ranged between 5 and 30 seconds. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of intraoperative in-vivo label-free HSI but further work is needed to fully integrate it into the current surgical workflow.

1. Introduction

Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to radically transform the way surgery is performed. These optical imaging modalities are ideally suited for clinical use because they are non-contact, non-ionizing and non-invasive. As such, in vivo multispectral and HSI systems can potentially provide significant advantages over existing, more invasive intraoperative imaging techniques. Multispectral and HSI exploit the ability to split light into multiple narrow bands beyond the three conventional red, green and blue (RGB) visible spectral bands, enabling analysis of images not seen with the naked eye.

Several medical research multispectral and HSI systems have focused on the technique's diagnostic capabilities. The ability to quantitatively measure oxygenation levels in tissue has been tested in various pathologies and clinical contexts, including; peripheral vascular disease [1, 2], retinal eye disease [3, 4], diabetic foot disease [5, 6] and wound healing [7, 8]. Numerous groups have also published work demonstrating the effectiveness of using HSI for cancer detection [9–20]. With the development of high-resolution and fast frame rate multispectral and HSI cameras, enabling the capture of a rich imaging dataset at a video rate, these imaging methods offer great potential for real-time non-invasive tissue characterization and surgical guidance [21].

In this paper, we present a brief summary of the physical basis of HSI followed by a systematic review evaluating the intraoperative surgical applications of in vivo label-free, non-fluorescence multispectral and HSI. We report the different types of imaging systems used in surgery from a clinical perspective, review their key technical features and specifications and evaluate their clinical aims, reported effectiveness and safety.

1.1. Principles and physical basis of HSI

1.1.1. Light and the hyperspectral cube

Every wavelength of light within the visible light spectrum is perceived as a spectral colour also referred to as mono-chromatic light. Standard cameras capture images comprising three broad spectral bands, namely red, green and blue whereas multispectral or hyperspectral systems capture images from multiple narrow spectral bands (typically some tens to hundreds) covering a well-defined range of the electromagnetic spectrum. Consequently, each pixel in the multispectral/hyperspectral image provides a rich representation of the full colour spectrum (in contrast to restricting three values for RGB), which can characterize tissue with greater colour, detail, and accuracy. Increasing number of spectral bands drives the transition from multispectral to HSI but the distinction is loosely defined. For the remainder of this article we shall thus refer to all types as HSI.

Single timepoint HSI datasets are termed a hyperspectral cube (HS cube). Hyperspectral cube data spans two dimensions (2D) in space (field of view) and one dimension in spectra (captured wavelength/quasi-monochromatic colours) (Figure 1). In label-free imaging, the spectral reflectance signature captured by each pixel enables HSI systems to quantitatively assess various tissue properties to provide an assessment of a tissue's microstructure, perfusion and presence of disease [21, 22].

Figure 1.

Hyperspectral cube. Note how spectral signatures may be generated for specific tissues. A.U.: Aribitary Units

1.1.2. Optical properties of tissues

All tissues reflect light allowing them to be seen (or imaged) in the visible spectrum. As light propagates across biological tissue it undergoes multiple absorption and scattering events [23]. Tissue optical characteristics are determined by the type of tissue and its constituents but may be altered by presence of disease and by surgery [22]. Tissue absorption is determined by molecular composition and can serve as a spectral fingerprint for diagnostic purposes. Tissue components absorbing light are called chromophores and may either be converted to heat or radiated in the form of luminescence (bioluminescence), including fluorescence and phosphorescence but play a limited role in label-free HSI. Scattering is principally caused by the heterogenous subcellular composition of biological tissue but it also reflects a tissue's structural variation due to its extracellular membranous proteins.

1.1.3. Hyperspectral acquisition methods

There are three main methods of acquiring HSI data; namely, spatial scanning, spectral scanning or snapshot imaging (Figure 2). Spatial scanning methods typically acquire HSI data by concurrently capturing the complete wavelength spectrum for each pixel (point-scanning) or for a line of pixels (line-scanning) and scanning through the field of view. Point-scanning and line-scanning methods are also referred to as “Whiskbroom” or “Pushbroom” techniques, respectively, and are currently unable to provide a live display of spectral images.

Figure 2.

Hyperspectral imaging acquisition methods. A and B, Spatial scanning. A, Point Scan. B, Line scan. C, Spectral scanning, for example, wavelength scan. D, Snapshot. Adapted from Wang et al [24]

Spectral scanning methods typically use a single exposure to concurrently acquire the whole scene at a single wavelength band with a 2D detector array; they then “step” through each wavelength sequentially to complete the datacube. Spectral scanning methods may also be termed “wavelength scanning” or “wide-field imaging methods” [24]. Advanced image detectors such as charged couple devices (CCDs) and scientific complementary metal-oxide semiconductors (sCMOS) can now enable rapid hyperspectral wide-field imaging by swiftly acquiring 2D images at multiple wavelengths over time. Wavelength scanning has the advantage of achieving high spectral resolution by using monochromatic filter sets such as rotating filter wheels, liquid crystal tunable filters (LCTF) and acousto-optic tunable filters (AOTFs) but the increased spectral selectivity of using a filter is tempered by low light throughput or prolonged imaging times.

Snapshot imaging approaches utilize image-diversion and dispersive elements to acquire the entire three dimensional datacube in a single exposure without using any scanning mechanism. Snapshot cameras may be divided into two categories according to their acquisition and image reconstruction strategies and include direct-measurement systems such as a snapshot mosaic system, or those utilizing computational systems. Similar to modern RGB image sensors that are arranged in a regular, mosaic-like Bayer 2 × 2 pixel filter pattern, the snapshot mosaic systems utilize refined n × n pixel filter arrays and may sometimes be referred to as snapshot mosaic cameras. Computational systems can either perform direct image reconstruction or iterative image reconstruction [25]. Snapshot systems have the potential to provide real-time HSI; however, at present, computational methods are often disadvantaged by their computational complexity which significantly slows datacube processing and direct-measurement systems currently require large-format cameras which typically have a low frame rate resulting in a lower spatial and spectral resolution [26].

2. Methods

2.1. Search strategy

A structured search of the Pubmed and Web of Science databases was undertaken over a 10-year period. The last date of the search was 11 May 2018 with no initial language restriction. Two independent researchers (JS and YX) applied the search criteria using the Boolean search terms:

1. (hyperspectral OR multispectral)

   AND

2. (surgery OR surgical OR intraoperative OR operative OR microscopy OR endoscopy OR laparoscopy)

To ensure all eligible publications were identified, reference lists of included articles were also reviewed, and expert opinion sought.

2.2. Study selection

All citations were evaluated for eligibility; the selection scheme is reported in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram [27] (Figure 3).

Figure 3.

PRISMA diagram. Flow diagram illustrating study selection

Titles and abstracts were screened to identify articles that satisfied the following eligibility criteria:

1. Clinical studies in which in vivo label-free non-fluorescence HSI techniques had been used intra-operatively in adult patients undergoing surgery

2. Studies that reported on the technique's safety or effectiveness

All peer-reviewed publications in which English-language manuscripts were available through electronic indexing comprising were eligible for inclusion in the systematic review. Full articles were obtained and further assessed for eligibility and any discrepancy was resolved through mutual review and involvement of the senior author.

2.3. Data extraction and outcomes

A data extraction template was used by two independent reviewers (JS and YX) to collect study-level information, including:

1. System name

2. Stated clinical aim of the HSI device, including the target tissue, patient group and operation

3. Key technical features of the device including hyperspectral cube acquisition time

4. Technical specifications, including; camera(s) used, imaged wavelength spectrum, light source, size and compatibility (if designed to be used with current surgical hardware)

5. Safety and effectiveness of the device

Corresponding authors and device manufacturers were contacted to provide supplemental data when required.

2.4. Quality assessment

We assessed methodological quality by using the Methodological Index for Non-Randomised Studies (MINORS) scoring system [28]. All studies are scored on the following criteria: (1) a stated aim of the study; (2) inclusion of consecutive patients; (3) prospective collection of data; (4) endpoint appropriate to the study aim; (5) unbiased evaluation of endpoints; (6) follow-up period appropriate to the major endpoint; (7) loss to follow-up not exceeding 5%; (8) prospective calculation of the study sample size. Comparative studies are also scored with respect to: (9) an adequate control group; (10) contemporary groups; (11) baseline equivalence of groups; (12) adequate statistical analyses. Rating scores out of 16 and 24 for non-comparative and comparative studies, respectively, were generated by the lead author. Studies of greater quality, that is, those with a higher MINORS score, were given greater weighting in the subsequent qualitative synthesis.

2.5. Study analysis

A descriptive analysis of the included studies is performed, summarizing the extracted data and comparing the clinical and technical features of the various HSI devices. The principal summary measure is the hyperspectral cube acquisition time.

3. Results

3.1. Studies included in final analysis

In total, 2473 articles were identified through the database searches and an additional record was identified through other sources. Following removal of duplicate and non-English language studies, 1837 manuscript titles and abstracts were screened. After applying the eligibility criteria 15 full text articles were reviewed. A further two articles were excluded because HSI analysis was only performed on tissue ex vivo in these studies. In all, 13 studies satisfied the inclusion criteria and were included in further qualitative analysis (Figure 3).

Thirteen studies satisfied the inclusion criteria and underwent qualitative analysis [29–41] (Table 1). Seven different multispectral or HSI systems have been used in human surgical studies including four studies reporting use of a digital light processing (DLP) HSI system in renal surgery [30–33] and four studies from the collaborative European HELICoiD (HypErspectraL Imaging Cancer Detection) project [37–40]. Studies comprised either individual case reports [29, 35, 41] or case series involving four to 37 patients [30–34, 36–40].

Table 1. Label-free in vivo intraoperative hyperspectral imaging systems.

System name	Reference(s)	Clinical aim	Target tissue	Operation	Number of Patients	Key technical features of HSI system	Acquisition mode	System specifications				1. Safety 2. Effectiveness 3. Limitations
								Camera(s)	Imaged wavelength spectrum	Light source	Size and compatibility
Multispectral spatial frequency domain imaging (SFDI) system	Gioux et al [29]	Intraoperative tissue oxygenation map	Skin	Reconstructive skin flap in breast surgery	Not specified Result from 1 patient reported	Multispectral data acquisition time: 3.6 s Spectrally-resolved spatiallymodulated near-infrared light system Based on the FLARE™ imaging system with a custom fibre-coupled multispectral laser diode light source permits fast (<1 s) measurement of optical properties at a single wavelength over large fields of view (>100 cm2)	Wavelength scan	2x NIR cameras (800-848 nm & 689-725n m bandpass) 1x RGB camera (400-650 nm bandpass)	Six discrete wavelengths: 670, 730, 760, 808, 860, and 980 nm	12 × laser diodes 4 × diodes required at each fluorescence-dedicated wavelength due to higher power demand of fluorescence imaging (670 and 760 nm)	14 × 12 cm imaging head mounted onto a cart	No safety concerns documented. First-in-human pilot study demonstrated that a HSI oxygenation map could be generated with subjective correlation to the operative procedure Time-consuming calibration Multiple acquisitions per image required SFDI significantly influenced by operating room lighting
Digital Light Processing (DLP) Hyperspectral Imaging	Holzer et al [30]; Best et al [31]; Olweny et al [32]; Liu et al [33]	Intraoperative tissue oxygenation map	Kidney	Open and laparascopic partial nephrectomy	21 [30]; 26 [31]; 18 [32]; 37 [33]	HS cube acquisition & processing time: <30 s 81 × 81 pixel area 126 spectral bands Entire kidney may be imaged but an optical biopsy is then performed on 81 to 100 pixel area to generate a representative sample of the kidney's oxygenation status	Wavelength scan	CCD camera	520 to 645 nm (1 nm steps)	500 W Xe arc lamp Digital micro-mirror device (DMD) used to select target wavelength spectrum	Compatible with standard operating microscope and laparoscope probe with separate data processing unit	No safety concerns documented. HSI oxygenation map can successfully characterize dynamic changes in renal oxygenation, correlating with intraoperative 6 of 14 surgical manoeuvres and may predict post-operative renal function Can only measure the percent of HbO2 at the parenchymal surface (microvascular level) and may not be representative of arterial oxygen saturation
Customized HSI system	Klaessens et al [34]	Intraoperative tissue oxygenation map	Skin	Reconstructive radial free arm flap in ENT oral cancer surgery	5	HS cube acquisition and processing time: Unk Limited technical detail	Wavelength scan	CMOS camera	370 to 880 nm	LED light source (370-880 nm) LED 2D flat panel consists of 600 LEDs, with 17 different wavelengths	Large standalone free-standing machine separate to standard microscope	No safety concerns documented. Subjective correlation of HSI oxygenation map with operative procedure Limited technical detail
Customized HSI system	Noordmans et al [35]	Intraoperative tissue oxygenation map	Brain	Epilepsy surgery	1	HS cube acquisition time: Unk 1392 × 1024 pixel area Four spectral bands Four wavelengths captured at 1.5 s intervals at discrete regions of interest	Wavelength scan	LCTF Monochrome camera with 1392 × 1024 resolution	Four discrete wavelength: 480, 570, 600 and 660 nm	Standard illumination from microscope used to image four discrete	Mounted onto standard surgical microscope	No safety concerns documented. Subjective correlation of HSI oxygenation map with localisation of epileptogenic zone Slow acquisition time (7 min scan)
HSC-1700	Mori et al [36]	Intraoperative tissue oxygenation map	Brain	STA-MCA bypass for Moyamoya Disease and ICA occlusion	4	HS cube acquisition time: 5 to 16 s 640 × 480 pixel area 80 spectral bands 2D oxygen saturation maps produced for 6 regions of interest Subset ROIs of 20 x 20 pixels extracted from each figure to calculate mean oxygenation data	Line scan	F1.4 50 mm lens + HSC1700 camera (Eba Japan)	400 to 800 nm (5 nm steps)	Xe light (XEF-152S Kenko)	No details provided	No safety concerns documented. HSI oxygenation map may be used to evaluate cortical metabolism and is representative of intraoperative dynamic changes No limitations reported
HELICoiD	Fabelo et al [37]; Ravi et al [38]; Fabelo et al [39]; Fabelo et al [40]	Tumour detection	Brain	Brain tumour resection	22 [37] 18 [38] 5 [39] 26 [40]	HS cube acquisition time: VNIR camera: ~80 s NIR camera: ~40 s VNIR camera: 826 spectral bands covering 400-1000 nm, spectral resolution of 2-3 nm, pixel dimension of 128.7 μm × 128.7 μm NIR camera: 172 spectral bands, covering 900-1700 nm, spectral resolution of 5 nm. pixel dimension of 0.48 μm × 0.48 μm. Data processing time: ~1 min	Line scan	CCD Hyperspec VNIR (400-1000 nm) InGaAs Hyperspec NIR (900-1700 nm)	VNIR: 400 to 1000 nm NIR: 900 to 1700 nm (5 nm steps)	150 W Quartz Tungsten-Halogen lamp (QTH)-broadband light emission from 400 to 2200 nm	Large standalone free-standing machine separate to standard microscope	No safety concerns documented. HSI can discriminate between normal tissue, tumour tissue, hypervascularised tissue and background with high specificity and sensitivity (>95%) and demonstrates the potential for real-time analysis Limitations: large hardware setup difficult to easily integrate system into standard neurosurgical workflow, limiting data capture fixed working distance limited clinical validation slow data processing
Customized HSI system	Calin et al [41]	Intraoperative tissue oxygenation map	Skin	Reconstructive skin flap performed for diabetic foot disease	1	HS cube acquisition time: Unk 940 spectral bands covering 380-800 nm, spectral resolution of 0.52 nm Manual selection of ROI with 329 spectral bands (500-660 nm), masking the outside pixels	Line scan	Xenoplan 1.4/17 objective lensDX4 CCD Camera	380 to 800 nm (0.4 nm steps)	2x 300 W Halogen lamps	Standalone machine	No safety concerns documented Subjective correlation of HSI oxygenation map with operative procedure Slow acquisition and processing time

Abbreviations: CCD, charged coupled device; HS cube, hyperspectral cube; HSI, hyperspectral imaging; InGaAs, indium gallium arsenide; LED, light-emitting diode; NIR, near-infrared; RGB, red green blue; VNIR, visible and near-infrared; Xe, xenon.

3.2. Study quality

Outcome reporting across studies with extremely variable (Table 2). Using the MINORS assessment tool, most studies scored between 7 and 9, out of a possible 16, with one notable exception being the only comparative study scoring 22 out of 24 [33]. None of the studies included in this review prospectively calculated their study size and with the exception of the study by Liu et al none included consecutive patients. Prospective data collection was inferred by the study design in most studies but was not explicitly mentioned in any.

Table 2. Quality of studies using MINORS criteria.

System name	Study	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)	Total
Multispectral SFDI system	Gioux et al [29]	2	0	0	1	0	2	2	0	n/a	n/a	n/a	n/a	7/16
Digital light processing hyperspectral imaging	Holzer et al [30]	2	0	0	1	0	2	2	0	n/a	n/a	n/a	n/a	7/16
	Best et al [31]	2	0	0	2	0	2	2	0	n/a	n/a	n/a	n/a	8/16
	Olweny et al [32]	1	0	0	1	0	0	1	0	n/a	n/a	n/a	n/a	3/16
	Liu et al [33]	2	2	2	2	2	2	2	0	2	2	2	2	22/24
Customized HSI system	Klaessens et al [34]	1	0	0	0	0	0	0	0	n/a	n/a	n/a	n/a	1/16
Customized HSI system	Noordmans et al [35]	1	0	0	0	0	0	0	0	n/a	n/a	n/a	n/a	1/16
HSC-1700	Mori et al [36]	2	0	0	1	1	2	2	0	n/a	n/a	n/a	n/a	7/16
HELICoiD	Fabelo et al [37]	1	0	0	2	1	2	2	0	n/a	n/a	n/a	n/a	8/16
	Ravi et al [38]	2	0	0	2	1	2	2	0	n/a	n/a	n/a	n/a	9/16
	Fabelo et al [39]	2	0	0	2	1	2	2	0	n/a	n/a	n/a	n/a	9/16
	Fabelo et al [40]	2	0	0	2	1	2	2	0	n/a	n/a	n/a	n/a	9/16
Customized HSI system	Calin et al [41]	2	0	0	1	1	2	2	0	n/a	n/a	n/a	n/a	8/16

Note: (1) A stated aim of the study; (2) inclusion of consecutive patients; (3) prospective collection of data; (4) endpoint appropriate to the study aim; (5) unbiased evaluation of endpoints; (6) follow-up period appropriate to the major endpoint; (7) loss to follow-up not exceeding 5%; (8) prospective calculation of the study sample size. Comparative studies are also scored with respect to: (9) an adequate control group; (10) contemporary groups; (11) baseline equivalence of groups; (12) adequate statistical analyses.

Abbreviations: HELICoiD, hyperspectral imaging cancer detection; HSC, hyperspectral cube; HSI, hyperspectral imaging; MINORS, Methodological Index for Non-Randomised Studies; SFDI, spatial frequency domain imaging.

3.3. Clinical aims

To date, HSI has only been used for intraoperative tumour detection in brain tumour surgery [37–40] whereas a variety of different surgeries have used HSI to assess tissue perfusion and oxygenation; including, an assessment of reconstructive skin flaps in breast [29] and oral cancer [34] surgery, to assess renal oxygenation during partial nephrectomy [30–33], and to assess brain perfusion within the epileptogenic zone during epilepsy surgery [35] and following cerebrovascular anastomosis [36].

3.4. Acquisition methods

The reported HSI systems used either a wavelength scanning [29–36] or line-scanning [37–41] acquisition mode. Two systems imaged a selection of discrete wavelengths [29, 35], whereas the remainder preferred to image a spectral range with 0.4 to 5 nm wavelength steps. A wide spectral wavelength range was employed by the HELICoiD group for the purposes of tumour detection (400-1700 nm) [37–40] whereas a narrower spectrum was used in studies generating an intraoperative tissue oxygenation map. A very narrow wavelength range of 520 to 645 nm used by the Dallas group [30–33] but other HSI oxygenation studies included the infrared wavelength spectrum.

Different bespoke configurations of cameras, and light sources were used in the construction of the reported HSI systems and these are listed in Table 1. The size of each system also varied considerably; some systems were designed to be compatible with existing surgical microscopes [30–33, 35] whereas others were large free-standing machines [29, 34, 37–41]. No safety concerns relating to the use of intraoperative HSI were reported in any of the studies.

3.5. Clinical correlation of HSI data

All studies generating a tissue oxygenation map reported good correlation between the HSI data and the operative procedure performed although this was frequently a subjective assessment [29, 34, 35, 41]. However, the Dallas group quantitatively analysed their HSI data by correlating intraoperative renal oxygenation measurements with patients' eGFR results in both open [30, 31, 33] and laparoscopic [32] renal surgery. Their pilot studies demonstrated that HSI renal oxygenation data may help to predict those patients at risk of postoperative renal insufficiency and may distinguish between patients with otherwise similar baseline characteristics such as eGFR [31, 32]. To date, the HELICoiD group are the only ones to have used HSI to perform in vivo intraoperative tissue characterization [37–40] and successfully demonstrated that HSI may be used to discriminate between normal brain tissue, tumour tissue, hypervascularised tissue, and background with high specificity and sensitivity using an intra-patient 10-fold cross-validation method.

The most frequently reported limitation of the current HSI systems relate to the time-consuming calibration required and the slow image acquisition and processing time. Based on the available data, the time taken to acquire a complete hyperspectral cube dataset was calculated to take between 5 and 30 seconds. The size and bulkiness of some of the standalone systems, such as the one used in HELICoiD studies [37–40], also made it difficult to easily integrate its use into the standard surgical workflow which in turn reduced the amount of training data acquired and the image processing speed.

4. Discussion

HSI can quantitatively assess biomarkers such as oxyhaemoglobin and deoxyhaemoglobin and can assess tissue type based on its spectral characteristics. Several medical research HSI systems quantitatively measure oxygenation levels to help guide the diagnosis and management of various vascular and circulatory conditions [1–8]. A second main area of HSI research has been focused on developing HSI systems for cancer detection. Tumours alter tissue absorption, scattering and fluorescence because of biochemical and morphological changes that occurs as a result of tumour infiltration. HSI systems analyse these changes and numerous groups have published encouraging work demonstrating the diagnostic success of HSI in in vivo animal tumour models and patient populations with breast [9], cervical [10–12], gastrointestinal [13, 14], ovarian [15, 42], urological [43], skin [44–47] and head and neck cancer [18–20, 48].

4.1. Clinical functions of HSI

The ability to deliver real-time hyperspectral images at a video-rate makes it ideally suited for intraoperative surgical use. HSI has the potential to enhance a surgeon's vision at the molecular, cellular and tissue level in four different ways; including (a) the identification of residual tumour tissue; (b) the classification of critical anatomical structures within the surgical field; (c) the ability to monitor tissue oxygenation during surgery and (d) by enabling surgeons visualize target structures under blood.

4.1.1. Identification of residual tumour tissue

It is often very difficult to differentiate a line of demarcation between a tumour and surrounding healthy tissue using the human eye. Panasyuk et al were one of the first to demonstrate that HSI may be used to guide surgery in vivo by using HSI to distinguish tumour tissue from normal breast and other tissues in an experimental rat breast tumour model [48]. Others have also evaluated the effectiveness of HSI to examine tumour resection margins in ex vivo surgical specimens in the operating room including patients undergoing head and neck cancer [20, 49], colon [50], and skin cancer [51] surgery and in vivo fluorescence-based HSI systems have been reported for guiding urological [52] and brain tumour surgery [53].

4.1.2. Identification of critical anatomical structures

In many operations, the target tissue or tumour is intricately associated with other critical anatomical structures which should be preserved during surgery. HSI offers the potential to create real-time anatomical maps that may be overlaid onto the surgeon's visualization system to highlight the location of such critical structures. Several animal studies have demonstrated this application of HSI for use in a variety of surgical procedures including; cholecystectomy [54, 55], hysterectomy [56] and parotidectomy [56].

4.1.3. Monitoring tissue oxygenation

Tissue perfusion and oxygenation are positive indicators of a tissue's viability. Assessing dynamic changes in tissue perfusion and oxygenation aids patient management in various clinical settings and several studies have already demonstrated its potential for guiding surgical procedures. The Dallas group developed HSI technologies to quantify tissue oxygenation in renal surgery [57, 58] whilst others have applied the technology to guide flap reconstruction [59–61] and assess vessel [62–64] and bowel [65, 66] anastomoses.

4.1.4. Visualizing structures beneath blood

Finally, HSI in the near infrared wavelength spectrum has been shown to successfully characterize target tissues submerged in a blood layer that could not be seen by the naked eye [67].

4.2. Intraoperative in vivo HSI functions

To date, intraoperative in vivo surgical HSI devices have either focused on generating an intraoperative tissue oxygenation map during surgery [29–36, 41] or by enabling tumour detection [37–40]. The intraoperative assessment of tissue and perfusion can assist the surgeon in various ways. The Dallas group developed a HSI system capable of imaging the entire kidney during in both open and laparoscopic surgery [30–33] and is a continuation of the preclinical work performed in a pig model [57, 58]. The customized DLP system used in their studies consists of a wavelength scanning system in the 520 to 645 nm range. An “optical biopsy” is subsequently performed on an 81 to 100 pixel square area to generate a representative sample of the kidney's oxygenation status. Their work demonstrated that oxygenation maps can successfully characterize the dynamic changes in renal perfusion observed during surgery. It also showed that intra-operatively acquired HSI data may be used as an independent predictor of post-operative renal function [31, 32].

Three other studies have evaluated how intraoperative tissue oxygenation maps generated from HSI data might guide skin flap reconstruction in breast [29], oral cancer [34] and vascular [41] surgery. These studies all demonstrated a correlation between the HSI-oxygenation data and the expected dynamic surgical changes however post-operative follow-up data relating to the end-organ's viability was not available. Thus, in future work, it would be helpful to compare the viability of skin flaps performed with and without the intraoperative use of HSI-data. Following on from existing pre-clinical work [62–66], it will also be exciting to see whether HSI-generated tissue oxygenation maps of vessel and bowel anastomoses may be acquired in human patients and whether its use helps to improve surgical technique and outcomes.

The second clinical focus of using intraoperative HSI has been to guide tumour resection. The feasibility of using HSI as a diagnostic technique for cancer has been comprehensively evaluated in multiple preclinical animal models and several recent studies have also demonstrated its use in assessing tumour margins in ex vivo tissue in the operating room [9–20, 43–47]. Fewer studies have reported the in vivo use of HSI during surgery but several groups have used HSI to highlight subtle changes in exogenous fluorescence [52, 53, 68]. Bravo et al demonstrated that hyperspectral data process improves PpIX contrast during fluorescence guided surgery of human brain tumours [53] and other laboratory studies have confirmed that HSI can quantitatively measure and visualize low concentrations of PpIX and that there is a significant correlation with the tissue's cellularity and nuclear-cytoplasmic ratio [69].

Recently, the HELICoiD group have been demonstrating the potential of label-free HSI in guiding brain tumour surgery [37–40]. Their customized supervised and unsupervised machine learning classifier method provided excellent results with specificity and sensitivity reaching 100% in most cases using a 10-fold intra-patient cross-validation method [39]. However, despite these very positive results, it is very difficult to validate in vivo HSI results. This limitation is particularly true in brain tumour surgery where the margins between tumour and normal brain are blurred and where normal brain tissue cannot be excised for routine testing.

Another limitation of the HELICoiD system was its size. The system's hardware was very large and cumbersome [39, 40] which made it difficult to integrate easily into the standard neurosurgical workflow. This, in turn, limited its use and the amount of data captured. High-speed computational processing is required to deliver real-time intraoperative image analysis but is reliant on a large training dataset and only 22 patients (36 captures) were obtained [37–40]. Consequently, the total image processing time, including acquisition and computational processing, took between 35.53 and 68.76 seconds even with the use of an accelerated processing algorithm [40]. A slow hyperspectral cube acquisition and processing time was common to most studies (5-30 seconds) and this remains the key obstacle to delivering a truly real-time HSI system suitable for intraoperative use.

Future work should focus on developing optimal, yet practical intraoperative HSI systems compatible with existing, commonly used surgical visualization systems that may then be tailored for use in different surgical procedures. Researchers should determine key design specifications to enable the easy integration of such a device into their desired surgical workflow in order to facilitate its widespread use, enabling the acquisition of larger training datasets required to generate reliable real-time models.

4.3. Limitations

The present systematic review was limited by various factors. Firstly, given the variety of ways HSI data was presented and the small numbers of available studies, it was not possible to perform a meta-analysis and quantitatively analyse the data. Consequently, we were unable to draw any firm conclusions concerning the effectiveness of the described HSI techniques. Secondly, the studies were of mixed methodological quality and reported a variety of customized HSI systems, limiting our discussion to a qualitative discussion of a small number of higher quality studies.

5. Conclusion

HSI has the potential to transform intraoperative surgical guidance. Numerous studies have evaluated its use in preclinical in vivo animal models and others have demonstrated the usefulness of intraoperative ex vivo tissue analysis. The use of in vivo label-free HSI systems has not been widely tested during surgery but the small number of studies demonstrate its current capabilities and highlight avenues for future research. However, in order to translate this promising imaging technique into regular surgical use, it must be seamlessly integrated into the surgical workflow. Hardware development therefore needs to leverage recent advances in the miniaturization of hyperspectral sensors and robust real-time image models must be developed.