Abstract
We present an automatic classification strategy for early and accurate assessment of burn injuries using terahertz (THz) time-domain spectroscopic imaging. Burn injuries of different severity grades, representing superficial partial-thickness (SPT), deep partial-thickness (DPT), and full-thickness (FT) wounds, were created by a standardized porcine scald model. THz spectroscopic imaging was performed using our new fiber-coupled Portable HAndheld Spectral Reflection Scanner, incorporating a telecentric beam steering configuration and an f- scanning lens. ASynchronous Optical Sampling in a dual-fiber-laser THz spectrometer with 100 MHz repetition rate enabled high-speed spectroscopic measurements. Given twenty-four different samples composed of ten scald and ten contact burns and four healthy samples, supervised machine learning algorithms using THz-TDS spectra achieved areas under the receiver operating characteristic curves of 0.88, 0.93, and 0.93 when differentiating between SPT, DPT, and FT burns, respectively, as determined by independent histological assessments. These results show the potential utility of our new broadband THz PHASR Scanner for early and accurate triage of burn injuries.
Subject terms: Terahertz optics, Imaging and sensing, Optical spectroscopy, Biophotonics
Introduction
In 2018, approximately 416,000 patients were treated for burn injuries in emergency departments in the United States1. The depth of injury in a cutaneous burn determines its potential for spontaneous healing or the need for surgical intervention, thus guiding the course of clinical management and the treatment plan. Burn injuries are designated into three categories: superficial (first-degree), partial-thickness (second-degree), and full-thickness (third-degree)2. The thermal insult in superficial burns is confined to the epidermis. Therefore, they will generally heal on their own through the re-epithelialization process3. In full-thickness burns the entire dermis layer, and often the underlying hypodermis, is necrotic, and therefore they require surgical intervention (skin grafting)4,5. Meanwhile, some of the partial-thickness burns heal spontaneously (i.e. superficial partial-thickness), whereas others will progress to a full-thickness state over the course of the natural inflammatory response to the burn (i.e. deep partial-thickness), which usually lasts a few days post-burn6,7. This intermediate group is largely responsible for inaccuracies in burn triage and therefore poor wound healing outcomes8. Early diagnosis and timely treatment, in the form of excision and grafting of deep burns, are therefore crucial in improving and expediting the healing process, reducing the infection rate, and avoiding hypertrophic burn scars9–11. Although the current clinical assessment technique, based on the visual examination of a burn injury, is effective in differentiation between superficial and full-thickness burns, classification of intermediate burns is largely subjective and remains the main source of error in burn triage8. The accuracy rate of clinical assessment for partial-thickness burns has remained as low as 60–75 several days post burn induction12, and significantly lower (about 50) for a diagnosis achieved within 24–48 h6. This is mainly due to the dynamic nature of the partial-thickness burns during the first 48–72 h after injury13.
The low accuracy in timely assessment of partial-thickness burns has motivated the development of a wide range of objective diagnostic modalities based on various imaging techniques. Different optical modalities, such as laser Doppler imaging14, indocyanine green (ICG) video angiography15, diffuse reflection near-infrared spectroscopy16,17, orthogonal polarization spectral imaging (OPSI)18, imaging by spatially-modulated structured light, also known as spatial frequency-domain imaging (SFDI)19, laser speckle imaging (LSI)20, photoacoustic microscopy (PAM)21, and polarization-sensitive optical coherence tomography (PSOCT)22, have been explored for that purpose. These techniques mostly focus on monitoring perfusion15,23, alterations in the cellular metabolism24, or the patency of the tissue microvasculature18. A major drawback of current optical techniques in the visible regime is their limited penetration depth, confining them to superficial blood flow. However, the analysis of deeper tissues would be more beneficial for differentiating the partial-thickness burns6,25. Additionally, most of these techniques have a small field of view, resulting in long acquisition times for larger wounds, which renders them more susceptible to the artifacts caused by the patient’s movements over time20,26. Among the aforementioned techniques, LDI offers the best estimates of burn injury severity27, identifying the full-thickness burns requiring surgical intervention with 95 accuracy28. However, LDI is less accurate when utilized within the first 24–48 h of injury14,27. Other wound healing studies indicated that it is only on day 3 post-burn that the severity assessment using LDI is significantly more accurate than the routine clinical assessment techniques14.
Terahertz time-domain spectroscopy (THz-TDS) has emerged as a noninvasive and objective tool for assessment of various biological tissue types, including determining corneal hydration29–32, diagnosis of brain tumors33–35, diabetic foot syndrome36, delineating breast cancer tumor margins37–41, and classification of burn injuries42,43. In burn diagnosis applications, it was shown that the reflectivity of the full-thickness burns in rodent models is higher compared to the normal skin over a frequency range between 0.2 and 1 THz44. This increased THz reflectivity was explained by the post-burn formation of the interstitial edema. Moreover, the double Debye dielectric relaxation model of healthy human skin45,46 was extended to model the THz response of the less severely burned tissue samples. More recently, it has been shown that THz waves are not only sensitive to the post-burn formation of interstitial edema, but also to the density of skin structures47. A linear combination of broadband THz reflectivity and spectral slope was correlated with the density of discrete scattering structures in the skin layers. Additionally, in an in vivo porcine pseudo-scald model, it was shown that a similar combination of THz hyperspectral parameters, i.e., the area under the reflectivity curve and the roll-off spectral slope, introduced as a novel THz Z-metric value, can differentiate between burn injuries with burn depth exceeding or below 350 m, as determined by vimentin immunohistochemistry48. The Z-metric was further optimized to differentiate burns into two categories: 1. depth of injury greater than 50 of the dermis thickness, and 2. depth of injury less than 50 of the dermis thickness49. Moreover, it was reported that the longitudinal variation of the Z-metric in superficial partial-thickness burns is different from deep partial-thickness and full-thickness burns, and therefore it can be used for monitoring the wound healing outcomes.
Previous THz-TDS studies for burn assessment have utilized point-spectroscopy measurements, which are unable to capture the heterogeneity of the tissue by forming spectral images. In this work, we have addressed this limitation by demonstrating broadband THz spectroscopic imaging of graded burn severities in an in vivo porcine model. To enable large animal imaging without suffering from motion artifacts, all optical components were placed inside a fiber-coupled alignment-free handheld housing. A motorized gimbal mirror and a f- scanning lens were used in the handheld housing to raster scan the THz waves over a 27 27 mm field-of-view. To examine the potential of THz-TDS for early diagnosis of partial-thickness burns, we compared the THz spectra measured 1 h after burn induction (Day 0) to histology measurements collected four days post-burn, when the dynamic burn wounds typically reach their maximum depth. Supervised machine learning algorithms using support vector machines, naive Bayes classifiers, discriminant analysis classifiers, and ensemble learning were developed and employed to classify the in vivo burn injuries into three groups, namely superficial partial-thickness (less than 40 dermal depth), deep partial-thickness (between 40 and 80 dermal depth), and full-thickness (above 80 dermal depth). The area under the curve (AUC) in the receiver operating characteristic (ROC) plots using support vector machines (SVM) with a Gaussian kernel reached 94 for classification of superficial partial-thickness burns, 89 for deep partial-thickness burns, and 91 for full-thickness burns. These results confirm that the THz-TDS measurements and machine learning techniques can be used together to yield accurate and early assessments in acute burn injuries.
Methods
THz time-domain spectroscopy
THz-TDS measurements were obtained using our PHASR (Portable HAndheld Spectral Reflection) Scanner, described previously50. Briefly, we incorporated the TERA ASOPS dual-fiber-laser spectrometer (Menlo Systems, Inc., Newton, NJ, USA) in our PHASR Scanner, where THz pulses are generated by optical excitation of a photoconductive antenna (PCA) using 1560 nm pulses of a femtosecond laser at 100 MHz repetition rate. A second synchronized femtosecond laser at a close repetition rate with a fixed 100 Hz offset generates the probe pulses. This offset rate allows for high-speed scanning over ten nanoseconds of time delay between pump and probe pulses without a mechanical delay line. The optical components of the spectroscopic imaging setup, schematically shown in Fig. 1a, were placed inside the handheld housing of the PHASR Scanner, shown in Fig. 1b. Broadband THz beams generated by the emitter PCA are collimated using a TPX lens (Menlo Systems, Inc., Newton, NJ, USA) with 50 mm focal length. A high-resistivity silicon beam splitter guides the collimated beams toward a mirror mounted on a 2-axis motorized heliostat system, composed of a goniometer and a rotational stage, which raster scans the beam over a custom-made f- lens51. In a telecentric f- scanning system, collimated beams passing through the front focus of an f- lens at a deflection angle are focused to a true planar surface at a distance from the optic axis52. Therefore, the focused beam remains parallel to the optic axis and has a constant spot-size at the target plane52. In this study, we used a high-density polyethylene (HDPE) f- lens with a 40 mm focal length. Upon reflection from the sample, THz beams retrace the path of the incident beams back to the beam splitter, where they are directed towards a focusing TPX50 lens and collected by the detector PCA. Using this THz PHASR Scanner design, an area of approximately 7.29 cm (27 27 mm) was scanned in about three minutes, while recording the full THz-TDS spectrum of each 1 mm pixel area. Table 1 provides the performance metrics of the PHASR Scanner.
Table 1.
Parameter | Value |
---|---|
Housing dimensions | 37.3 14 25.1 cm |
Focus size | 3.1 2.4 mm |
Focus depth | 7.25 mm |
Field of view | 27 27 mm |
Pixel size | 1 1 mm |
Number of time averages | 20 averages per pixel |
Scan time | 3 min per field of view |
A custom MATLAB (Mathworks Inc., Natick, MA, USA) software was used for controlling the hardware and acquiring the THz-TDS measurements.
Burn protocol
We conducted an acute in vivo scald study in a porcine model. The experimental protocol used in this study was approved by the Institutional Animal Care and Use Committee (IACUC) at the Stony Brook University. All methods reported here were performed in accordance with the relevant guidelines and regulations of the IACUC and follows the recommendations in the ARRIVE guidelines53. The animal model used was female Yorkshire pigs, weighing 45–50 kg. Prior to inducing the burns on the first day of the study (Day 0), the animal was sedated through an intramuscular (IM) injection of a pre-anesthetic cocktail consisting of ketamine (20 mg/kg), xylazine (2.2 mg/kg), acepromazine (0.1 mg/kg), and atropine (0.02 mg/kg), and then anesthetized with a continuous flow of 0.5–5 isoflurane. The animal was kept on isoflurane throughout the imaging process and monitored continuously. On Day 0, multiple scald or contact burns were created on the dorsal side of the subject54, following the pattern shown in Fig. 1c. The scald device, which is shown schematically in Fig. 1d, was composed of a foam-wrapped steel pipe, an inflow hot water tube, and an outflow suction tube. When water in a heating reservoir reached the desired temperature using an immersion heating circulator, it was pumped into the steel pipe placed on the pig skin. After being in contact with the skin for a specific duration, e.g., ten seconds in this study, the hot water was removed using a vacuum pump (Adafruit Industries, New York, NY, USA), attached to a vacuum flask. The contact device, shown schematically in Fig. 1e, was composed of a metallic brass bar, a heating element, a thermocouple, and a spring loaded tube to maintain constant pressure (2 kg/6.25 cm). Burn severity grades were created at multiple sites by exposure to hot water or brass bar at 70 C, 80 C, and 98 C, shown in Fig. 1c by black, blue, and red, respectively. To control for the anatomical and skin thickness variations, the locations of each burn induction temperature group were uniformly varied and distributed on the dorsal side of the animal subject. Moreover, the locations of the burns were chosen such that they were 4 cm apart in all horizontal and vertical directions. This distance is deemed sufficient to avoid the effect of inflammatory response of each burn on its adjacent burns55. Following the burn induction, buprenorphine (0.005-0.02 mg/kg) was administered IM and a 72-h slow-release transdermal fentanyl patch (50 g/g) was placed proximal to the tail. The THz-TDS measurements on Day 0 were obtained after the burns were debrided by gently scraping with the blunt end of forceps, consistent with the wound debridement step in routine clinical burn care. After obtaining the THz-TDS measurements using the PHASR Scanner, as is shown in Fig. 1f, triple antibiotic ointment was applied to each burn wound to prevent infection. The burns were then bandaged by covering the wounds with a transparent Tegaderm sheet (3M, Saint Paul, MN, USA). Finally, the midsection of the pig was wrapped with flexible gauze bandage and adhesive Tensoplast (BSN Medical, Hamburg, Germany). On Day 4 of the study, 96 h post burn induction and prior to imaging, the animal was sedated and anesthetized with the same procedure used on Day 0.
Histopathology
Four-millimeter punch biopsies were collected on Days 0 and 4 of the study (corresponding to the peak of the inflammatory response56) for histological assessment. The excised biopsies were fixed in 10 buffered formalin for 24 h and then paraffin embedded. They were then stained with hematoxylin and eosin (H&E). Microscopic images of the H&E-stained biopsies for healthy skin and representative burn wounds are shown in Fig. 2a–d. The burn depth was assessed by measuring the deepest point of injury, which can be characterized by microvascular occlusion, collagen discoloration, or necrosis of follicular, mesenchymal, and adipocyte cells54, as identified in Fig. 2b–d. Blind evaluation was performed by a board-certified histopathologist. The depth of injury was normalized by the total dermal thickness and presented in percentage of dermis thickness. Figure 2e, f show the dermal burn depth percentage on Days 0 and 4, respectively, using box plots, where the red lines show the group medians and each box represents the 25 to 75 of the burn depth range in each temperature group, and the whiskers covering the full range of the obtained data. It can be noted that progressively deeper burns are observed with increasing scald temperature on Day 4 post-burn, when the peak inflammatory response of the tissue to thermal damage occurs and the burns reach their maximum depth56. Accordingly, tissues with less than 40 burn depth in the dermis layer were grouped as the superficial partial-thickness (SPT) category. Tissues with burn depth between 40 and 80 of dermal thickness formed the deep partial-thickness (DPT) group, and samples having larger than 80 dermal burn depth, including those with a damaged hypodermis, were grouped as the full-thickness (FT) category. It should be noted that our burn induction devices, which are designed and fabricated following the techniques demonstrated in57,58, result in highly consistent and fairly homogeneous burns over the same location. In addition, the 40 burn thickness range encompassed by each burn severity group is large enough to account for slight variations in the burn thickness over the same burn location.
Signal processing
Signal processing in both time and Fourier domains was important in extracting relevant spectroscopic information for burn assessment. In in vivo experiments, because of the relatively small number of possible averages (twenty averages per pixel in this case), the noise contribution before and after the main THz pulse was noticeable. Because the acquisition time is directly proportional to the number of time averages, acquiring many averages at each pixel is time-consuming and can result in mechanical and motion artifacts. Therefore, we implemented two data-driven signal processing steps to achieve a higher signal to noise ratio (SNR) over a larger bandwidth even with the fewer number of time averages. First, the THz waveforms were denoised using wavelet shrinkage59,60. The wavelet denoising algorithm was designed based on the maximal overlap discrete wavelet transform of the time-domain THz signals and the decomposition level-dependent wavelet thresholding61. Accordingly, at each decomposition level, one threshold was set based on two time windows in the signal, i.e., before and after the main THz pulse, and signals were modified using wavelet hard thresholding59,62. In the second pre-processing step, the THz pulse reflected from the imaging window-skin interface was deconvolved using a reference measurement obtained from the window-air interface. Here, we implemented a Wiener deconvolution scheme, where the SNR at each frequency point is accounted for to prevent large spectral value artifacts caused by the ill-posed deconvolution implementation59,63. A summary of the signal processing steps and example spectral data are shown as part of the flowchart in Fig. 3.
Machine learning classifier
Figure 3 exhibits the machine learning workflow for THz spectral analysis. The first two steps in the signal processing flowchart, i.e., wavelet denoising and Wiener deconvolution, were described in the earlier section. THz images are formed using the bandwidth-limited area under the reflection spectral amplitude curves. To create the predictor space, fifteen 4 4-pixel regions of interest (ROI) were selected randomly over each of the ten burns, where each pixel represented the THz spectral response of a 1 mm burned tissue. It has been shown that the Mie scattering by granular particulate structures gives rise to significant spectral artifacts in the THz regime64, which can be mitigated using signal processing65,66 or ensemble averaging67 techniques. To account for the scattering effects caused by the heterogeneous skin appendages, the average THz reflectivity of the sixteen pixels in each ROI was used. In other words, the deconvolved spectrum averaged over pixels in each ROI in the frequency range between 0.1 and 0.5 THz, was used as a single observation in later classification stages. Additionally, the random selection of the ROIs, in contrast to a deterministic selection, and repeating the random trials for ten iterations, avoid the classifiers’ performance to be biased toward a specific configuration of the ROIs. Burn severity classification labels were assigned using the dermal burn percentage in biopsies collected on Day 4.
Different machine learning algorithms, including support vector machine (SVM), naive Bayes (NB) classifier, and linear discriminant analysis (LDA) classifier, were modeled separately to predict the severity designation of a burn tissue based on its THz spectral response. The SVM algorithm has been used for classification of THz spectra of various pathological tissues, such as breast cancer39,68, colon cancer69, and gastric cancer70. The SVM classifier utilizes an iterative approach to maximize the boundary between different classes, while using the minimal amount of the support vectors71. In this study, we compare the performance of SVM classifiers with linear, polynomial (n), and Gaussian kernel functions72. The NB classifier leverages the Bayes theorem, with the two assumptions that predictors within each class are independent, and predictors have similar effects on the outcome73. The NB classifier estimates the prior density function of the predictors within each class, and models the posterior probabilities following the Bayes theorem74. Accordingly, an observation is assigned to the class that yields the maximum posterior probability. The LDA classifier assumes predictors within each class form a Gaussian mixture distribution74,75. In LDA, the covariance matrix of the Gaussian mixture models in different classes remains the same, while their means vary. The classifier assigns an observation to the class that minimizes an expected classification cost74. Finally, we examine Ensemble learning techniques, combination of multiple learning algorithms to improve the classification performance74, based on the boosted discriminant classifiers76. In the adaptive boosting (AdaBoost) of multiple classifiers, an equal weight is given to all the training instances initially. However, subsequent learners are tweaked to favor those instances misclassified by the previous classifiers76,77.
In experiments with limited sample size, cross-validation is used to avoid over-fitting and to estimate how accurately a predictive model will perform on an independent set not included in training74. In this work, we also used fivefold cross validation to avoid over-fitting and to evaluate the predictive ability of our classification models. In fivefold cross validation, each classifier hyper-parameters are set based on only 80 of the samples, while 20 of the samples are reserved for calculating the test error rate. Furthermore, by iterating this process for 5 times, while completely different samples are used as the test set of each iteration, we avoid the dependence of classifiers’ performance metrics on how the samples are divided into the train and test sets. Each model’s specific hyper-parameters were optimized over the training set using Bayesian optimization78. In the results section, we compare the performance of various machine learning techniques based on the area under the receiver operating characteristic (ROC) curves, i.e., the true positive rate (TPR) versus the false positive rate (FPR), obtained for differentiation between the three burn severity groups.
Results
Figure 4 compares digital images obtained from the burns on Day 0 with THz images on Day 0 formed using the area under the reflected spectral amplitude curves between 0.1 and 0.5 THz at each pixel. Images in Fig. 4a–c illustrate representative burns created using the scald device at 70, 80, and 98, respectively. The length of the scale bars in Fig. 4a–c is 10 mm. The dashed black lines show the scanner’s field-of-view on each burn, carefully placed so that the four-millimeter punch biopsies collected earlier were identified in the upper right quadrant of each image. Figure 4d–f show the corresponding THz images of the burns in Fig. 4a–c, respectively. The dashed black lines in Fig. 4d–f also delineate the same field-of-view in Fig. 4a–c. Additionally, the dashed red circles outline the location of biopsies in each image. Figure 4g–i show the representative THz-TDS pulses measured from the burn (solid red line) and the biopsy (solid blue line) locations in Fig. 4d–f, respectively. The biopsy locations. which serve as fiducial markers for imaging coregistration, can be identified using the phase of the Fabry–Perot internal reflections in the imaging window using the time-domain THz pulses, as illustrated in Fig. 4g–i. Briefly, the air gaps formed between the imaging window and the tissue at the biopsy locations result in the phase change in Fabry–Perot reflections, which are utilized to delineate the pixels associated with the biopsy locations, as shown in Fig. 4j–l.
Figure 4j–l show binary images corresponding to Fig. 4d–f, where pixels identified as biopsy are equal to one, whereas pixels elsewhere equal to zero. The biopsy pixels are excluded from the classification algorithm, since the burned tissue in those pixels is removed. Figure 4m–o show the microscopic images (5 magnification) of the H&E-stained biopsies from the burns in Fig. 4a–c, respectively. The dermal burn depth, which is determined based on the deepest point with microvascular occlusion (shown using the black arrows), is also reported for each burn in Fig. 4m–o. It can be noted that despite the lower scald temperature, the 70 burn in Fig. 4a has resulted in a 0.1 mm-deeper dermal burn in comparison to the 80 scald burn shown in Fig. 4b. This discrepancy between the induction temperature, shape, and depth of the burn wounds results from the complex heterogeneity of the skin and its constituents, and therefore, it necessitates the use of non-invasive and objective modalities that enable a deeper monitoring of the skin condition, such as the THz time-domain spectroscopy.
Figure 5 shows representative results in different steps of the signal processing flowchart explained earlier. Figure 5a shows a representative THz image of a SPT burn created using 70 scald device. The field of view shown by the x and y axes are 27 by 27 mm, and the color axis represents the normalized area under the reflection spectral amplitude curves between 0.1 and 0.5 THz. Here, the burn depth was determined to be 22 of the dermis thickness. Figure 5b, c shows the wavelet denoising outcome in an example THz-TDS pulse selected from a pixel marked by the red star in Fig. 5a. It can be noted that although the noise surrounding the main THz pulse is prominent in the raw signal (red trace) in Fig. 5b, wavelet shrinkage has effectively removed all such contributions in the denoised signal (blue trace), while the main pulse remains nearly intact.
Moreover, it can be observed in Fig. 5c that while spectral fluctuations abound in the Fourier transform of the raw THz pulse, they are removed in the wavelet shrinkage outcome. Such spectral artifacts can diminish the accuracy in automatic classification, and their removal enhances the precision in differentiation between the burn grades. Figure 5d compares the conventional and Wiener implementations of deconvolution in a 4 4-pixel ROI selected in the burn, as shown in Fig. 5a with the red box. It can be noted that the Wiener implementation effectively avoids the higher-frequency spikes after 0.6 THz, which are not diagnostic of the burn tissue and only appear due to the ill-posed deconvolution59. Figure 5e compares the Wiener-deconvolved spectral amplitude of reflectivity for three ROIs, each selected randomly from a different burn grade example, i.e., SPT, DPT, and FT burn groups. Each line in Fig. 5e illustrates the average spectral amplitude over the pixels of an ROI plotted with the standard deviation as the error bars. It can be noted that the spectral amplitude of FT burns is higher than the spectral amplitude of DPT and SPT burns. The higher spectral amplitude in deeper burns is mainly associated with the formation of the interstitial edema, which results in a change in the skin’s refractive index, yielding larger reflections from the imaging window-skin interface. Figure 5f compares the area under the spectral amplitude curves within 0.1–0.5 THz for the three burn designations in all ten burns (composed of thirty 4 4-pixel ROIs randomly selected from the burns in each group) using box plots. Here, the black circles and the red lines indicate the mean and median of each group, respectively. The bottom and top edges of each box indicate the 25th and 75th percentiles in each group, and the whiskers extend to the data points within the 1.5 times the interquartile range from either edge of the box. The outliers outside this range are plotted individually using the red ‘+’ symbol. A one-way analysis of variance (ANOVA) at level of significance confirms that the difference between SPT and FT groups ( < .001), and also the difference between DPT and FT groups ( < .001) is statistically significant, while the difference between SPT and DPT was not significant ( = 0.061). Comparing the area under the spectral amplitude curves in Fig. 5f with the dermal burn percentage reported in Fig. 2f suggests that THz spectral amplitudes measured immediately post-burn can potentially predict the eventual burn depth on Day 4, which typically increases in response to the peak inflammatory processes of the tissue to a thermal insult and occurs 72–96 h post burn induction56. It should be noted that results in Fig. 5f confirm the need for using full spectroscopic information in conjunction with supervised machine learning algorithms to improve the accuracy of burn classification using THz-TDS measurements. In the final section of this paper, we will explore such a possibility using supervised machine learning techniques.
Figure 6a–d show the ROC curves, by plotting the True Positive Rate (TPR) versus the False Positive Rate (FPR) as the discrimination threshold on a classifier’s scores is varied. The performance of the classifiers in identification of SPT (black lines), DPT (blue lines), and FT burns (red lines), i.e., differentiation of each group from the other two groups, or one versus all, are plotted separately. In the methods section, we explained that to account for tissue heterogeneity, fifteen 4 4-pixel ROIs were selected randomly over each burn, as shown in Fig. 3, and the average spectral amplitude in each ROI was utilized as a single observation for building the classifiers. Here, each ROC curve shows the average and standard deviation over ten iterations of each classifier, with different ROIs randomly selected in each run.
The dotted diagonal lines show the ROC curve of a random classifier with no discrimination power. An ROC curve with a higher area under the curve (AUC) indicates a classifier with a higher accuracy rate, with an ideal classifier having an AUC of 100. Therefore, the AUC value can be used to evaluate and compare the performance of different classifiers. In Fig. 6a–d, the average and the standard deviation of AUC values of each classifier over the ten iterations are given in the legends, where different training and test sets have been utilized in each trial. Figure 6a demonstrates the ROC curves of the SVM classifier with a Gaussian kernel function, where the AUC in the classification of SPT, DPT, and FT burns is 88 ± 3, 93 ± 3, and 93 ± 4, respectively. Figure 6b–d shows the ROC curves obtained using the NB, LDA, and AdaBoosted LDA classifiers, respectively. It can be noted that the NB classifier has the lowest overall AUC values as compared to the other algorithms. Additionally, comparing the LDA and boosted LDA curves in Fig. 6c–d reveals that does not improve the AUC for classification of either of the three groups. Comparison between the SVM classifier in Fig. 6a with the other classifiers in Fig. 6b–d, considering the range of variation in the AUC values given as the standard deviation in the legends, reveals that the Gaussian SVM outperforms other classification models. To assess the effect of the kernel function in an SVM classifier model, Fig. 6e compares the AUC values for SVM classifiers with Gaussian, polynomial, and linear kernel functions. Here, the error bars show the standard deviation of AUC over the ten iteration tests. The AUC in classification of the SPT, DPT, and FT burns using the SVM classifier with a Gaussian kernel function is 88 ± 3, 93 ± 3, and 93 ± 4, respectively. These values for the SVM with a polynomial kernel function are 88 ± 3, 92 ± 2, and 94 ± 3, and for the SVM with a linear kernel function are 77 ± 6, 84 ± 4, and 82 ± 7. It should be noted that the power of the polynomial kernel function was set as an optimization parameter during the cross-validation. Comparing different SVM kernels based on the average and standard deviation in AUC, the performance of Gaussian and Polynomial kernels is identical, whereas the performance of the SVM classifier with a linear kernel function is inferior, signifying the importance of selecting an appropriate kernel function when using this algorithm. Finally, Fig. 6f shows the sensitivity, specificity, and accuracy in classification of each burn severity group using the SVM classifier with a Gaussian kernel function. The average sensitivity rates in differentiating SPT, DPT, and FT burns were determined to be 74, 74, and 76, respectively. The average specificity rates for classifying SPT, DPT and FT burns were found to be 88, 92, and 95, respectively, while the overall average accuracy rates were 78, 84, and 88. The high specificity rate achieved using our technique means that there are fewer false positive results in each group, which is highly desirable when the diagnostic test may be used to recommend invasive or costly procedures.
Discussion
Our study shows the potential of the THz-TDS imaging for early and accurate assessment of burn injuries. We used a handheld THz time-domain scanner to classify burn injuries in an acute in vivo study of porcine scald burns. Utilization of our new PHASR Scanner based on THz asynchronous optical sampling (ASOPS) and a telecentric beam scanning system in a 3D-printed housing enabled alignment-free and high-speed broadband spectroscopic measurements inside an operating room environment. We also showed that by implementing appropriate signal pre-processing techniques, including wavelet denoising and Wiener deconvolution, common machine learning algorithms are able to classify different burn grades using THz spectroscopic features collected 1-h post burn induction. In particular, the support vector machine classifier with a Gaussian kernel function achieved above 89 area under the ROC curve for differentiation of SPT, DPT, and FT burns using the THz spectra collected only 1 h post burn induction. Importantly, the depth of burn injuries was determined by an independent and blinded histopathologist using biopsies obtained 4 days post burn induction, corresponding to the peak of the inflammatory response of the tissue. Because in routine burn care physicians often delay surgical interventions for a few days until the true depth of each burn can be discerned, the ability to predict the burn depth using THz spectra on Day 0 will allow for early and accurate determination of the treatment plan, which in turn can result in improved wound healing outcomes and infection prevention5,9.
One limitation of this study was the total number of observations available for training and testing the machine learning algorithms. Although cross-validation techniques and randomized iterative runs are well established counter-measures to potential overfitting in data scarce scenarios, it can impact quantification of sensitivity and specificity. Another limitation of studies performed with standardized burn induction protocols is that the robustness of the classification technique to various types of burn etiologies, such as burns caused by chemical or electrical energy, are not tested. Future work includes expanding experimental data sets, utilizing more advanced learning techniques, such as deep neural networks, and enhanced PHASR Scanner designs to improve the classification performance in differentiating burn injuries with various etiologies.
Acknowledgements
Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM112693.
Author contributions
M.H.A. conceived the experiments, M.H.A. and A.J.S. supervised the study, M.E.K., Z.B.H., O.B.O., J.W.Z., and A.C. conducted the experiments, M.E.K. analyzed the results, M.E.K. prepared the manuscript, all authors discussed and reviewed the results and contributed to the final manuscript preparation.
Data availability
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
Competing interests
MHA discloses intellectual property owned by the University of Washington, US Patent No. US9295402B1. The rest of the authors have no conflict of interest79.
Footnotes
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Cairns, C., Kang, K. & Santo, L. National Hospital Ambulatory Medical Care Survey: 2018 Emergency Department Summary Tables. https://www.cdc.gov/nchs/data/nhamcs/web_tables/2018-ed-web-tables-508.pdf (2018).
- 2.Mertens DM, Jenkins ME, Warden GD. Outpatient burn management. Nurs. Clin. N. Am. 1997;32:343. [PubMed] [Google Scholar]
- 3.Baxter CR. Management of burn wounds. Dermatol. Clin. 1993;11:709–714. [PubMed] [Google Scholar]
- 4.Hettiaratchy S, Papini R. Initial management of a major burn: II—Assessment and resuscitation. BMJ. 2004;329:101–103. doi: 10.1136/bmj.329.7457.101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Papini R. Management of burn injuries of various depths. BMJ. 2004;329:158–160. doi: 10.1136/bmj.329.7458.158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Devgan, L., Bhat, S., Aylward, S. & Spence, R. J. Modalities for the assessment of burn wound depth. J. Burns Wounds5 (2006). [PMC free article] [PubMed]
- 7.Singh V, Devgan L, Bhat S, Milner SM. The pathogenesis of burn wound conversion. Ann. Plast. Surg. 2007;59:109–115. doi: 10.1097/01.sap.0000252065.90759.e6. [DOI] [PubMed] [Google Scholar]
- 8.Monstrey S, Hoeksema H, Verbelen J, Pirayesh A, Blondeel P. Assessment of burn depth and burn wound healing potential. Burns. 2008;34:761–769. doi: 10.1016/j.burns.2008.01.009. [DOI] [PubMed] [Google Scholar]
- 9.Davis SC, Mertz PM, Bilevich ED, Cazzaniga AL, Eaglstein WH. Early debridement of second-degree burn wounds enhances the rate of epithelization-an animal model to evaluate burn wound therapies. J. Burn Care Rehabil. 1996;17:558–561. doi: 10.1097/00004630-199611000-00014. [DOI] [PubMed] [Google Scholar]
- 10.Ong YS, Samuel M, Song C. Meta-analysis of early excision of burns. Burns. 2006;32:145–150. doi: 10.1016/j.burns.2005.09.005. [DOI] [PubMed] [Google Scholar]
- 11.Singer AJ, et al. Early versus delayed excision and grafting of full-thickness burns in a porcine model: A randomized study. Plast. Reconstr. Surg. 2016;137:972e–979e. doi: 10.1097/PRS.0000000000002161. [DOI] [PubMed] [Google Scholar]
- 12.Godina M, Derganc M, Brčič A. The reliability of clinical assessment of the depth of burns. Burns. 1977;4:92–96. [Google Scholar]
- 13.Watts A, Tyler M, Perry M, Roberts A, McGrouther D. Burn depth and its histological measurement. Burns. 2001;27:154–160. doi: 10.1016/s0305-4179(00)00079-6. [DOI] [PubMed] [Google Scholar]
- 14.Hoeksema H, et al. Accuracy of early burn depth assessment by laser doppler imaging on different days post burn. Burns. 2009;35:36–45. doi: 10.1016/j.burns.2008.08.011. [DOI] [PubMed] [Google Scholar]
- 15.Kamolz L-P, et al. Indocyanine green video angiographies help to identify burns requiring operation. Burns. 2003;29:785–791. doi: 10.1016/s0305-4179(03)00200-6. [DOI] [PubMed] [Google Scholar]
- 16.Sowa MG, et al. Classification of burn injuries using near-infrared spectroscopy. J. Biomed. Opt. 2006;11:1–6. doi: 10.1117/1.2362722. [DOI] [PubMed] [Google Scholar]
- 17.Cross KM, et al. Clinical utilization of near-infrared spectroscopy devices for burn depth assessment. Wound Repair Regen. 2007;15:332–340. doi: 10.1111/j.1524-475X.2007.00235.x. [DOI] [PubMed] [Google Scholar]
- 18.Goertz O, et al. Orthogonal polarization spectral imaging: A tool for assessing burn depths? Ann. Plast. Surg. 2010;64:217–221. doi: 10.1097/SAP.0b013e3181a1473b. [DOI] [PubMed] [Google Scholar]
- 19.Nguyen JQM, et al. Spatial frequency domain imaging of burn wounds in a preclinical model of graded burn severity. J. Biomed. Opt. 2013;18:1–8. doi: 10.1117/1.JBO.18.6.066010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ponticorvo A, et al. Quantitative assessment of graded burn wounds in a porcine model using spatial frequency domain imaging (SFDI) and laser speckle imaging (LSI) Biomed. Opt. Express. 2014;5:3467–3481. doi: 10.1364/BOE.5.003467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Zhang HF, Maslov K, Stoica G, Wang LV. Imaging acute thermal burns by photoacoustic microscopy. J. Biomed. Opt. 2006;11:1–5. doi: 10.1117/1.2355667. [DOI] [PubMed] [Google Scholar]
- 22.Srinivas SM, et al. Determination of burn depth by polarization-sensitive optical coherence tomography. J. Biomed. Opt. 2004;9:207–212. doi: 10.1117/1.1629680. [DOI] [PubMed] [Google Scholar]
- 23.Jaskille AD, Ramella-Roman JC, Shupp JW, Jordan MH, Jeng JC. Critical review of burn depth assessment techniques: Part II. Review of laser doppler technology. J. Burn Care Res. 2010;31:151–157. doi: 10.1097/BCR.0b013e3181c7ed60. [DOI] [PubMed] [Google Scholar]
- 24.Sowa MG, Leonardi L, Payette JR, Fish JS, Mantsch HH. Near infrared spectroscopic assessment of hemodynamic changes in the early post-burn period. Burns. 2001;27:241–249. doi: 10.1016/s0305-4179(00)00111-x. [DOI] [PubMed] [Google Scholar]
- 25.McGill D, Sørensen K, MacKay I, Taggart I, Watson S. Assessment of burn depth: A prospective, blinded comparison of laser doppler imaging and videomicroscopy. Burns. 2007;33:833–842. doi: 10.1016/j.burns.2006.10.404. [DOI] [PubMed] [Google Scholar]
- 26.Kaiser M, Yafi A, Cinat M, Choi B, Durkin AJ. Noninvasive assessment of burn wound severity using optical technology: A review of current and future modalities. Burns. 2011;37:377–386. doi: 10.1016/j.burns.2010.11.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Pape SA, Skouras CA, Byrne PO. An audit of the use of laser doppler imaging (LDI) in the assessment of burns of intermediate depth. Burns. 2001;27:233–239. doi: 10.1016/s0305-4179(00)00118-2. [DOI] [PubMed] [Google Scholar]
- 28.Jeng J, et al. Laser doppler imaging determines need for excision and grafting in advance of clinical judgment: A prospective blinded trial. Burns. 2003;29:665–670. doi: 10.1016/s0305-4179(03)00078-0. [DOI] [PubMed] [Google Scholar]
- 29.Taylor ZD, et al. THz and mm-wave sensing of corneal tissue water content: In vivo sensing and imaging results. IEEE Trans. Terahertz Sci. Technol. 2015;5:184–196. doi: 10.1109/TTHZ.2015.2392628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Sung S, et al. THz imaging system for in vivo human cornea. IEEE Trans. Terahertz Sci. Technol. 2018;8:27–37. doi: 10.1109/TTHZ.2017.2775445. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Chen A, et al. Investigation of water diffusion dynamics in corneal phantoms using terahertz time-domain spectroscopy. Biomed. Opt. Express. 2020;11:1284–1297. doi: 10.1364/BOE.382826. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Chen A, et al. Non-contact terahertz spectroscopic measurement of the intraocular pressure through corneal hydration mapping. Biomed. Opt. Express. 2021;12:3438–3449. doi: 10.1364/BOE.423741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Ji YB, et al. Terahertz reflectometry imaging for low and high grade gliomas. Sci. Rep. 2016;6:36040. doi: 10.1038/srep36040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Yamaguchi S, et al. Brain tumor imaging of rat fresh tissue using terahertz spectroscopy. Sci. Rep. 2016;6:30124. doi: 10.1038/srep30124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Zou Y, et al. Terahertz spectroscopic diagnosis of myelin deficit brain in mice and rhesus monkey with chemometric techniques. Sci. Rep. 2017;7:5176. doi: 10.1038/s41598-017-05554-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hernandez-Cardoso GG, et al. Terahertz imaging for early screening of diabetic foot syndrome: A proof of concept. Sci. Rep. 2017;7:42124. doi: 10.1038/srep42124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Fitzgerald AJ, et al. Terahertz pulsed imaging of human breast tumors. Radiology. 2006;239:533–540. doi: 10.1148/radiol.2392041315. [DOI] [PubMed] [Google Scholar]
- 38.Bowman T, El-Shenawee M, Campbell LK. Terahertz transmission vs reflection imaging and model-based characterization for excised breast carcinomas. Biomed. Opt. Express. 2016;7:3756–3783. doi: 10.1364/BOE.7.003756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Grootendorst MR, et al. Use of a handheld terahertz pulsed imaging device to differentiate benign and malignant breast tissue. Biomed. Opt. Express. 2017;8:2932–2945. doi: 10.1364/BOE.8.002932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Chavez T, Bowman T, Wu J, Bailey K, El-Shenawee M. Assessment of terahertz imaging for excised breast cancer tumors with image morphing. J. Infrared Millim. Terahertz Waves. 2018;39:1283–1302. doi: 10.1007/s10762-018-0529-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Cassar Q, et al. Terahertz refractive index-based morphological dilation for breast carcinoma delineation. Sci. Rep. 2021;11:6457. doi: 10.1038/s41598-021-85853-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Bajwa N, et al. Terahertz imaging of cutaneous edema: Correlation with magnetic resonance imaging in burn wounds. IEEE. Trans. Biomed. Eng. 2017;64:2682–2694. doi: 10.1109/TBME.2017.2658439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Tewari P, et al. Methods for registering and calibrating in vivo terahertz images of cutaneous burn wounds. Biomed. Opt. Express. 2019;10:322–337. doi: 10.1364/BOE.10.000322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Arbab, M. H., Dickey, T. C., Winebrenner, D. P., Chen, A. & Mourad, P. D. Characterization of burn injuries using terahertz time-domain spectroscopy. In Advanced Biomedical and Clinical Diagnostic Systems IX, vol. 7890, 109 – 115 (SPIE, 2011).
- 45.Pickwell E, Cole B, Fitzgerald A, Wallace V, Pepper M. Simulation of terahertz pulse propagation in biological systems. Appl. Phys. Lett. 2004;84:2190–2192. [Google Scholar]
- 46.Pickwell E, Cole BE, Fitzgerald AJ, Pepper M, Wallace VP. In vivo study of human skin using pulsed terahertz radiation. Phys. Med. Biol. 2004;49:1595. doi: 10.1088/0031-9155/49/9/001. [DOI] [PubMed] [Google Scholar]
- 47.Arbab, M. H. et al. A noninvasive terahertz assessment of 2nd and 3rd degree burn wounds. In Proceedings of Conference on Lasers and Electro-Optics, CTu3B.3 (OSA, 2012).
- 48.Osman OB, et al. Differentiation of burn wounds in an in vivo porcine model using terahertz spectroscopy. Biomed. Opt. Express. 2020;11:6528–6535. doi: 10.1364/BOE.397792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Osman, O. B. et al. In vivo assessment and monitoring of burn wounds using a handheld terahertz hyperspectral scanner. Adv. Photonics Res. 2100095 (2022). [DOI] [PMC free article] [PubMed]
- 50.Harris ZB, Khani ME, Arbab MH. Terahertz Portable Handheld Spectral Reflection (PHASR) scanner. IEEE Access. 2020;8:228024–228031. doi: 10.1109/access.2020.3045460. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Harris ZB, Katletz S, Khani ME, Virk A, Arbab MH. Design and characterization of telecentric f- scanning lenses for broadband terahertz frequency systems. AIP Adv. 2020;10:125313. doi: 10.1063/5.0030110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Harris ZB, Virk A, Khani ME, Arbab MH. Terahertz time-domain spectral imaging using telecentric beam steering and an f- scanning lens: Distortion compensation and determination of resolution limits. Opt. Express. 2020;28:26612–26622. doi: 10.1364/OE.398706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The arrive guidelines for reporting animal research. PLoS Biol. 2010;8:e1000412. doi: 10.1371/journal.pbio.1000412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Singer AJ, et al. Comparison of comparable scald and contact burns in a porcine model: A preliminary report. Wound Rep. Reg. 2020;28:789–796. doi: 10.1111/wrr.12848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Goldsmith K, Goradia E, McClain SA, Sandoval S, Singer AJ. The effect of tadalafil on reepithelialization and scarring of partial thickness porcine burns. Wound Repair Regen. 2020;28:26–32. doi: 10.1111/wrr.12770. [DOI] [PubMed] [Google Scholar]
- 56.Nanney L, Wenczak B, Lynch J. Progressive burn injury documented with vimentin immunostaining. J. Burn Care Rehabil. 1996;17:191–198. doi: 10.1097/00004630-199605000-00003. [DOI] [PubMed] [Google Scholar]
- 57.Singer AJ, et al. Validation of a vertical progression porcine burn model. J. Burn Care Res. 2011;32:638–646. doi: 10.1097/BCR.0b013e31822dc439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Andrews CJ, Kempf M, Kimble R, Cuttle L. Development of a consistent and reproducible porcine scald burn model. PLoS One. 2016;11:1–18. doi: 10.1371/journal.pone.0162888. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Chen Y, Huang S, Pickwell-MacPherson E. Frequency-wavelet domain deconvolution for terahertz reflection imaging and spectroscopy. Opt. Express. 2010;18:1177–1190. doi: 10.1364/OE.18.001177. [DOI] [PubMed] [Google Scholar]
- 60.Khani ME, Winebrenner DP, Arbab MH. Phase function effects on identification of terahertz spectral signatures using the discrete wavelet transform. IEEE Trans. Terahertz Sci. Technol. 2020;10:656–666. doi: 10.1109/tthz.2020.2997595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Ebrahimkhani, M. & Arbab, M. H. Extraction of THz absorption signatures obscured by rough surface scattering using discrete wavelet transform. In Proceedings of 43rd International Conference on Infrared, Millimeter, THz Waves (IRMMW-THz), 1–2 (2018).
- 62.Khani ME, Arbab MH. Chemical identification in the specular and off-specular rough-surface scattered terahertz spectra using wavelet shrinkage. IEEE Access. 2021;9:29746–29754. doi: 10.1109/access.2021.3059424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Neelamani R, Choi H, Baraniuk R. Forward: Fourier-wavelet regularized deconvolution for ill-conditioned systems. IEEE Trans. Signal Process. 2004;52:418–433. [Google Scholar]
- 64.Khani ME, Osman OB, Arbab MH. Diffuse terahertz spectroscopy in turbid media using a wavelet-based bimodality spectral analysis. Sci. Rep. 2021;11:1–13. doi: 10.1038/s41598-021-02068-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Khani, M. E., Harris, Z. B. & Arbab, M. H. Terahertz spectral imaging through turbid media: A wavelet approach to scattering mitigation. In Proceedings of Conference on Lasers and Electro-Optics, SM2F.5 (2020).
- 66.Osman OB, Arbab MH. Mitigating the effects of granular scattering using cepstrum analysis in terahertz time-domain spectral imaging. PLoS One. 2019;14:1–14. doi: 10.1371/journal.pone.0216952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Arbab, M. H., Chen, A., Thorsos, E. I., Winebrenner, D. P. & Zurk, L. M. Effect of surface scattering on terahertz time domain spectroscopy of chemicals. In Terahertz Technology and Applications, vol. 6893, 92 – 99 (SPIE, 2008).
- 68.Liu W, et al. Automatic recognition of breast invasive ductal carcinoma based on terahertz spectroscopy with wavelet packet transform and machine learning. Biomed. Opt. Express. 2020;11:971–981. doi: 10.1364/BOE.381623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Eadie LH, Reid CB, Fitzgerald AJ, Wallace VP. Optimizing multi-dimensional terahertz imaging analysis for colon cancer diagnosis. Expert Syst. Appl. 2013;40:2043–2050. [Google Scholar]
- 70.Hou D, et al. Terahertz spectroscopic investigation of human gastric normal and tumor tissues. Phys. Med. Biol. 2014;59:5423–5440. doi: 10.1088/0031-9155/59/18/5423. [DOI] [PubMed] [Google Scholar]
- 71.Yin X, Ng BW-H, Fischer BM, Ferguson B, Abbott D. Support vector machine applications in terahertz pulsed signals feature sets. IEEE Sens. J. 2007;7:1597–1608. [Google Scholar]
- 72.Cristianini, N. et al.An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (Cambridge University Press, 2000).
- 73.Schütze, H., Manning, C. D. & Raghavan, P. Introduction to Information Retrieval, vol. 39 (Cambridge University Press, 2008).
- 74.Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, 2009).
- 75.Guo Y, Hastie T, Tibshirani R. Regularized linear discriminant analysis and its application in microarrays. Biostatistics. 2007;8:86–100. doi: 10.1093/biostatistics/kxj035. [DOI] [PubMed] [Google Scholar]
- 76.Lu J, Plataniotis KN, Venetsanopoulos AN, Li SZ. Ensemble-based discriminant learning with boosting for face recognition. IEEE Trans. Neural Netw. Learn. Syst. 2006;17:166–178. doi: 10.1109/TNN.2005.860853. [DOI] [PubMed] [Google Scholar]
- 77.Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 1997;55:119–139. [Google Scholar]
- 78.Snoek, J., Larochelle, H. & Adams, R. P. Practical Bayesian optimization of machine learning algorithms. Adv. Neural Inf. Process. Syst.25 (2012).
- 79.Arbab MH, et al. Methods and systems for assessing a burn injury. US Pat. 2016;9:295–402. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.