Semi-automated analysis of supraclavicular thermal images increases speed of brown adipose tissue analysis without increasing variation in results

Abstract

Interest in brown adipose tissue remains high a decade after it was determined to be present outside of the neonatal period. In vivo imaging, however, has remained a challenge due to the lack of a imaging modality suitable for large healthy-volunteer studies, post-prandial investigations and vulnerable groups, such as children. Infrared thermography is increasingly accepted as a valid, non-invasive and flexible alternative but there is a wide approach to analysis between different groups. Defining the region of interest with anatomical borders rather than using a simple polygon may have advantages in terms of consistency but makes image analysis slower, limiting some applications. Our novel semi-automated method, using a custom-built graphical user interface, allows an 86% improvement in speed of image analysis (54.9 (38.3–71.4) seconds/image) without increases in variation between analysers or with repeated analysis. The improved efficiency demonstrated makes feasible larger studies, longer imaging periods or increased image acquisition frequency, providing an opportunity to study novel features of brown adipose tissue function.

Graphical abstract

Highlights

• •Brown adipose tissue is a key heat-generating tissue but is difficult to measure.
• •Thermal imaging can measure brown adipose tissue response without radiation.
• •A semi-automated approach increases image analysis efficiency.
• •Thermal video analysis and imaging over longer periods is now feasible.

1. Introduction

Brown adipose tissue (BAT) is a heat-generating tissue which oxidises glucose and fatty acids through mitochondrial respiration without the usual resultant production of adenosine triphosphate. This is due to the presence of a unique protein, uncoupling protein (UCP)-1, in the mitochondrial membrane (Cannon and Nedergaard, 2004). The last decade has seen a renewed interest in BAT following the demonstration of its presence outside of the neonatal period, hinting at a potentially novel mechanism to aid weight management by increasing calorie expenditure, beyond the usual increase in voluntary physical activity (Celi, 2009; Virtanen et al., 2009; Cypess et al., 2009; van Marken Lichtenbelt et al., 2009).

Measuring BAT activity remains challenging with the gold standard method of PET-CT, using either glucose (18F-fluorodeoxyglucose) or fatty acid (e.g. 18F-fluro-6-thiaheptadecanoic acid or 11C-acetate) isotopes (Labbé et al., 2016; Ouellet et al., 2012). PET-CT provides a static measurement of what is a dynamic tissue and, also, cannot be used to look at the effects of meals due to the rapid uptake of the isotope into muscle in the postprandial period (Hankir et al., 2017). In addition, the high dose of ionising radiation makes PET-CT unsuitable for use in large studies of healthy volunteers and in groups such as children.

Due to the heat-generating property of BAT, infrared thermography is a natural alternative to PET-CT for measuring BAT activity, especially given the relatively superficial location of the supraclavicular depot which is the largest BAT depot in humans (Leitner et al., 2017). Infrared thermography measures infrared radiation emitted by an object and converts infrared data to temperature which is then presented as a false-colour image using a user-defined colour-map. Images can then be analysed to determine temperature properties of all, or part of, the image. By measuring the heat signature of the supraclavicular region before and during the introduction of a suitable stimulus, such as cold, the response of the supraclavicular BAT to the stimulus can be ascertained (Law et al., 2019; Ang et al., 2017; Lee et al., 2011; Haq et al., 2017). Infrared thermography has been validated as an alternative to, and shows a strong correlation with, PET-CT (Law et al., 2018c; Lee et al., 2016; Boon et al., 2014; Jang et al., 2014; Haq et al., 2017).

A variety of image analysis methods have been used by groups with some opting to define the contour of the region of interest (ROI) using simple regular shapes and others utilising anatomical borders (Robinson et al., 2014, 2016). The latter has advantages in terms of reproducibility and accuracy but was time-consuming and exerted a practical limit on the number of images that could be analysed.

We, therefore, custom-built an image analysis application (using the same ROI, defined by anatomical borders, as the original method and the same underlying process for determining the output) to streamline the analysis process (Law et a., 2018c). We present here a comparison of the efficiency and variability of the original, manual method and the semi-automated method.

2. Methods

A pragmatic retrospective evaluation was undertaken. All thermal images were acquired on a FLIR B425 camera (FLIR Systems AB, Danderyd, Sweden) using standard settings (emissivity 0.98) as previously described (Law et al., 2018c, 2019; Robinson et al., 2014, 2016). Members of the research group were trained in two methods of image analysis, a manual method using FLIR's proprietary software, ResearchIR version 4 (FLIR Systems) (Robinson et al., 2014, 2016), and a semi-automated method using a custom-designed graphical user interface (GUI) within the MATLAB (MathWorks, Natick, MA, USA) programming environment (Law et al., 2018c, 2019). Training on both methods was delivered consistently by a single individual to all researchers. Analysers practised analysing training images until they were confident in both techniques and then analysed further sets of images to demonstrate competence and consistency. Each analyser analysed two sets of 31 images using both methods and then reanalysed a subset of eight images from each set a further two times. Ethical permission was granted for the acquisition of the images for each of the original studies (Nottingham-2 NHS Research Ethics Committee reference 13/EM/0102; University of Nottingham Faculty of Medicine and Health Sciences Research Ethics Committee reference 26/299/05/2017) which included consent for storage and future use. No additional ethical approval was required.

2.1. Images

Each analyser was asked to analyse two sets of images (Set A and Set B) on three occasions (Visit 1, Visit 2 and Visit 3). On the first visit, an extended set of 31 images were analysed in each set (62 images in total). On the subsequent two visits, a subset of eight images was analysed from each set (16 images in total). Set A comprised a set of images where the participant had moved a moderate amount between frames; set B comprised a set of images where the subject had moved very little between frames. On the first visit, analysers were randomly assigned to analyse images using the manual (M) or the semi-automated (SA) method first and then alternated the methods used first at the second and third visit. The order of images was kept constant throughout.

2.2. Manual (M) Method

Thermal images of the supraclavicular region were acquired and saved in the proprietary FLIR JPEG format, as previously described (Robinson et al., 2014). This format saves the infrared data in the file metadata along with the values of variables required to calculate the temperature of the object in the image (Tattersall, 2015). The image was opened in ResearchIR which displayed the temperature data in an alterable and processable format.

The inbuilt polygon tool was used to define a contour around a region of interest (ROI). Two ROIs were drawn: one around the left supraclavicular region and one around the right (Fig. 1A). The region was defined as (a) a straight line from the contour of the next adjacent to the acromoclavicular process to the sternal notch, (b) a straight line from the sternal notch to the intersection of the sternocleidomastoid muscle with the contour of the neck and (c) the contour of the neck from the acromoclavicular point to the intersection of the sternocleidomastoid muscle. Once the ROIs had been defined, the temperature data of the pixels within each was exported as a comma-separated value (.csv) file which was given a file name indicating the image it related to and whether it was the left or right side. Finally, a FLIR session file (.irs) was saved. The session file saved the workspace, including the polygon (and any other measurement tools), and a link to the image being viewed to allow the analysis to be reproduced later. If errors were later found to have been made during the ROI-saving process, the session file allowed the workspace to be reopened and the ROI saved using the analyser's original polygon.

Fig. 1.

(A) Example of analysis using FLIR ResearcherIR software. The borders of the contour (cyan line) were a straight line approximating the sternocleidomastoid muscle, from the point where it intersects the contour of the neck to the suprasternal notch; a straight line approximating the clavicle, from the suprasternal notch to the acromioclavicular process; and the contour of the neck between the preceding two lines, manually approximated by the research by a series of short straight lines. (B) Example of analysis from MATLAB app showing apices (white stars) identified by analyser.

The process was then repeated for the next image. The polygon defining the ROI was maintained by the software when the next image was selected. The analyser chose to adapt the previous polygon (by translation along the x- and y-axes and by the movement of points) to fit the ROIs on the new image or delete it and define a new one. Where the image was similar, small changes to the original polygon were quicker and acceptable but tended to become increasingly less appropriate with each subsequent image and worked poorly if there was more movement between images.

Once all the images in the set had been analysed, the analyser opened each comma-separated value file in Microsoft Excel. Each cell of the spreadsheet corresponded to a pixel in the original image. The analyser was then required to process the data using a short succession of keystrokes to run a custom-built macro. The macro ranked the cells from warmest to coolest and then identified the output required (Symonds et al., 2012), namely 95th percentile (Law et al., 2018c). The analysers than saved the file output as a new file.

An automated checking process was developed using R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria) to confirm that all the expected filenames were present and that the file labelled as containing the data from the left ROI was different to the one labelled as containing data from the right ROI (i.e. that the same ROI had not been exported twice) (Law, 2021). Other errors, such as cross-labelling images would not have been detected and cannot be excluded.

2.3. Semi-automated (SA) Method

As previously published (Law et al., 2018a, 2018c), thermal images, saved in FLIR's proprietary JPEG format, were converted into an open, portable network graphic (PNG) format with the raw radiometric data stored as a greyscale image and the variables required for conversion to thermal data stored in the metadata.

Using the MATLAB GUI (Fig. 1B), the analyser identified the five apices of the region of interest (i.e. the sternal notch, the acromioclavicular processes, and the intersections of the sternocleidomastoid muscles with the contour of the neck) on each image. The “threshold” temperature was set to a default of 30°C to distinguish the person in the image (temperature above the threshold) from the background (temperature below the threshold) and could be adjusted by the researcher if required.

The points from the previous image remained in position when the next image was displayed and could be adjusted by either dragging or clicking with the mouse.

Once the apices had been identified for all images in the set, the ROI was automatically defined and processed, and the output saved as a single comma-separated file with each row corresponding to an image and outputs in columns. In our case, an output of the 95th percentile of the ROI was used (Law et al., 2018c) but the code can be easily adapted to produce any output(s) required.

2.4. Statistics

As above, the BAT “hotspot” was defined as the pixels with temperature values in the upper decile within each ROI and the output value (T_SCV) was the median value of the hotspot, equivalent to the 95th percentile of the region (Law et al., 2018c).

On the first visit, analysers noted how long it had taken them to analyse each complete set of 31 images for each method to determine the mean length of time to analyse each image using the different methods and whether movement of the imaged participant affected the analysis rate. To determine the effect of Method (M or SA) and Set (A: moderate movement or B: little movement) on analysis speed (seconds per image), a 2-way repeated-measures ANOVA was undertaken.

The intra-analyser variability for each researcher and the inter-analyser variability for each image was determined for both methods. Use of the co-efficient of variation was not appropriate as temperature (unless measured in Kelvin) is not measured using a ratio scale. Since, in this context, the mean temperatures are very similar, the population standard deviation, $\sigma =\sqrt{\frac{\sum {\left(X-\mu \right)}^2}{n}}$, of the repeated measures was used as the measure of consistency.

The intra-analyser variability was calculated as the standard deviation of T_SCV for the three repeated measures of the eight images in Set A and the eight images in Set B. Two 3-way repeated-measures ANOVAs were undertaken, one for the left ROI and one for the right, to determine the effect of Method, Set and Analyser. To check the robustness of the results, 2-way repeated measures ANOVA were then undertaken for each combination of Set A and Set B and of the left ROI and right ROI to review the effect of Method and Analyser.

Similarly, the inter-analyser variability, defined as the population standard deviation of T_SCV from the third visit of the seven researchers, was calculated for each method for the sixteen images (8 in Set A and 8 in Set B). Two 3-way repeated-measures ANOVAs were undertaken, one for each ROI, to determine the effect of Method, Set and Image. The robustness of the results was checked via the same four 2-way repeated measure ANOVAs as above, to review the effect of Method and Image.

For each analysis, a full factorial model was initially fitted with the equivalent main effect model used subsequently if the interaction term(s) was/were not statistically significant (p > 0.05). Tukey's post-hoc comparison test was used to determine which levels were statistically significantly different and the direction of the difference.

Statistics were performed using R: A language and environment for statistical computing and saved to a publicly-accessible repository (Law, 2021).

3. Results

Seven analysers – all post-graduate research students affiliated with the Division of Child Health, Obstetrics and Gynaecology of the University of Nottingham Medical School – completed all three visits.

3.1. Speed and errors

The interaction between Method and Set in the 2-way repeated measures ANOVA full factorial model was not significant (p = 0.28). The subsequent main effects model showed Method was highly significant (p < 0.0001) and Set tended towards significance (p = 0.08) (Fig. 2). Post-hoc analysis showed SA was significantly faster than M (SA: 8.6 s/image; M: 63.5 s/image; difference 54.9 (38.3–71.4) sec/image; p < 0.0001) and that the set of images with movement (Set B) trended towards being significantly faster than the set with more movement between images (Set A: 45.5 s/image; Set B: 30.8 s/image; difference 15 s/image (−2 to 32); p = 0.08).

Fig. 2.

Mean time taken per image for each researcher to analyse 31 images using the manual (filled) and semi-automated (open) methods for two sets of images. Set A (circles) had a moderate amount of movement of the participant between images; Set B (triangles) had very little movement. Lines indicate mean ± SD.

In addition, errors were commonly made by analysers when using M. Errors included missing files (in four sets), the same region of interest saved as both the left and right file (in eight sets), and files being duplicated (in two sets). In total, 12 out of 42 (29%) sets of images analysed using M had errors identified. One error was identified for the sets analysed using SA, where the threshold temperature had not been set correctly.

3.2. Intra-analyser variation

The initial 3-way repeated measures ANOVA full factorial models showed significant interactions between all combinations of Method, Analyser and Set (p < 0.0001) for both the left and right ROIs. Method (p < 0.0001), Set (p = 0.007) and Analyser (p < 0.0001) were all independently highly significant for the left ROI and Method (p < 0.0001) and Analyser (p < 0.0001), but not Set (p = 0.733), were highly significant for the right ROI. All four 2-way repeated measures ANOVA full factorial models (Set A: left ROI and right ROI; Set B: left ROI and right ROI) confirmed a significant interaction between Method and Analyser (p < 0.0001), i.e. that the effect of Method was different for different analysers, and that Method and Analyser were independently highly significant for both sets and both ROIs (Set A: left ROI p = 0.040, right ROI p = 0.0001; Set B: left & right ROI p < 0.0001).

Post-hoc analysis, as well as simple observation (Fig. 3), demonstrated one analyser was a significant outlier and had performed poorly with M (interactions between the outlier and other analysers: p < 0.0001; all other interactions: p > 0.1). To test the robustness of the results, the analysis was rerun with that data excluded. A further two analysers had high variability in one and two images using M and a third analyser had high variability in one image using SA (Table 1) which were not excluded.

Fig. 3.

Boxplot of variation (population standard deviation) in repeated measures of the same image by the each researcher on three occasions (intra-analyser variation) comparing the manual (M) and semi-automated (SA) methods for two sets of images (Set A: minimal movement between images; Set B: moderate movement between images). (A) right region of interest and (B) left region of interest. Column shading represents individual researchers.

Table 1.

Images where intra-analyser variation (population standard deviation) was greater than 0.1.

Analyser	Image	Method	Set	Variation (right ROI)	Variation (left ROI)
1	4	M	A	0.141	0.189
1	7	M	A	0.043	0.141
2	7	M	A	<0.001	0.141
5	5	SA	A	0.073	0.108

M: manual method; SA: semi-automated method; ROI: region of interest.

Repeat analysis of the 3-way full factorial model, with the outlier excluded, showed no significant effect of Method for either ROI (right: p = 0.377; left: p = 0.596). Set was significant for the right (Set A: σ = 0.019; Set B: σ = 0.007; difference: 0.012 (0.007–0.017); p < 0.0001), but not left (p = 0.125), ROI. Analyser trended towards significance on the right ROI (p = 0.055) and was significant on the left (p = 0.021). Only Set & Analyser showed a significant interaction on the right (p = 0.021) but all interactions were significant for the left except Method & Set (Supplementary Material). Two-way ANOVAs confirmed that the interaction between Method & Analyser was not significant for the right ROI (Set A: p = 0.779; Set B: p = 0.939) but was on the left for Set A (p = 0.005) and trended toward significance for Set B (p = 0.088). Two-way ANOVAs were, therefore, run both with and without the interaction term and the results consistently showed there was no effect of Method on intra-analyser variation but there was an effect of Analyser, except on the left for Set A (Supplementary Material).

3.3. Inter-analyser variation

The initial 3-way repeated measures ANOVA full factorial models for both the left and right ROIs showed a significant effect of Method (p < 0.0001) and Set (p < 0.0001) but not Image (p > 0.2) and a significant interaction only between Method & Set (p < 0.0001) (Fig. 4A & C). After removal of the outlier, as above, there remained a significant effect of Set on both the right (set A: σ = 0.035; set B: σ = 0.012; difference 0.023 (0.009–0.037); p = 0.003) and left (set A: σ = 0.048; set B: σ = 0.016; difference 0.032 (0.012–0.051); p = 0.003) but not of Method (p > 0.4) (Fig. 4B & D) and there were no significant interactions (Supplementary Material). Therefore, the main effects model was calculated with consistent results. Similarly, no effect of Method, Image or the interaction term was seen when 2-way full factorial models were calculated for the left and right ROIs for each set separately nor for Method or Image in the 2-way main effects model.

Fig. 4.

Boxplot of variation (population standard deviation) in measures of the same image by multiple analysers (inter-analyser variation) comparing the manual method (filled bars) and semi-automated method (open bars) for two sets of images (Set A: minimal movement between images; Set B: moderate movement between images). (A) right region of interest including all analysers and (B) exlcuding outlier; (C) left region of interest from all analysers and (D) excluding outlier.

After removal of the outlier analyser, one image had high variability (Table 2).

Table 2.

Images where inter-analyser variation (population standard deviation) was greater than 0.1.

Image	Method	Set	Variation (right ROI)	Variation (left ROI)
7	M	A	0.037	0.141

M: manual method; SA: semi-automated method; ROI: region of interest.

4. Discussion

SA showed an 86% increase in speed compared to M with no loss of consistency either between analysers or on repeated analysis by the same researcher. Images with more movement between them increased inter and intra-analyser variability and trended towards slower analysis. The relevance of the increase in efficiency is not limited to making it easier for analysers, but rather makes practical the analysis of larger sets of images. This makes longer imaging sessions, studies with more participants and, perhaps most excitingly, studies with increased image acquisition frequency a realistic prospect. The latter has the potential to include analysis of thermal videos. Both MRI and CT acquire high-resolution images by averaging the raw pixel data of rapidly repeated images to improve the signal-to-noise ratio. In doing so, the noise can be reduced enhancing the detection of subtler signals and there is the potential to achieve similar benefits in the field of thermal imaging utilising the increased efficiency in image analysis presented here. Smaller temperature changes and cyclical patterns within the overall trend may then be able to be detected, presenting opportunities to detect hitherto undetectable responses. Further work will develop the automated analysis to create a fully automated system, capable of analysing hundreds of frames per minute. As well as the efficiency savings, the fully automated system will precisely track the anatomical position of the apices image-by-image, where the human user must decide whether the point has moved from the previous image and whether an adjustment is required.

Despite training prior to analysing these sets of images, one of the seven analysers had a distinctively higher variability when using M and a further two analysers had higher variability (σ > 0.1) in one or more images. In contrast, only one analyser had a high variability for a single image analysed using SA. Even after removal of the outlier analyser, one image analysed with M had a high inter-analyser variation, but none analysed using SA did. There is, therefore, an indication of fewer outliers being generated when using SA. Although the difference in favour of SA was lost when the outlying analyser was excluded, there was no evidence of any increase in either in inter or intra-analyser variability, accepting that the possibility of a false-negative conclusion must be considered due to the pragmatic nature of the analysis.

There were some significant differences between the findings for the left and right ROI, particularly in the interaction terms. The pragmatic retrospective design and size mean it is not possible to draw conclusions about the causes for the differences between the sides and, due to the small sample size, there should be caution determining that an interaction is not significant due to a non-significant p-value, hence both full and main effects models were compared for robustness. However, the intra-analyser interaction between Set & Analyser was significant for both sides, suggesting how well each analyser performed was variable depending on whether they were analysing images where the participant moved a little or a moderate amount. Indeed, Set was frequently found to be significant with both more inter/intra-analyser variability and slower analysis when the participant was moving between images. This emphasises the importance of optimising image quality and consistency during the acquisition process as results can be degraded in an unpredictable manner and may be one of the factors differentiating groups who are able to consistently utilise thermal imaging successfully. Furthermore, Analyser was significant in all intra-analyser 2-way ANOVAs, except the left ROI for set A, indicating that Analysers tended to be consistent in the amount of variability between repeated analyses. Further work could examine whether analysers with higher variability could improve their results with further training but, again, this highlights a further source of variability which may affect results. A further limitation is that analysers self-reported the time taken to undertake the image analysis and could have over or under-reported their analysis time. However, approximate times calculated by the file created and modified details were consistent with the times reported and would not have altered the overall findings. Although analysers were not blinded and were able to view the output of their analysis, they had no indication of where their outcome would lie within the range of other analysers. In addition, the nature of the analysis makes it impractical to try and manipulate the output by adjusting the boundary of the ROI.

Thermal imaging is increasingly used in a wide range of medical and physiological applications, both in humans (Law et al., 2018b) and animals, for instance to monitor stress and welfare remotely in a non-invasive manner (McManus et al., 2016). There remains the potential to adapt the method presented here to enable it to be applied in alternative-use scenarios.

Further work will consider the options to reduce variability further. On some images, body habitus can make it difficult to identify the anatomical landmarks retrospectively. We have, therefore, developed thermally opaque skin markers which can be placed on the participant prior to the start of image acquisition. We expect this to increase the accuracy and consistency of the choice of apical placement by analysers at a cost of slowing down analysis if analysers make multiple small adjustments to markers because all little movements between images are more evident. We are, therefore, developing a fully automated method capable of detecting marker placement.

In conclusion, our novel semi-automated method drastically improves the speed of image analysis and reduces the risk of errors without increasing variability and should, therefore, be used in preference to manually defined complex polygons. Use of this method allows larger studies, longer imaging periods and increased image acquisition frequency which will allow novel study designs to further understand BAT function.

Funding

No funding was received for this work.

Author contributions

J.M.L. conceived and designed the study, undertook the acquisition, analysis and interpretation of the data, and drafted, revised and approved this work. D.E.M. designed the analysis methods used and revised and approved this work. L.R. designed the study, contributed to the analysis methods used and revised and approved this work. M.E.S. co-designed the study, interpreted the data, revised and approved this work. H.B. conceived and co-designed the study, interpreted the data, revised and approved this work.

Data availability

The datasets generated during and analysed during the current study are available from the corresponding author upon reasonable request. Original images are not available to preserve the confidentiality of participants.

Resource availability

The software used for infrared thermographic analysis in the current study is freely available from the corresponding author on a collaborative basis upon reasonable request.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: