Comparative accuracy testing of non-contact infrared thermometers and temporal artery thermometers in an adult hospital setting

Abstract

Background

NCIT are non-invasive devices for fever screening in children. However, evidence of their accuracy for fever screening in adults is lacking. This study aimed to compare the accuracy of non-contact infrared thermometers (NCIT) with temporal artery thermometers (TAT) in an adult hospital.

Methods

A prospective observational study was conducted on a convenience sample of non-infectious inpatients in 2 Australian hospitals. NCIT and TAT devices were used to collect body temperature recordings. Participant characteristics included age, gender, skin color, highest temperature, and antipyretic medications recorded in last 24-hour.

Results

In 265 patients, a mean difference of ± 0.26°C was recorded between the NCIT (36.64°C) and the reference TAT (36.90°C) temperature devices. Bland-Altman analysis showed that NCIT and TAT temperatures were closely aligned at temperatures <37.5°C, but not at temperatures >37.5°C. NCIT had low sensitivity (16.13%) at temperatures ≥37.5°C. An AUROC score of 0.67 (SD 0.05) demonstrated poor accuracy of the NCIT device at temperatures ≥37.5°C.

Conclusion

This is the first study to compare accuracy of NCIT thermometers to TAT in adult patients. Although mass fever screening is currently underway using NCIT, these results indicate that the NCIT may not be the most accurate device for fever mass screening during a pandemic.

INTRODUCTION

Body temperature is a vital sign that is regularly measured to assess the status of a patients’ health, facilitate diagnosis, and target treatments in a hospital setting. Normal body temperatures range from 36.5°C to 37.5°C.¹ Upward deviation from this range reflects a fever and may require treatment. Health care professionals must use the most accurate and precise measurement devices available to reproduce stable results reflecting core body temperature. The gold standard method of determining core body temperature measures blood temperature.^{2 , 3} Other invasive methods measure core body temperature at various sites including the oesophagus, the nasopharynx, or the urinary bladder.^{4 , 5} Although these methods provide accurate representation of core body temperature, their invasive nature limits their application in a wide range of patients during outbreak or pandemic situations such as COVID-19. Thus, new technological tools that avoid contact and prevent the transmission and spread of viruses are emerging.

The World Health Organisation instigated public health measures including body temperature screening for rapid identification of potential Coronavirus cases and infection prevention.^{6 , 7} Not surprisingly, a quick, non-contact, reliable, cost-efficient, and easy to use approach for temperature assessment is a pressing need for screening individuals in the current COVID-19 era. One potential method is an indirect estimate of core temperature measurement using Non-Contact Infrared Thermometers (NCITs).

The NCIT is a non-contact, rapid, and portable body temperature measuring tool. Measurements are taken from the frontal bone or the temporal artery. The NCIT does not require sterilisation between individuals, therefore making it a strong contender for mass-screening in pandemic situations.

The NCITs have been used in a wide-range of settings where the temperature of an object needs to be measured without touching it. They infer temperature by capturing the thermal radiation emitted from a focused spot, concentrating it onto a sensing element, and converting it to an electronic signal. This signal is then digitally processed, and the temperature value is presented to the user. The conversion between thermal radiations to an electrical signal was originally described by Macedonio Melloni in the early 1800s through the discovery of the thermopile. Since Melloni's original discovery, vast improvements in fabrication and measurement processes have enabled modern devices to provide highly accurate and precise non-contact temperature measurement capabilities in a handheld unit. However, similar to many other thermometers, these devices are highly reliant on correct operator use in order to obtain accurate results. A key consideration is the distance-to-spot ratio, as the size of the area measured is directly related to the distance to the operator. Therefore, the medical use of the NCITs always recommends a specific distance between the device and the point being measured to ensure accurate results. If this distance is not fulfilled, it is impossible to guarantee that the measurements reported from the device are actual readings from the required location.

Insufficient evidence exists that compares the measurement accuracy and precision of the NCITs with gold standard core temperature measurement devices in adults. Most studies conducted to date, determine the accuracy and reliability of the NCITs in children, and these studies have reported conflicting results. In children, a 0.2-0.4°C mean difference in body temperatures at or above 37.5°C was reported between the NCIT temperature when measured at forehead as compared to the core temperature measurements conducted using Temporal Artery Thermometers (TATs).^{8 , 9} Also in paediatric populations, studies have compared the NCITs with alternate devices including axillary digital thermometers,8, 9, 10 tympanic artery thermometers,^{8 , 11 , 12} and rectal thermometers.¹² Thus, in the current pandemic, this degree of error could result in inappropriate referrals for treatment, undetected infection transmissions across health care organizations and associated impacts on patient safety and clinical decision-making.

Studies comparing the correlation and differences in body temperature measured using the NCITs and other temperature measurement methods in adults are scarce. Thus, it is necessary to collect scientific evidence using robust research methods on the accuracy of the NCITs when used in adult populations. Therefore, this study compared the accuracy of the NCIT with the TAT in hospitalized adults. Characteristics of the patients’ age, gender, skin color, antipyretic medication use, and highest temperature in the last 24 hours are assessed to determine their influence on body temperature measurement.

METHOD

Study design

A prospective observational study design was used to determine the accuracy of temperature measurement of the NCIT in relation to the TAT.

Setting

This study was administered at 2 metropolitan hospitals in Melbourne, Australia. The first, a tertiary hospital that has approximately 500 beds and provides comprehensive health care services in the state of Victoria, Australia. It has the busiest emergency and trauma centre in Australia, one of the largest intensive care units in Australia, and provides 14 state-wide services. The second hospital has approximately 140 beds and provides state-wide rehabilitation services including brain injury rehabilitation, neurological rehabilitation, spinal rehabilitation, aged care, and care for amputee patients.

Sampling and eligibility criteria

The study used a convenience-sampling method for recruitment of participants. All hospitals’ inpatients were invited to participate in this study. A Registered Nurse explained the study aims and requirements of participation prior to obtaining verbal consent and data collection. Patients aged 16 years and older who were willing to have their temperature taken were eligible to participate. Patients in COVID-19 infection screening clinics, the emergency department, hospital in the home, radiology, outpatients, and wards with diagnosed COVID-19 cases were excluded. Patients in isolation for any other reason were also excluded.

Sample size estimation

Sample size was calculated using the Australian Bureau of Statistics sample size estimator.¹⁴ Using Tay et al.’s¹⁵ study as a reference (N = 430 participants, 7.9% fever reports), a sample size of 265 participants, were required at a prevalence rate of 5% febrile cases in metropolitan hospitals’ patients, and a standard error of 0.01.

Instruments

Two temperature measuring devices were used.

• 1.Non-contact infrared thermometer (NCIT)

The NCIT (Cocoon-NC9900) is a non-contact battery operated temperature measurement device that records body temperature or ambient temperature. It has a built-in infrared laser pointer, and records temperature in 2 different units (Celsius and Fahrenheit). NCIT is a factory-calibrated instrument that automatically calibrates in first 15 seconds when the device is switched on for use.

• 2.Temporal artery thermometer (TAT)

The TAT (Exergen TAT 5000) also uses infrared technology to record temperature. Temperature is measured by gently moving the TAT across the forehead, and includes a momentary touch of the probe to the neck area behind the ear lobe, to account for any cooling of the forehead as a result of diaphoresis. The temperature of the skin surface (over the temporal artery) is measured used Arterial Heat Balance Technology (AHB). The TAT takes rapid sequential readings at up to 1000 measurements per second, and reports the highest temperature detected (peak) during the measurement course. The calibration of the TAT is established by automation process through which the TAT instrument automatically self-calibrates each time the instrument is turned on. The biomedical engineering team at the tertiary hospital conducts performance testing annually using the manufacturer instructions to assess calibration of the instrument.

Demographics variables

The demographic variables recorded were age, gender and skin color. These variables were recorded on a case report form for each individual. The temperature using both devices was also recorded for each individual and information from the patient's electronic medical record: highest temperature in last 24 hours, and antipyretic medications taken in last 24 hours.

Procedures

Prior to data collection, the Nurse Manager or Resource Nurse identified any patients or areas who were infectious to exclude them from the study. Following patient consent, body temperatures were taken using the standard TAT and the NCIT on the forehead. The registered nurse taking the temperature followed the manufacturer's instructions for the use of TAT and NCIT. She reviewed the manufacturer's instructions and was observed to follow the correct technique prior to commencing the data collection. Personal protection equipment of masks and gloves were worn by the data collector. The temperature evaluation was performed during the hours of 8:00 AM-6:00 PM for a period of 4 days.

Data analysis

The STATA statistical software package (Version 16) was used for statistical analysis.¹⁶ For all tests, the level of significance was set at 0.05. Frequency distributions and mean (standard deviations) were calculated for characteristics of age, gender, skin color, highest temperature, and the antipyretic medication used in the last 24 hours, and body temperature recordings using the 2 devices. An independent sample t-test was used to calculate the mean difference between the body temperature recordings of the TAT and the NCIT. Box plots were constructed to determine the differences in NCIT body temperature amongst TAT fever and non-fever groups. Bland Altman analysis was plotted, with mean difference of the TAT and the NCIT on the y-axis and the TAT temperatures on the x-axis. A bivariate analysis using the χ² test was performed to determine the association between the patients’ characteristics and body temperature measurements. The Area Under the Receiver Operating Characteristics (AUROC) was calculated. For this study AUROC estimated the accuracy of NCIT ability to separate febrile and non-febrile patients. An AUROC value of 1 indicates the tool is strong; whereas an area of 0.5 or less indicates that the test fails to differentiate. Sensitivity and specificity score calculations were made to test the capability of NCIT to correctly identify patients with fever and without fever, respectively.

Ethical considerations

Strict confidentiality of information relating to patients was maintained to protect patient identity, and to ensure no disclosure of private information. The data collected for this study was stored on a password protected RedCap database in a secure server with access available only the investigator team. The study was approved as an evaluation project as it did not meet the trigger for the ethical review as stated in the National Health and Medical Research Council Ethical Considerations in Quality Assurance and Evaluation (2014).¹³ The manuscript was reviewed by the Executive Director for Nursing Services and approved for publication.

RESULTS

Characteristics of participants

Two hundred and sixty-five patients, aged 16 years and older were enrolled in this study. Table 1 shows the characteristics of participants. The mean age of participants was 58.77 years (SD 5.04). Almost 51% participants in this survey were 60 years and older and the rest 49% participants were equally distributed amongst age groups of <30 years, 30-40 years, 40-50 years and 60-70 years respectively. There were 18.86% participants of 70-78 years, 15.84% of 80-90 years and 3.39% of 90+ years. Over 59% of the sampled population were males. More than 91% of the participants had light skin color. The mean highest temperature measured using the reference TAT in the last 24 hours was 36.98°C (SD 0.12). Almost 66% of participants had taken an antipyretic medication in last 24 hours, the majority had taken a mean dose of 1gm of paracetamol.

Table 1.

Characteristics of participants (N = 265)

Characteristics of participants	n	%
Age (years)
• <30	32	12.07
• 30-40	29	10.94
• 40-50	31	11.69
• 50-60	38	14.33
• 60-70	34	12.83
• 70-80	50	18.86
• 80-90	42	15.84
• 90+	9	3.39

	Mean	SD
Age (years)	58.77	5.04
Highest temperature measured in last 24 hours (°C)	36.98	0.12
Anti-pyretic medication dosage
• Paracetamol (grams)	1.01	0.20
• NSAIDs (grams)	0.4	0.00

Gender	n	%
• Male	155	58.49
• Female	110	41.51
Skin color
• Light	242	91.32
• Medium and Dark	23	8.68
Anti-pyretic medication last 24 hrs
• Yes	174	65.66
• No	91	34.34
Paracetamol in last 24 hrs
• Yes	174	65.66
• No	91	34.34
NSAIDS in last 24 hrs
• Yes	1	0.38
• No	264	99.62

NSAIDs, non-steroidal anti-inflammatory drugs; SD, standard deviation.

Mean temperature difference between the NCIT and the TAT

Table 2 shows the comparison between the NCIT and the TAT mean temperatures and mean differences. The mean temperature measured using the TAT and the NCIT were 36.98°C and 36.64°C, respectively. Temperatures recorded using the NCIT (0.26, P< .001) had a significant mean difference as compared to the reference temperature using the TAT.

Table 2.

Mean temperature and mean temperature differences found using the TAT, the NCIT

TAT (N = 265)		NCIT (N = 265)		TAT versus NCIT (N = 265)
Mean°C	SD	Mean°C	SD	MD	SE
36.90	0.10	36.64	0.08	0.26*	0.33

SD, standard deviation; SE, standard error.

^⁎P < .001, MD – mean difference.

Figure 1 shows a significant difference was found between NCIT and TAT temperatures for fever and non-fever groups. The difference in temperature was greater in the fever group (0.18, P < .001) as compared to the non-fever group (0.81, P < .001).

Fig 1.

The distribution of differences of NCIT and TAT between non-fever group (<37.5°C) and fever group (≥37.5°C).

Bland-Altman analysis

A Bland-Altman analysis compared the NCIT with the reference TAT device (Fig 2 ). The margin of error was narrow. The average mean difference (bias) for comparison between the NCIT and the TAT ranged from 0-0.3°C. At body temperatures below 37.5°C, the NCIT measurements were closely clustered to that of the TAT body temperatures. The difference between the NCIT and the TAT was more dispersed at temperatures above 37.5°C (0.50, 95% Confidence Interval 0.5-1.0). However, this result should be interpreted with caution due to low numbers of febrile cases (TAT febrile group n = 31; NCIT febrile group n = 7) found in the studied cohort.

Fig 2.

Bland-Altman plot of mean temperature differences between NCIT and TAT devices (N = 265).

Sensitivity, specificity and area under the receiver operator characteristic (AUROC) curve

Table 3 presents the sensitivity, specificity and AUROC for NCIT (N = 265). The NCIT sensitivity and specificity scores at temperatures ≥37.5°C were estimated as 16.13% and 99.15%, respectively. The sensitivity scores were observed to increase at temperatures of ≥37°C (25.81%), ≥36.5°C (84.41%) and ≥36°C (99.62%). Whilst the specificity scores had decreasing temperatures ≥37°C (94.44%) and ≥36.5°C (51.22%), respectively. The specificity score was not calculated for temperature ≥36°C, due to having only 1 participant at temperature <36°C (reference temperature). An AUROC score of 0.67 (SD 0.05) was found at temperature ≥37.5°C suggesting a poor accuracy of the NCIT device at temperatures ≥37.5°C.

Table 3.

The sensitivity, specificity and AUROC for NCIT (N=265)

Temperature (°C)	Frequency	Sensitivity (%)	Specificity (%)	AUROC (SE)
≥37.5	7	16.13	99.15	0.67 (0.05)
<37.5	258
≥37.0	21	25.81	94.44	0.72 (0.12)
<37.0	244
≥36.5	204	82.14	51.22	0.75 (0.14)
<36.5	61
≥36.0	264	99.62	0.00	0.87 (0.30)
<36.0	1

Comparison between participant characteristics and temperature measurements

Table 4 compares the temperature mean differences (the NCIT vs the TAT) in relation to participants’ characteristics (N = 265). Female participants had a higher mean difference in temperature (0.32°C) compared to male participants (0.21°C). Participants with a light skin color had a higher mean difference in temperature (0.27°C) compared to participants with medium and dark skin color (0.12°C).

Table 4.

Participant characteristics and differences in mean temperatures between NCIT and TAT (N = 265)

Characteristics of participants	n	Mean difference (SD)	P
Age
• <50 years	92	0.22 (0.14)	0.31
• ≥50 years	173	0.27 (0.13)
Gender
• Male	155	0.21 (0.36)	<0.05
• Female	110	0.32 (0.47)
Skin color
• Light	242	0.27 (0.41)	<0.05
• Medium + Dark	23	0.12 (0.07)
Anti-pyretic medication used in last 24 hrs
• Yes	174	0.26 (0.17)	0.42
• No	91	0.26 (0.15)
Paracetamol in last 24 hrs
• Yes	174	0.26 (0.17)	0.42
• No	91	0.26 (0.15)
NSAIDS in last 24 hrs
• Yes	1	0.26 (0.10)	-
• No	264	-0.20 (-)

Note: - no value obtained in the analysis.

DISCUSSION

NCITs are increasingly used for mass temperature screening due to its non-contact, rapid processing, and hygiene maintenance attributes. Yet there is little evidence analyzing the accuracy of the NCIT for use in adults. To address this gap, this study compared the accuracy of the NCIT to the TAT, a noninvasive infrared thermometer widely used in hospitals. Four key findings were found in this research.

First, the NCIT's mean body temperature measurement (36.64°C) was in agreement with that of the TAT's (36.90°C). A mean difference of ±0.26°C was recorded between the NCIT and the reference TAT. Second, the Bland-Altman analysis showed that temperatures measured using the NCIT and the TAT were closely aligned at temperatures below 37.5^°C but at temperatures ≥37.5°C, the mean differences widened considerably. Third, an AUROC score of 0.67 (SD 0.05) demonstrates poor accuracy of the NCIT device at temperatures ≥37.5°C. Fourth, patient characteristics were associated with significant mean differences in temperature.

Pediatric studies have suggested that the NCIT is an accurate and reliable device based on its sensitivity and specificity scores, and lower mean difference when compared to alternate temperature measuring devices.^{13 , 17} Participants in all these studies were individuals of age 0-18 years. Türe & Yazar's study of 317 pediatric emergency department patients found high agreement between NCIT and the TAT temperatures with a mean difference of 0.002°C.¹⁷ Similar to our study, Teran et al.’s¹³ (2012) study of 434 paediatric patients found the mean temperature difference of the NCIT and the TAT was ±0.30°C.

In contrast to our study, Türe & Yazar's pediatric study found a higher sensitivity (79.25%) and specificity (98.10%) for NCIT with the reference TAT device for temperatures ≥37.5°C.¹⁷ Although they also found the NCIT identified less temperatures >37.5°C than the TAT. Studies that measured the NCIT against other reference devices also reported higher sensitivity and specificity scores for NCIT for temperatures >38°C.^{18 , 13}. Chiappini et al.’s¹⁸ prospective multicentre study (N = 251) on children and adolescents compared NCIT with a reference axillary thermometer. The study reported a high diagnostic accuracy of NCIT as compared to axillary thermometer temperature >38°C (sensitivity 89% and specificity 90%). Similarly, Teran et al.’s¹³ prospective analytical cross-sectional study (N = 434) in Bolivian pediatric patients, aged 1-48 months, reported a high sensitivity (97%) and specificity (99.6%) for NCIT in comparison to a reference rectal mercury thermometer. However, in one of the few studies on an adult population, Tay et al. (2015) also found a similar low sensitivity (29.4%) for fever screening in Singapore. These results may indicate that the accuracy of NCIT is dependent on age of the patient.

The AUROC score (0.67, SD 0.05) calculated in our study for NCIT fever diagnostic accuracy was lower than that reported in previous studies. Türe et al.¹⁷ study reported the NCIT had an AUROC score of 0.91 to a reference TAT temperature of ≥37.5°C. Similarly, Chiappini et al.¹⁸ reported AUROC score of 0.968 for the NCIT to a reference axillary temperature of ≥38°C. The differences in diagnostic accuracy in ours and previous studies may result from differing patient characteristics, device attributes and environmental factors (eg, ambient temperature). Strict attention to the standard operating procedures based on user guidelines is required to be followed for accurate measurement of temperature using NCIT devices.¹⁹ If standard operating procedures are not adhered to and incorrectly executed, then nonaccurate results may occur.¹⁹ In our study, the same RN collected the temperatures and strictly attended to the standard operating procedures for each patient measured. Other studies used multiple data collectors (Tay et al, 2015; Türe & Yazar, 2019) that might have resulted in measurement error or reporting bias (Chiappini et al, 2011; Teran et al., 2012).

Patient characteristics of gender and skin color were significantly associated with the mean difference in temperatures. Higher mean temperature differences for females compared to males is consistent with the literature that reports females have greater thermal responses to exogenous and endogenous heat load/loss. A larger ratio of body surface to body mass, greater subcutaneous fat, and lower exercise capacity are factors that influence female temperature regulation.²⁰ Hormonal modulation during the menstrual cycle may also modify thermoregulation in women.²⁰

In the current study, participants with light skin color had a higher mean difference in body temperature as compared to participants with medium and dark skin color. To the authors’ knowledge, this is the first study that has compared the temperature differences based on skin color. There is a level of subjectivity deciding on skin tone so we chose to group into 2 groups. However there were small numbers in the sample so the results must be interpreted cautiously.

Other factors, not analyzed in this study, may also influence body temperature discrepancies between the NCIT and the TAT. These factors include patient- and environmental-related factors. We did not attempt to determine some patient-related factors such as skin thickness, blood flow under the skin, metabolic rate, cardiac output, and hormonal levels. Other patient characteristics, such as forehead perspiration¹³ and antipyretic medication taken, may influence the NCIT measurements. Other environmental factors that could lead to false readings include suboptimal training of operators, inappropriate positioning of the thermometer, and uncontrolled temperature within the rooms. The ambient temperature of each ward was not recorded in this study. However, pre-operational training by the research nurses in regards to the device use and device cleaning, were undertaken to follow the device manufacturer instructions.

Strengths and limitations

To the authors’ knowledge, this is the first study that determined the accuracy of the NCIT in comparison to the TAT in adult inpatient hospital wards. The study followed strict protocol and infection control guidelines. There were several limitations for this study. First, a convenience sampling method was used for recruitment of participants in 2 hospitals. Although this is a simple, cost effective and easy to use method, the sample recruited using this method may not representative of the population. Second, repeated measures were not recorded to check precision of the NCIT. Third, the number of febrile cases (≥37.5°C) were limited to 31(11.7%) cases only.

Recommendations for research, education, and practice

The NCIT is a fast, noninvasive, nontouch, and hygienic device. NCIT temperature device training is required to increase the accuracy of the measurements. However, variable levels of accuracy in pediatric and adult populations highlighted the need for further research, particularly to validate the use in adults. In this instance, a larger sample size, use of a random sampling method, and repeated measurement of temperatures using NCIT and multiple reference devices is required. Furthermore, the response of NCIT devices to patient and environment factors such as patient physical characteristics, physical exercise, and variable ambient temperatures needs further study. Given the low sensitivity for temperatures over 37.5°C it may be preferable to use a direct temperature measurement device for temperatures greater than 37.5°C to check accuracy in health care settings. Nurses frequently have various temperature measurement options to use in clinical settings, this article offers a comparison of 2 different types that both indirectly rely on skin temperature measurement. This study provides health professionals with information on the reliability of the NCIT and cautions them about the low sensitivity for temperatures over 37.5°C.

CONCLUSION

This is the first study to compare accuracy of NCIT to a reference device TAT in adult hospitalized patients. Although mass fever screening is currently underway using NCIT, these results indicate that the NCIT may not be the safest device for mass fever screening in adults if used in isolation during a pandemic. It should be noted that fever screening is often conducted in less controlled conditions than those presented in this study, which may result in further decreased accuracy than our findings. Additional research is required to compare its accuracy and precision to other invasive and non-invasive core body temperature testing methods.

Authors contribution

TB conceptualied the project and design; TB SK BS and KD were involved in the protocol development; TB BS and RD were involved in data collection and TB SK and RD conducted data analysis. SK, TB, BS, SA, AK and KD were involved in the data interpretation and manuscript writing.