An Evaluation of Portable Wet Bulb Globe Temperature Monitor Accuracy

Abstract

Context: 

Wet bulb globe temperature (WBGT) is the gold standard for assessing environmental heat stress during physical activity. Many manufacturers of commercially available instruments fail to report WBGT accuracy.

Objective: 

To determine the accuracy of several commercially available WBGT monitors compared with a standardized reference device.

Design: 

Observational study.

Setting: 

Field test.

Patients or Other Participants: 

Six commercially available WBGT devices.

Main Outcome Measure(s): 

Data were recorded for 3 sessions (1 in the morning and 2 in the afternoon) at 2-minute intervals for at least 2 hours. Mean absolute error (MAE), root mean square error (RMSE), mean bias error (MBE), and the Pearson correlation coefficient (r) were calculated to determine instrument performance compared with the reference unit.

Results: 

The QUESTemp° 34 (MAE = 0.24°C, RMSE = 0.44°C, MBE = –0.64%) and Extech HT30 Heat Stress Wet Bulb Globe Temperature Meter (Extech; MAE = 0.61°C, RMSE = 0.79°C, MBE = 0.44%) demonstrated the least error in relation to the reference standard, whereas the General WBGT8778 Heat Index Checker (General; MAE = 1.18°C, RMSE = 1.34°C, MBE = 4.25%) performed the poorest. The QUESTemp° 34 and Kestrel 4400 Heat Stress Tracker units provided conservative measurements that slightly overestimated the WBGT provided by the reference unit. Finally, instruments using the psychrometric wet bulb temperature (General, REED Heat Index WBGT Meter, and WBGT-103 Heat Stroke Checker) tended to underestimate the WBGT, and the resulting values more frequently fell into WBGT-based activity categories with fewer restrictions as defined by the American College of Sports Medicine.

Conclusions: 

The QUESTemp° 34, followed by the Extech, had the smallest error compared with the reference unit. Moreover, the QUESTemp° 34, Extech, and Kestrel units appeared to offer conservative yet accurate assessments of the WBGT, potentially minimizing the risk of allowing physical activity to continue in stressful heat environments. Instruments using the psychrometric wet bulb temperature tended to underestimate WBGT under low wind-speed conditions. Accurate WBGT interpretations are important to enable clinicians to guide activities in hot and humid weather conditions.

Key Points

• Environmental monitoring using wet bulb globe temperature (WBGT) sensors is an increasingly important component of heat-safety policies in athletics.

• Commercially available sensors determine WBGT using different algorithms and data inputs.

• The degree of accuracy and bias among commercially available WBGT sensors vary. Sensors that used the psychrometric wet bulb temperature tended to underestimate the WBGT, whereas those that explicitly measured the globe, wet bulb, and dry bulb temperatures or incorporated wind in the calculation of the wet bulb temperature had superior performance or provided more conservative WBGT estimates.

Exertional heat illness (EHI) has become more prevalent in the past 10 to 15 years in the United States.^1–3 Geographically, the higher risk for EHI has been in the eastern half of the country and specifically the Southeast region.⁴ Despite the reported increased prevalence in the Southeast, the catastrophic consequences associated with EHI in athletic populations make this condition a concern across the United States. Additionally, although deaths occur across all ages, the majority are among football athletes who are aged 18 or younger.¹

Because EHIs are highly preventable and treatable with proper precautions and care, regulatory organizations have focused on policy governance. Several states have taken steps to mitigate EHI-related concerns; many of these policies were based on the National Athletic Trainers' Association's “Preseason heat-acclimatization guidelines for preseason secondary school athletics.”⁵ New Jersey was the first state to implement such laws in high schools,⁶ and 15 other states established heat-acclimatization policies that meet the minimum standards set forth by the Korey Stringer Institute.⁷ Also, the Texas High School University Interscholastic League has implemented preseason practice guidelines in an attempt to mitigate EHIs and prevent related deaths.^6,7 Besides high school governing bodies and state legislatures, the National Collegiate Athletic Association has instituted preseason practice guidelines in an effort to keep players safe while participating in the heat in an effort to reduce the incidence of EHIs.⁸

Many of these new laws, policies, and procedures use the wet bulb globe temperature (WBGT) to determine practice conditions. The WBGT has been considered the criterion standard for environmental stress assessment during physical activity as it is the measurement of choice of the American College of Sports Medicine (ACSM) and the US Department of Defense.^9,10 The WBGT requires specialized measurements for the wet bulb and globe temperatures and thus is not typically assessed at weather stations, such as those monitored by the National Weather Service. Many WBGT devices are commercially available, yet no recent studies have been completed to determine the reliability of these devices compared with a specification unit. The devices vary in design and price, ranging from approximately $100 to $3000 (Table 1). Most commercially available WBGT devices directly measure the dry and globe temperatures, but how the wet bulb temperature is determined differs. Traditionally, the natural wet bulb temperature is measured with a wetted thermometer placed in the natural environment.¹¹ It is influenced by ambient air temperature, humidity, wind, and radiant energy. Other instruments estimate the wet bulb temperature using air temperature and humidity via an algorithm.¹² Some combine multiple variables, such as wind speed and relative humidity, to estimate the natural wet bulb temperature.¹³ Other instruments use the psychrometric or thermodynamic wet bulb temperature, which can be computed from the dry bulb temperature and humidity measurements.¹² Importantly, the natural wet bulb temperature and the psychrometric wet bulb temperature are similar for wind speeds above 3 m/s but may differ substantially in low wind-speed environments. Indeed, the psychrometric wet bulb devices may provide values up to 10°C lower than the natural wet bulb temperature outdoors in the sun.¹² Finally, for measuring the globe (radiant) temperature, many portable WBGT instruments use black globes that are smaller than the standard diameter of 0.15 m but often adjustments are made to match those readings (eg, QUESTemp° 34 [3M Detection Solutions, St Paul, MN], Kestrel 4400 Heat Stress Tracker [Neilsen-Kellerman Co, Boothwyn, PA], WBGT-103 Heat Stroke Checker [Kyoto Electronics Manufacturing Co, Ltd, Kyoto, Japan]).

Table 1. .

Costs and Descriptions of Wet Bulb Globe Temperature Devices Assessed

Unit	Manufacturer	Reported Accuracy			Size of Unit (Height × Width × Depth), cm	Diameter of Black Globe, cm	Corrects Black Globe to 15 cm?	Retail Value,a $
		Wet Bulb Globe Temperature, °C	Air Temperature, °C	Relative Humidity, %
General WBGT8778 Heat Index Checker	General Tools & Instruments LLC, Secaucus, NJ	Unreported	±0.6	±3	27.8 × 7.5 × 7.5	7.5	No information	189.95b
REED Heat Index WBGT Meter model SD-2010	REED Instruments, Wilmington, NC	Unreported	±0.8	±3	17.7 × 6.8 × 4.5	7.5	No information	219.00c
Extech HT30 Heat Stress Wet Bulb Globe Temperature Meter	FLIR Commercial Systems Inc, Nashua, NH	±2.0	±1.0	±3	29.2 × 8.9 × 8.6	4.0	No information	169.99d
QUESTemp° 34	3M Detection Solutions, St. Paul, MN	Unreported	±0.5	±5	23.5 × 18.3 × 7.5	5.1	Yes	2926.60e
Kestrel 4400 Heat Stress Tracker	Neilsen-Kellerman Co, Boothwyn, PA	±0.7	±0.5	±3	16.5 × 5.9 × 2.8	2.5	Yes	469.00f
WBGT-103 Heat Stroke Checker	Kyoto Electronics Manufacturing Co, Ltd, Kyoto, Japan	±2.0	±1.0	±5	24.0 × 4.0 × 3.2	2.4	Yes	99.00g

^a Retail value at the time of the assessment.

^b General Heat Index Tracker manufacturer's information.

^c Reed Instruments manufacturer's information.

^d Extech manufacturer's information.

^e Quest Technologies manufacturer's information.

^f Kestrel manufacturer's information.

^g Kyoto manufacturer's information.

Coaches and athletic trainers rely on these units to make important decisions regarding participation, so accurate and reliable instruments are needed to monitor environmental conditions. Unreliable and inaccurate readings could lead health care providers and coaches to put athletes at risk of sustaining heat-related illnesses and being exposed to potentially dangerous situations. Selecting an accurate WBGT instrument will ensure that WBGT readings are reliable and that the activity modification is appropriate for the particular weather conditions. Therefore, the purpose of our study was to determine the accuracy of 6 commercially available WBGT monitoring devices compared with a standardized reference unit (RU) on days with weather conditions sufficiently oppressive to require activity modification according to the ACSM.⁹

METHODS

We examined a variety of commercially available sensors, ranging in cost from around $100 to several thousand dollars. The General WBGT8778 Heat Index Checker (General Tools & Instruments, New York, NY), REED Heat Index WBGT meter (model SD-2010, REED Instruments, Wilmington, NC), Extech HT30 Heat Stress WBGT Meter (FLIR Commercial Systems Inc, Nashua, NH), QUESTemp° 34, Kestrel 4400 Heat Stress Tracker, and WBGT-103 Heat Stroke Checker were assessed against the RU. The specifications for each unit are provided in Table 1. The QUESTemp° 34 explicitly measured the natural wet bulb temperature, and the Kestrel unit computed the natural wet bulb temperature from relative humidity and wind speed.¹³ The remaining instruments determined the psychrometric wet bulb temperature for use in computing the WBGT.

The RU was custom designed and built by Kestrel engineers to meet specifications for a WBGT monitor determined by the International Organization for Standardization (ISO). (The term ISORU will refer to the RU.) These guidelines indicate that the natural wet bulb sensor should be cylindrical in shape (6 ± 1 mm diameter and 30 ± 5 mm long) with a measuring range of 5°C to 40°C and accuracy of ±0.5°C. The wick that fits over the sensor must be white and constructed of a water-absorbent material such as cotton. The globe temperature sensor must have a matte black globe 0.15 m in diameter and measure the range of 20°C to 120°C with accuracies of 0.5°C for temperatures ranging from 20°C to 50°C and ±1°C for temperatures >50°C. Finally, the dry bulb temperature sensor should have a radiation shield and measure the range 10°C to 60°C with an accuracy of ±1°C.¹¹

Although some manufacturers indicated that their units could have a specific measurement error for the WBGT (up to ±2°C), others did not disclose a value for instrument accuracy (Table 1).

Procedures

The devices were attached to camera tripods with the top of each black bulb positioned in line with the top of the reference bulb at a height of 1.52 m (5 ft). The center of each device was placed 0.91 m (3 ft) from the ISORU, in hexagonal fashion (Figure 1). Before data collection, all devices were turned on and stabilized per the manufacturers' directions. For heat-monitoring devices that measured wind speed, each device was opened as indicated by the manufacturer's guidelines and oriented at 0° north. Monitors were placed on the fields such that shadows did not interfere with any device throughout data collection. One stopwatch was used for data collection, and each monitor was observed by 1 volunteer to ensure that simultaneous recording was maintained. All recordings occurred in direct sunlight with cloud cover providing the only shade.

Figure 1. .

Photograph of the instrument setup. The reference wet bulb globe temperature unit is located in the center.

Data were recorded for 3 sessions (1 in the morning and 2 in the afternoon) at 2-minute intervals for at least 2 hours. Measurements were taken September 7, 2012 (7:30 am to 9:45 am and 3:45 pm to 6:00 pm), and September 27, 2012 (4:00 pm to 6:00 pm), to sample conditions that might occur during morning and afternoon practices. Measurements were taken over grass on the morning of day 1 and afternoon of day 2 and over field turf (Sports Turf, Dalton, GA) for the afternoon session of day 1. Ambient weather conditions were assessed by the nearest weather-observing station, located at the Climatology Research Laboratory on the University of Georgia campus. On September 7, morning weather conditions included temperatures ranging from 21.1°C to 23.9°C, dewpoint temperatures ranging from 20.6°C to 22.2°C, and wind speeds ranging from 2.6 to 3.6 ms⁻¹, with cloud cover increasing from clear to mostly cloudy over the study session. In the afternoon, air temperatures ranged from 31.7°C to 32.2°C, with dewpoint temperatures of 20.6°C, mostly calm conditions, and periods of clear and scattered clouds. Afternoon conditions on September 27 were slightly cooler and less humid than during the previous session, with air temperatures ranging from 28.0°C to 28.3°C, dewpoint of 14.0°C, wind speeds ranging from calm to 2 ms^–1, and cloud cover ranging from clear to partly cloudy over the study session. Based on current ACSM guidelines,⁹ recommended activity modification for the weather conditions ranged from “normal activity” to “limiting intense exercise and total daily exposure to heat and humidity.” Thus, our study captured a wide range of environmental conditions in which the risk for heat injury varied from moderate to high.

Statistical and Graphical Analyses

Statistical and graphical analyses were used to evaluate the performance of the various WBGT sensors. Summary statistical measures of mean absolute error (MAE), root mean square error (RMSE), mean bias error (MBE), and the Pearson correlation coefficient (r) were used to quantify instrument performance relative to the ISORU. Graphical analysis was completed using scatter plots of the various WBGT sensors versus the ISORU.

RESULTS

We identified a range of values among the various sensors (Table 2). The difference in the average WBGT among the sensors over the study periods was about 2°C, ranging from 25.54°C ± 2.13°C for the General unit to 27.51°C ± 2.64°C for the Kestrel unit, with a mean of 26.67° ± 2.18°C for the ISORU. Compared with the ISORU, the best overall performances were from the QUESTemp° 34 and Extech units, with the lowest overall values of <1°C for MAE and RMSE and MBEs of well under 1%. The General WBGT instrument performed the poorest with MAE and RMSE values >1.1°C and an MBE of 4.25%. The remaining instruments demonstrated intermediate performance values. Further, the MBE indicated positive biases (underestimates) for the General, REED, Extech, and WBGT-103 units and negative biases (overestimates) for the QUESTemp° 34 and Kestrel units. Finally, WBGTs for all instruments had high correlations (r > 0.93) with the ISORU, indicating that the WBGTs varied in a similar manner.

Table 2. .

Evaluation Statistics

	General WBGT8778 Heat Index Checkera	REED Heat Index WBGT Meter model SD-2010b	Extech HT30 Heat Stress Wet Bulb Globe Temperature Meterc	QUESTemp° 34d	Kestrel 4400 Heat Stress Trackere	WBGT-103 Heat Stroke Checkerf	Reference Unitg
Mean ± SD, °C	25.54 ± 2.13	26.1 ± 1.97	26.55 ± 2.34	26.85 ± 2.21	27.51 ± 2.64	26.08 ± 2.11	26.67 ± 2.18
Mean absolute error, °C	1.18	0.88	0.61	0.24	0.95	0.78	NA
Root mean square error, °C	1.34	1.00	0.79	0.44	1.23	0.97	NA
Mean bias error, %h	4.25	2.14	0.44	−0.64	−3.14	2.21	NA
ri	0.95	0.93	0.94	0.98	0.95	0.94	NA
Countj	171	171	171	171	171	171	171

Abbreviation: NA, not applicable.

^aGeneral Tools & Instruments LLC, Secaucus, NJ.

^bREED Instruments, Wilmington, NC.

^cFLIR Commercial Systems Inc, Nashua, NH.

^d3M Detection Solutions, St Paul, MN.

^eNeilsen-Kellerman Co, Boothwyn, PA.

^fKyoto Electronics Manufacturing Co, Ltd, Kyoto, Japan.

^gCustom designed and built by Kestrel engineers (Nielsen-Kellerman Co) to meet the specifications for a wet bulb globe monitor determined by the International Organization for Standardization Organization.

^hPositive or negative mean bias error indicates an underestimation or overestimation, respectively, relative to the reference unit.

ⁱPearson product moment correlation coefficient.

^jNumber of observations.

A graphical analysis was used to further examine the bias pattern of the instruments with regard to the ISORU (Figure 2). The QUESTemp° 34 unit, followed by the Extech unit, showed the least bias about the 1-to-1 line. The General and WBGT-103 sensors almost uniformly underestimated the WBGT compared with the ISORU, whereas the Kestrel unit overestimated WBGT. The REED sensor, however, underestimated WBGTs higher than 28°C and tended to overestimate those values between 24°C and 28°C.

Figure 2. .

Scatter plots of wet bulb globe temperatures from the instruments versus the reference unit. A, General WBGT8778 Heat Index Checker (General Tools & Instruments LLC, Secaucus, NJ). B, REED Heat Index WBGT Meter (model SD-2010; REED Instruments, Wilmington, NC). C, Extech HT30 Heat Stress Wet Bulb Globe Temperature Meter (FLIR Commercial Systems Inc, Nashua, NH). D, QUESTemp° 34 (3M Detection Solutions, St Paul, MN). E, Kestrel 4400 Heat Stress Tracker (Neilsen-Kellerman Co, Boothwyn, PA). F, WBGT-103 Heat Stroke Checker (Kyoto Electronics Manufacturing Co, Ltd, Kyoto, Japan).

Overall, the sensors performed well, with no errors (MAE or RMSE) >1.34°C (Table 2). The magnitude of such errors may not seem especially large, but small differences near the margins of safety thresholds may alter activity suggestions. To assess these effects, we examined the results categorically in terms of the ACSM WBGT activity guidelines with an emphasis on those categories requiring activity modification (Figure 3). Relative to the values from the reference sensor, a far greater percentage of observations from the General, REED, and WBGT-103 sensors were in the lower activity categories. Importantly, these sensors greatly underestimated the 30.1°C to 32.2°C range, when exercise should be limited. In contrast, the Kestrel unit and, to a lesser extent, the Extech and QUESTemp° 34 units had biases toward the higher activity categories (>27.9°C), where intense activity should be carefully monitored or limited.

Figure 3. .

Comparison of frequency of measurements in different American College of Sports Medicine heat-safety categories based on wet bulb globe temperature (WBGT). The WBGT guidelines for activity modification were WBGT ≤27.8°C (normal activities), 27.9°C to 30.0°C (limit practice time), 30.1°C to 32.2°C (very limited practice time), and ≥32.3°C (cancel). ^a The reference unit was custom designed and built by Kestrel engineers (Nielsen-Kellerman Co, Boothwyn, PA) to meet the specifications for a wet bulb globe monitor as determined by the International Organization for Standardization. ^b General WBGT8778 Heat Index Checker (General Tools & Instruments LLC, Secaucus, NJ). ^c REED Heat Index WBGT Meter (model SD-2010; REED Instruments, Wilmington, NC). ^d Extech HT30 Heat Stress Wet Bulb Globe Temperature Meter (FLIR Commercial Systems Inc, Nashua, NH). ^e QUESTemp° 34 (3M Detection Solutions, St Paul, MN). ^f Kestrel 4400 Heat Stress Tracker (Neilsen-Kellerman Co, Boothwyn, PA).^g WBGT-103 Heat Stroke Checker (Kyoto Electronics Manufacturing Co, Ltd, Kyoto, Japan).

DISCUSSION

We observed both positive and negative biases of different magnitudes among the various instruments. Overall, the QUESTemp° 34 unit, followed by the Extech unit, had the best overall performance. From a safety standpoint, those with positive biases, such as the Kestrel unit, would provide more conservative estimates of WBGT. Indeed, in our study, about 40% of observations taken by the Kestrel unit would have resulted in alteration of training activities compared with about 27% of observations taken by the ISORU. The General, REED, and WBGT-103 units had either uniformly negative biases or, in the case of the REED unit, a bias for the higher WBGTs. Thus, these sensors reported WBGTs lower than those measured with the ISORU. Only about 17% of the observations from the General unit, for instance, would have merited planning activities with discretion or even limited training, compared with 27% for the ISORU.

Some of the biases we observed may have been related to the meteorologic conditions present during the study or to the device design. A total of 96% of the wind-speed data obtained from the University of Georgia weather station were ≤3 m/s. The underestimates of WBGT provided by the sensors using the psychrometric wet bulb temperature were likely related to these low wind speeds, which have been shown to cause underestimates of the natural wet bulb temperature.¹² Similarly, a specific manufacturer's device design could play a role; for example, black bulb size ranged from 2.4 to 7.5 cm, with some of the devices correcting to 15 cm (Table 1).

A literature search revealed few current assessments of WBGT instrument reliability or accuracy.^14–17 Previous work was limited in that it either focused on instruments that were not commonly used by clinicians or coaches^14–17 or compared only a small number of instruments.¹⁴ As the need for a more comprehensive policy to affect the occurrence rates of EHIs among athletes becomes apparent, the sports medicine community must look for ways to properly and accurately measure environmental stresses. As stated earlier, many prestigious organizations use the WBGT index to determine environmental exposure. Therefore, it is imperative that sports medicine practitioners and the athletic community be able to make informed decisions as to which weather-monitoring equipment is the most accurate. Independent comparison tests on a variety of units is 1 method of aiding these important decisions.

Accurate weather-monitoring data are essential to sports medicine practitioners. The variability of the WBGT instruments in assessing the environment is shown in Table 2. The purchase decision becomes less complex when a particular unit is known to underestimate weather conditions, which could be detrimental to the health and well-being of athletes.

Our study had several possible limitations. First, units from additional manufacturers were available for commercial purchase. It was not our intent to compare all available units but to instead assess a representative sample. Second, assessing temperature extremes or a variety of surfaces may yield different results. Future researchers may want to test the WBGT units under extremely high temperatures or on various surfaces (eg, concrete, dirt) to identify their performance under other circumstances. Third, we assessed the instruments under meteorologic conditions with low wind speeds, which may have affected the accuracy of the instruments using the psychrometric wet bulb temperature. Yet we believe it is important for instruments to operate under a variety of meteorologic conditions, especially those with low wind speeds that could reduce cooling and enhance the risk for a heat illness.

CONCLUSIONS

The use of portable environmental weather-monitoring equipment has become an essential tool for health care and sports medicine professionals as a means for determining whether environmental conditions are safe for activity. The use of WBGT monitors has been advocated by numerous organizations, such as the ACSM, US Department of Defense, American Pediatric Society, and National Athletic Trainers' Association.^9,10,18,19 Currently, numerous products are available for purchase, and choosing one that is accurate is an important decision. The purchasing decision on a WBGT monitor should be based on the reliability of the unit and the level to which it accurately assesses the WBGT, thereby enabling a health care professional to decide whether to cancel or alter practice or continue practice without modifications.

Instruments that consistently underestimate WBGT values could endanger the health of athletes when in the upper WBGT categories (eg, WBGT >28°C). Our study suggested that instruments relying on psychrometric wet bulb temperatures may be prone to underestimate the true WBGT under low wind-speed conditions. From a safety standpoint, the Extech, QUESTemp° 34, and Kestrel units provided accurate or conservative (or both) estimates of WBGT under our study conditions. Using information provided by an independent evaluation will help clinicians and coaches make the most informed choice when purchasing a WBGT monitor and developing practice policies designed to provide for the overall safety and health of active populations.