Visual Indicators on Vaccine Boxes as Early Warning Tools to Identify Potential Freeze Damage

Abstract

Background

The aim of this study was to determine whether the use of visual freeze indicators on vaccines would assist health care providers in identifying vaccines that may have been exposed to potentially damaging temperatures.

Methods

Twenty-seven sites in Connecticut involved in the Vaccine for Children Program participated. In addition to standard procedures, visual freeze indicators (FREEZEmarker^® L; Temptime Corporation, Morris Plains, NJ) were affixed to each box of vaccine that required refrigeration but must not be frozen. Temperatures were monitored twice daily.

Results

During the 24 weeks, all 27 sites experienced triggered visual freeze indicator events in 40 of the 45 refrigerators. A total of 66 triggered freeze indicator events occurred in all 4 types of refrigerators used. Only 1 of the freeze events was identified by a temperature-monitoring device. Temperatures recorded on vaccine data logs before freeze indicator events were within the 35°F to 46°F (2°C to 8°C) range in all but 1 instance. A total of 46,954 doses of freeze-sensitive vaccine were stored at the time of a visual freeze indicator event. Triggered visual freeze indicators were found on boxes containing 6566 doses (14.0% of total doses). Of all doses stored, 14,323 doses (30.5%) were of highly freeze-sensitive vaccine; 1789 of these doses (12.5%) had triggered indicators on the boxes.

Conclusions

Visual freeze indicators are useful in the early identification of freeze events involving vaccines. Consideration should be given to including these devices as a component of the temperature-monitoring system for vaccines.

Vaccines play an integral role in reducing the risk for human diseases.¹ In 2010, approximately 82 million vaccine doses were administered to an estimated 40 million children in the United States in the Vaccines for Children (VFC) program at a cost of $3.6 billion.² The VCF program is administered by the US Centers for Disease Control and Prevention (CDC).

For vaccines to be effective, they must be properly stored and handled from the time of manufacture through delivery to and storage in the provider’s office.³ Whereas exposure to any inappropriate conditions, including excessive heat or cold exposure, can affect potency of refrigerated vaccines, a single exposure to freezing temperatures will destroy some vaccines, such as liquid vaccines that contain an aluminum adjuvant.³ Should loss of potency occur, patients who receive the vaccine will not be protected against vaccine-preventable diseases.

In recent years, there have been indications that storage and handling of vaccines within required temperature ranges, including shipment of vaccine from the distributor to a hospital,⁴ were an ongoing problem in the United States. A survey conducted in 2008 by the Association of Immunization Managers (AIM) demonstrated considerable variability among city and state immunization projects in the United States.⁵ A field survey conducted by the California Department of Public Health found that a hospital in Santa Ana, Calif, had stored various vaccines at lower than freezing temperatures for 7 months in 2009. As a result, 1641 newborns were given potentially defective hepatitis B vaccine.⁶ In 2010, another investigation found a significant 76% correlation between the 25% of vaccine refrigerators in Houston’s community health centers that experienced prolonged freezing temperatures and that region’s rate of pertussis.⁷ In 2014, the Hartford (Connecticut) HealthCare Medical Group reported that the effectiveness of 5003 vaccinations given to 3833 patients since January 2013 may have been compromised because of poor temperature control. Patients, including those vaccinated against pneumonia or pertussis, were to be revaccinated.⁸

These earlier findings led to preliminary discussions in July 2010 with AIM to identify ways in which issues of improper storage and handling of vaccines at providers’ offices might be addressed. Subsequent discussions occurred with the Connecticut Department of Public Health in collaboration with the Connecticut Chapter of the American Academy of Pediatrics and Foundation for Children. These discussions resulted in the design and implementation of an in-field observational study to determine whether the use of visual freeze indicators would help VFC providers better manage vaccines to avoid wastage and the inadvertent administration of ineffective vaccines to children. The study was funded by Temptime Corporation, Morris Plains, NJ.

MATERIALS AND METHODS

Study Objectives

The primary study objective was to determine whether the use of visual freeze indicators on vaccine boxes would assist health care providers in identifying vaccines that may have been exposed to potentially damaging temperatures and those incorrectly suspected of having been exposed to potentially damaging temperatures. The secondary objective was to assess vaccines at risk for exposure to potentially damaging temperatures.

Participants

The study was conducted for 24 weeks at 27 sites in Connecticut that participate in the VFC program, including group and solo practices, public health centers, and hospital pediatric clinics. Study locations were recruited and selected by the Connecticut Chapter of the American Academy of Pediatrics and the Connecticut Department of Public Health. Vaccines were either provided through the VFC program or privately purchased.

Endpoints

The primary endpoints measured in the study were (1) freeze events identified by triggered visual freeze indicators and (2) freeze events identified by temperature readings. Secondary endpoints included (1) locations of triggered visual freeze indicators within vaccine refrigerators; (2) type, number of doses, and value of vaccine with triggered visual freeze indicators; (3) total inventory and value of vaccine that experienced a freeze event; and (4) site and vaccine refrigerator characteristics (mean number of vaccine refrigerators per site, types of refrigerators, types of temperature-monitoring devices used, and location of temperature-monitoring devices within the refrigerators).

Equipment and Procedures

A variety of devices were used to monitor the temperature within refrigerators, including continuous monitoring devices, some of which were state-issued data loggers, thermometers, and digital displays on the outside of the refrigerator unit, plus freeze indicators. Freeze indicators are either chemical or electronic in nature and provide a visual indication if the temperature falls lower than a preset temperature. The visual freeze indicator used in this study (FREEZEmarker^® L; Temptime Corporation, Morris Plains, NJ) is a small, chemical freeze indicator composed of microscopic particles dispersed within a colloid. When the temperature reaches 32°F ± 1.8°F (0°C ± 1°C), the particles become unstable, overcoming the repulsive forces that keep them separate. As a consequence, the particles coagulate irreversibly and form an opaque white color that indicates a freeze event has occurred (Fig. 1).

FIGURE 1.

FREEZEmarker^® L visual freeze indicators maintained at temperatures higher (left) or lower (right) than their specified freeze set point. Left, Colloid is clear, allowing the green background to be easily visible. Right, Colloid has irreversibly agglomerated, preventing the green background from being visible, thereby indicating exposure to temperatures at or lower than the set point.

A visual freeze indicator was affixed to each box of stored vaccine. To identify cold spots, visual freeze indicators were also placed in up to 76 locations throughout each vaccine refrigerator, including walls, shelves, and door. On-site staff was trained on the use and placement of visual freeze indicators, monitoring requirements, appropriate actions to take if a visual freeze indicator was triggered, and other requirements; current CDC guidelines were also reinforced. Existing vaccine storage and handling procedures adopted by the Connecticut Department of Public Health and directed by the CDC remained in effect.

Triggered freeze indicators inside the refrigerators and/or on boxes of vaccine were identified by on-site staff or field study assistants during weekly site visits. Total vaccine doses by type, number of doses by type with triggered freeze indicators, locations of triggered freeze indicators within the refrigerator, temperature reading at the time of the event, and prior reading on the Vaccine Daily Temperature Log were recorded.

Weekly site visits were made for the purposes of monitoring and data collection. The field study assistants also visited a site within 48 hours of a reported freeze event. All study data were collected on prespecified study forms and subsequently entered into an Excel spreadsheet and transferred into SAS software. Summary tables and data listings were then generated using SAS Version 9.3 software. Summaries for quantitative variables include descriptive statistics (mean, median, SD, minimum, maximum). Summaries for qualitative variables include the number and percentage for each outcome.

RESULTS

All 27 sites (100%) experienced at least 1 and as many as 6 triggered freeze indicator events (Table 1). Vaccines were exposed to potentially damaging temperatures per triggered freeze indicator in 40 of 45 refrigerators (Table 2). Only 5 refrigerators (4 household, 1 pharmacy/medical type) had no freeze indicator event.

TABLE 1.

Occurrence of a Triggered Freeze Event

TABLE 2.

Triggered Freeze Indicator Event by Type of Refrigerator Used for Vaccine Storage

A total of 66 triggered freeze indicator events occurred, only 1 of which was detected by a temperature-monitoring device, and this one was a remote, continuous temperature monitoring data logger located in a hospital pharmacy. Most of these events (64.6%) were found both in the refrigerators and on vaccine boxes; 18.5% were found on vaccine boxes only and 16.9% were found in refrigerators only. Of the 66 events, 34.8% were identified by office staff; 57.6%, by the field study assistant; 6.1%, by both office staff and the field study assistant; and 1.5%, by others. A total of 46,954 doses of vaccine valued at $1,587,353 were stored in refrigerators experiencing freeze indicator events during the study. Triggered visual freeze indicators were found on boxes containing 6566 doses (14.0% of total doses) valued at $187,335 (11.8% of the total value). Of all doses stored, 14,323 doses (30.5%) were of highly freeze-sensitive vaccine valued at $388,820 (24.5%); 1789 of these doses (12.5%) valued at $46,670 (12.0%) had triggered indicators on the boxes. The mean vaccine inventory in a single refrigerator at the time of a freeze indicator event was $24,051, with values ranging from $563 to $110,275.

Although triggered freeze indicators were found in locations throughout the refrigerators, 80.7% were on boxes of vaccine located on the top or second shelf (Table 3). The number of locations within individual refrigerators with a triggered freeze indicator was as high as 32 in locations identified as “cold spots.” The temperatures recorded on vaccine temperature logs before freeze indicator events were within the 35°F to 46°F (2°C to 8°C) range in 55 (98.2%) of 56 instances; 1 recording of 47°F was observed. Data indicating temperatures immediately before triggered freeze indicator events were not available for 10 events.

TABLE 3.

Location of Vaccine Boxes With Triggered Visual Freeze Indicators

Within 43 of the 45 refrigerators for which the location of the temperature-monitoring device was reported, the main temperature-monitoring device was most frequently located in the front of the refrigerator (22/43, 51.2%) and least frequently in the middle (6/43, 14.0%) (Table 4). The device was most frequently located on the top shelf (18/43, 41.9%). Secondary temperature-monitoring devices were reported in 17 (37.8%) of 45 refrigerators.

TABLE 4.

Location of Main Temperature-Monitoring Device Within Vaccine Refrigerators (Number [Percentage])

DISCUSSION

The present study found that triggered freeze indicator events are common and that only 1 of the freeze events was identified by a temperature-monitoring device. These results confirm the value of the visual freeze indicator device used in this study as an early warning tool to detect the potential risk for vaccines to become ineffective because of exposure to freezing temperatures.

The findings from the present study corroborate reports of vaccine wastage within provider offices and health care facilities. A 1998 to 1999 survey of 64 public-sector state and local health department immunization programs reported rates of vaccine wastage ranging from 1% to 5% (mean, 2.6%). Most wastage was due to cold chain lapses.⁹ A wastage rate of 1% to 5% was also observed in a survey of 61 VFC program coordinators in 2002.¹⁰ It is possible, however, that the rates reported in these surveys are low because another study found that 15% of refrigeration units had temperatures of 33.8°F (1°C) or lower.¹¹

The findings of the present study are also consistent with a study conducted in 2011 by the Office of Inspector General (OIG) of the US Department of Health and Human Services.² Although most storage temperatures measured by monitoring devices during a 2-week period of the OIG study were within required temperature ranges, VFC vaccines stored by 76% of the providers were exposed to inappropriate temperatures for 5 or more cumulative hours during that time. In addition, all 45 providers had 1 or more recorded temperatures that differed from those measured independently by OIG investigators during the same 2-week period. These findings showed that VFC providers generally did not meet vaccine management requirements or maintain required documentation and prompted the OIG to recommend that the CDC work with grantees and providers to improve compliance with vaccine storage requirements. Specific recommendations regarding the handling and storage of vaccines were issued by the CDC in 2012 and subsequently updated in May 2014.³

Together, the findings of the OIG study and the present study suggest that freeze events are far more common than previously reported. The reporting of patients being given vaccine that may have been ineffective because of temperature damage in Hartford, Conn, in 2014 demonstrates that difficulties with vaccine storage persist.⁸

The present study also showed that the variety of vaccine refrigeration units, even costly pharmacy/medical refrigerators, and the monitoring systems currently used are not infallible, indicating a need for complementary monitoring devices that can help providers identify when vaccines are at risk for damage due to freezing. This would give providers the opportunity to intervene to avoid both vaccine waste and the inadvertent administration of vaccines that have lost potency. Visual freeze indicators can serve as early warning tools to complement temperature-monitoring devices such as data loggers. Digital data loggers are recommended by the CDC for continuous temperature monitoring because these are capable of recording thousands of individual temperature readings.³ Data loggers are generally battery-operated electronic devices with a sensor and an internal microprocessor that records data at intervals set by the user; many also have an audible or visual alarm. A data logger thermometer is typically placed in 1 location inside the refrigerator.³ “As a general rule, the closer the thermometer is placed to the actual location where vaccines are stored, the closer its readings will be to actual vaccine temperatures.”¹² This suggests that the temperature-monitoring device is less likely to accurately reflect the temperature of vaccines not placed adjacent to it. Because only 1 temperature-monitoring device is required, most vaccines are not located in close proximity. In contrast, visual freeze indicators, which are placed on boxes of vaccine, are more likely to reflect the temperature of the vaccine to which they are affixed. Visual freeze indicators have the added benefit of helping providers identify where 1 or more “cold spots” exist within a specific refrigerator and avoid storing vaccines in those locations where there is greater risk for freezing. Visual freeze indicators can also be used to alert providers when temperature-monitoring devices are not functioning properly. The visual freeze indicator used in this study is currently being used to monitor shipments of selected vaccines to providers for the VFC program; this device has also been used for distribution of the H1N1 vaccine within the United States.

The reliability of the visual freeze indicator used in this study (FREEZEmarker^® L) has been verified in independent testing of the leading chemical and electronic freeze indicators in 2010 by Thermal Packaging Solutions. The test was conducted by Micom Laboratories, an International Safe Transit Association–certified laboratory in Montreal, Canada, using standards adopted by the National Institute of Standards and Technology, US Department of Commerce.⁴ Results from 20 samples showed that FREEZEmarker^® L performed within its temperature specification of 32°F ± 1.8°F (0°C ± 1°C), with a performance range of 30.7°F to 32.5°F (−0.7°C to 0.3°C).

At the 27 sites in the present study, vaccine doses valued at approximately $1.6 million (VFC and private pay combined) were exposed to potentially damaging freeze temperatures. Because it was not an objective of the study to assess the potency of vaccine identified as being exposed to a freeze event, it is unknown whether vaccine potency was compromised. Limited data show that some vaccines are stable for weeks or months at freezing temperatures, whereas other vaccines, such as aluminum-adsorbed vaccines, are sensitive to freeze-thaw damage.^13,14 Hepatitis B vaccine, for example, has been shown to partially freeze with structural damage from 1 to 6 hours at 14°F (−10°C) but not at 21.2°F (−6°C) for up to 72 hours. However, agitation may hasten freezing, resulting in partial freezing with structural damage within 1 hour at 21.2°F (−6°C).¹⁵

The potential wastage of vaccine has important implications, particularly for providers and patients. Many providers not only report losing money on vaccinations because of the soaring prices of vaccines they purchase outside the VFC program and limited reimbursement but also are often responsible for the cost of vaccines lost because of freezing, as well as the expenses associated with revaccination.¹⁶ Some providers have, in fact, curtailed or stopped administering vaccines, making it difficult for some patients to be vaccinated.^16,17 Patient access to vaccines may be further compromised by vaccine wastage due to freezing. This may place children at avoidable risk for preventable diseases.

Beyond determining the frequency of a freeze event and identifying the amount of vaccine put at risk because of a freeze event, the present study demonstrated other findings that have important implications for clinical practice. Although the standard of practice is to monitor temperatures using temperature-monitoring devices, all except 1 of the triggered freeze indicator events were identified by the visual freeze indicator rather than a temperature-monitoring device or visual inspection of vaccine. As identified by the OIG study, thermometers used by providers did not accurately measure temperatures or providers did not accurately record the temperature readings.² On average, refrigerator temperatures recorded by providers varied by 2°F (1.1°C) from the temperatures independently measured by the study investigators, suggesting improperly calibrated thermometers. Normal wear and tear of a temperature-monitoring device may cause its accuracy to drift over time. Visual inspection is an unreliable method for determining vaccine potency, particularly for inactivated vaccines, which may give no visible indication of reduced or lost potency when frozen.³ By contrast, the shake test, first introduced in the late 1980s, has been used in the field to identify vaccines damaged by freezing temperature. Until recently, the shake test had never been validated as a reference test by comparison with a gold standard. A 2010 study involving 475 vials of 8 different types of freeze-sensitive vaccines demonstrated that a properly conducted shake test has 100% sensitivity, 100% specificity, and 100% positive predictive value for detecting freeze damage to aluminum-based freeze-sensitive vaccines.¹⁸ The shake test requires that a frozen standard be prepared. Once this is done, it takes approximately 10 minutes to properly conduct the test. A limitation of the shake test is that it is not appropriate for acellular vaccines.

Office of Inspector General study investigators also found that thermometers were not always correctly placed within the refrigerators. The CDC recommends not placing the device in the door, near or against walls, close to cooling vents, or on the floor of the refrigerator. The CDC also recommends determining the location of the most reliable and consistent temperature within the refrigerator and storing the vaccines at this location with the temperature-monitoring device located nearby.³ In the present study, the temperature-monitoring device was most frequently located on the top shelf (41.9%), which is where cooling vents are often located. Moreover, central placement of a single temperature-monitoring device is unlikely to detect temperature gradients in refrigerators, as was observed in this study.¹² Prospective investigation by the National Institute of Standards and Technology, US Department of Commerce, also noted temperature variability within refrigerator units depending on location, particularly during frequent opening and closing of the door, as often occurs in clinical practice.¹²

The OIG and other studies identified additional factors that place vaccines at risk for exposure to damaging temperatures within a provider’s office.^2,3,12 These factors include failure of the thermometer, battery, or refrigerator as well as frequent opening of the refrigerator door, refrigerator defrost cycles, room temperature increases, power outages, and use of inappropriate refrigerator equipment. In the present study, 7 of the 45 refrigerators were dormitory style, a type not recommended by the CDC.³

Although there was no selection bias in the sites included in the study and a variety of provider sites, providers per site, vaccine doses administered per week, and refrigerator types were involved, it is not known whether the results of this study are generalizable to all providers within the VFC program or those who purchase vaccines privately. Moreover, the total number of vaccine doses involved in this study represents less than 1% of all VFC doses distributed annually in the United States. Furthermore, the course of action regarding use of triggered vaccines was a subjective decision based on visual inspection of the vaccine, previous temperature readings recorded on the temperature logs, and unpublished data regarding impact of exposure to freezing temperatures. These actions may or may not be reproducible under the guidance of a different inspector. Finally, our study was conducted around the time of the release of the 2012 CDC vaccine recommendations. Thus, comparison of provider practices observed during the study with those recommended in the 2012 CDC recommendations is meant for illustrative purposes only because adoption of guidelines into clinical practice often takes several years. Consistent with the findings of the 2008 AIM survey, in which providers expressed a strong interest in guidance related to vaccine storage and handling, we found providers eager to implement mitigating measures upon triggering of freeze indicators.⁵ These included moving vaccines away from cold spots and implementing CDC recommendations included in the Vaccine Storage and Handling Toolkit.³

In conclusion, visual freeze indicators have the potential to serve as an early warning of a freeze event and to assist providers as they work to protect patients with the most effective vaccines available. This visual tool can help providers better manage every dose of vaccine stored in their offices and thereby expand access to vaccine by avoiding unnecessary wastage. It is recommended that the CDC, in collaboration with grantees, providers, and industry, consider including a recommendation to affix visual freeze indicators to individual boxes of vaccine as an early warning tool to complement the storage and temperature-monitoring devices and equipment currently being used.