Laboratory testing of sunscreens on the US market finds lower in vitro SPF values than on labels and even less UVA protection

Abstract

Background

New research has attributed increased significance to the causal link between ultraviolet A (UVA) radiation and immunosuppression and carcinogenesis. In the United States, sunscreens are labeled with only their sun protection factor (SPF) and an imprecise term “broad‐spectrum protection.” Sunscreen marketing and efficacy evaluations continue to be based primarily on skin redness (sunburn) or erythema. We sought to evaluate the ultraviolet (UV) protection offered by common sunscreen products on the US market using laboratory‐measured UV‐absorption testing and comparing with computer‐modeled protection and the labeled SPF values. This approach enables an investigation of the relationship between the labeled SPF and measured UVA protection, a factor that is ignored in current regulations.

Methods

Fifty‐one sunscreen products for sale in the United States with SPF values from 15 to 110 and labeled as providing broad‐spectrum protection were tested using a commercial laboratory. All products were evaluated using the ISO 24443:2012 method for sunscreen effectiveness. The final absorbance spectra were used for analysis of in vitro UV protection.

Results

In vitro SPF values from laboratory‐measured UV absorption and computer modeling were on average just 59 and 42 percent of the labeled SPF. The majority of products provided significantly lower UVA protection with the average unweighted UVA protection factor just 24 percent of the labeled SPF.

Conclusion

Regulations and marketplace forces promote sunscreens that reduce sunburn instead of products that provide better, more broad‐spectrum UV protection. The production and use of products with broad spectrum UV protection should be incentivized, removing the emphasis on sunburn protection and ending testing on people.

SUMMARY STATEMENT.

Our manuscript is an original research study of fifty‐one sunscreens purchased in the US and tested in a commercial laboratory. The measured efficacy indicates that US sunscreens significantly underperform relative to the labeled SPF. Underperformance is most notable for UVA radiation, which is linked to immunosuppression and carcinogenesis; thus, the current practice of prioritizing sunburn protection in products is potentially detrimental to the goal of reduced health harm through balanced UV protection. From practice, regulatory, and product development perspectives, sunscreens with broad spectrum UV protection should be incentivized, removing the emphasis on sunburn protection and ending testing on people.

1. INTRODUCTION

Skin cancer is the most commonly diagnosed cancer in the United States, and the majority of cases are considered preventable. ¹ Reducing the incidence of skin cancers caused by excess solar exposure has become a major health initiative around the world. Sunscreen products have become an important tool promoted by health agencies and consumer product manufacturers, based on the premise that the reduction of any ultraviolet radiation (UVR) exposure benefits health. ²

Exposure to UVR is associated with three types of skin cancers: basal cell, squamous cell, and melanoma. ³ UVA radiation contributes just 10%‐13% of erythema risk when the sun is directly overhead ⁴ but has been linked to melanoma, ⁵ , ⁶ , ⁷ generation of reactive oxygen species and carcinogenesis, ⁸ , ⁹ , ¹⁰ immunosuppression, ¹¹ DNA damage under the surface of the skin, ¹² and photoaging. ¹³ Significant biological damage, including delayed cyclobutane pyrimidine dimers, has been documented from exposure to radiation occurring at the UVA/visible boundary. ¹⁴

Sunscreen products available to US consumers significantly differ with regards to the spectral uniformity or UVA protection provided, even when comparing products with identical labeled SPF. ¹⁵ Use of products with suboptimal UVA attenuation can increase lifetime UVA burden. Sunscreen products can be improved to provide more balanced protection through changes to standards and stronger correlation between labeled SPF and measured UVR reduction. ¹⁵

Some recent publications on sunscreens focus on sunburn reduction as the marker of efficacy and clinical benefit, ¹⁶ , ¹⁷ , ¹⁸ yet sunburn protection alone leads to increased UVA exposure comparable to that from tanning beds. ¹⁹ Figure 1 plots the wavelength‐dependent erythema, ²⁰ reactive oxygen species, ²¹ and immunosuppression of recall immunity as measured using a nickel contact hypersensitivity elicitation model in humans, ²² weighted to solar effectiveness during noon summer sunlight at 40 degrees north of the Equator. ²³ In addition to suppressing the elicitation arm of immune reactions, UVA has been shown to impact aspects of induction of the immune response as well, which may have more biological relevance for skin carcinogenesis. ²⁴ Investigation of sunscreens on their ability to inhibit both induction and elicitation of the immune response indicates that sunscreens, which only protect against UVB and erythema, do not provide equal immune protection, indicating UVA radiation can be immunosuppressive in humans. ²⁵ The biological significance of the suppression of the sensitization phase in the UVA region has not been fully established and subsequent research found the erythema action spectrum to be predictive of UVR‐induced immunosuppression across the UV spectrum. ²⁶ Observed photodamage to cell viability, DNA, and differential gene expression from exposure to radiation at the UVA/visible threshold further supports the need for broad‐spectrum protection. ¹⁴

FIGURE 1.

Biological responses to summer noonday sunlight with the maxima normalized to one. Erythema is primarily caused by radiation in the UVB portion of the spectra, whereas UVA radiation is more likely to induce free radical generation or immunosuppression of recall immunity using a contact hypersensitivity elicitation model [Colour figure can be viewed at wileyonlinelibrary.com]

Within the United States, melanoma incidence rates have increased nearly 1.5% a year unabated for the past decades. ³ The National Cancer Institute reports that melanoma incidence has increased by more than 300% since 1975, with an estimated 100 350 cases diagnosed in 2020. ³ The focus of scientific attention on increasing melanoma incidence has been on the role of UVR exposure although a recent publication highlights that while sunburn is known to increase the risk, a significant increase in melanoma incidence may be due to increased diagnostic scrutiny. ²⁷

Sunscreen development, regulation, and marketing are based largely on SPF value in the United States, which is determined by a reduction in skin redness (sunburn) or erythema a day after UVR exposure. This testing is required for nearly every sunscreen product sold around the world. ¹⁵ , ²⁸ , ²⁹

Consumer preference is for higher SPF products, ³⁰ incentivizing a trend of products with increasingly higher SPF numbers. ³¹ Increased SPF can be accomplished by changes to product formulations, such as adding active and/or inactive ingredients that reduce erythema or through changes to how the measured UV absorption testing is conducted, such as optimizing the solar spectra of the lamp ⁴ , ³² or optimizing lamp intensity. ³³ SPF values derived from a single laboratory, as required for regulatory compliance and labeling, have been described as unreliable due to significant variation in results from different laboratories. ³² , ³⁴ SPF testing is based on a solar spectrum equivalent to midday sun near the equator, overemphasizing the important of UVB protection. ³⁵ Experimental testing also finds that SPF values measured in a laboratory do not reflect the actual protection provided by the sunscreen in an outdoor setting. ¹⁵ , ³⁶

The perception that SPF values are a reliable marker of product effectiveness is rooted in history, because sunburn reduction was the original intent of sun protection products. ³⁷ However, since 1989, numerous papers on melanoma have questioned the role of sunscreen ³⁸ , ³⁹ , ⁴⁰ and the impact of UVB, ⁴¹ UVA, ⁴² , ⁴³ , ⁴⁴ and increased UVA exposure relative to UVB. ⁴⁵ Use of tanning beds has been associated with an increased melanoma risk. ⁷ , ⁴⁶ Tanning beds were designed to minimize the potential for sunburns and maximize tanning through the use of solar lights that emit up to 10‐15 times greater UVA radiation compared to a typical sun spectrum. ⁴⁷ , ⁴⁸ Tanning beds were classified by the World Health Organization as a known human carcinogen because of a dose‐response increase in melanoma for young women, ⁴⁹ and the association has been reaffirmed in newer tanning beds that rely more strongly on long wavelength UVA. ⁶

With a continued reliance on the SPF to communicate sunscreen efficacy, various methods have been developed to characterize and set a minimum standard for protection from UVA radiation not captured in the SPF erythema test. The current methodologies used and proposed by the US Food and Drug Administration and within the ISO 24443:2012 involve laboratory measurement of the UVR absorption of a sunscreen applied to a slide. Both of these in vitro tests are used not to calculate an accurate SPF but to supplement in vivo SPF testing. A new in vitro method is currently being developed to replace in vivo SPF testing entirely. ⁵⁰ The UVA protection factor (UVAPF) calculated using the ISO 24443:2012 methodology is intended to approximate persistent pigment darkening with a peak response at approximately 365 nm. The UVAPF is less sensitive to UVR between 364 and 385 nm and the observed immunosuppression peak as measured by suppression of the elicitation phase of contact hypersensitivity. ²² The US FDA broad‐spectrum standard and the EU 1/3 UVA/SPF standard allow products that provide significantly lower UVA than UVB protection, whereas ideal protection has been described as equal parts UVA and UVB, ⁵¹ similarly to what covering up with a shirt would provide. The Boots star system provides greater incentive for balanced protection with a UVA/UVB ratio of 0.9 required to label a product with 5 stars for UVA protection. Additionally, the Boots system is based on uniform protection across the UVA spectra unlike the ISO 24443:2012 method used in the EU, which is based on persistent pigment darkening, which is most sensitive to radiation centered on approximately 355‐360 nm. For aesthetic reasons, sunscreen formulations typically achieve full transmission at 400 nm, the threshold between visible and UV light, limiting the UV protection near the threshold, although recent research has documented DNA damage occurs from exposure at these wavelengths. ¹⁴

The US FDA and ISO 24443:2012 methodologies share the assumption that the SPF value is the true measure of protection, but recent work has hypothesized that correlating the magnitude of the measured UVR absorption to SPF can improve products. ¹⁵ In the United States, there is no requirement for the magnitude of SPF protection measured on people to correlate with the measured UVR reduction on a slide. Within ISO 24443:2012, in addition to the use of control SPF 15 formulation, the measured UV absorption is scaled to the labeled SPF using an adjustment coefficient factor, C. The limits for this adjustment factor defined by ISO 24443:2012, while maintaining some correlation between labeled and measured SPF, also allows for the reported UVA protection to be significantly higher than the measured in vitro UVA protection. The US FDA is concerned that the use of high SPF value products with inadequate UVA protection could lead to excessively large UVA doses. ²

In this paper, we evaluate the correlation between labeled SPF, measured UV absorption, and computer‐modeled protection. Computer‐modeled or simulated SPF is a calculation of expected UV absorption using active ingredient concentrations. ⁵² We discuss and evaluate the implications of the assessment of the magnitude of UV reduction and UVA protection in comparison to ideal UV protection and the current and proposed methods used by the FDA and within ISO 24443:2012.

2. METHODS

A total of 51 sunscreen products were analyzed. Thirty‐one products were purchased online or in Connecticut stores from Amazon, CVS, Target, and Walmart stores in late Fall 2018, and 20 were purchased from the same stores in Fall 2019. The products were chosen based on the active ingredients used, the SPF values, and the dedicated shelf space of products. All of the products were lotions and included a range of formulation types, including mineral and nonmineral active ingredients. All products were labeled as providing broad‐spectrum protection. To ensure blind testing, all samples were transferred to standardized sample containers and labeled by sample number before being shipment to the laboratory.

To determine the expected SPF from the measured UV absorption, products were tested in accordance with protocols described in the ISO 24443:2012 method. ⁵³ The 20 products purchased in 2019 were also tested according to the FDA 1978N‐0038 method, Final Rule 21 CFR§ 201.327. Within the ISO 24443:2012 methodology, the initial absorbance spectra are scaled to match the labeled SPF using a correction factor coefficient, which is then applied to the final spectra. An SPF 15 standard was evaluated alongside test products as required in the ISO 24443:2012 method, which itself was not specifically developed to provide SPF values. In our analysis, the final unweighted absorbance spectra were used to calculate the measured SPF and measured UVA protection factor (UVAPF).

The modeled sunscreen protection was calculated using the active ingredients concentrations input into the free online BASF Sunscreen Simulator available at www.sunscreensimulator.basf.com/Sunscreen_Simulator/. ⁵ , ¹⁹ This approach relies on the ingredient molar absorption spectra ⁵² , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ with thickness modeled on a gamma distribution function. ⁵⁸ The results of the BASF Sunscreen Simulator have been shown to correlate with in vivo testing although use of the results is currently limited to product development and research and not for regulatory purposes. ⁵² , ⁵⁹

3. RESULTS

The sunscreen products fell into five separate groups of product categories based on active ingredients: 12 products with only zinc oxide (ZnO) as the active ingredient, 8 with ZnO and titanium dioxide (TiO₂), four with only TiO₂, six with ZnO or TiO₂ in combination with organic filters, and 21 with only organic active ingredients. Two products were not tested to completion: an SPF 50 product with ZnO+TiO₂ actives had phase separation and could not be tested, and an SPF 50 ZnO product failed the initial step of testing. Table 1 shows a summary of the product types and average SPF values from product labels, measured UV absorption, and computer‐modeled results. Supporting Information provides a comprehensive data set, including details about the percentage of each active ingredient used in each product, comprehensive measured UV absorption laboratory results, computer‐modeled results, and final absorbance spectra.

TABLE 1.

Summary of 49 tested products and the magnitude of SPF protection. The labeled SPF is being used as an indicator of the US FDA in vivo SPF test results that are required of all products on the market

Products		Labeled SPF				Average SPF
	Total	<30	30‐49	50,50+	51‐110	Labeled SPF (% of measured in vitro SPF)	In vitro SPF (from measured UV absorption using ISO 24443:2012)	Modeled (% of measured in vitro SPF)
Mineral filter (ZnO)	11	0	6	6	0	40 (228%)	17.6	13.1 (74%)
Mineral filters (ZnO+TiO2)	7	0	2	4	2	48.6 (109%)	44.6	16.0 (36%)
Mineral filter (TiO2)	4	0	3	1	0	35 (93%)	37.7	20.8 (55%)
Mineral +organic	6	1	2	3	0	40.2 (198%)	20.3	18.6 (92%)
Organic filters	21	1	8	4	8	58 (195%)	29.8	26.6 (89%)

For products with ZnO, mineral +organic, and organic active ingredients, the average measured UV absorption‐derived SPF, calculated using the final absorbance spectra, was approximately half of the labeled SPF. The average computer‐modeled SPF for those products was similar to the measured UV absorption‐determined values.

On average, the labeled SPF for TiO₂ based sunscreens was similar to the measured UV absorption‐determined value but significantly lower than the SPF determined from simulations. These TiO₂‐based sunscreens, which had the greatest discrepancy between simulation and measured UV absorption‐derived SPF, are relatively uncommon in the US market. ³¹

Using the results from ISO 24443:2012 tests, we determined if the products would meet any of the three standards for broad‐spectrum: the current US FDA standard, which requires a critical wavelength of 370 nm or higher; the proposed US FDA standard, which required a ratio of UVA I/UV greater than 0.7; and the European Union requirement of UVAPF/SPF ratio greater than 1/3. Overall, 94% (48/51) of the products would be expected to pass the current US FDA test; 67% (34/51) expected to pass the UVA1/UV test; and 35% pass the EU requirement with which they were officially evaluated. The low percentage of US products passing the EU requirements is consistent with previous reports. ⁶⁰ ZnO‐based products were most likely to pass, as shown in Figure 2. Table S1 provides a summary by product type of the Boots star ratings as calculated from the final absorbance spectra using the ISO 24443:2012 method.

FIGURE 2.

Using the ISO 24443:2012 testing methodology the number of products expected to fail different UVA standards is compared. *The term expected is used here because the ISO 24443:2012 standard uses slightly different testing specifications than the US FDA standard which can results in some differences [Colour figure can be viewed at wileyonlinelibrary.com]

Of the twenty products tested using the method currently required by the US FDA to determine broad spectrum, 1978N‐0038, all passed. On average, the US FDA measured critical wavelength for these products was 1.5 nm higher than that measured using the ISO 24443:2012 test.

One ZnO and one ZnO+TiO₂ based product had a critical wavelength under 370 nm using the ISO 24443:2012 test, but neither was assessed using the US FDA test. One ZnO product tested outside the allowable range of allowable correction factor coefficients, with a measured UV absorption calculated SPF of 7 vs the labeled SPF 50. The US FDA has no requirement for correlation between the measured UV absorption and the labeled SPF, and this product would have likely passed the US FDA test. All products with a labeled SPF value over 60 failed the EU 1/3 ratio test, yet passed both the current and previously proposed US FDA tests.

While the ISO 24443:2012 test uses labeled SPF to calculate the ratio of UVA protection to SPF, the method normalizes the measured SPF to the labeled SPF using a term, “C.” This adjustment factor is used even though a standard SPF 15 is tested at the same time to ensure the magnitude of protection is within an acceptable range. The original intent of the correction factor coefficient was to account for small variability between laboratories, but as implemented, it allows for the measured UV absorption to deviate significantly from the SPF value on a product label. ⁶¹ Tables S2 and Table S3 provide an analysis of the range of acceptable “C” factor values as implemented within the ISO test. This calculation, using the “C” factor in reverse to determine the acceptable in vivo SPF values from an in vitro absorption spectra, finds that a product with an in vitro SPF of 23 could have a labeled SPF of 122. As originally published by Colipa, the hypothetical product would have been limited to a labeled SPF of 40. ⁶¹ It is noted that while the in vitro 24443:2012 method is not validated to provide an SPF value, the method does require an in vitro SPF to verifying that the “C” factor correlating the in vitro SPF and in vivo SPF is within a specific range. A future validated method for determining SPF in vitro would eliminate any discussion of “C” factor.

Here, we analyze the results without the correction coefficient to assess the ratio of the unadjusted UV absorption spectra to the labeled SPF. Figure 3 shows the ratio of the unadjusted UVAPF to the labeled SPF for all tested products. According to the final laboratory reports, 18 of 51 products passed the EU UVA test, but without the correction factor coefficient, only 9 of 51 would have passed. Only two of forty‐seven products with a labeled SPF over 30 had a measured UVAPF that was more than a third of the labeled SPF.

FIGURE 3.

Ratio of measured UVA protection factor (UVAPF) to the labeled SPF of tested US sunscreens. The UVAPF is calculated from in vitro testing and designed to correlate with persistent pigment darkening. The shaded region represents the area where the ratio of the UVAPF/labeled SPF is 1/3 or greater, as required for sunscreen products sold in the European Union [Colour figure can be viewed at wileyonlinelibrary.com]

Figure 4 shows the average UV protection factor for 49 products as a function of wavelength, with the shaded area representing a standard deviation. The UV protection factor is defined as 1 divided by the transmission. At a given wavelength, a transmission factor of 0.1 would equate to a tenfold reduction in incident UVR and a protection factor of 10. The average labeled SPF for the 49 products tested was 48, whereas the average measured UV absorption protection factor had a peak under 30 in the UVB portion of the spectra. The protection decreased to between 10 and 4 in the 365‐385 nm range, which represents the peak of UV‐radiation‐induced immune suppression. ¹¹ Even less protection is provided in between 385‐405 nm, which has been shown to cause DNA damage in both in vitro and in vivo studies. ¹⁴

FIGURE 4.

Average wavelength‐dependent UV protection factor of forty‐nine tested US sunscreens. The shaded area represents a standard deviation on the average UV protection factor. A straight line or uniform protection across the UV spectrum, including the entire UVA portion, has been proposed as the ideal metric ⁵¹ [Colour figure can be viewed at wileyonlinelibrary.com]

4. DISCUSSION

Measured UV absorption spectra are a direct metric of radiation reduction performance across the entire range of UVR. In contrast, current SPF testing is a measure of sunscreen performance that relies on a biological response, erythema, which is caused by exposure to just a small portion of the UV spectra. The ISO 24443:2012 method used here allows direct measurement of in vitro UV protection but is limited with respect to not being developed with validation data to correlate directly with in vivo SPF values. The limitation is partially ameliorated through the use of an SPF standard and the required calculation of a correlation coefficient between the in vitro SPF and the in vivo SPF values.

Our data demonstrated that computer‐modeled UV protection correlates well with the measured UV absorption for products without TiO₂. The modeled TiO₂ data significantly underestimated the in vitro measured and labeled SPF values. Outside than products with TiO2, the labeled SPF was significantly higher than what would be expected from the measured UV absorption data. Modeling efficacy based on the active ingredients has the drawback of being unable to account for ingredient dispersion differences and inactive SPF boosters that can increase the pathlength of radiation or inactive ingredients that attenuate UVR or reduce skin redness. Studies have raised concerns about the ability of sunscreen ingredients to act as anti‐inflammatory agents in vivo and reduce skin redness without reducing UVR and enabling increased time in the sun. ⁶² Other studies have found that these ingredients do not impact the measured SPF. ⁶³ Ending our reliance on in vivo testing, and instead using laboratory tests that directly measure UV protection, ⁵⁰ would dissuade manufacturers from adding ingredients solely to boost SPF values.

The use of measured UV absorption should be prioritized, because it offers a direct measure of UVR attenuation across the entire spectrum without exposing persons to high levels of UVR as done in SPF tests or persistent pigment darkening tests. The emission profile used in testing should also be more representative of typically outdoor solar exposures that present a larger ratio of UVA to UVB radiation. ³⁵

Current sunscreen products in the United States advertise SPF values not supported by direct measurement of UVR attenuation in a laboratory. The lack of concordance between the erythema observations on human skin and measured UV absorption exacerbates the poor quality of UVA protection in US products, which already lags behind that of the rest of the world. Ending the in vivo SPF testing of sunscreens along with increased market access of new UVA filters would aid public health efforts to reduce harmful UV exposure.

In 2007 and again in 2011, the US FDA refused to replace the human test with a measured UV absorption one over concerns that the substrates did not adequately mimic human skin and would not until data demonstrated equivalency. ²⁸ This argument is centered on the false assumption of erythema being the only endpoint of health concern. Sun protection products in the United States focus on erythema reduction, and numerical SPF value attributes most of interest for those with light color skin. However, that focus may limit the value of UV reduction products currently sold in the United States for people of color whose naturally occurring melanin reduces their comparative risk from sunburn, even though they may still benefit from the use of sunscreen. ⁶⁴ New sunscreen products that provide UVA protection equivalent to SPF protection are urgently needed in the United States market, to ensure health protection for all, especially for children and adolescents. ⁶⁵ Numerous methods in various stages of development and validation are available to replace the SPF test, including diffuse reflectance spectroscopy method in vitro SPF testing and in silico modeling. ⁵⁰ , ⁶⁶ Eliminating the emphasis on SPF values and human testing in favor of newly validated UV absorption testing ⁵⁰ should offer an improvement in public health benefits of sunscreen products.

CONFLICT OF INTEREST

Environmental Working Group is a nonprofit organization and the current employer of DQA, KR, CB, ES, AMT, and OVN and previous employer of SP‐G. and NL All authors declare no financial conflict of interest. Environmental Working Group evaluates and publicly ranks sunscreens based on the modeled efficacy and an evaluation of ingredient hazards. Environmental Working Group accepts sunscreen sample product donations for inclusion in annual fundraisers. In the past three years, the authors have also prepared and provided written and oral comments on behalf of the Environmental Working Group to the US Food and Drug Administration regarding the regulation of sunscreens.

AUTHOR CONTRIBUTIONS

David Andrews: conceptualization; methodology; formal analysis; writing—original draft; visualization; writing—review and editing (equal). Kali Rauhe: validation (supporting); visualization (supporting); writing—review and editing (equal). Carla Burns: methodology (supporting); visualization (supporting); validation (supporting); writing—review and editing (equal). Emily Spilman: validation; visualization (supporting); writing—review and editing (equal). Alexis Temkin: validation (supporting); visualization (supporting); writing—review and editing (equal). Sean Perrone‐Gray: methodology (supporting); visualization (supporting); writing—review and editing (equal). Olga Naidenko: visualization (supporting); writing—review and editing (equal). Nneka Leiba: methodology (supporting); visualization (supporting); writing—review and editing (equal). All authors have read and agreed to the published version of the manuscript.

STATEMENT ON ANY PRIOR PRESENTATION

This work has not been published previously and is not under consideration for publication elsewhere.

Supporting information

Supplementary Material

Supplementary Material

Andrews DQ, Rauhe K, Burns C, et al. Laboratory testing of sunscreens on the US market finds lower in vitro SPF values than on labels and even less UVA protection. Photodermatol Photoimmunol Photomed. 2022;38:224–232. 10.1111/phpp.12738

DATA AVAILABILITY STATEMENT

The data that supports the findings of this study are available in the supplementary material of this article