Reliability of tanning behaviors and sunburns history among sororities and fraternities students

Abstract

The need for both valid and reliable measurement is crucial for the assessment of sun behavior. We used test-retest reliability of a self-administered survey to examine the reliability of reporting on behavior relevant to artificial ultraviolet tanning, sunburns and sun sensitivity among sorority and fraternity-affiliated university students. Subjects were members of sororities and fraternities who participated in an initial survey and skin examination. Students were prompted by specific recall queries. The students completed a second survey 2-4 weeks later. High reliability on test-retest for questions evaluating the number of artificial UV tanning sessions and the number of sunburns during specific time periods was found. Moderate reliability for measures reporting the use of self-tanning creams, can be largely explained by only one third of the students reporting they had ever used self-tanning creams. Overall this study suggests that members of sororities and fraternities report lifetime artificial ultraviolet tanning consistently when required to recall time period specific exposures prior to estimating their lifetime exposure.

Environmental ultraviolet radiation (UVR) has been linked to skin cancer rates over the past few decades (1, 2). This may reflect changes in population tanning habits. Tanning occurs through UVR which may be modified by an individual’s skin color or their skin’s reaction to UVR. The factors that link skin cancer and UVR clearly include intermittent intense sun exposures, typically measured as sunburns, but also are likely to include cumulative exposure to sunlight. Total UVR is also increased by artificial ultraviolet (UV) from indoor tanning. For both sun exposure and artificial UV tanning, the wide variation in ways to measure and summarize exposure may lead to lack of comparability across studies for either measure.

The potential for misclassification of sun exposure or UVR obtained using different questions that are summarized or pooled across a lifetime history is high. Few prior studies have reported the reliability of the measurement of cumulative sun exposure or the measurement of artificial UV tanning. For studies involving sunburn histories, reliability studies with over a year between comparison measures have reported Kappas ranging from 0.2-0.7 among skin cancer cases and 0.4-0.6 among controls or population based samples (3-6). Higher agreement (Kappas from 0.6-0.8) was found in studies examining consistency between measures 2-11 weeks apart (7-10), but only one of these studies examined the reliability sunburn histories prior to age 18. Measures that produce more consistent results are needed. The purpose of this study was to examine the consistency of recalled tanning behaviors among sorority and fraternity students in the Midwest.

MATERIALS AND METHODS

We recruited university students affiliated with sororities and fraternities (the so-called “Greek” houses). Greek houses with more than 10 members were asked to participate in an educational session on skin care but unrelated to tanning and sun exposure and were surveyed during the spring semester of 2004. Eight of 28 Greek houses (sororities and fraternities) listed in the campus telephone book and registered with 10 or more members at the University of Iowa were contacted and agreed to participate. Within the contacted Greek houses, we recruited 163 students attending their monthly house meeting. Among these students, 63 were randomly chosen to complete a second survey 2-4 weeks after taking the initial survey and 37 provided completed surveys. A comparison of the 37 students who completed both surveys with all 163 students who completed the first survey by demographic characteristics and sun sensitivity factors was conducted using a chi-square test or Fisher’s exact test.

Intramethod reliability, or consistency of our measurements on the same subjects (Armstrong, White et al. 1992), was examined by a test-retest of the original questionnaire with the second survey. Both surveys were identical. The resulting Kappa values were used in our assessment of the reliability of these categorical measures (11). Weighted Kappas were used for ordered categories, and simple Kappas for yes/no categories (11). Kappas were calculated using SAS software with Cicchetti-Allison weights used (12, 13).

Under the parallel tests assumption, the validity of factors related to tanning habits were estimated as the square-root of the reliability coefficient (11). In cases where the assumption of uncorrelated errors in the two measurements did not hold, the square-root of reliability provided an upper-limit of the validity of the measure (11). We estimated validity based on the reliability coefficient in all subjects and conservatively present this estimate as an upper-limit of validity of tanning habit measures in this study.

RESULTS

Participants who completed both surveys were representative of students in the overall study. Table 1 compares the demographic characteristics and sun sensitivity factors for the resurveyed students to those of the overall study population that participated in the original survey. In both cases, the study population was predominantly white and female. Neither chi-square tests nor Fisher’s exact test detected significant differences between the students in this study and the main population of surveyed students. Over 85 percent of the subjects had ever used artificial UV tanning devices, with 73 percent of the 37 subjects (84 percent of ever users) reporting artificial UV tanning use more than 20 times in their lifetime. Very similar rates of artificial UV tanning were seen in the full population.

TABLE 1.

Distribution of demographic and sun sensitivity factors comparing the study population to those randomly selected to be resurveyed

Characteristics		Overall survey (n=167)	Reliability survey (n=37)	Chi Square
Gender
	Male	27.5%	18.9%
	Female	72.5%	81.1%	p=0.28 Race
Race
	White	98.2%	94.6%	p=0.23*
	Non-White	1.8%	5.4%
Year in College
	Freshman	28.9%	27.0%
	Sophomore	50.0%	54.0%
	Junior	9.6%	5.4%
	Senior	11.4%	13.5%	p=0.86*
Sun Sensitivity Factors
Eye color
	Blue	34.3%	29.7%
	Green	13.9%	24.3%
	Hazel	19.3%	18.9%
	Brown	32.5%	27.0%	p=0.46
Natural hair color
	Blond	18.1%	13.5%
	Red/Blond	1.8%	0%
	Red/Auburn	2.4%	2.7%
	Light Brown	36.1%	35.1%
	Dark Brown	38.0%	43.2%
	Black	3.6%	5.4%	p=0.93*
Untanned skin color
	Fair	50.9%	64.9%
	Medium	46.1%	35.1%
	Dark	3.0%	0%	p=0.26*

^*Two-sided p-value from Fisher’s exact test.

Test-retest reliability of tanning behavior was compared using both percent agreement and Kappa which adjusts for agreement by chance (Table 2). Because the subjects were college students, the questionnaire focused on the periods “before high school”, “during high school”, “after high school” and “during the past year”. For tanning bed use, reliability ranged from a Kappa of 0.63 for use prior to high school to 0.97 for lifetime exposure. Our strategy of first inquiring about specific time periods was intended to stimulate the subject’s recall regarding prior use before asking for an estimate of their lifetime use. No gender specific differences in Kappa values were found. For self-tanning creams, only one-third of the subjects (12 of 37) reported any use. Among students that reported use of tanning creams, use in the past of tanning cream was infrequent. As a result of the lack of infrequent use, lower reliability was seen. The lowest constancy was seen for self-tanning cream use before high school (Kappa=0.45) with only three subjects reporting such use. Reliability coefficients ranged from 0.63 to 0.68 for the other three time periods (Table 2). The highest reliability was seen for their estimated lifetime use of self-tanning creams.

TABLE 2.

Test–retest reliability of sun exposure and tanning habits

Component	# of Categories (n=37)	Percent Agreement	Kappa* (lower limit)	Upper limit of validity
Artificial UV tanning:
Have you ever used a sunlamp, tanning lamp or sunbed in order to get a tan?	Yes/no	100%	1.00 (1.00, 1.00)	1.00
How many times would you estimate that you have ever used sunlamps, tanning lamps or sunbeds… †	6 categories each
… before high school (under age 14)		78%	0.63 (0.43, 0.84)	0.79
… during your high school years (ages 14-18)		65%	0.83 (0.72, 0.93)	0.91
… after age 18 (n=35)		69%	0.83 (0.72, 0.93)	0.91
… during this past year		62%	0.79 (0.68, 0.91)	0.89
… between New Years 2004 & Spring Break 2004		54%	0.74 (0.63, 0.86)	0.86
Based on your answer above, … in your lifetime?		95%	0.97 (0.93, 1.00)	0.98
Sunless tanning Creams:
Have you ever used self-tanning creams in order to get a tan?	Yes/no	81%	0.61 (0.36, 0.87)	0.78
How often did you use tanning creams… †	6 categories each
… before high school (under age 14) ‡		68%	0.45 (0.22, 0.67)	67%
… during your high school years (ages 14-18)		65%	0.65 (0.45, 0.84)	81%
… after age 18		62%	0.63 (0.45, 0.80)	79%
… during your lifetime		70%	0.68 (0.48, 0.88)	82%
Sunburns:
How many sunburns did you have so severe that they produced blisters or pain lasting 2 or more days… §	4 categories each
… before high school (under age 14)	(n=33)	82%	0.75 (0.56, 0.94)	0.87
… during your high school years (ages 14-18)	(n=36)	72%	0.62 (0.40, 0.85)	0.79
… after age 18	(n=35)	74%	0.52 (0.28, 0.76)	0.72
… during this past year		89%	0.73 (0.47, 0.98)	0.85
… during Spring Break 2004		95%	0.80 (0.54, 1.00)	0.90
Sun Sensitivity:
Tendency to burn:
When your skin is exposed to strong sunlight for 30 minutes for the first time each summer with no protection, does it get… a severe and painful sunburn, moderate sunburn, mild sunburn, burn first then tan afterwards, or no sunburn?	5 categories	54%	0.51 (0.29, 0.72)	0.71
Inability to tan
After repeated and prolonged exposure to the sun, does your skin become… deeply tanned, moderately tanned, mildly tanned, or have no tan?	4 categories	76%	0.59 (0.35, 0.83)	0.77
Skin Color
How would you describe your untanned skin color on your upper inner arm? (fair, medium or dark)	3 categories	89%	0.78 (0.59, 0.97)	0.88

^*This is a weighted Kappa for ordered categories, simple Kappa for dichotomous factors.

^†Response categories of 0, 1-5, 6-10, 11-15, 16-20, >20 times

^‡Less than 30% of study participants used sunless tanning creams prior to high school and among those participating in the test-retest, subjects had only used them 5 times or less prior to high school.

^§Response categories of 0, 1-2, 3-5, 6+ sunburns

The reliability of reporting past sunburns was also relatively high, with Kappa values ranging from 0.52 to 0.80. Consistency in the reporting of sun sensitivity was found to be variable (Table 2). The lowest reliability was seen for the question on the subject’s tendency to burn (Kappa=0.51) with highest consistency for self-reported skin color (Kappa=0.78). The reporting of eye color (Kappa =0.96) and hair color (Kappa=0.88) was found to be highly consistent. As expected, we found perfect agreement on questions regarding gender, race and year in college (not shown).

We estimated an upper limit of validity based on our reliability scores. Since some reliability subjects may remember what they previously reported, the random errors of the two measurements are likely correlated. Thus the estimates of validity reported in Table 2 should be considered an upper limit of validity. The upper limits of the validity coefficients, which range between 0.79-0.98 for frequency artificial UV tanning, 0.67-0.82 for self-tanning creams and 0.72-0.90 for history of sunburns, show that the questionnaire items have the potential to accurately estimate the intended parameters.

DISCUSSION

The potential for misclassification of sun exposure or determining amount of UVR obtained using different questions that are summarized or pooled across a lifetime history is high. We examined younger individuals, at a time closer to relevant exposures, and examined the consistency with which they answer questions regarding the amount of tanning and number of sunburns. Few studies have reported the reliability of their data. Overall, the Kappa values for each factor in our study showed reasonably consistency measures of self report of behaviors at two points in time. Moderate to high Kappa scores were seen for self reported recalled artificial UV tanning and sunburns with specific proximal time periods and overall lifetime periods of time. Our findings demonstrate that reliability studies nested within case-control studies provide specific estimates of the consistency of measurement in the study population, and that the use of memory cues helps with lifetime estimates of exposures.

Prior reliability reports for artificial UV tanning have been for “ever use” (8, 14), “how often do you use a sunbed” (7), number of times per year (never, 1-10 times, 10+ times) (3), to an open ended “best estimate of how many times you have tanned indoors in the last 12 months (15), with Kappas ranging from 0.7 to 1.0 as summarized in Table 3. Similarly, we found perfect agreement for ever use, and Kappa values ranging from 0.6 to 0.8 for the number of artificial UV tanning sessions during specific periods. The relatively low Kappa of 0.6 for artificial UV tanning prior to high school was due, in part, to less than 25 percent of subjects reporting such exposure and most reporting 10 times or less. The Kappa values for subsequent time periods were each about 0.8. Once subjects had provided estimates of the number of artificial UV tanning sessions covering the periods beginning before high school and ending at the past year (and prior to spring break), they were able to provide a reliable overall estimate of their lifetime number of artificial UV tanning sessions (Kappa ~ 1.0). Thus, using time period specific recall followed by asking the subject to estimate their total lifetime exposure resulted in high Kappa values for lifetime estimates. Similar memory cues have been used in a variety of studies to improve recall. As examples, calendars have been used to effectively aid recall (11, 16). And, more specifically, as a means of improving recall of birth control methods one study (17) used a month-by-year matrix calendar on which marriages and cohabitations were first recorded, followed by pregnancies, and then by birth control methods. We have used this “memory recall” theory to inquire about different time periods of exposure then we asked subjects to estimate their lifetime exposure. Our approach of first requesting exposure information prior to high school, during high school, after age 18, and during the past year then an estimate of lifetime exposure appears to have assisted in the recall of lifetime exposure. Thus, we saw the highest reliability of lifetime recall of artificial UV tanning exposure. Although our questionnaire did not attempt to ask subjects about lifetime sunburns, future studies may do so using an approach similar to that used for the artificial tanning in this study.

TABLE 3.

Intra-method reliability (typically test-retest) for sunburns and artificial UV use

Measurement of exposure	Author & year	Population (n)	age/ location	Interval between tests	Reliability*
Artificial UV tanning
Ever (Yes, no)  “Have you ever used a sunlamp, tanning lamp or sunbed  in order to get a tan?”	Beane Freeman, 2005 (8)	Melanoma cases and colorectal controls; Self-administered questionnaire; Caucasian only. (n=76)	age 40+ Iowa, USA	2-4 weeks	Cases: Controls: Both:	1.00 0.71 0.83
Ever (Yes, no)  Indoor tanning use	Pichon, 2005 (14)	Female college students (convenience samples); Telephone interview. (n=32)	age 18-19 USA	1 week	Sample	1.00
Frequency (Often, rather often, seldom, stopped using, never  used)  “How often do you use a sunbed?”	Branstrom, 2002 (7)	Swedish nurses in 2000; Self administered questionnaire. (n=52)	age 24-59 Sweden	3 weeks	Sample	0.82
Ever (yes, no), # times (Never, 1-10 times, > 10 times)  “Have you ever used or do you ever use sunbeds? If yes,  number of times per year?	Westerdahl, 1996 (3)	Swedish women; Self administered questionnaire. (n=670)	age 29-68 in 1994 Sweden	1-3 years	Sample	0.73
# times in last 12 months (open-ended)  Estimate of frequency of tanning salon use in general  during the past year	Hillhouse, 2000 (15)	Male & female college students; Selfadministered questionnaire. (n=197)	college students SE city, USA	2 weeks	Sample	г=0.95
Childhood Sunburns
# before age 14 (0, 1–2, 3–5, 6+, don’t know)  “# of sunburns so severe to produce blisters or pain lasting  2 days or longer”	Beane Freeman, 2005 (8)	Melanoma cases and colorectal controls; Self-administered questionnaire; Caucasian only. (n=76)	age 40+ Iowa, USA	2-4 weeks	Cases: Controls Both:	0.77 0.68 0.73
Ever before age 16 (yes, no)  First sunburn before 16 years of age	Rosso, 2002 (5)	Random sample from basal cell or squamous cell carcinoma cases and population controls. (n=236)	age 20-70 Italy, Spain, France	18-26 months	Cases: Controls: Controls:Both:	0.65 0.69 0.67
Adolescent Sunburns
# between age 14-18 (0, 1–2, 3–5, 6+, don’t know)  # of sunburns so severe to produce blisters or pain lasting  2 days or longer	Beane Freeman, 2005 (8)	Melanoma cases and colorectal controls; Self-administered questionnaire; Caucasian only. (n=76)	age 40+ Iowa, USA	2-4 weeks	Cases: Controls: Both:	0.67 0.67 0.67
Adult or recent Sunburns
# after age 18 (0, 1–2, 3–5, 6+, don’t know)  # of sunburns so severe to produce blisters or pain lasting  2 days or longer	Beane Freeman, 2005 (8)	Melanoma cases and colorectal controls; Self-administered questionnaire; Caucasian only. (n=76)	age 40+ Iowa, USA	2-4 weeks	Cases: Controls: Both:	0.73 0.78 0.69
Ever, # after age 19 (If yes, 0, 1-5, >5)  “Have you ever been sunburned causing erythema and  pain for a few days? If yes, how many times after age 19?”	Westerdahl, 1996 (3)	Swedish women; Self administered questionnaire. (n=670)	age 29-68 in 1994 Sweden	1-3 years	Sample	0.54
# in past year (> 10 times, 6-10 times, 3-5 times, 1-2 times,  none, haven’t sunbathed)  “How many times have you been sunburnt (redness and  smarting pain) during the past year in Sweden?”	Branstrom, 2002 (7)	Swedish nurses in 2000; Self administered questionnaire. (n=52)	age 24-59 Sweden	3 weeks	Sample	0.52
Lifetime Sunburns
Ever (Yes, No)  “Have you ever been sunburned severely enough to  cause pain for 2 or more days or blisters?”	Berwick, 1995 (4)	Random sample of melanoma cases and population controls; Caucasian only. (n=82)	age 20-87 Connecticut (USA)	1-3 years	Ever: Cases: Controls:	0.37 0.57
# during lifetime (0,1, 2, 3, 4, 5, 6)  “How many times have you been sunburned severely  enough to cause pain for 2 or more days or blisters?”					# sunburns: Cases: Controls:	r=0.69 r=0.59
# during lifetime (0, 1–2, 3–5, 6–10, >10)  Sunburn history, scored	Glanz, 2003 (9)	Waiting room sample from primary care practices in March 1999; Excluded those being treated for skin cancer; 72-77% Caucasian subjects. (n=165)	age 20-65 Hawaii & New York (USA)	4 weeks	Sample	0.77
# per year (0, 1-2, 3+)  “When you sunbathe do you get sunburned? If yes, how  many times per year?”	Westerdahl, 1996 (3)	Swedish women; Self administered questionnaire. (n=670)	age 29-68 in 1994 Sweden	1-3 years	Sample	0.52
# during lifetime (continuous)  # of Sunburns	Rosso, 2002 (5)	Random sample from basal cell or squamous cell carcinoma cases and population controls. (n=236)	age 20-70 Italy, Spain, France	18-26 months	Cases: Controls: Both:	г=0.19 г=0.39 г=0.25
# during lifetime (categorical: 0, 1, 2, 3+)  # of Sunburns					Cases: Controls: Both:	0.42 0.44 0.43
# during lifetime (never; once;2-5; 6-10, >10)  Lifetime burns	Van der mei, 2006 (10)	Multiple sclerosis cases and community controls; Interviewer-administered questionnaire. (n=104)	age <60 Tasmania	11 weeks	Cases: Controls:	0.73 0.61
Lifetime frequency (every year, sometimes but not every  year, a few times, never, I have never sunbathed)  “How often have you been sunburnt (redness and  smarting pain) in Sweden during your whole lifetime?”	Branstrom, 2002 (7)	Swedish nurses in 2000; Self administered questionnaire. (n=52)	age 24-59 Sweden	3 weeks	Sample	0.58
# during lifetime (0, 1-2, 3-10, 11+)  # of lifetime painful sunburns  # during lifetime (0, painful burn only, 1-2, 3+)  Lifetime blistering sunburns	English, 1998 (6)	Case control study of non-melanocytic skin cancer; Subjects were born in Australia, with European ancestry but not southern European ancestors. (n=190)	age 40-64 Western Australia	5 years (1988 & 1993)	# painful burns: Controls: 0.46 # blistering burns: Controls: 0.47

^*Cohen’s Kappa (K) unless otherwise specified as a correlation coefficient (r)

Effects of sunless tanning creams have not been extensively studied to date. One previous study reported high consistency for ever use of self-tanning cream both in melanoma patients (Kappa=1.0) and colorectal cancer controls (Kappa=0.9) (8). We found high agreement with a moderate Kappa for ever use. Since less than a third of subjects had used self-tanning creams, the remaining Kappa values were only moderately high. The lower reliability might also suggest that better wording of questions on sunless tanning are needed. The reliability of sunless tanning cream use might be increased by adding language that clearly distinguishes artificial tanning using these products from those involving UV lamps particularly in older subjects. To better understand how subjects view these questions, studies using cognitive interviewing are needed (18). Such studies ask subjects about how they interpret questions and make modifications accordingly.

Moderate to high consistency in reporting was seen for sunburn histories. Several prior studies have reported on ever being sunburned, but few have examined number of sunburns and many do not define “severe” sunburns. Among studies examining number of sunburns, two defined sunburns as “blistering or pain lasting 2 days or more” (4, 8) reported more consistently higher Kappas (0.6 to 0.8) than those (3, 7) that defined a sunburn as redness or pain (Kappas of 0.5) or those (4, 5, 9, 10) that did not state their definition (Kappas from 0.4 to 0.8). A Western Australia study (6) looked at lifetime painful burns and lifetime blistering burns but found similar reliability for both (Kappa=0.5). Studies reporting lower reliability of measures of sunburns tended to have over a year between comparison measures, while studies with less than three months between measures reported higher consistency (Table 3). Only one prior study of sunburn history for specific time periods has reported on their reliability providing Kappa values ranging from 0.7 to 0.8 suggesting that the reliability is moderately high (8). Our study found Kappa values ranging from 0.5 to 0.8 for number of sunburns which is consistent with the prior assessment. Since several reviews of melanoma risk and sunburns (19-21), pooled odds ratios for “ever sunburned” with those for the highest categories of number of sunburns or the highest severity of sunburns, this advocates that more consistent measures of sunburns are needed. Using memory cues of time period specific exposures may increase the reliability of recalling lifetime exposures.

Reliability of self-reported sun sensitivity factors is less well understood and seems to vary by population. Most sun sensitivity measures seem to have moderate consistency, with kappas ranging from 0.4 to 0.8, among different populations for freckling, hair color, skin color, tendency to burn, inability to tan, and self-reported skin type (3, 4, 8, 9, 22). We found similar consistency for skin color and inability to tan. Tendency to burn was less reliable (Kappa=0.5) in our population of sorority and fraternity students who report high levels of tanning. The lower reliability may reflect difficulty in answering questions about tendency to burn after chronic tanning, among students with a desire to have darker skin.

As artificial tanning increases in the population and more studies examine artificial UV tanning, reliable measures are needed. The open-ended question about artificial UV tanning in the past year by Hillhouse & Turrisi (15), provides high reliability of recent exposure. While open ended questions about recent exposure may be easy to answer, it is unclear if lifetime exposure can be answered in a similar manor. With current high reports of artificial UV tanning among 18-30 year olds, accurately measuring lifetime use in populations with high exposure levels may get difficult as such populations age. We found that using time period specific recall followed by asking the subject to estimate their total lifetime exposure resulted in high reproducibility of lifetime estimates in a population with high levels of artificial UV tanning exposure.

In summary, we found that first requesting time period specific recall and then estimates of lifetime exposure resulted in high reliability of recall of lifetime exposures, particularly with respect to the reporting of lifetime artificial UV tanning sessions.

Abbreviations

UV

ultraviolet

UVR

ultraviolet radiation