Refinement of Measures to Assess Psychosocial Constructs Associated with Skin Cancer Risk and Protective Behaviors of Young Adults

Abstract

The study’s purpose was to select/refine measures assessing psychosocial constructs associated with skin cancer risk/protective behaviors. Cognitive interviewing was conducted with twenty participants locally, and a survey was conducted with 965 adults aged 18–25 years at moderate to high risk of developing skin cancer, recruited nationally online. Psychosocial measures assessed variables from the Integrative Model of Behavior Prediction. As a result of expert review and cognitive interviewing, items were removed, added, and/or made simpler, more personal, consistent, and less ambiguous. A factor analysis resulted in 14 scales and adequate model fit. Internal reliability and test-retest reliability was acceptable to good. Correlations among the psychosocial and behavioral variables were generally significant and in expected directions, demonstrating convergent validity. We have refined measures that assess important psychosocial constructs associated with skin cancer-related behaviors, that research participants can understand and complete successfully, and that are reliable and demonstrate evidence for validity.

Introduction

Invasive melanoma is the second most diagnosed cancer among young adults (Bleyer & Barr, 2009). It is common for young adults to expose themselves to large amounts of ultraviolet radiation (UV) and protect their skin poorly (Buller et al., 2011; Coups, Manne, & Heckman, 2008; Heckman, Coups, & Manne, 2008; Stanton, Janda, Baade, & Anderson, 2004). Thus, young adulthood is an important window for skin cancer risk reduction interventions. However, young adults tend to be resistant to public health recommendations, and they also tend to be experimenters and risk-takers highly influenced by peers (Andrews, Tildesley, Hops, & Li, 2002; Freedman, Nelson, & Feldman, 2012; Galliher & Kerpelman, 2012). In order to develop interventions to address this issue appropriately, it is important to have valid and reliable measures to assess skin cancer prevention-related behaviors and their psychosocial correlates appropriate for the unique developmental issues of young adults.

A prior project that involved expert consensus and cognitive interviewing produced items assessing UV exposure and protective behaviors such as sun exposure, indoor tanning, sunless tanning, skin examination, and skin protection (Glanz et al., 2008; Lazovich et al., 2008). To build upon the prior work, we focused on measures of psychosocial constructs from the Integrative Model of Behavior Prediction (IM)(Fishbein, Hennessy, Yzer, & Douglas, 2003) that have been found to be associated with these behaviors (Arthey & Clarke, 1995; Holman & Watson, 2013; Keeney, McKenna, Fleming, & McIlfatrick, 2009; Tripp, Vernon, Gritz, Diamond, & Mullen, 2013). The IM includes background constructs such as demographics and knowledge; psychosocial constructs such as behavioral beliefs and outcome evaluations, normative beliefs and perceived normative pressure, self-efficacy, perceived behavioral control, and behavioral intentions; as well as behavior. Jackson and Aiken (Jackson & Aiken, 2000) had conducted psychometric testing on several measures of these constructs previously with white female psychology students. However, newer measures have become available since then and have not yet been thoroughly psychometrically evaluated in a national sample of women and men. It is important to assess psychosocial constructs in addition to behaviors, because these factors can influence UV exposure and protective behaviors and “mediate” the effects of skin prevention interventions.

The current paper describes the refinement of measures that assess constructs from the IM for use with young adults. We expected to demonstrate adequate internal and test-retest reliability as well as convergent validity. Regarding validity, we expected pro-exposure variables to be significantly positively correlated with one another, pro-protection variables to be positively correlated with one another, and pro-exposure variables to be negatively correlated with pro-protection variables. Use of appropriate, reliable, and valid psychosocial measures across studies and samples may build consensus in measures and ultimately aid in better understanding and modifying skin cancer risk and protective behaviors.

Materials and Methods

Overview

The process of measure refinement included expert review, cognitive interviewing, and psychometric evaluation.

Expert Group Meetings – Part 1

The authors, who include experts in skin cancer prevention and detection, psychology, assessment, health literacy, and the young adult population selected measures assessing psychosocial constructs associated with UV protection and exposure behavior. With regard to UV protection and exposure, the constructs assessed included knowledge, behavioral beliefs and outcome evaluations, normative beliefs and perceived normative pressure, self-efficacy, perceived behavioral control, and behavioral intentions. The team met biweekly over the course of approximately nine months and revised items and scales to improve their focus, clarity, and simplicity based on best practices in survey design, psychometrics, and health literacy (Berrigan et al., 2010; Dillman, Smyth, & Christian, 2008; Fortune-Greeley et al., 2009; Glanz et al., 2008; Patrick et al., 2011; Pinard, Uvena, Quam, Smith, & Yarvoch, 2015; Subar et al., 1995; G.B. Willis, 2005; G. B. Willis & Artino, 2013; Wolfe, Frongillo, & Cassano, 2001).

Cognitive Interviewing

Cognitive interviewing is commonly used to help improve comprehension of items and enhance surveys by identifying cognitive problems that may occur when respondents complete surveys (G.B. Willis, 2005). Young adults aged 18–25 years from a Northeastern metropolitan area were invited to participate in this portion of the study. Email ads were sent to employees and trainees at a cancer center and an affiliated general hospital, flyers were posted in the local community, and an announcement was posted to Craigslist.com. Interested respondents were directed to a website, in which they completed items from the Brief Skin Cancer Risk Assessment Tool (BRAT) (Glanz et al., 2003). Twenty young adults at moderate or high risk of skin cancer based on the BRAT attended an in-person session. Sixty percent of the sample was female, and all of the participants except one were White.

At the beginning of each individual session, participants signed informed consent and HIPAA forms, and the purpose of the session was explained. A cognitive interviewing protocol was developed and utilized, based on recommendations by Willis (G.B. Willis, 2005). Participants were asked to either complete the entire online survey from start to finish as they normally would and to notify the interviewer of any questions, problems, or confusion OR “think aloud” while answering selected items on paper that the authors had newly developed or been concerned about, with interviewers following up with specific probes to identify problems with reading, clarity, memory retrieval, sensitivity, consistency, or some other problem (G.B. Willis, 2005). Half the sample completed each task rather than doing both due to the length of the questionnaire and potential participant burden. Sessions typically took about 30 minutes to complete, and participants were compensated for their time. All sessions were audio recorded, and the interviewer took notes. The session leader compiled a report based on notes and the recording from each session.

Expert Group Meetings – Part 2

The expert group reconvened to discuss the report and potential changes to be made to survey items. When more than one participant had made a suggestion, the expert group was more likely to make the suggested change. Revisions to items themselves, instructions, item formatting, and response options were made, yielding a final survey (see the eMeasures section). For the purpose of this paper and future studies that will utilize cognitive interviewing, we identified problems with each item that was revised, and a coding scheme for these revisions was then developed and utilized, as per recommendation by Willis (G.B. Willis, 2005). Item revisions were coded by two of the authors, with disagreements resolved by a third author.

Revisions were made to most of the pre-existing items by the expert group and as a result of cognitive interviewing. Efforts were made to keep the instructions, item stems, and response options as simple and similar as possible across the measures in order to reduce respondent confusion (Berrigan et al., 2010; Dillman et al., 2008; Fortune-Greeley et al., 2009; Patrick et al., 2011; Subar et al., 1995; G.B. Willis, 2005; G. B. Willis & Artino, 2013; Wolfe et al., 2001). For example, we chose to use a 5-point Likert-type scale for many of the items to make them consistent with one another and since an odd (rather than even) number of response options is recommended (Krosnick & Presser, 2010). The following types of revisions were made: retained original item but made revisions; broke original item into multiple new items (e.g., an item addressing wearing protective clothing was broken into items designed to assess wearing sunglasses, long pants, and a shirt that covers the arms individually); added a new item that was deemed missing from or not addressed in pre-existing measures to our knowledge (e.g., an item that addressed tanning in order to feel the warmth of the sun); removed original item due to redundancy, lack of age-appropriateness, or difficulty. When items were retained but revised, the following types of revisions were coded: clarity and specificity (e.g., adding more information to make items more clear and/or specific), made language simpler (e.g., shortened items to make them more readable), made language more relatable (i.e., wrote items consistently in first person), and other (e.g., made items more appropriate for the age group rather than for children or older adults). See eTable 2 for an example of each type of change.

Psychometric Evaluation of the Revised Measures

Psychometric qualities of the scales were assessed in a separate national sample of adults enrolled in a randomized controlled trial of an online skin cancer prevention intervention. To be eligible for the study, participants had to be aged 18–25 years and report being at moderate to high risk of developing skin cancer as assessed by the BRAT(Glanz et al., 2003). Participants were recruited online by Survey Sampling International (SSI) using their US consumer opinion panel and partnerships with other panels and online communities. SSI panelists were exposed to brief web ads about the study from which they could click to link to the study website. Using automated online enrollment, 5,015 people were screened, 2,384 were eligible as described above, and 1,234 provided consent and submitted surveys.

Prior to the psychometric analyses, quality metrics for use with online data (e.g., unusually short or long survey completion time) were evaluated using methods recommended in the literature (Bowen, Daniel, Williams, & Baird, 2008; Meade & Craig, 2012). Based on a latent class analysis (Linzer & Lewis, 2011, 2013) of these metrics, poor quality surveys were discarded, and the high quality surveys from 965 participants was included in subsequent analyses.

We conducted a factor analysis (Taherdoost, Sahibuddin, & Jalaliyoon, 2014) including 103 items. We calculated the correlation matrix using all pairwise complete observations. The number of factors was chosen using the Horn’s parallel analysis from the nScree procedure in the R package “nFactors” (Raiche, 2010). The factor analysis was fit using maximum likelihood and a varimax rotation using the factor analysis procedure in the R package “psych” (Revelle, 2015). Varimax rotation was selected because it produces simpler, more easily interpretable, and more generalizable factors. Because of the large number of anticipated factors (scales), it was decided that finding factors with few highly related variables was more important than maximizing fit (Rennie, 1997). The criterion for an item fitting on a scale was 0.50 (Matsunaga, 2010). This resulted in the loss of the perceived control scale. Factors with two items were not retained due to their anticipated low reliability and validity.

Measures

Psychosocial Measures

The following measures of psychosocial aspects of UV protection and exposure are based on the items derived from the expert review and cognitive interviewing and scales derived from the factor analysis. Items were averaged to produce a total scale score, unless otherwise noted. Higher scores on any scale indicate higher levels of the variable. See eTable 1 for a summary of the original measures.

Table 1.

Demographics of the Psychometric Testing Sample (N = 965; USA, 2014).

Variable	n (%)
Age [M(SD)]	21.8 (2.2)
Female Sex	637 (66.1)
Race
White	825 (85.7)
Non-White	63 (6.5)
Other/mixed	75 (7.8)
Hispanic	92 (9.6)
Skin color
Very fair	263 (27.3)
Fair	570 (59.1)
Olive	119 (12.3)
Light brown to black	13 (1.3)
Family history of skin cancer	339 (35.2)
Education
HS/GED or less	411 (42.7)
Partial college/Associates degree/Vocational training	338 (35.1)
College graduate	181 (18.8)
Graduate degree or professional training	32 (3.3)
Hours worked per week
0	272 (28.2)
Part-time	517 (53.6)
≥ 40	175 (18.2)
Ability to live on income
Not hard at all	179 (19.2)
Somewhat hard	440 (47.2)
Hard to extremely hard	314 (33.6)
Receives public assistance	173 (18.8)
US geographical region
South	354 (36.7)
West	253 (26.3)
Midwest	184 (19.1)
Northeast	172 (17.8)

Knowledge about skin cancer and its risk factors was measured using items adapted from Buller and colleagues (Buller et al., 2006), Irwin and colleagues (Irwin, Mauriello, Hemminger, Pappert, & Kimball, 2007), and one investigator-created item (“Even if a sunscreen says it’s “water-resistant,” you still need to re-apply after swimming.”) Response options for each item were true, false, and not sure with each item scored as correct or not. This produced a proportion correct ranging from 0 to 1.

Behavioral Beliefs and Outcome Evaluations

Perceived benefits of UV exposure. Items adapted from Ingledew and colleagues (Ingledew, Ferguson, & Markland, 2010) was used to construct a measure of perceived benefits of/reasons for UV exposure. The factor analysis supported Ingledew and colleagues’ (Ingledew et al., 2010) subscales of appearance, health/well-being, and social benefits for UV exposure. Perceived Benefits of Skin Protection. Items adapted from Ingledew and colleagues (Ingledew et al., 2010) and the photoaging subscale of the Physical Appearance Reasons for Tanning Scale by Cafri and colleagues (Cafri et al., 2006) comprised the perceived benefits of skin protection scale. Perceived barriers to UV exposure. Items adapted from the photoaging subscale of the Physical Appearance Reasons for Tanning Scale by Cafri and colleagues (Cafri et al., 2006) and two investigator-created items were used as a measure of perceived barriers to or disadvantages of UV exposure (“I don’t tan as much as I would like because it would give me age spots/make my skin look leathery.”). Perceived susceptibility. Perceived susceptibility to skin damage was assessed using two items adapted from Gibbons and colleagues (Gibbons, Gerrard, Lane, Mahler, & Kulik, 2005) and one investigator-created item (“I already have some long-term damage from the sun or tanning.”)

Normative beliefs and perceived normative pressure

Several sets of items were used to construct three scales related to normative beliefs and perceived normative pressure associated with UV exposure subscale. Items adapted from Jackson and Aiken (Jackson & Aiken, 2000, 2006) and Hillhouse and colleagues (Hillhouse, Turrisi, & Kastner, 2000) were included in order to assess normative beliefs associated with the views of one’s immediate social circle. Items adapted from Hillhouse and colleagues (Hillhouse et al., 2000) were included in order to assess perceived normative pressure (e.g., stem of “I am motivated to comply with…” with an example of an item being “MY FRIENDS’ opinions on tanning”) of media/society and one’s social circle.

Sunscreen self-efficacy

Items for assessing self-efficacy for using sunscreen were adapted from a measure by Maddock and colleagues (Maddock, Redding, Rossi, & Weinstock, 2005).

Perceived control

We adapted one item by Hillhouse and colleagues (Hillhouse et al., 2000) for assessing perceived control over indoor UV exposure (e.g., “In the next year, how easy/difficult would it be for you to use a tanning salon?”) and created additional items about outdoor exposure and skin protection.

Intentions to avoid sun and cover up

Items adapted from Maddock and colleagues (Maddock et al., 2005) and Mahler and colleagues (Mahler, Kulik, Gerrard, & Gibbons, 2010) were used to assess intentions for sun avoidance and wearing protective clothing.

Subsequent psychometric analyses included internal reliability, three-week test-retest reliability, and correlations with other psychosocial and behavioral measures. Internal consistency is reported using Cronbach’s alpha, and intra-class correlation coefficients (ICC) were used to indicate three-week test-retest reliability. ICCs were calculated using a subsample of the overall sample (n = 307) that did not receive any intervention during the three weeks between assessments.

In order to assess the convergent validity of the psychosocial measures, we assessed correlations between the psychosocial variables and UV protection and exposure behaviors. Specifically, we examined outdoor UV exposure behaviors, UV protection behaviors, and indoor tanning in the past month.

Measures of Behavior

The following sun protection behaviors were assessed, using items adapted from Glanz and colleagues (Glanz et al., 2008): wearing sunscreen with an SPF of 15 or more on the face, wearing sunscreen with an SPF of 15 or more on other parts of the body, wearing a shirt with sleeves that cover the shoulders, wearing long pants, wearing a hat, wearing sunglasses, and staying in the shade. Participants indicated how often they engaged in these behaviors over the past month (1 = “Never”; 5 = “Always”). This measure was internally consistent (alpha = 0.76).

For indoor tanning, participants were asked to indicate the number of days in the past month that they used a tanning bed or booth, using language adapted from a measure of self-reported indoor tanning by Lazovich and colleagues (Lazovich et al., 2008). Since this variable was skewed, it was dichotomized into those who indoor tanned in the past month vs. those who did not.

In addition, participants were asked to indicate how often they engaged in four other UV exposure behaviors [wearing clothes that expose the skin to the sun, sunbathing, getting a tan just by being outdoors (i.e., unintentional tanning), and using products to get a faster or deeper tan] over the past month (1 = “Never”; 5 = “Always”), using a scale adapted by Ingledew and colleagues (Ingledew et al., 2010). This measure had acceptable internal consistency (alpha = 0.68).

Results

Factor analysis of the items revised based on cognitive interviewing and expert review revealed a 17 factor model. The RMSEA Index was 0.051, indicating good fit, the Root Mean Square of the Residuals was 0.03, indicating acceptable fit; although, the Tucker Lewis index was 0.76. The model explained 51% of the variance of the correlation matrix, which is acceptable. Two-item scales that were removed included perceived barriers to wearing hats, perceived benefits of UV exposure, perceived benefits of sun protection on health, perceived barriers to protection, and media and societal norms for tanning. Scales were then modified for conceptual reasons. This included removing three items from the benefits of skin protection scale that overlapped with the barriers to UV exposure scale, including only sunscreen items in the self-efficacy scale, removing a self-efficacy item from the intentions scale, and removing a three-item intentions to use sunscreen and check one’s skin for skin cancer scale. The final survey items for the 11 scales, as well as instructions, item stems, and response options, can be viewed in the eMeasures section.

Table 1 describes the demographic characteristics of the participants included in the psychometric testing. Participant characteristics include: mean age of 21.8, 85.7% white, and 66.1% female.

Descriptive and psychometric qualities of the final scales are shown in Table 2. Most scales had internal reliability coefficients of 0.75 or higher. The only scales with a lower Cronbach’s alpha were perceived susceptibility (a = 0.73) and self-efficacy (a = 0.73). Intra-class Correlation Coefficients for three-week test-retest reliability were acceptable, ranging from 0.72–0.85, except for perceived health/well-being benefits of UV exposure at 0.68.

Table 2.

Characteristics of and correlations between psychosocial variables (N = 965; USA, 2014).

Variables	M (SD)	Possible Range	Cronbach’s alpha	ICC (n = 307)	2	3	4	5	6	7	8	9	10	11
1. Knowledge	0.79 (0.27)	0–1	0.84	0.84	0.101*	−0.005	−0.219***	0.254***	0.295***	−0.049	−0.041	−0.135***	0.202***	−0.077*
2. Appearance benefits of UV exposure	4.07 (0.72)	1–5	0.78	0.76	–	0.370***	0.355***	0.261***	0.097*	0.124**	0.246***	0.147**	−0.056	−0.159***
3. Health/well-being benefits of UV exposure	3.65 (0.85)	1–5	0.87	0.68		–	0.288***	0.173***	0.045	0.112*	0.165***	0.114*	−0.017	−0.022
4. Social benefits of UV exposure	2.87 (1.14)	1–5	0.92	0.80			–	0.159**	0.051	0.264***	0.325***	0.423***	−0.084	0.058
5. Perceived benefits of skin protection	3.75 (0.84)	1–5	0.92	0.78				–	0.621***	0.141***	−0.002	0.063	0.277***	0.113**
6. Barriers to UV exposure	3.38 (1.11)	1–5	0.92	0.72					–	0.205***	0.094**	0.052	0.218***	0.089***
7. Susceptibility	2.58 (0.98)	1–5	0.73	0.80						–	0.216***	0.235***	0.045	0.054
8. Social circle norms for tanning	2.91 (0.86)	1–5	0.80	0.85							–	0.425***	−0.010	−0.091**
9. Normative pressure to tan	2.47 (1.02)	1–5	0.82	0.74								–	−0.053	0.013
10. Sunscreen self-efficacy	3.19 (0.97)	1–5	0.73	0.74									–	0.219***
11: Intentions to avoid sun and cover up	3.00 (1.01)	1–5	0.77	0.75										–

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

With regard to convergent validity, correlations between the final scales, as well as correlations with behavioral measures, are reported in Tables 2 and 3, respectively. Most psychosocial variables were significantly correlated with each other. The highest correlation was between benefits of skin protection and barriers to UV exposure at 0.62. Many of the psychosocial variables were significantly correlated with skin protection and indoor and outdoor UV exposure behaviors. For example, variables significantly positively associated with skin protection included benefits of skin protection, sunscreen self-efficacy, and intentions to avoid sun and cover up. Variables significantly negatively associated with skin protection included knowledge, perceived appearance benefits of exposure, and perceived societal and social circle norms for tanning. Variables significantly positively associated with indoor tanning included perceived benefits of UV exposure: appearance, health/well-being, social; perceived susceptibility; perceived social circle norms for tanning; and social pressure for tanning. Knowledge was significantly negatively associated with indoor tanning behavior. Variables significantly positively associated with other UV exposure included perceived benefits of exposure: appearance, health/well-being, social; benefits of skin protection; perceived susceptibility; perceived social circle norms for tanning; and social pressure for tanning. Variables significantly negatively associated with other UV exposure included knowledge, sunscreen self-efficacy; and intentions to avoid sun and cover up.

Table 3.

Associations between psychosocial variables and behaviors (N = 965; USA, 2014).

Variables	UV Protection in Past Month		Indoor Tanning in Past Month				Other UV Exposure in Past Month
	r	p	OR	(95% CI)		p	r	p
Knowledge	−0.19	<.001	0.26	0.13	0.51	<.001	−0.08	0.01
Benefits of exposure:
Appearance	−0.13	<.01	1.63	1.12	2.39	0.01	0.24	<.001
Health/well-being	−0.00	NS	2.14	1.51	3.04	<.001	0.30	<.001
Social	0.04	NS	1.80	1.41	2.28	<.001	0.29	<.001
Benefits of skin protection	0.13	<.01	1.20	0.84	1.72	NS	0.10	0.01
Barriers to UV exposure	0.01	NS	1.07	1.45	2.36	NS	0.04	NS
Susceptibility	−0.01	NS	1.85	2.14	4.09	<.001	0.24	<.001
Social circle norms for tanning	−0.08	0.02	2.96	0.67	1.11	<.001	0.32	<.001
Social pressure to tan	0.06	0.08	1.79	1.04	1.77	<.001	0.24	<.001
Sunscreen self-efficacy	0.14	<.001	0.95	0.76	1.19	NS	−0.08	0.02
Intentions to avoid sun and cover up	0.29	<.001	1.20	1.12	4.95	NS	−0.15	<.001

Conclusions

There are numerous versions of items and scales that have been used to assess psychosocial constructs associated with skin cancer risk and protective behaviors. Many of these scales had not been cognitively tested previously nor had been comprehensively assessed psychometrically in a large national sample (see eTable 1) (Pinard et al., 2015; Tripp et al., 2013). It is important to have reliable and valid psychosocial measures in order to better understand and modify UV exposure and protective behaviors associated with skin cancer. For the current project, we selected the items and scales that we believe best assessed selected constructs from the Integrative Model of Behavior Prediction (IM) with the best available psychometric data. Based on expert review and cognitive interviewing, these existing items and scales were revised. The types of revisions made to items included dividing, deleting, simplifying, clarifying, personalizing, making them more specific, and so on. When items were not available for a particular construct, we modified and/or developed our own and tested them. The process of identifying and reviewing items, developing the cognitive interviewing protocol, gathering and processing feedback, and discussing and deciding on changes is labor-intensive but a necessary step in ensuring that items are well-understood, especially in specific populations; in this case, young adults.

Most of the final scales derived from the factor analysis were found to possess good internal reliability. The three-week test-retest reliability was acceptable for all but one of the scales, indicating that scores obtained on these scales are consistent across this length of time. Perceived health/well-being benefits of UV exposure had an ICC of 0.68. Perhaps the test-retest reliability for this scale was slightly lower since it assessed both mental and physical reasons for tanning.

The hypotheses about which variables would be positively or negatively correlated with each other were confirmed for the most part, demonstrating evidence for convergent validity. As expected, most psychosocial variables were significantly but not highly correlated with each other, suggesting that they measure related but not completely overlapping constructs. In addition, many of the psychosocial variables were significantly correlated with skin protection and indoor and outdoor UV exposure as expected.

However, a few surprising findings were observed. Higher perceived benefits of skin protection was significantly associated with higher perceived benefits of UV exposure and higher UV exposure behaviors. A potential explanation for this finding is that the reasons for both tanning and protection were for perceived appearance and health enhancement. Though UV exposure would be perceived to enhance appearance and feelings of health in the immediate future, skin protection would enhance appearance and health in the distant future. Other interesting findings were that higher knowledge was significantly associated with lower intentions to avoid sun and cover up as well as lower UV protection behavior. One potential explanation for the knowledge finding is that individuals with greater knowledge might report lower UV protection intentions and behavior because they don’t want to prevent vitamin D production.

This study had strengths, such as the overall comprehensiveness of the approach. Specifically, it involved both qualitative and quantitative refinement and evaluation based on psychometric best practices. Additionally, it utilized a large national sample of young adults to assess the psychometric characteristics of the measures; though, the sample included only individuals at moderate to high risk of skin cancer, most of whom were white, potentially limiting generalizability. A limitation is that the survey was administered solely online, so the results could differ if another mode of administration (e.g., paper and pencil, telephone, face-to-face) were used. Based on their behaviors, young adults are a population at risk for skin cancer, but future research should be conducted similarly with other populations longitudinally and assess other administration modes. Additionally, the perceived control items did not fit with any of the other items, thus perceived control is not included in the final measure. The barriers to skin protection scales were also removed because they included only one or two items.

Finally, it is important to remember that the measures can still be improved. For example, the current measures focused primarily on cognitive variables, but it is well-known that emotional and physiological variables affect UV and protection behaviors, which should also continue to be researched. Additionally, some researchers might be interested in distinguishing between descriptive and injunctive norms or cognitive and affective components of perceived susceptibility, which was not done here. Also, we attempted to simplify items so that they refer to only one idea. However, after data collection was concluded, it was brought to our attention that one item asks participants to decide about skin protection on both cold and cloudy days (see eMeasures). This item might have been better broken into two items, one referring to cold and one referring to cloudy days. Similarly, some of the items were not fully consistent with one another (e.g., long-sleeve shirts, shirts that cover my arms or shoulders, shirt with sleeves that cover the shoulders). Additionally, though the study was relatively comprehensive, there may be some potentially important psychosocial variables that were not assessed (e.g., fatalism, reactance), and the behavioral measures did not include every possible exposure or protection behavior (e.g., avoiding sun near solar noon or summer solstice).

Although labor-intensive, the expert review and in-depth cognitive interviewing process produced items deemed to be more understandable, clear, and specific, thus facilitating more accurate responding from research participants. The study resulted in scales that demonstrated adequate to good internal consistency, acceptable test-retest reliability, and evidence for convergent validity. The resulting items and scales assess important psychosocial constructs or mediators of sun protection/sun exposure behaviors from the IM. As its name suggests, the Integrative Model includes constructs from several prior models of health behavior (e.g., the Health Belief Model, the Theory of Reasoned Action, Social Cognitive Theory) that have been found to accurately describe and explain a variety of health behaviors among a variety of populations. Similarly, these scales are expected to be widely applicable for research with young adults related to UV exposure and protective behaviors. Appropriate and consistent assessment of such psychosocial constructs can aid in better understanding and modifying skin cancer risk and protective behaviors.