Development and Psychometric Properties of a Theory-Guided Prescription Stimulant Misuse Questionnaire for College Students

Abstract

A theory-guided instrument for examining prescription stimulant misuse in the college population was developed and its psychometric properties were evaluated from 2011–2012 at one Pacific Northwest (United States) university. Study methods included instrument development, assessment by five health and measurement professionals, group interviews with six college students, a test-retest pilot study, and a paper-based, in-classroom, campus study using one-stage cluster sampling (N = 520 students, 20 classrooms, eligible student response rate = 96.30%). The instrument demonstrated reliability (i.e. internal consistency and stability) and validity (i.e. face, content, and predictive). Limitations and implications are discussed.

Introduction

Prescription stimulants are a class of drugs with high abuse and adverse health effect potential (National Institute on Drug Abuse [NIDA], 2009, 2008; Nissen, 2006; White, Becker-Blease, & Grace-Bishop, 2006), and the illicit use of prescription stimulants [IUPS; see glossary] has emerged as a substance use behavior of the 21^st century college student (Johnston, O'Malley, Bachman, & Schulenberg, 2011). To date, the field of IUPS lacks an instrument that 1) comprehensively defines “illicit use of prescription stimulants”, and 2) includes a comprehensive set of theory-guided intrapersonal, social, and environmental measures (Bavarian, Flay, Ketcham, & Smit, 2012). As a result, the reported prevalence of IUPS and risk and protective factors associated with the behavior have varied across studies (Bavarian et al., 2012). Addressing these research gaps, through the development of a new instrument, is essential for accurately assessing the scope of IUPS and identifying the multitude of factors associated with IUPS on any one campus.

We aim to contribute to the growing field of IUPS study by describing the development and psychometric evaluation of a preliminary instrument that addresses the two aforementioned research gaps. This instrument is specific to the college population, and measures, amongst other things, behaviors, expectancies, normative beliefs, attitudes, and intentions. We engaged in a five-phase study at one university located in the Pacific Northwest (United States) and hypothesized that the systematic approach to measurement development would result in an instrument with strong psychometric properties (i.e. internal consistency and stability reliability, and face, content, and predictive validity). Our goal is for campuses to be able to use this instrument to better understand the risk factors that are both unique to IUPS and shared across substances, which should strengthen broad-based strategic plans for substance use prevention and intervention.

Method and Results

Design and Participants

The study included five phases of research that were completed over the course of two years: I) Instrument development, II) Instrument review by health and measurement professionals, III) Instrument review by college students via group interviews, IV) Pilot testing with college students, and V) a campus study. The study was approved by the Oregon State University Institutional Review Board, and participants in phases II through V were provided a monetary incentive, with funding via the Dr. Joel Grinold’s Research Grant from the Pacific Coast College Health Association. The authors have no conflicts of interest to declare. After summarizing this study’s theoretical guide and instrument measures, we present, by study phase, procedures and results.

The Theory

Development of the Behaviors, Expectancies, Attitudes and College Health Questionnaire (BEACH-Q; Appendix A) was guided by the Theory of Triadic Influence (TTI; Flay, Snyder, & Petraitis, 2009; Flay & Petraitis, 1994). Although a multitude of meritorious theories of health behavior exist, the ecological-based TTI allows for various intrapersonal, social, and environmental theories to be unified into a single framework (Flay et al., 2009). Accordingly, the decision to use the TTI as the theoretical guide was made based on its comprehensive framework, as well as our preliminary research illustrating the multi-etiological nature of IUPS (Bavarian et al., 2012).

The TTI (Figure 1) includes three streams of influence (i.e., intrapersonal, social situation/context, and sociocultural environment) and three levels of causation (i.e., ultimate underlying causes [ultimate], distal predisposing influences [distal], and proximal immediate predictors [proximal]). According to the TTI, ultimate and distal factors in each stream influence self-efficacy, social normative beliefs, and attitudes towards a behavior. The combination of self-efficacy, social normative beliefs, and attitudes toward a behavior then influence a person’s intent to perform a behavior, and the experiences gained from trial behavior influence not only frequency of the behavior, but also, related behaviors. If one considers the TTI as a matrix of behavioral influences, our goal in developing the BEACH-Q was to ensure constructs from each “square” in the matrix would be included so that a comprehensive picture of IUPS risk and protective factors could be developed.

Figure 1.

The Theory of Triadic Influence

Note: Figure adapted with permission from an online PowerPoint

The Instrument

In developing the BEACH-Q, a review of surveys used in studies of IUPS in the college population yielded several pre-existing items that tap into various TTI-guided constructs. Whenever possible, instruments with established reliability and/or validity were retained or adapted (e.g., abbreviated, reworded, recall period revised) and included in the BEACH-Q; permission was obtained to use items retained without adaptation (e.g., items from the National College Health Assessment II (American College Health Association, 2009)). New items were developed in instances where constructs embedded within specific TTI domains did not pre-exist in the literature.

The measures included in the BEACH-Q are summarized below, and Appendix B provides, for each construct included in the final (Phase V) version of the BEACH-Q, the number of items per variable, response options, and the original source. The final instrument included 14 theoretical correlates from the intrapersonal stream of influence, eight from the social situation/context stream of influence, 14 from the sociocultural environmental stream of influence, and one immediate precursor. Of these 37 TTI-based covariates, 26 were adapted from pre-existing measures.

Intrapersonal Stream of Influence

With respect to the intrapersonal stream of influence, ultimate-level variables include inattention, hyperactivity, sensation seeking, gender, race/ethnicity, age, year in school, international student status, and enrollment credits. Psychological distress, academic concerns, grades, and Attention Deficit Hyperactivity Disorder [ADHD] diagnoses are distal-level measures in the BEACH-Q. Lastly, prescription stimulant avoidance self-efficacy is a proximal-level measure in the BEACH-Q.

Social Situation/Context Stream of Influence

Residence is included as an ultimate-level variable in the social situation/context stream of influence. Measures included at the distal level are Greek life participation, varsity sports participation, relationship status, strength of relationships with friends, family, and faculty/staff, perceptions of IUPS by friends, family, faculty/staff, and endorsement of IUPS by friends, family, and faculty/staff. Perceived prevalence of IUPS by friends and the overall campus were included as proximal-level variables in the social situation/context stream.

Sociocultural Environment Stream of Influence

Ultimate-level variables of the sociocultural environment stream are financial-related stress, participation in religious activities, exposure to prescription drug media on television and in print media, perception of academic demand, perception of substance use during college, and perception of health care providers’ prescription drug writing. Distal-level variables include interactions with social institutions (i.e., academic faculty), interactions with social institutions influencing values (i.e. value of academic performance), information (i.e. provided by health care providers regarding prescription stimulants), information influencing knowledge, and IUPS expectancies. Knowledge about prescription stimulants and perceived costs/benefits of IUPS were included as proximal-level variables.

Immediate Precursors and Related Behaviors

As the TTI posits that immediate precursors and related behaviors also influence use, IUPS intentions (immediate precursor) and use of other substances (alcohol, tobacco, marijuana, cocaine, and prescription pain killers), are included in the BEACH-Q.

Additional Items

Additional items that do not fall within the TTI matrix, but that nonetheless may be of interest to researchers and student affairs and health professionals were included in the BEACH-Q. These measures reflect hours working, volunteering, or interning, problems with sleepiness, self-reported health rating, primary health care provider, and academic major.

Illicit Use of Prescription Stimulants

To date, the field of IUPS study lacks a unified definition of IUPS (Arria & Wish, 2006). Several existing instruments screen out students with legitimate prescriptions for medical stimulants from studies on misuse (e.g., Advokat, Guidry, & Martino, 2008; McCabe, Knight, Teter, & Wechsler, 2005), in spite of literature showing IUPS to be more likely among students with a prescription (e.g., Judson & Langdon, 2009; Novak, Kroutil, Williams, & Van Brunt, 2007; Tuttle, Scheurich, & Ranseen, 2010). Additionally, some surveys ask only about methylphenidate use (e.g., Babcock & Byrne, 2000; DuPont, Coleman, Bucher, & Wilfod, 2008) even though other classes of prescription stimulants are available. Lack of a unified definition is problematic, as it may lead to an underestimation of prevalence and flawed/biased conclusions about correlates of use.

As we aimed to create an instrument that comprehensively assesses IUPS (our dependent variable), a number of questions were developed so that students could be properly classified as illicit users of any type of prescription stimulant, irrespective of whether they have a prescription for the drug. First, students are asked if, during their time in college, they have ever (0 = No; 1 = Yes) used prescription stimulants “without a prescription from a health care provider,” “for nonmedical purposes (i.e. to help with studying, to stay awake, to get high),” or “in excess of what was prescribed to you.” To assess frequency and initiation of use, students are then asked on how many occasions per academic term they have engaged in the behavior (1 = Never; 7 = 40 or more occasions), and when they first initiated the behavior. Students who report ever engaging in IUPS during their lifetime are directed to items regarding route of ingestion, source of prescription stimulants, motives for use, and whether they experienced the outcome they desired.

Phase I

Procedures

Phase I (Instrument Development) required a systematic review of the IUPS literature and resulted in the development of a preliminary 97-instrument (discussed above). This preliminary version of the BEACH-Q was informally reviewed by N.B.’s five-person doctoral committee.

Results

Changes to survey flow, formatting, and content were made based on feedback. For example, one item (transfer status), was removed from the survey, resulting in a 96-item survey.

Phase II

Procedures

In Phase II, the BEACH-Q was assessed for content validity (i.e., the degree to which a measure represents a concept; Trochim, 2006; Singleton & Straits, 2005). A convenience sample of health and measurement professionals with at least one year of experience in the college setting was recruited via e-mail to participate in a brief assessment of the survey. Of the six assessment packets distributed, five completed questionnaires were returned using pre-addressed envelopes.

Professionals were asked to indicate how much they agreed (1 = Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree) that the item(s) being used to measure a specific construct accurately encompassed that construct. At the conclusion of the survey, respondents were asked, “If you marked ‘Strongly Disagree’ or ‘Disagree’ for any question, please specify why you did so and how you think the items can be improved”.

Survey responses (from this and all subsequent phases) were analyzed using Stata version 12.1 (StataCorp, 2011). Each respondent contributed one score per measure, and a median and mean score between 3 and 4 was determined to reflect high content validity as such a score would fall between “Agree” and “Strongly Agree”. Any measure that received a rating below three was reviewed to determine how the measure could be improved before beginning phase III.

Results

The mean and median ratings based on the five returned assessments are provided in Table 1. Of the 37 TTI-based covariates examined, 35 received a median rating between “Agree” and “Strongly agree”, and 34 received a mean rating between “Agree” and “Strongly Agree”. Minor revisions were made with respect to both content and formatting based on feedback from participants. For example, some items (e.g., enrollment status) were made into continuous measures. Other items were replaced completely (e.g., the three items intended to measure academic concern) based on reviewer recommendation.

Table 1.

Phase II and III Assessments, Content and Face Validity analysis, Summary Statistics (N = 5 Professionals in Phase II and 6 Students in Phase III)

	Phase II (Content Validity)		Phase III (Face Validity)
Instrument Measures	Median	Mean	Median	Mean
Intrapersonal Stream
Ultimate Underlying Causes
Inattention	3.00	3.20	4.00	3.67
Hyperactivity	3.50	3.50	3.00	3.33
Sensation-Seeking	3.50	3.50	3.00	3.17
Gender	3.00	3.40	4.00	3.67
Race/Ethnicity	4.00	3.80	4.00	3.83
Age	4.00	3.80	4.00	4.00
Year in School	4.00	3.60	4.00	3.83
International Student Status	4.00	3.80	4.00	3.83
Enrollment Status	4.00	3.60	4.00	4.00
Distal Predisposing Influences
Psychological Distress	3.00	3.00	3.50	3.50
Academic Concern	4.00	3.75	4.00	3.83
Grades	4.00	3.80	4.00	3.83
ADHD Diagnosis	4.00	3.60	4.00	3.83
Proximal Immediate Predictors
Refusal Self-Efficacy	3.00	3.20	3.50	3.50
Social Situation/Context
Ultimate Underlying Causes
Residence	4.00	3.80	4.00	3.83
Distal Predisposing Influences
Greek Life	4.00	3.80	3.50	3.50
Varsity Sports	4.00	3.80	4.00	3.67
Relationship Status	3.50	3.50	4.00	3.83
Strength of Relationships	3.50	3.50	4.00	3.83
Perceptions of IUPS by Socializing Agentss	3.50	3.50	4.00	3.83
Endorsement of IUPS by Socializing Agent	3.00	3.20	4.00	3.83
Proximal Immediate Predictors
Perception of Prevalence	3.00	3.20	4.00	4.00
Sociocultural Environment
Ultimate Underlying Causes
Socioeconomic Status/ Financial-Related Stress	4.00	3.60	4.00	3.67
Religiosity	3.00	3.20	4.00	4.00
Exposure to Prescription Drug Media	4.00	3.40	4.00	3.67
Campus Culture – Perception of Academic Rigor	3.00	3.25	4.00	3.67
Campus Culture – Perception of Substance Use During College	4.00	3.40	4.00	4.00
Campus Culture – Perception of Campus Health Center Personnel	3.00	3.25	4.00	3.50
Distal Predisposing Influences
Interactions with Social Institutions	2.50	2.75	3.50	3.33
Interactions with Social Institutions Influencing Values	2.50	2.75	4.00	3.67
Information	3.00	3.00	4.00	3.67
Information Influencing Knowledge	3.00	3.00	4.00	3.67
IUPS Expectancies	4.00	3.60	4.00	3.67
Proximal Immediate Predictors
Knowledge about Prescription Stimulants	3.50	3.25	4.00	4.00
Costs/Benefits of IUPS	3.00	2.80	4.00	4.00
Immediate Precursors
IUPS Intentions	3.00	3.00	4.00	4.00
Related Behaviors
Substance Use	3.50	3.50	4.00	3.83
Behavior
IUPS	4.00	3.60	3.50	3.50
IUPS Frequency	4.00	3.80	4.00	3.83
IUPS Initiation	4.00	3.80	4.00	3.83
Route of Administration	4.00	3.60	4.00	4.00
Source of Prescription Stimulants	4.00	3.80	4.00	4.00
IUPS Motives	4.00	3.80	4.00	3.83
IUPS Reinforcement	4.00	3.60	4.00	3.83
Additional Items
Work Hours	4.00	3.60	4.00	3.67
Sleep Difficulties	3.00	3.00	4.00	4.00
Health Care Provider	4.00	3.60	4.00	4.00
Health Rating	4.00	3.80	4.00	3.83
Academic Major	4.00	3.80	4.00	3.83

Note: 1 = Strongly Disagree; 2 = Disagree; 3 = Agree; 4 = Strongly Agree.

Phase III

Procedures

The purpose of the semi-structured student group interviews (Phase III) was to determine whether college students believe BEACH-Q items are measuring what they are intended to measure, also known as face validity (Trochim, 2006; Singleton & Straits, 2005). An additional purpose of the student interviews was to obtain critical survey feedback from the intended target audience with respect to survey readability, cultural appropriateness and clarity.

A convenience sample of six students volunteered to participate in a 60-minute group interview about the BEACH-Q. For each set of items intended to capture a particular domain, participants were asked, “(1) Do you think this item/these items measure(s) ‘(intended outcome)’?” (Response Options: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree). Students were also asked whether they found items offensive and/or in need of revision.

Each respondent contributed one score per variable/scale and a median and mean score between 3 and 4 were determined to reflect high face validity, as such a score would reflect a response of “Agree” or “Strongly Agree”. Revisions were made to the BEACH-Q based on ratings and verbal feedback.

Results

Of the 37 TTI-based covariates reviewed by students, all 37 received a median and mean rating equal to or above 3.00 (Table 1). Overall, students called the BEACH-Q “straightforward”, and no student found any item offensive. Moreover, the students interviewed did not feel overwhelmed by the length of the survey.

Phase IV

Procedures

Phase IV involved a test-retest pilot study, not only to determine the time and resources needed to implement larger-scale data collection, but also to investigate the reliability of the BEACH-Q. One course instructor allowed the BEACH-Q to be administered in class on a voluntary basis. Of the 45 students who completed the survey at the first administration, 39 volunteered to re-take the survey at the second administration (time elapsed = two weeks; response rate of eligible participants = 88.64%).

The internal consistency reliability of each scale that consisted of three or more items was assessed using the standardized Cronbach coefficient alpha. For constructs with two items, Pearson’s correlation (i.e. r) between the two items was calculated. Stability reliability was calculated using the Pearson’s r correlation between test-retest responses for continuous outcomes, and tetrachoric rho for binary outcomes; these test-retest statistics are reported at the item level for single-item measures and at the level of summated scales for composite measures. As has been done in other studies examining the reliability of an instrument (e.g., Bringham et al., 2009), reliability was rated as modest for those test statistics between 0.30 to 0.49, moderate for those test statistics between 0.50 to 0.69, and high for test statistics between 0.70 and 1.00. In addition to using psychometric results to revise the BEACH-Q, responses to a survey addendum regarding readability, clarity, and overall thoughts on the BEACH-Q were used to improve the instrument.

Results

Survey administration (i.e. explanation of the study, distribution of surveys, return of surveys, and distribution of incentives) at both test- and retest required 20 minutes to complete. With respect to internal consistency reliability, the 12 TTI-based correlates for which Cronbach’s alpha could be calculated were moderate to high (Table 2). Stability reliabilities ranged from modest to high. Review of the results from psychometrics and the survey addendum led to minor revisions (e.g., revision of response options for the expectancy items) to the BEACH-Q.

Table 2.

Phase IV Pilot Test, Reliability Analysis for Theoretical Correlates and Behavior (N = 39 students)

Variables	Time 1 Internal Consistency Reliabilitya	Time 2 Internal Consistency Reliabilitya	Stability Reliabilityb
Intrapersonal Stream
Ultimate Underlying Causes
Inattention (composite)c	α = 0.81	α = 0.81	r = 0.89
Hyperactivity (composite)c	α = 0.51	α = 0.68	r = 0.76
Sensation-Seeking (composite)c	α = 0.72	α = 0.86	r = 0.83
Gender	NA	NA	r = 1.00
Race/Ethnicity	NA	NA	r = 0.95
Age, in years	NA	NA	r = 1.00
Year in School	NA	NA	NA
International Student Status	NA	NA	NA
Enrollment Status, in credit hours	NA	NA	r = 0.87
Distal Predisposing Influences
Psychological Distress (composite)d	α = 0.82	α = 0.86	r = 0.73
Academic Concern (composite)d	α = 0.77	α = 0.83	r = 0.65
Grades	NA	NA	r = 0.88
ADHD Diagnosis	NA	NA	Tetrachoric rho = 1.00
Proximal Immediate Predictors
Avoidance Self-Efficacy (composite)e	α = 0.89	α = 0.92	r = 0.95
Social Situation/Context
Ultimate Underlying Causes
Housing	NA	NA	r = 0.98
Distal Predisposing Influences
Greek Life Participation	NA	NA	Tetrachoric rho = 1.00
Varsity Sports Participation	NA	NA	Tetrachoric rho = 1.00
Relationship Status	NA	NA	r = 0.98
Strength of Relationshipsf
Friends	NA	NA	r = 0.74
Family	NA	NA	r = 0.83
Campus Faculty and Staff	NA	NA	r = 0.78
Perceptions of IUPS by Socializing Agentsg
Friends	NA	NA	r = 0.79
Family	NA	NA	r = 0.68
Campus Faculty and Staff	NA	NA	r = 0.50
Endorsement of IUPS by Socializing Agentsh
Friends	NA	NA	r = 0.68
Family	NA	NA	r = 0.80
Campus Faculty and Staff	NA	NA	NA
Proximal Immediate Predictors
Perception of Prevalence
Use by close friends, in percent	NA	NA	r = 0.91
Use by student population, in percent	NA	NA	r = 0.79
Sociocultural Environment
Ultimate Underlying Causes
Financial-Related Stressd	NA	NA	r = 0.86
Participation in Religious Activitiesd	α = 0.97	α = 0.98	r = 0.99
Exposure to Prescription Drug Mediac	r = 0.81	r = 0.78
Television			r = 0.73
Print media			r = 0.78
Campus Culture #1c	r = 0.27	r = 0.10
Courses academically demanding			r = 0.66
Students compete for best grades			r = 0.73
Campus Culture #2c	NA	NA	r = 0.40
Perception of Health Center Personnelc	NA	NA	r = 0.64
Distal Predisposing Influences
Interactions with Social Institutionsc	NA	NA	r = 0.72
Interactions with Social Institutions Influencing Valuesc	NA	NA	r = 0.70
Informationc	NA	NA	r = 0.50
Information Influencing Knowledgec	NA	NA	r = 0.51
IUPS Expectancies-Positivei	α = 0.79	α = 0.83	r = 0.56
IUPS Expectancies-Negativei	α = 0.79	α = 0.91	r = 0.55
Proximal Immediate Predictors
Knowledge about Prescription Stimulantsc	α = 0.85	α = 0.87	r = 0.81
Costs/Benefits of IUPSc	α = 0.63	α = 0.58	r = 0.75
Immediate Precursors
Intentions to Engage in IUPSi	NA	NA	r = 0.90
Behavior
IUPS (Ever Use During College)	NA	NA	Tetrachoric rho = 1.00
IUPS Frequencyj	NA	NA	r = 0.95

^aCronbach alpha or Pearson’s r used for internal consistency

^bPearson’s r or tetrachoric rho used for stability reliability

^cResponse options for items and composite range from 1 = Strongly Disagree to 5 = Strongly Agree

^dResponse options for items and composite range from 1 = None of the time to 5 = All of the time

^eResponse options for items and composite range from 1 = Not at all confident to 5 = Completely confident

^fResponse options for items range from 1 = Very Weak to 5 = Very strong

^gResponse options for items range from 1 = Very Negatively to 5 = Very Positively

^hResponse options range from 1 = None to 5 = All

ⁱResponse options range from 1 = Very Unlikely to 4 = Very Likely

^jResponse options range from 1 = Never to 7 = 40 or more times

NA: Not Applicable

Phase V

Procedures

Phase V involved one-stage cluster sampling. Of the 151 randomly selected undergraduate classes, 144 (95%) instructors were successfully contacted via e-mail. Of those 144 instructors contacted, 27 allowed entry into the class, and funding allowed for 20 classes to be surveyed. At each survey administration, all students were informed of the purpose of the study, notified that participation was voluntary and anonymous, and told that students choosing not to participate would not be penalized. Of the 20 participating classes, the maximum survey administration time was 20 minutes. Of the 540 surveys distributed to eligible students, 520 were completed (response rate = 96.30%); these 520 students were representative of the total undergraduate population (results not shown).

When applicable (e.g., when a measure was encompassed by three or more items), confirmatory factor analysis [CFA] was used (using Stata’s “sem” command) to examine factor loadings and model fit acceptability (Ford & Schroeder, 2009; Brown, 2006). As per convention, items with factor loadings greater than 0.40 were considered to be major constituents of that particular factor (Hanson, Imperatore, Bennett, & Knowler, 2002). Given phase V’s relatively large sample size, and the fact that having a large sample size increases the likelihood of a significant chi-square value (Ford & Schroeder, 2009; Brown, 2006), the Comparative Fit Index (CFI) and Root Mean Square Error Approximation (RMSEA) were used as model fit indices. According to Hu and Bentler (1999, as cited in Brown, 2006), RMSEA values below 0.06 and CFI values in the range of 0.90–0.95 or above could indicate acceptable model fit.

To examine the BEACH-Q’s predictive validity (Trochim, 2006), crude logistic regression (Long & Freese, 2006) was performed whereby each construct in the BEACH-Q that lies within the TTI’s matrix of behavioral influences (i.e. ultimate, distal, proximal, immediate precursor, and related behaviors) served as an independent variable and a dichotomous measure of IUPS (1 = Ever used during college) was the dependent variable. Unadjusted odds ratios were examined to determine if the level of influence was significant and in the hypothesized direction. These analyses were run for each of the 37 TTI-based hypothesized correlates.

Results

Results from the confirmatory factor analyses are presented in Table 3. In fitting a one-factor solution for each applicable scale, factor loadings for all items encompassing each respective scale were significant at the 0.01 level. For all measures excluding “positive IUPS expectancies” the factor loadings were above 0.40; this was also the only measure where the CFI and RMSEA did not meet recommendations. Results suggest this measure may more accurately encompass two factors (i.e. academic and non-academic positive expectancies). Also, Stata could not converge on a solution for the four item “costs/benefits” measure, and post-hoc analyses (results not shown) suggested future analyses using these items should treat them as two separate measures (i.e. “Costs” and “Benefits”).

Table 3.

Phase V Campus Study, Confirmatory Factor Analyses on Applicable Measures (N = 520 students)

Measure/Items	Standardized Loadings	Comparative Fit Index	Root Mean Square Error of Approximation
Intrapersonal Stream
Ultimate Underlying Causes
Inattentiona		1.00	0.00
It is difficult for me to pay attention during classes	0.71**
It is difficult for me to concentrate on academic work	0.88**
I have difficulty keeping track of school assignments	0.55**
Hyperactivitya
I often feel restless	0.44**	1.00	0.00
I am an impulsive person	0.70**
I rarely plan ahead	0.50**
Sensation-Seekinga		1.00	0.00
I like “wild” parties	0.61**
I enjoy getting into situations where I do not know how things will turn out	0.85**
I prefer friends who are unpredictable	0.59**
Distal Predisposing Influences
Psychological Distressb		1.00	0.00
Sad or blue	0.67**
Anxious	0.85**
Worried	0.86**
Academic Concernb		1.00	0.00
Worried about academic performance	0.82**
Helpless about academic performance	0.71**
Stressed about academic performance	0.74**
Proximal Immediate Predictors
Avoidance Self-Efficacyc		1.00	0.07
Confident would not use more than was prescribed	0.66**
Confident would refuse if offered	0.87**
Confident would not ask if knew someone with prescription stimulants	0.87**
Confident would not misuse if had large deal of work	0.80**
Sociocultural Environment
Ultimate Underlying Causes
Participation in Religious Activitiesb		1.00	0.00
Attend a place of worship	0.84**
Rely on religious teaching when have a problem	0.95**
Turn to prayer or mediation with personal problem	0.92**
Rely on religious beliefs for day to day living	0.93**
Distal Predisposing Influences
IUPS Expectancies-Positived		0.85	0.23
I would get better grades	0.85**
I would find studying more enjoyable	0.84**
I would be able to concentrate/focus better	0.70**
I would be able to stay awake for a long time	0.90**
I would lose weight	0.48**
I would be able to party longer	0.39**
IUPS Expectancies-Negatived		0.91	0.19
I would feel anxious	0.75**
I would feel dizzy/lightheaded	0.87**
My heart would race	0.86**
I would not be able to sleep	0.75**
I would get in trouble	0.57**
I would get headaches	0.75**
Proximal Immediate Predictors
Knowledge about Prescription Stimulants (PS)a		1.00	0.00
Aware of recommended dosage levels for PS	0.66**
Aware of what may happen to body if use PS	0.87**
Knowledgeable about the side effects of PS	0.89**
Substance Usee
Tobacco	0.65**	0.94	0.10
Alcohol	0.66**
Marijuana	0.70**
Cocaine	0.45**
Prescription pain killers	0.48**

^*Estimator = Maximum Likelihood;

p < 0.05

^**p < 0.01

^aResponse options range from 1 = Strongly Disagree to 5 = Strongly Agree

^bResponse options range from 1 = None of the time to 5 = All of the time

^cc-Response options range from 1 = Not at all confident to 5 = Completely Confident

^dd-Response options range from 1 = Very Unlikely to 4 = Very Likely

^ee-Response options range from 1 = Never to 7 = 40 or more times

Results from the crude logistic regression analyses are provided in Table 4. Of the 37 covariates examined, 24 had a significant (p < 0.05 or p < 0.01) association with IUPS, with all significant associations in the expected direction.

Table 4.

Phase V Campus Study, Predictive Validity using Crude Logistic Regression Analyses (N = 520 students)

Variables	Odds Ratio (95% Confidence Interval)
Intrapersonal Stream
Ultimate Underlying Causes
Inattentiona	2.08 (1.63, 2.67)**
Hyperactivitya	2.42 (1.80, 3.24)**
Sensation-Seekinga	2.07 (1.62, 2.65)**
Gender
Female	1.00
Male	1.18 (0.79, 1.79)
Race/Ethnicity
White	1.00
Asian or Pacific Islander	0.22 (0.07, 0.72)*
Other	0.92 (0.50, 1.69)
Age	1.00 (0.95, 1.06)
Year in School
1st year	1.00
2nd year	2.29 (0.87, 6.03)
3rd year	3.37 (1.33, 8.55)**
4th year	4.07 (1.63, 10.17)**
5th year or postbac	4.65 (1.75, 12.36)**
International Student Status
Domestic	1.00
International	0.80 (0.26, 2.47)
Enrollment Credits	0.94 (0.85, 1.04)
Distal Predisposing Influences
Psychological Distressb	1.45 (1.13, 1.86)**
Academic Concernb	1.73 (1.34, 2.24)**
Grades
A	1.00
B	3.29 (1.85, 5.85)**
C	6.21 (2.91, 13.27)**
ADHD Diagnosis
Never Diagnosed	1.00
Ever Diagnosed	2.92 (1.61, 5.31)**
Proximal Immediate Predictors
Avoidance Self-Efficacyc	0.24 (0.18, 0.31)**
Social Situation/Context
Ultimate Underlying Causes
Residence
Off-Campus Housing	1.00
Campus Housing	0.21 (0.10, 0.48)**
Distal Predisposing Influences
Greek Life
Non-member	1.00
Member	1.42 (0.84, 2.40)
Varsity Sports
Non-member	1.00
Member	1.43 (0.83, 2.48)
Relationship Status
Not in a relationship	1.00
In a relationship and NOT living together	0.96 (0.61, 1.50)
In a relationship and living together	0.75 (0.39, 1.42)
Strength of Relationshipsd
Friends	1.09 (0.84, 1.43)
Family	0.93 (0.72, 1.19)
Faculty/ Staff	0.81 (0.66, 1.01)
Perceptions of IUPS by Socializing Agentse
Friends	4.02 (2.88, 5.63)**
Family	3.16 (2.34, 4.26)**
Faculty/ Staff	1.05 (0.82, 1.35)
Endorsement of IUPS by Socializing Agentsf
Friends	4.06 (3.08, 5.35)**
Family	3.72 (2.30, 6.02)**
Faculty/ Staff	1.93 (1.21, 3.08)**
Proximal Immediate Predictors
Perception of Prevalence Rates
Friends IUPS	1.06 (1.04, 1.07)**
Campus IUPS	1.03 (1.02, 1.04)**
Sociocultural Environment
Ultimate Underlying Causes
Financial-Related Stressb	1.31 (1.11, 1.54)**
Participation in Religious Activitiesb	0.66 (0.55, 0.79)**
Exposure to Prescription Drug Media on Televisiona	1.03 (0.85, 1.23)
Exposure to Prescription Drug Print Mediaa	0.87 (0.72, 1.04)
Campus Culture – Perc. of Academic Demand #1a	1.39 (1.07, 1.80)*
Campus Culture – Perc. of Academic Demand #2a	1.10 (0.90, 1.35)
Campus Culture – Perc. of Sub. Use During Collegea	1.46 (1.16, 1.83)**
Campus Culture – Perc. of HC Providers Rx writinga	1.17 (0.95, 1.44)
Distal Predisposing Influences
Interactions with Social Institutionsa	1.28 (1.04, 1.58)*
Interactions with Soc. Inst. Influencing Valuesa	0.73 (0.52, 1.03)
Informationa	1.10 (0.91, 1.32)
Information Influencing Knowledgea	1.18 (0.98, 1.42)
IUPS Expectanciesg
Positive Expectancies	3.53 (2.68, 4.65)**
Negative Expectancies	0.74 (0.60, 0.92)**
Proximal Immediate Predictors
Knowledge about Prescription Stimulantsa	1.34 (1.08, 1.67)**
Costs/Benefits of IUPSa	5.91 (3.72, 9.38)**
Immediate Precursors
IUPS Intentionsh	8.17 (5.66, 11.80)**
Related Behaviors
Substance Useh	3.62 (2.82, 4.66)**

^*p < 0.05

^**p < 0.01

^aResponse options for items and composite range from 1 = Strongly Disagree to 5 = Strongly Agree

^bResponse options for items and composite range from 1 = None of the time to 5 = All of the time

^cResponse options for items and composite range from 1 = Not at all confident to 5 = Completely confident

^dResponse options for items range from 1 = Very Weak to 5 = Very strong

^eResponse options for items range from 1 = Very Negatively to 5 = Very Positively

^fResponse options range from 1 = None to 5 = All

^gResponse options range from 1 = Very Unlikely to 4 = Very Likely

^hResponse options range from 1 = Never to 7 = 40 or more times

Discussion

The systematic and multi-method approach to development of the BEACH-Q resulted in an instrument with strong psychometric properties. In assessments completed by health and measurement professionals (Phase II) and students (Phase III), the instrument demonstrated content and face validity, respectively; moreover, reviewers agreed that the definition of IUPS used in the BEACH-Q was appropriate. In addition, internal consistency reliabilities ranged from moderate to high, and stability reliabilities ranged from modest to high (Phase IV). Items loaded strongly onto their respective measures, and the BEACH-Q also demonstrated construct validity, as 24 of the 37 covariates had a significant association with IUPS in the expected direction (Phase V).

Study Limitations and Strengths

This study was not without its limitations. With respect to Phase I, in order to reduce respondent burden, the number of items in pre-existing scales with known reliability and/or validity were often reduced and/or adapted. Altering pre-existing scales may have altered their psychometric properties; this was addressed by examining the psychometrics of the BEACH-Q during each subsequent study phase.

With respect to Phases II and III, responses on assessment sheets were subjective in nature. Nonetheless, the use of five health and measurement professionals with different educational degrees (Phase II) allowed for the professional opinions of persons from various qualified backgrounds to be used to improve the BEACH-Q. With respect to Phase III, the benefit of receiving feedback from the target population (i.e. college students) arguably outweighed the cost of potential subjectivity. Moreover, Phases II and III were followed by pilot-testing in Phase IV, to obtain objective measures of reliability, and a campus study in Phase V, to obtain an objective measure of validity.

Phases II through IV included the use of convenience samples. As such, the thoughts of all health and measurement professionals and students may not have been encompassed by those self-selecting to participate. Nonetheless, the feedback given by participants provided key insights and information that did result in a psychometrically sound instrument, as demonstrated by the results. Moreover, the campus study (Phase V) used one-stage cluster sampling, and resulted in a random sample that was representative of the total undergraduate population.

As is common after a final implementation phase (Phase V), we learned that some covariates could have been better defined/classified, and additional questions should have been included. For example, rather than classifying perceived costs/benefits as types of attitudes towards the behavior, it can be argued that new items could have been used to better capture attitudes towards the different types of IUPS. In addition, the phase V BEACH-Q did not include questions on diversion, which is of key interest to a number of stakeholders. We have revised the BEACH-Q accordingly for future use (available upon request).

While the BEACH-Q allows for ultimate-level correlates (e.g., sensation seeking, social-group participation, and participation in religious activities) of other substances (e.g., alcohol and marijuana) to be examined, a test of the full TTI based on the BEACH-Q is currently limited to IUPS. Accordingly, future research directions include not only examining the ultimate-level differences and commonalities between different forms of substance use, but also testing a full model of the TTI as it applies to IUPS.

Strengths of the study include the strategic approach to measurement development which resulted in an instrument that comprehensively defines IUPS, includes a multitude of theory-guided measures, is psychometrically strong, and can be completed relatively briefly. As theory is critical for predicting and preventing high-risk health behaviors, an instrument such as the BEACH-Q should help stakeholders (e.g., researchers, student affairs, health professionals) better understand the risk factors for IUPS present on their respective campuses. In addition, results from campus studies using the BEACH-Q can help stakeholders develop the most appropriate policies and programs to address IUPS and related behaviors.

Conclusions

The purpose of this study was to develop a psychometrically-sound, theory-guided instrument that can be adapted for use at universities in need of assessing the scope of IUPS and understanding risk and protective factors, both unique to IUPS, and shared with other forms of substance use. The purpose was achieved through an intensive five-phase process. We hope that development of this measure will facilitate prevention and intervention activities on college campuses.

Glossary

Illicit use of prescription stimulants (IUPS)

Use of any class of prescription stimulants (i.e. amphetamines such as Adderall©, dextroamphetamines such as Dexedrine©, and methylphenidates such as Ritalin©) without a prescription from a health care provider, use for nonmedical purposes (e.g., to stay awake, to exacerbate the effects of alcohol, etc.) and/or use in excess of what is prescribed.