Validation of a Brief Pornography Screen across multiple samples

Abstract

Background and Aims

To address current gaps around screening for problematic pornography use (PPU), we initially developed and tested a six-item Brief Pornography Screen (BPS) that asked about PPU in the past six months.

Methods and Participants

We recruited five independent samples from the U.S. and Poland to evaluate the psychometric properties of the BPS. In Study 1, we evaluated the factor structure, reliability, and elements of validity using a sample of 224 U.S. veterans. One item from the BPS was dropped in Study 1 due to low item endorsement. In Studies 2 and 3, we further investigated the five-item the factor structure of the BPS and evaluated its reliability and validity in two national U.S. representative samples ( N = 1,466, N = 1,063, respectively). In Study 4, we confirmed the factor structure and evaluated its validity and reliability using a sample of 703 Polish adults. In Study 5, we calculated the suggested cut-off score for the screen using a sample of 105 male patients seeking treatment for compulsive sexual behavior disorder (CSBD).

Results

Findings from a principal components analysis and confirmatory factor analysis supported a one-factor solution which yielded high internal consistency ( α = 0.89–0.90), and analyses further supported elements of construct, convergent, criterion, and discriminant validity of the newly developed screen. Results from a Receiver Operating Characteristic (ROC) curve suggested a cut-off score of four or higher for detecting possible PPU.

Conclusions

The BPS appears to be psychometrically sound, short, and easy to use in various settings with high potential for use in populations across international jurisdictions.

Introduction

Currently, there is considerable debate among clinicians and researchers regarding how best to classify excessive/problematic engagement in sexual behaviors ( Kraus, Voon, & Potenza, 2016b ), and scholars have proposed classifications including hypersexual disorder ( Kafka, 2010 ), impulse-control disorder ( Grant et al., 2014 ; Kraus et al., 2018 ), non-paraphilic compulsive sexual behavior disorder (CSBD) ( Coleman, Raymond, & McBean, 2003 ), or behavioral addiction ( Kor, Fogel, Reid, & Potenza, 2013 ). Problematic pornography use (PPU) may be grouped with other sexual behaviors meeting diagnostic criteria for CSBD as defined in the ICD-11 ( Kraus et al., 2018 ). CSBD is described as a persistent pattern of failure to control intense, repetitive sexual impulses or urges, resulting in repetitive sexual behavior over an extended period (e.g., 6 months or more) that generates marked distress or impairment in social, occupational, or other important areas of functioning ( Kraus et al., 2018 ; World Health Organization, 2018) . The current study evaluated the psychometric properties of a newly developed self-report screen designed to assess for probable PPU in five samples comprised of non-clinical and clinical adults.

Prevalence estimates of CSBD among clinical and non-clinical populations remain elusive ( Gola & Potenza, 2018 ; Kraus, Voon, et al., 2016b ). A recent study of 2,325 U.S. adults found that 8.6% of the representative sample (7.0% of women and 10.3% of men) endorsed clinically relevant levels of distress and/or impairment associated with concerns controlling sexual feelings, urges, and behaviors ( Dickenson, Gleason, Coleman, & Miner, 2018 ). Specific to pornography use, data from a U.S. nationally representative sample of 2,075 Internet users found that approximately half ( n = 1,056) reported past-year use of pornography, and 11% of men and 3% of women reported “feeling addicted to pornography” ( Grubbs, Kraus, & Perry, 2019b ). Preliminary evidence collected from U.S. military veterans suggested an elevated rate of compulsive sexual behavior ( Smith et al., 2014 ); however, studies have typically not examined PPU among U.S. veterans, a group noted with high clinical comorbidities and impulsivity ( James, Strom, & Leskela, 2014 ).

Further, among individuals seeking treatment for CSBD, most (>80%) report concerns with pornography use ( Gola et al., 2018 ; Kraus, Potenza, Martino, & Grant, 2015b ; Reid et al., 2012 ; Scanavino et al., 2013 ). For these individuals, PPU is often characterized by craving, diminished self-control, impairments in functioning, and use of pornography to cope with anxiety or dysphoric mood ( Kraus, Martino, & Potenza, 2016a ; Wordecha et al., 2018 ). Individuals seeking treatment for PPU and other sexual behaviors often report psychiatric concerns including, depression, anxiety, and substance use disorders ( Kraus, Potenza et al., 2015b ).

To identify PPU, multiple self-report scales have been developed and tested including the Problematic Pornography Use Scale (PPUS) ( Kor et al., 2014 ), Compulsive Pornography Consumption Scale (CPC) ( Noor, Rosser, & Erickson, 2014 ), Cyber Pornography Use Inventory (CPUI/CPUI-9) ( Grubbs, Sessoms, Wheeler, & Volk, 2010 ; Grubbs, Volk, Exline, & Pargament, 2015 ), Pornography Consumption Inventory (PCI) ( Reid, Li, Gilliland, Stein, & Fong, 2011b ), Pornography Craving Questionnaire (PCQ) ( Kraus & Rosenberg, 2014 ), and Problematic Pornography Consumption Scale (PPCS) ( Bothe et al., 2018 ), and Problematic Pornography Consumption Scale (PPCS-6) ( Bőthe, Tóth-Király, Demetrovics, & Orosz, 2020 ). While each has strengths, many of these self-report questionnaires have limitations and have often not been subjected to rigorous psychometric testing (see Fernandez & Griffiths, 2019 for discussion on pornography measures). For instance, they typically have been developed and tested on non-clinical, convenience samples in Western countries, often lack a unified theoretical or diagnostic framework, assess multiple and discrepant symptom domains, and do not have a suggested clinical cut-off score for determining who should be further evaluated by mental health professionals. Although these problems are concerning in their own right, they are even more concerning in light of the diagnostic recognition of CSBD. In June 2019, CSBD was officially added to the ICD-11 ( World Health Organization, 2018 ) and with the high co-occurrence of PPU, the development of short, robust, and psychometrically sound screening instruments for PPU is greatly needed to address current gaps in the field.

Aims of the present study

In light of the described limitations above, the present work describes the development of a brief screening instrument Brief Pornography Screen (BPS) to identify PPU in five independent studies. In Study 1, we examined 283 U.S. military veterans' ratings of agreement with the proposed items, conducted a principal components analysis, and assessed the internal reliability and validity of the BPS. In Study 2, we used the Omnibus service provided by Qualtrics Survey Software to recruit 2,075 U.S. adults matched to U.S. representative norms in order to reconfirm the single-factor structure of the screen, assess its internal reliability, and examine the relationships between the BPS and measures of psychopathology. In Study 3, we used the Turkprime panel service to reassess the BPS factor structure in 1,063 U.S. adults again matched to representative norms and examined correlations with measures of psychopathology. In Study 4, we recruited 703 community-based Polish adults to confirm the factor structure further in a non-U.S. sample and assess internal consistency and validity. In Sample 5, we examined the clinical characteristics of 105 male patients in Poland seeking treatment for PPU to establish the recommended clinical cut-off score. Recruitment for all studies is discussed in further detail in the Supplemental Materials .

Statistical analyses for studies 1–5

In Studies 1 and 4, we used SPSS-19 for descriptive statistics, chi-squares, principal components analysis, Pearson Product-moment correlations, ANCOVAs, and independent t -tests.

In Studies 2 and 3, we conducted our CFA models using the lavaan ( Rosseel, 2011 ) package for R, using diagonally weighted least squares estimation, which does not assume normality or homoscedasticity of residuals and is preferable for ordinal data ( Flora & Curran, 2004 ). For Study 5, we used SPSS-19 for conducting Receiver Operating Characteristic (ROC) curve analyses.

Study 1

Method

Procedure and participants

Study 1 was conducted with data from the Survey of Experiences of Returning Veterans (SERV) project, which recruited military veterans across the U.S. ( Kraus et al., 2017 ; Smith et al., 2014 ). The general procedures employed to recruit participants and conduct the SERV project have been described elsewhere ( Kraus et al., 2017 ). Study eligibility requirements were as follows: (a) separated (discharged) from the U.S. military; (b) a veteran of Iraq, Afghanistan, or surrounding eras; (c) at least 18 years of age; (d) English speaker; and, (e) living in the U.S. Portions of this dataset have been published previously in the following papers ( Decker et al., 2019 ; Moisson et al., 2019 ; Scoglio et al., 2017 ; Turban, Potenza, Hoff, Martino, & Kraus, 2017 , Turban, Shirk, Potenza, Hoff, & Kraus, 2020 ), but none of these papers focused on the structure or validity of the BPS.

Sample Characteristics

Of 283 participants surveyed, most were male (70.6%, n = 197) with a mean age of 35.1 ( SD = 9.2) years. Sample characteristics are listed in Supplemental Table 1 .

Measures

The first author developed the initial six items on the BPS as a possible measure of PPU in U.S. veteran samples. These items were initially generated when the first author was completing a postdoctoral fellowship in psychology. Items were generated based on clinical interactions with patients and continued work from prior studies investigating clinical correlates of PPU (see Kraus, Martino et al., 2016a ; Kraus & Rosenberg, 2014 ). Next, the proposed items were crossed examined by two other team members before being examined in Study 1.

In Study 1, participants were given the BPS, which was designed to identify individuals reporting problems managing their use of pornography. The initial scale consisted of six items. Participants were asked, “In the last 6 months, have any of these situations happened to you in regards to your use of pornography?” Item responses were 0 (never), 1 (occasionally), and 2 (very often), with a scoring range from 0 to 12. See Table 1 for the exact wording of the BPS.

Table 1.

Study 1, Frequency count of agreement for the six items of the Brief Pornography Screen (BPS) among U.S. veterans ( N = 222)

Items	Never (%)	Occasionally (%)	Very Often (%)	M (SD)	Component matrix
You find yourself using pornography more than you want to.	60.5	29.6	9.9	1.49 (0.67)	0.80∗
You have attempted to “cut back” or stop using pornography, but were unsuccessful.	73.5	18.8	7.2	1.33 (0.61)	0.82∗
You find it difficult to resist strong urges to use pornography.	61.9	28.7	9.0	1.47 (0.66)	0.84∗
You find yourself using pornography to cope with strong emotions (e.g., sadness, anger, loneliness, etc.).	68.6	20.2	10.8	1.42 (0.68)	0.73∗
You continue to use pornography even though you feel guilty about it.	61.4	25.6	12.6	1.51 (0.71)	0.76∗
People have expressed concern about your use of pornography.	90.6	5.8	3.1	1.12 (0.41)	0.49

Note . Component loadings in boldface indicate higher loadings on that component. Missing data on two participants.

Component 1 = 3.75; Percent of variance = 62.5%.

^∗ Bolded items were retained in the final version.

M = mean; SD = standard deviation.

We also used the Sexual Behaviors and Pornography History Questionnaire ( Rosenberg & Kraus, 2014 ) to assess participants' sexual history and pornography use characteristics, the PCQ ( Kraus & Rosenberg, 2014 ) to assess craving for pornography ( α = 0.83), and the PPUS ( Kor et al., 2014 ) to assess features associated with PPU ( α = 0.83). The UPPS-P Impulsive Behavior Scale ( Cyders, Littlefield, Coffey, & Karyadi, 2014 ; Lynam, Smith, Whiteside, & Cyders, 2006 ) is a 45-item questionnaire that assessed overall impulsivity ( α = 0.80) and Premeditation (lack of) ( α = 0.84), Negative Urgency ( α = 0.81), Positive Urgency ( α = 0.81), Sensation-Seeking ( α = 0.84), and Perseverance (lack of) components ( α = 0.83), and the Hypersexual Behavior Inventory (HBI) ( Reid, Garos, & Carpenter, 2011a ) to measure characteristics of hypersexuality ( α = 0.82). An additional question assessed veterans' interest in receiving treatment for specific CSBD behavior(s) (e.g., compulsive pornography, casual/anonymous sex, etc.).

Ethics

The Institutional Review Board of the Department of Veterans Affairs approved the study. All participants provided informed written consent prior to involvement in the study.

Results

Pornography use and sexual practices among veterans

Twenty-one percent ( n = 59) of participants reported that they never viewed pornography. Approximately 51% ( n = 42) of women indicated that they never used pornography compared to 8.6% of men ( n = 17), χ ² (5) = 96.15, P < 0.001, Cramer's V = 0.59. Because the current study focused on the psychometric evaluation of the BPS to assess PPU, we removed these 59 pornography non-users from the study, leaving 220 individuals for subsequent analyses.

Item reduction and factor structure of the Brief Pornography Screen (BPS)

We first conducted item reduction by examining the item-total correlations of the initial six items ( Table 1 ). All items were moderately correlated ( r s = 0.31–0.70, P < 0.001), suggesting that none could be eliminated on this basis. Second, we examined the frequency counts for each level of agreement for each of the six items on the BPS to identify any items which were ‘unbalanced’ ( Clark & Watson, 1995 ). Using this decision rule, one item (“People have expressed concern”) was suitable for elimination; however, we subjected all six items to principal components analysis (un-rotated) for further item-reduction purposes.

Principal component analysis (PCA) is often used for item reduction in scale development, and PCA and exploratory factor analysis (EFA) often yield similar results ( Schneeweiss & Mathes, 1995 ). Because of the simplicity of the BPS (originally 6 items) and its single underlying factor, our goal was simply to reduce the number of items while retaining as much of the original variance as possible ( Conway & Huffcutt, 2003 ). However, had the BPS included multiple factors, and we were interested in the relationship between those factors, EFA or structural equation modeling (SEM) would have been considered. Below we report the results for the PCA.

Results yielded only one component with an eigenvalue of 3.75, which accounted for 62.5% of the total variance ( Table 1 ). Only the previously identified unbalanced item did not have high loadings (≥0.50) and communalities (>0.40); using this decision rule ( Costello & Osborne, 2005 ), the item was dropped. The five remaining items had high internal consistency coefficient ( α = 0.89), composite reliability (0.92), and a moderate mean inter-item correlation ( r = 0.62), supporting the unidimensionality of the BPS ( Clark & Watson, 1995 ).

Construct, convergent, criterion, and discriminant validity of the BPS

To evaluate one element of construct validity, we first examined whether BPS scores varied as a function of the amount of pornography watched, after adjusting for gender. ANCOVA results indicated a significant main effect for pornography use frequency, F (3, 216) = 14.32, P < 0.001, partial η ² = 0.12. Using post-hoc comparisons (Bonferroni-corrected) we found that daily pornography users ( M = 4.39, SD = 2.10, SE = 0.48) had significantly higher BPS scores than did weekly users ( M = 2.53, SD = 0.73, SE = 0.29), who in turn had higher BPS scores than did monthly users ( M = 1.45, SD = 0.36, SE = 0.25). We also calculated Pearson Product-moment correlations to assess the relationships between study variables, and in support of convergent validity, we found a positive and robust correlation between the PPUS and BPS scores (see Table 2 for bivariate correlations by gender). In support of criterion validity, we found positive but moderate correlations between the BPS, HBI, and PCQ scores. In support of discriminant validity, the BPS was largely unrelated to impulsivity, although for men, and not women, negative and positive urgency were positively associated, albeit weakly, with BPS scores.

Table 2.

Study 1, Correlations and means and standard deviations for study variables of interest for U.S. veterans

Variable	Brief Pornography Screen				Range
	Women ( n = 40)		Men ( n = 180)
	r	M (SD)	r	M (SD)
Brief Pornography Screen	–	0.80 (1.73)	–	2.55 (2.87)	0–10
Pornography Craving Questionnaire	0.32∗	2.03 (0.95)	0.45∗∗	2.95 (1.34)	1–7
Problematic Pornography Use Scale	0.77∗∗	1.27 (0.50)	0.75∗∗	1.92 (0.98)	1–5.7
Hypersexual Behavior Inventory	0.66∗∗	27.1 (9.0)	0.60∗∗	34.8 (15.4)	18–95
UPPS-P Negative Urgency	0.29	2.27 (0.51)	0.30∗∗	2.36 (0.52)	1.3–3.9
UPPS-P Lack of Premeditation	0.11	2.07 (0.44)	−0.03	2.08 (0.40)	1.2–3.3
UPPS-P Lack of Perseverance	0.18	1.79 (0.42)	0.11	1.94 (0.48)	1.0–3.4
UPPS-P Sensation Seeking	−0.02	2.61 (0.48)	0.05	2.87 (0.37)	1.2–4.0
UPPS-P Positive Urgency	0.22	1.94 (0.44)	0.22∗∗	2.23 (0.48)	1.1–3.6

Note . ∗ P < 0.05, ∗∗ P < 0.01.

M = mean; SD = standard deviation.

Sexual behavior treatment

Of the 220 veterans surveyed about their pornography viewing (see Supplemental Table 1 ), nine indicated they were interested in treatment for PPU. All individuals were male (9 of 180 males, 5%). The BPS mean score on the remaining five items for the nine men was 6.67 ( SD = 2.95). All subsequent Studies (2–5) used the five-item BPS for their analyses since they were conducted after Study 1.

Study 2 ¹

Method

Procedures and Participants

Using the Omnibus service provided by Qualtrics Survey Software, we recruited a U.S. nationally representative (non-probability sample based on 2010 census norms for age, gender, race, ethnicity, income, and U.S. Census region) for a cross-sectional study of adults ( N = 2,075; 51% women [ n = 1,059], 49% men [ n = 1,016]; M _age = 44.8, SD = 16.7).

Portions of this dataset have been described elsewhere in the following papers, but neither of the papers focused on the structure or validity of the BPS (see Grubbs, Kraus et al., 2019b ; Grubbs, Kraus, Perry, Lewczuk, & Gola, 2020 ).

Measures

Analyses were limited to adults who acknowledged viewing pornography within the past year ( N = 1,058, 66% men). Pornography-use behaviors were assessed via three items. Specifically, we asked participants how often they had intentionally viewed pornography alone within the past year. We also asked participants how often they had masturbated to pornography over the past year. For both questions, responses ranged from 1 ( not at all ) to 8 ( once a day or more ). A single item asked participants to report, in minutes, how much time they had spent daily, on average, viewing pornography.

Specific to this sample and in addition to the BPS, we also assessed psychological distress by including three depression-related items and two anxiety-related items from the Cross-Cutting Symptom Measure for the DSM-5 ( Narrow et al., 2013 ). We administered three CPUI-9 items ( Grubbs et al., 2015 ) to assess specific responses to or beliefs about pornography use. Each item was scored on a scale of 1 ( strongly disagree ) to 7 ( strongly agree ). These face-valid items were taken from the CPUI-9 subscales: Perceived Compulsivity (e.g., “I believe I am addicted to pornography”), Access Efforts (e.g., “I have put off things I needed to do to view pornography”), and Emotional Distress (e.g., “I feel depressed after viewing pornography”). All three items are substantively related to pornography-use behaviors ( Grubbs, Wilt, Exline, & Pargament, 2018a ; Grubbs, Wilt, Exline, Pargament, & Kraus, 2018b ).

Ethics

The Institutional Review Board of the Department of Bowling Green State University approved Study 2 as exempt. All participants provided electronic informed consent prior to involvement in the study.

Results

We conducted a Confirmatory Factor Analysis (CFA) using diagonally-weighted least-squares (DWLS) estimation with robust variances, as DWLS estimation does not assume normality or homoscedasticity of residuals and is preferable for ordinal data ( Flora & Curran, 2004 ). This analysis revealed excellent BPS fit for a unidimensional factor structure (Robust χ ² (5) = 3.06, P = 0.69; CFI = 1.00, TLI = 1.00, RMSEA < 0.001, SRMR = 0.01). Average BPS score was low ( M = 1.56, SD = 2.53), and analysis of internal reliability revealed high internal consistency ( α = 0.90). Men obtained higher BPS scores ( M = 2.24, SD = 2.81) than did women ( M = 1.70, SD = 2.60); t (2, 1,056) = 3.05, P < 0.001, Cohen's d = 0.20).

BPS scores were positively associated with multiple measures in expected directions. BPS scores were positively associated with statements of, “I am addicted to pornography” ( r = 0.620, P < 0.001), “I feel depressed after viewing pornography” ( r = 0.47, P < 0.001), and “I have put off things I needed to do to view pornography” ( r = 0.59, P < 0.001). BPS scores were positively associated with frequency of pornography viewing over the past year ( r = 0.39, P < 0.001), masturbating to pornography over the past year ( r = 0.40, P < 0.001), average daily minutes spent viewing pornography ( r = 0.23, P < 0.001), and generalized feelings of psychological distress ( r = 0.34, P < 0.001).

Study 3 ²

Method

Procedures and Participants

Data from 470 internet-using adults with past-year pornography use were analyzed from a larger sample of 1,063 U.S. adults matched on U.S. nationally representative norms based on 2010 U.S. nationally representative norms (based on U.S. Census data) for age, gender, ethnicity, race, U.S. Census region, and income. This non-probability sample was recruited and compensated by the Turkprime panel service ( Litman, Robinson, & Abberbock, 2017 ).

Portions of this dataset have been published previously in the following papers ( Grubbs et al., 2020 ; Grubbs & Gola, 2019 ; Grubbs, Grant; Engelman, 2019a ; Grubbs, Warmke, Tosi, James, & Campbell, 2019d ); however, none of the studies focused on the structure or validity of the BPS.

Measures

Consistent with Study 2, we restricted analyses to those who reported pornography use within the past year ( N = 470; M _age = 44.9; SD = 15.9; 72% men). Pornography use behaviors were assessed, as in Study 2, using the BPS and measures of frequency of solitary pornography use, frequency of masturbation to pornography, and average daily use of pornography in minutes. Generalized distress was measured via the same DSM-5 Cross-Cutting measure described in Study 2. Self-reported feelings of addiction to pornography were assessed with the CPUI-9 ( α = 0.91; Grubbs et al., 2010 ; Grubbs et al., 2015 ) and its component subscales assessing Perceived Compulsivity ( α = 0.93), Emotional Distress ( α = 0.92), and Access Efforts ( α = 0.87).

Ethics

The Institutional Review Board of the Department of Bowling Green State University approved Study 3 as exempt. All participants provided electronic informed consent prior to involvement in the study.

Results

A CFA using Robust DWLS estimation revealed excellent BPS fit for unidimensionality (χ ² (5) = 8.64, P = 0.12; CFI = 0.99, TLI = 0.98, RMSEA = 0.03, SRMR = 0.02). The mean BPS score was low ( M = 1.92, SD = 2.69) and internal reliability was high ( α = 0.91). Men ( M = 2.25, SD = 2.75) scored higher than did women ( M = 1.12, SD = 2.39; t [1,48] = 4.04, P < 0.001, Cohen's d = 0.40).

BPS scores were correlated with scores on the total CPUI-9 ( r = 0.72, P < 0.001) and Perceived Compulsivity ( r = 0.75, P < 0.001), Access Efforts ( r = 0.64, P < 0.001), and Emotional Distress ( r = 0.47, P < 0.001) subscales. BPS scores were positively associated with frequency of pornography use over the past year ( r = 0.47, P < 0.001), frequency of masturbation to pornography over the past year ( r = 0.43, P < 0.001), average daily use of pornography in minutes ( r = 0.33, P < 0.001), and generalized feelings of distress ( r = 0.33, P < 0.001).

Study 4

Method

Procedure and participants

The sample ( Supplemental Table 4 ) was comprised of 703 Polish adults (512 women, 72.8%) aged 18–54 years ( M = 26.04, SD = 6.07). A subset of this dataset (191 males) comes from the dataset described in Kowalewska, Kraus, Lew-Starowicz, Gustavsson, and Gola (2019) .

All adults were recruited from the Polish population through a web-based advertisement on gumtree.pl (Polish version of Craigslist) and hiperseksualnosc.pl (the website of the research team). Participants who completed the online survey and left their email address were eligible to win one of the following prizes, five bookstore vouchers of 30, 15, or 5 USD and 30 tickets to a movie theater. All email addresses were stored in the separate database and not associated with questionnaire data to help ensure anonymity.

Measures

In addition to using the BPS, we assessed impulsivity using the Polish adaptation of UPPS-P ( Poprawa, 2014 ). We measured obsessive-compulsive features using the Polish adaptation of the Obsessive Compulsive Inventory – Revised (OCI-R) ( Foa et al., 2002 ; details on translation provided in; Gola et al., 2017a ) and the Polish adaptation of the Sexual Addiction Screening Test – Revised (SAST-R) ( Gola et al., 2017a ) to assess (1) preoccupation with sex, (2) affect, (3) relationship disturbance by sexual behaviors, and (4) feeling of losing control over sexual behavior (SAST-R total α = 0.80).

Ethics

All procedures were approved by the Ethical Committee of Institute of Psychology, Polish Academy of Sciences. All participants were provided informed written consent prior to involvement in the study.

Results

Psychometric properties of the Polish-adapted BPS

An additional CFA using Robust DWLS estimation yielded an excellent fit for the one-factor solution (Robust χ ² (5) = 2.12, P = 0.83; CFI = 1.00, TLI = 1.00, RMSEA = 0.00, SRMR = 0.02). Similar to the previous studies, the Polish adaptation of the BPS had high internal consistency ( α = 0.89) and a moderate mean inter-item correlation ( r = 0.62). Both, internal consistency and mean inter-item correlation were higher in males ( α = 0.88; r = 0.61) than females ( α = 0.85; r = 0.54).

As depicted in Table 3 , for the full sample, the average BPS score was 1.92 ( SD = 2.65). Men ( M = 3.56, SD = 3.11) had higher BPS scores compared to women ( M = 1.12, SD = 1.92), t (701) = 10.12, P < 0.001, Cohen's d = 0.76). The number of minutes spent watching pornography was weakly correlated with BPS scores but only for men. In support of criterion validity, BPS scores were positively correlated with symptom severity as measured by the SAST-R. In support of discriminant validity and similar to Study 1, we found no correlation between BPS scores and UPPS-P sensation-seeking and lack of premeditation and weak positive correlations between BPS scores and negative urgency, positive urgency, and perseverance. BPS scores were weakly correlated with obsessive-compulsive features (see Table 3 for all correlations).

Table 3.

Correlations of BPS scores with other measures in a sample of Polish community adults ( N = 703)

Variable	Brief Pornography Screen				Range
	Women ( n = 512)		Men ( n = 191)
	r	M (SD)	r	M (SD)
Brief Pornography Screen	–	1.12 (1.92)	–	3.56 (3.11)	0–10
Amount of pornography use during last week (min.)	0.07	60.46 (108.93)	0.17∗	124.66 (179.12)	1–1,200
Sexual Addiction Screening Test – Revised	0.43∗∗	3.81 (2.99)	0.61∗∗	5.51 (4.23)	0–18
Obsessive Compulsive Inventory – Revised	0.17∗∗	18.03 (10.38)	0.25∗∗	19.21 (9.72)	0–58
UPPS-P Negative Urgency	0.22∗∗	29.26 (7.16)	0.29∗∗	27.02 (7.79)	2–48
UPPS-P Lack of Premeditation	0.06	22.28 (5.26)	0.14	21.83 (5.86)	2–41
UPPS-P Lack of Perseverance	0.14∗∗	20.25 (5.18)	0.15∗	20.24 (4.92)	2–37
UPPS-P Sensation Seeking	−0.06	31.22 (7.75)	−0.004	34.39 (7.99)	4–48
UPPS-P Positive Urgency	0.12∗∗	28.02 (9.54)	0.27∗∗	28.90 (10.03)	9–56

Note. ∗ P < 0.05, ∗∗ P < 0.01.

M = mean; SD = standard deviation.

Study 5

Method

Procedures and participants

To examine the BPS cut-off score, we assessed an additional 105 Polish men aged 18–55 years ( M = 32.94; SD = 7.45) who were seeking treatment for CSBD, the majority of whom reported PPU (see Supplemental Tables 5 and 6 ). The treatment-seeking group includes datasets from the following studies: Wordecha et al. (2018) (9 males); Gola, Lew-Starowicz, Draps, and Kowalewska (2019) (57 males); Draps et al. (2020) (26 males); Holas, Draps, Kowalewska, Lewczuk, & Gola (2020) (13 males). The control group consisted of 191 male adults aged 18–54 ( M = 26.04; SD = 6.07) from Study 4.

Treatment-seeking patients were recruited among males seeking treatment for PPU in two sexology clinics in Warsaw between June 2014 and November 2017. All patients seeking treatment for PPU met four out of five diagnostic criteria for hypersexual disorder as proposed by Kafka (2010) for DSM-5.

Measures

After completing an initial interview, patients were screened for inclusion/exclusion criteria. Inclusion/exclusion criteria consisted of being exclusively or predominantly heterosexual (as assessed using the Polish adaptation of the Kinsey Scale; Kinsey, Pomeroy, & Martin, 1948 ) and not meeting diganostic criteria for alcohol use disorder ( Saunders, Aasland, Babor, De la Fuente, & Grant, 1993 ) or gambling disorder (scores<5 on the South Oaks Gambling Screen (SOGS α = 0.70) ( Lesieur & Blume, 1987 ). All male patients were additionally assessed with the Structured Clinical Interview for DSM-IV (SCID) ( Gibbon, Spitzer, Williams, Benjamin, & First, 1997 ) for obsessive-compulsive, impulse-control, bipolar, anxiety, psychotic, and substance use disorders and sexual behaviors ( Supplemental Table 6 ). Male patients meeting at least three CSBD criteria ( Kraus et al., 2018 ) and four for hypersexual disorder ( Kafka, 2010 ) and none of the above-mentioned disorders were invited to participate in this study.

Ethics

All procedures were approved by the Ethical Committee of Institute of Psychology, Polish Academy of Sciences. All participants were provided informed written consent prior to involvement in the study.

Results

The average BPS score for treatment-seeking men was 7.50 ( SD = 2.58) and was significantly higher than in non-treatment-seeking men, 3.56 ( SD = 3.12), z = 14.66, P < 0.001, Cohen's d = 1.38. We assessed the classification quality of the a priori selected group of patients ( n = 105) against all males from the control group (Study 4, n = 191) (see Fig. 1 for ROC curve). The ROC curve captured an area of 82.2% of the 5 test items ( SE = 0.02; P < 0.001), characterized by 95% confidence intervals with limits of 77.5% and 86.9%. As shown in Table 4 , the proposed cut-off value is 4, for which sensitivity is 58.42%, specificity 90.48%, positive predictive value 91.74% (95% CI 85.88%–95.30%), negative predictive value 54.60% (95% CI 50.12%–59.00%), and accuracy 69.83% (95% CI 64.24%–75.02%). A cut-off value of 5 is characterized by a sensitivity of 68.42% and a specificity of 83.81% (see Table 4 ).

Fig. 1.

Study 5, ROC curve for the Polish-adapted BPS for those seeking treatment for problematic use of pornography (Score of 4 or higher)

Table 4.

ROC analysis for proposed Brief Pornography Screen (BPS) with suggested cut-off scores

Statistic	Value of 4 on the BPS		Value of 5 on the BPS
	Value	95% CI	Value	95% CI
Sensitivity	58.4%	51.1–65.5%	68.4%	61.3–75.0%
Specificity	90.5%	83.2–95.3%	83.8%	75.6–90.3%
Positive Likelihood Ratio	6.13	3.36–11.20	4.23	2.71–6.60
Negative Likelihood Ratio	0.46	0.38–0.55	0.38	0.30–0.47
Disease prevalence	64.4%	58.7–69.9%	64.4%	58.7–69.9%
Positive Predictive Value	91.7%	85.8–95.3%	88.4%	83–92.3%
Negative Predictive Value	54.6%	50.1–59%	59.5%	53.9–64.8%
Accuracy	69.8%	64.2–75%	73.9%	68.5–78.8%

To examine changes in PPU among treatment-seeking patients, we compared BPS scores of 57 men from our clinical sample before and after two months of pharmacotherapy with naltrexone or paroxetine ( Gola et al., 2019 ) using a dependent sample t -test. BPS scores differed following treatment ( t (56) = 6.75; P < 0.001, Cohen's d = 1.80), with higher BPS scores before therapy ( M = 8.54; SD = 1.77) than after two months of therapy ( M = 5.75; SD = 2.97).

Discussion

The current study evaluated the BPS, a brief screening tool, for identifying probable PPU. The robust sampling technique used in our studies has not been previously used when developing scales designed to assess PPU. Overall, the BPS is psychometrically sound as demonstrated by measures of reliability and validity across multiple samples, providing initial support for its use in clinical practice, though additional research is needed to determine its clinical utility for treatment-seeking individuals fully.

Prior work has consistently shown that men, relative to women, view and masturbate to pornography more regularly ( Bothe et al., 2018 ; Grubbs, Wilt, Exline, & Pargament, 2018a ; Wright, 2013 ), and this finding was observed across all five of the samples. Consistent with past research, we found that men, as compared to women, reported more concerns with pornography use ( Bothe et al., 2018 ; Kor et al., 2014 ). Our study is unique in that we examined the psychometric properties among five different samples (e.g., U.S. veterans, two U.S. general adult samples, Polish adults, and Polish male patients undergoing treatment for CSBD). Given the diversity of the samples we recruited to evaluate the psychometric properties of the BPS, we believe the findings have generalizability for both clinical and non-clinical groups from different countries. However, with that said, caution is still advised, and we recommend further research to validate the BPS for clinical populations, particularly among women and sexual and gender minorities seeking treatment for PPU.

Our initial examination of the proposed six-item screen in Study 1 revealed that one item was unbalanced, and further analysis suggested removing it. Across the studies, the five-item screen demonstrated high internal consistency as well as construct, convergent, discriminant, and criterion validity. As expected, BPS scores correlated strongly with other preexisting scales assessing PPU (e.g., the CPUI-9 ( Grubbs et al., 2015 ) and PPUS ( Kor et al., 2014 )) while only moderately correlating with symptom severity measures assessing hypersexuality ( Reid, Garos et al., 2011a ; Reid, Li et al., 2011b ) or sexual addiction ( Gola et al., 2017b ). Thus, the screen is more closely associated with measures assessing dimensions of PPU but is still associated with general measures related to CSBD (e.g., impaired control, failed attempts to quit). We did not intend the BPS to serve as a proxy of CSBD. Research suggests, however, that PPU is one of the most commonly reported problems among individuals seeking mental health treatment for CSBD ( Kraus, Meshberg-Cohen, Martino, Quinones, & Potenza, 2015a ; Kraus, Potenza et al., 2015b ; Reid et al., 2012 ). Therefore, the BPS may be a useful tool for detecting possible PPU among individuals seeking treatment for CSBD. Additional clinical interviews are needed to determine the presence of CSBD, which could be manifested as PPU among treatment-seeking individuals with different clinical presentations ( Kraus & Sweeney, 2019 ).

We also found that, in general, BPS scores were weakly correlated with impulsivity ( Cyders et al., 2014 ; Lynam et al., 2006 ) and obsessive-compulsive features ( Foa et al., 2002 ). In support of prior work ( Bőthe et al., 2018 , 2019 ), BPS scores were moderately correlated with measures of generalized feelings of distress and depression; we also found moderate correlations between BPS scores and measures of feeling addicted to pornography and prioritizing pornography viewing over other activities ( Grubbs, Perry, Wilt, & Reid, 2019c ). As noted elsewhere ( Kor et al., 2014 ), we also found a modest correlation between pornography viewing and PPU as measured by the BPS, although the relationship appeared stronger between BPS scores and masturbation frequency. We anticipated such associations between pornography viewing behavior and BPS scores. As discussed in other works ( Gola, Lewczuk, & Skorko, 2016 ; Kraus, Martino, et al., 2016a ; Bőthe et al. 2020 ), we also found that frequency of pornography viewing is not necessarily an indicator of PPU. Among both U.S. national samples, we found high proportions of individuals (predominately men) scoring at least four or greater on the BPS. ¹

Additional research is needed around establishing norms for the BPS for pornography use, which may vary based on gender, age, and possibly other socio-economic factors. Further, research is still evolving on the study of pornography use, and more work is needed to identify both risk and protective factors associated with PPU. Further, the recruitment of large, female samples would allow for greater examination of possible gender effects when studying PPU in nonclinical and clinical samples. There is a particular need for investigating PPU among women reporting high levels of pornography use (i.e., daily, several times a day). This group was not equally represented in our samples and across the board, women using pornography typically reported lower levels as compared to men. Results specific to women, in general, should be cautioned, since our results were likely impacted by the small sample size, and further research examining gender-related differences in PPU women is recommended. As has been done in a recent study ( Bőthe et al. 2020 ), we also recommend that gender invariance testing with the BPS be conducted to investigate further its psychometric properties with women or other diverse groups.

A primary strength of our current study is that we included a sample of treatment-seeking men for CSBD to determine the sensitivity and specificity of a brief screen for PPU. Specifically, in Study 5, we independently examined PPU among 105 men enrolled in a randomized clinical trial for CSBD. After comparing CSBD patients to non-affected control participants, we determined the initial clinical cut-off score on the BPS to be four. As we interpret it currently, a score of four or higher on the BPS should warrant further evaluation by a health care professional for PPU. However, scores among Polish treatment-seeking men (self-identified as heterosexual) and veterans interested in treatment for PPU reported scores well above 6. It is possible that the clinical cut-off is minimally at four, with a score of six or higher, possibly reflecting the need for clinical services. Further refinement with clinical and non-clinical samples to determine the optimal cut-off score on the BPS is warranted. The proposed cut-off score should be interpreted cautiously at present.

Although promising, the study has multiple limitations. First, although four of the five samples included women, additional research on PPU among women and diverse populations are needed to address gender- and diversity-related considerations. Preliminary data suggest that women are seven times less likely than men to seek treatment for PPU ( Lewczuk, Szmyd, Skorko, & Gola, 2017 ). An additional limitation is that we only recruited a sample of heterosexual Polish men for determining the clinical cut-off score for the BPS, and future work is needed to determine the threshold for women as well as clinical populations from other countries and individuals of different sexual orientations. At this time, we do not have evidence to suggest that there should be different cut-off scores for men and women or other specific groups. We suspect that further study of PPU among large, diverse samples of men and women, sexual and gender minority populations, and other groups, including clinical and nonclinical samples, will help identify optimal cut-off scores for identifying individuals with probable PPU.

Further, we acknowledge that additional research is also needed to validate the BPS and other PPU measures in non-Western countries and in samples with ethnic diversity and in sexual minority groups. An overrepresentation of research from Western countries has limited our understanding of PPU among diverse cultures and ethnic groups. It is possible that the suggested cut-off score on the BPS may vary based on gender or cultural considerations, and additional work is needed to determine appropriate thresholds for clinical and nonclinical groups. Building on this, future multicultural and multi-sample studies assessing utility and measurement invariance of the BPS are needed. An additional limitation is that we did not use clinical interviews for four of the five studies since we relied on web-based designs given the costs and difficulties of recruiting large groups of men and women from various backgrounds. Scores and responses may vary to some degree when the scale is administered face-to-face by a clinician. Furthermore, in future studies with larger, more diverse samples with clinical confirmation via interviews, item-response theory (IRT) could be used to better determine where individuals are positioned across the continuum of PPU, and pornography use more generally, using the BPS and providing further clarity and refinement of potential cut-off scores. Additionally, because Study 5 only contained recruited men who self-identified as heterosexual, we recommend further research with the BPS to include gay and bisexual men and other sexual minorities when determining possible cut-off scores for PPU.

The utility of the BPS as a clinical tool should be considered separate from its utility as a tool for understanding PPU in population studies. More to the point, future work should specifically examine and describe the best use and interpretation of BPS scores in clinical versus non-clinical samples. As discussed elsewhere ( Kraus & Sweeney, 2019 ), it is important to investigate PPU among treatment-seeking individuals and understand the motives underlying treatment-seeking behavior. Both motivations and barriers to care for PPU have not yet been fully examined and require additional attention. Currently, we propose that a positive screen on the BPS should not be interpreted as diagnostic of an underlying mental health disorder. As the BPS does not query regarding interference in major areas of life functioning as detailed in the diagnostic criteria for CSBD, such assessment should be conducted clinically for individuals screening positive on the BPS. Future research is needed to test and validate the BPS among diverse populations using both web-based and in-person designs. Other factors, such as moral incongruence and psychiatric (substance use, bipolar) and medical (dementia, Parkinson) conditions, should be considered when assessing PPU and considering treatment recommendations ( Brand, Antons, Wegmann, & Potenza, 2019 ; Grubbs & Perry, 2019 ; Grubbs, Perry, Wilt, & Reid, 2019c ; Grubbs, Wilt, Exline, Pargament, & Kraus, 2018b ; Kraus & Sweeney, 2019 ). Researchers ( Štulhofer, Bergeron, & Jurin, 2016a ; Štulhofer, Jurin, & Briken, 2016b ) have also noted that factors such as high sexual desire remained difficult to tease apart from hypersexuality, which raises concerns with regard to how PPU is conceptualized. Further research examining high sexual desire and/or behaviors among diverse groups is needed as researchers and clinicians develop tools to assess PPU accurately. Similar considerations exist for assessing moral incongruence as described in the criteria for CSBD.

Most notably, further research is needed to assess the test-retest and sensitivity and specificity among clinical and non-clinical samples using the BPS. Given the brevity of the BPS (1–2 min to complete), additional research should pilot its use in medical and health settings for identifying individuals with PPU who would benefit from treatment. In conclusion, our initial examination of the BPS suggests that it is psychometrically sound, short, and easy to use in clinical and non-clinical settings with high potential for use in populations across international jurisdictions.

Funding sources

The authors disclosed receipt of the following financial support for the research, authorship, and publication of this article. Study 1 was funded by support from the Department of Veterans Affairs Office of Research and Development, Clinical Science Research and Development (ZDA1, PI Rani A. Hoff) and VISN 1 New England MIRECC (PI Shane W. Kraus). Studies 2 and 3 were supported by institutional funds provided by Bowling Green State University (PI Joshua Grubbs). Studies 4 and 5 were supported by the National Science Centre of Poland (2014/15/B/HS6/03792; PI M. Gola).

Steven D. Shirk, Steve Martino, and Rani A. Hoff are full-time employees of the Department of Veterans Affairs. Dr. Potenza has received support from the Connecticut State Department of Mental Health and Addiction Services, the Connecticut Mental Health Center, and the Connecticut Council on Problem Gambling. Drs. Kraus, Potenza and Shirk have received support from the National Center for Responsible Gaming. The funding agencies did not provide input or comment on the content of the manuscript, and the content of the manuscript reflects the contributions and thoughts of the authors and do not necessarily reflect the views of the funding agencies.

Authors' contribution

SWK conceptualized and wrote the initial draft. SWK, RAH, MNP, and SM contributed to the data collection and data analysis of Study 1. JBG contributed to the data collection and analysis of Studies 2 and 3. MG, EK, and ML contributed to the data collection and analysis for Studies 4 and 5. SDS provided statistical oversight for Study 1 and guidance for the other studies. All authors provided input, read, and reviewed the manuscript prior to submission. SWK and the other authors approved the final draft of the manuscript.

Conflict of interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and publication of this article.

Supplementary Material

jba-9-259-s001.docx ^{(63.4KB, docx)}

Appendix A.

Brief Pornography Screen (BPS)	Date:
	ID#:
Directions : In the last 6 months, have any of these situations happened to you in regards to your use of pornography?	Never	Occasionally	Very Often
You find yourself using pornography more than you want to.	0	1	2
You have attempted to “cut back” or stop using pornography, but were unsuccessful.	0	1	2
You find it difficult to resist strong urges to use pornography.	0	1	2
You find yourself using pornography to cope with strong emotions (e.g., sadness, anger, loneliness, etc.).	0	1	2
You continue to use pornography even though you feel guilty about it.	0	1	2

Scoring . A score of 4 ≥ is considered a positive screen for possible problematic pornography use. Additional examination for possible problematic pornography use is encouraged.