Development of the Psychological Insight Questionnaire among a sample of people who have consumed psilocybin or LSD

Abstract

Background:

Several measures have been developed to examine acute psychedelic effects (e.g. mystical-type and challenging experiences), but no measure assesses acute psychologically insightful experiences that may occur during psychedelic experiences.

Aim:

The purpose of this study was to develop and examine the psychometric properties of the Psychological Insight Questionnaire.

Method:

A cross-sectional survey study among psilocybin and LSD users. Respondents (n=1661; M_age=22.9, standard deviation=8.5; Caucasian/White=83%; non-Hispanic=91%; men=72%; United States resident=66%) completed an Internet-based survey.

Results:

The Psychological Insight Questionnaire consists of 23 items with two subscales: (a) Avoidance and Maladaptive Patterns Insights and (b) Goals and Adaptive Patterns Insights. Construct validity of the Psychological Insight Questionnaire was supported by strong correlations of the Psychological Insight Questionnaire (and Avoidance and Maladaptive Patterns Insights and Goals and Adaptive Patterns Insights subscales) scores with the insight subscale of the Session Impacts Scale, and weak-to-moderate correlations with the Mystical Experiences and Challenging Experiences Questionnaires. Furthermore, Psychological Insight Questionnaire (and Avoidance and Maladaptive Patterns Insights and Goals and Adaptive Patterns Insights subscales) scores were moderately-to-strongly correlated with retrospectively reported increases in psychological flexibility, and well-being/life satisfaction that were attributed to a memorable psychedelic experience. Lastly, incremental validity was established showing that the Psychological Insight Questionnaire (and Avoidance and Maladaptive Patterns Insights subscale) scores predict unique variance in changes in psychological flexibility, and Psychological Insight Questionnaire (and Avoidance and Maladaptive Patterns Insights and Goals and Adaptive Patterns Insights subscales) scores predict changes in well-being and life satisfaction, beyond measures of acute mystical-type and challenging effects.

Conclusions:

The Psychological Insight Questionnaire has the potential to extend the understanding of the acute and enduring effects of psychedelics. Further longitudinal research is necessary to determine the long-term predictive validity of the Psychological Insight Questionnaire and to examine the role of psychological insight in predicting therapeutic outcomes.

Introduction

Several measures have been developed for assessment of acute subjective psychedelic drug effects that occur in the laboratory and in naturalistic settings (Davis et al., 2018, 2020; Griffiths et al., 2006, 2011; Kometer et al., 2015; Roseman et al., 2018; Russ et al., 2019). For example, the States of Consciousness Questionnaire (Griffiths et al., 2006), Hallucinogen Rating Scale (Strassman et al., 1994), and Altered States of Consciousness Rating Scale (Studerus et al., 2010) were developed as standardized assessment tools to capture as many domains of the psychedelic experience as possible, including sensorial effects, cognitive effects, effects on perception, feelings, and beliefs. Although these measures examine the breadth of states occasioned by psychedelics, they are time consuming (typically consisting of hundreds of items or more), making it burdensome for repeated assessment. Furthermore, these extensive measures require that researchers, who seek to examine a specific aspect of the psychedelic experience (e.g. mystical effects or challenging experiences), administer the entire pool of items thus increasing assessment burden. Therefore, shorter measures of specific psychedelic experiences were developed using these larger questionnaires.

One of the most researched psychedelic effects is the “mystical” experience (Barrett et al., 2015; Griffiths et al., 2006, 2011). Initially based on the work of Stace’s conceptual model (1960) and Hood's Mysticism Scale (Hood, 1975; Hood et al., 1993), the Mystical Experiences Questionnaire (MEQ) was created using the mystical-type experience items from the SOCQ (MacLean et al., 2012) and has become a widely used measure of acute mystical-type experiences. The MEQ originally had 43 items (MEQ43), and a psychometric examination of the MEQ retained 30 items and showed that it had a four-factor structure: (a) mystical (freedom from the limitations of one’s personal self and feeling a unity or bond with what was felt to be greater than one’s personal self), (b) positive mood (a sense of awe or awesomeness), (c) transcendence of time and space (the sense of being outside of time, beyond past and future), and (d) ineffability (the sense that the experience cannot be described adequately in words) (Barrett et al., 2015; MacLean et al., 2012). The MEQ total scale score and the factor scores have demonstrated good internal consistency reliability, alphas ranged between 0.80–0.93 (MacLean et al., 2012) and 0.86–0.97 (Barrett et al., 2015), and scores were moderately-to-strongly correlated with the Hood Mysticism Scale, demonstrating convergent validity (MacLean et al., 2012). Furthermore, the MEQ has been shown to predict long-term positive therapeutic outcomes in a variety of laboratory and naturalistic studies of people with alcohol use problems, depression, and anxiety (Davis et al., 2020; Garcia-Romeu et al., 2019; Griffiths et al., 2016; Ross et al., 2016), and across a variety of psychedelic substances (Barsuglia et al., 2018; Bouso et al., 2016; Davis et al., 2019; Griffiths et al., 2008; Liechti et al., 2017; Schmid and Liechti, 2018).

Although mystical-type experiences are often occasioned by ingesting a psychedelic substance, challenging affective, physiological, and cognitive experiences also sometimes occurs (Barrett et al., 2016, 2017). To examine this feature of the psychedelic experience more precisely, the Challenging Experience Questionnaire (CEQ; Barrett et al., 2016) was created based on the Hallucinogen Rating Scale (HRS) (Strassman et al., 1994), the Altered States of Consciousness Rating Scale and the Five-Dimensional Altered States of Consciousness Rating Scale (5DASC) (Dittrich, 1975, 1998; Studerus et al., 2010). The CEQ was developed by examining the psychometric properties of items completed in responses to an Internet survey of people who reported having had a challenging experience after ingesting psilocybin (Barrett et al., 2016). This procedure identified seven factors of challenging experiences: (a) fear or feeling frightened, (b) grief or feelings of despair and sadness, (c) physical distress, (d) isolation or loneliness, (e) paranoia, (f) feelings of insanity or loss of sanity, and (g) feeling as if one was dead or dying (Barrett et al., 2016). Results from that study further demonstrated the reliability, internal validity, and convergent validity for the CEQ. The CEQ was subsequently included in several studies of acute psychedelic effects, demonstrating that the CEQ can provide a standardized measure of acute challenging effects across different substances and settings (Barrett et al., 2016, 2017; Carbonaro et al., 2016; Davis et al., 2018, 2019; Haijen et al., 2018; Roseman et al., 2019; Russ et al., 2019).

Recent studies have examined the role of insight experiences that may occur as part of a psychedelic experience (Garcia-Romeu et al., 2019; Roseman et al., 2018; Studerus et al., 2010). Garcia-Romeu et al. (2019) conducted an anonymous online survey assessing the potential therapeutic effect of psychedelics used in the natural environment among people with an alcohol use disorder. Measurement included ratings of persisting effects (Persisting Effects Questionnaire (PEQ); Doblin, 1991; Griffiths et al., 2006; Pahnke, 1969) wherein participants were asked to rate the personal meaning, psychological challenge, psychological insight, and spiritual significance of their psychedelic experience (Garcia-Romeu et al., 2019). Results indicated a significant reduction or cessation of alcohol consumption following the psychedelic experience and there were positive correlations between mystical-type and insight experiences and between these acute psychedelic experiences and reductions in alcohol use (Garcia-Romeu et al., 2019).

In another recent study, Roseman et al. (2018) hypothesized that Oceanic Boundlessness (OBN), one of the factors of the Altered States of Consciousness Questionnaire (ASC), would predict outcomes of psilocybin-assisted therapy of treatment-resistant depression. Roseman et al. (2018) speculated that overcoming psychological resistance during a psilocybin experience may lead to emotional breakthrough and insight as measured by the “insightfulness” sub-factor. The results indicated that OBN was predictive of decreased depression at follow-up, and that ratings on the “insightfulness” sub-factor of the OBN were significantly higher in the treatment responder group (Roseman et al., 2018). Additionally, qualitative reports of acute psychedelic experiences have been published (Belser et al., 2017; Gasser et al., 2015; Watts et al., 2017), indicating that insight is a common feature of meaningful or therapeutic psychedelic experiences. Nevertheless, despite the preliminary findings highlighting the association between insight (not otherwise specified) and positive outcomes in individuals with alcohol use or depression after ingesting psychedelics, no measure of specific insights has been developed or psychometrically validated for assessing these acute effects. For example, the insightfulness subscale of the OBN is comprised of three items, and the persisting effects questionnaire has one item, yet neither measure assesses the broad range of different types of insight experiences that may occur during a psychedelic experience (Doblin, 1991; Griffiths et al., 2006; Pahnke, 1969; Roseman et al., 2018; Studerus et al., 2010). Similarly, recent development of the Emotional Breakthrough Inventory (Roseman et al., 2019) captures qualities of emotional catharsis which may be related to psychological insights as they pertain to discoveries about emotional capacities but lacks inclusion of other types of acute insight experiences (e.g. awareness about relationships, past events, goals, values, etc.).

Given that insight is one of the most commonly discussed constructs in the general psychotherapy literature (Gibbons et al., 2007), we considered adapting an existing measure for use in psychedelic studies. However, disparate operational definitions of insight limited such application of existing measures (Gibbons et al., 2007; Johansson et al., 2010; Lacewing, 2014; Moro et al., 2012; Sondhi et al., 2016). For example, the term insight has been used in psychiatric settings to convey a patient’s recognition of having an illness, understanding of contributing psychological factors, or belief that treatment could be helpful (Cleary et al., 2014; Rosenbaum et al., 1956). It has also been defined as the degree of clarity of understanding of one’s thoughts, feelings and behavior (Grant et al., 2002), awareness of internal states (Sutton, 2016), overconfidence in interpreting experiences (Beck et al., 2004), or capacity for introspection and self-examination (Conte et al., 1996). Accordingly, several measures with disparate underlying operational definitions of insight have been developed, including the Birchwood Insight Scale for Psychosis (Cleary et al., 2014), Self-Awareness Outcomes Questionnaire (Sutton, 2016), Self-Reflection and Insight Scale (SRIS; Grant et al., 2002), Beck Cognitive Insight Scale (BCIS; Beck et al., 2004), and Psychological Mindedness Scale (Conte et al., 1996).

Although several psychotherapy insight measures exist, none of these measures assess acute insight experiences in the context of a discrete event (i.e. gaining awareness, knowledge, discovery of relationships, emotions, past events, etc.) which further limits their application to research on psychedelic effects and, for that matter, their application to acute psychotherapy effects as well. The only measure that was found to gauge the acute type of insight associated with a discrete event is the Session Impacts Scale (SIS), which contains a five-item subscale self-report measure of the “impacts” of psychotherapy sessions (i.e. a client’s specific reaction to a therapy session) (Elliott and Wexler, 1994). Elliott and Wexler’s (1994) psychometric analyses found that the SIS demonstrated internal reliability and convergent validity. However, a later comment by Mallinckrodt (1994) questioned whether the measure had test-retest reliability. Although the SIS assesses acute discrete event-associated insight similar to that which would be desirable for measuring psychedelic effects, it does not assess the various types of insight one might have during a psychedelic experience, thereby decreasing its utility in measuring psychedelic effects across different psychedelic substances and populations of users.

Current study

The present study was designed to develop and test the psychometric properties of a new measure of acute experiences of psychological insight, defined as realizations or discoveries about personality, relationships, behavioral patterns or emotions. Using an Internet-based recruitment and data collection procedure, the primary aim of this study was to create a brief measure of acute insight that could be used in future clinical trials, and to test whether it demonstrates good internal consistency reliability, and whether data supported the discriminant, convergent, and predictive validity of the newly devised questionnaire.

Method

Development of Psychological Insight Questionnaire (PIQ)

Because we could find no other measures in the published literature that assessed psychological insight defined as a range of realizations or discoveries about personality, relationships, behavioral patterns or emotions that are often subsumed under the construct of psychological insight, we began development of this measure through conversations with psilocybin session facilitators on our study team. We drafted a list of 38 potential items believed to assess different aspects of insight that participants have described following psilocybin sessions. Further consideration of the content of these items was conducted by several of the authors (AKD, TCS, RRG) who met several times to revise items for clarity of expression. Throughout this revision process, items were removed that appeared to overlap substantially with other items, or when items did not appear to measure a specific type of insight. The final draft of the initial measure for this study included 28 items (see Table 1).

Table 1.

Component loadings and item selection in the exploratory dataset.

Item	Item mean (standard deviation)	PC1	PC2	h 2
Awareness of uncomfortable or painful feelings I previously avoided	2.8 (1.8)	0.82	0.03	0.67
Discovered I could explore uncomfortable or painful feelings I previously avoided	3.0 (1.7)	0.77	0.21	0.64
Awareness of dysfunctional patterns in my actions, thoughts, and/or feelings	3.3 (1.6)	0.74	0.16	0.57
Realized the nature and/or origins of my defenses or other coping strategies	2.7 (1.7)	0.74	0.31	0.64
Realized how current feelings or perceptions are related to events from my past	3.3 (1.6)	0.72	0.17	0.55
Discovered a clear pattern of avoidance in my life	2.4 (1.8)	0.70	0.20	0.53
Gained resolution or clarity about past traumas or hurtful events	2.4 (1.9)	0.67	0.29	0.53
Gained a deeper understanding of events/memories from my past	3.2 (1.6)	0.66	0.35	0.56
Realized I could experience memories previously too difficult to experience	2.0 (1.8)	0.65	0.29	0.50
Discovered how aspects of my life are affecting my well-being	3.7 (1.4)	0.64	0.32	0.51
Realized ways my beliefs may be dysfunctional	3.3 (1.7)	0.62	0.17	0.42
Discovered clear similarities between my past and present interpersonal relationships	2.5 (1.8)	0.61	0.36	0.51
Realized how critical or judgmental views I hold towards myself are dysfunctional	3.1 (1.7)	0.59	0.34	0.47
Realized certain actions I should take in regards to important relationships in my life	3.2 (1.7)	0.50	0.44	0.45
Discovered new feelings or perspectives about significant relationships in my life	3.5 (1.6)	0.46	0.45	0.41
Realized the point of view or actions of others that had been difficult to understand previously	3.0 (1.7)	0.45	0.36	0.33
Awareness of my life purpose, goals, and/or priorities	3.6 (1.5)	0.09	0.72	0.53
Awareness of beneficial patterns in my actions, thoughts, and/or feelings	3.1 (1.5)	0.23	0.70	0.54
Experienced validation of my life, character, values, or beliefs	3.5 (1.5)	0.10	0.67	0.46
Realized the importance of my life	3.6 (1.6)	0.09	0.67	0.45
Discovered clarity or creative solutions about how to solve a problem in my life a	3.0 (1.7)	0.36	0.63	0.53
Awareness of information that helped me understand my life	3.9 (1.3)	0.23	0.62	0.44
Discovered new actions that may help me achieve my goals	3.1 (1.6)	0.32	0.60	0.46
Discovered ways to see my problems with more clarity	3.5 (1.6)	0.45	0.58	0.54
Discovered new insights about my work or career	2.2 (1.9)	0.23	0.53	0.33
Discovered specific techniques for coping with difficulties	2.8 (1.7)	0.40	0.51	0.41
Gained a deeper understanding of previously held beliefs and/or values	3.7 (1.3)	0.23	0.51	0.31
Discovered a vivid sense of the paradoxes in life	3.5 (1.6)	0.17	0.50	0.28

PC1: loading onto the first principal component; PC2: loading onto the second principal component; h²: item communality, or the sum of the squared factor loadings for a given item.

Items means (standard deviations) were not statistically different in the psilocybin and LSD subsamples (data available upon request). Items are sequenced by loading on PC1. Italic font indicates five items that were removed due to loading 0.4 or more on both factors after the first or second principal component analysis. Bolded coefficients indicate items retained in the final questionnaire from each principal component. The full questionnaire with items, response options, and scoring instructions are available in supplementary materials or from the corresponding author.

^aThis item loaded ⩾0.4 on both factors based on the second principal component analysis after removing the first four items based on the first principal component analysis.

Study procedure

Volunteers were invited to complete an anonymous Internet-based cross-sectional survey of individuals who reported having an experience of gaining psychological insight as a result of consuming a psychedelic substance. Recruitment included the use of electronic message postings and advertisements on several websites (e.g. www.erowid.com; www.facebook.com; www.reddit.com), electronic mail announcements, and via an article published by Motherboard/VICE (www.vice.com). Advertisements led potential respondents to a secure Web-based survey (www.qualtrics.com) that began with a consent document detailing the inclusion criteria (i.e. being at least 18 years old, able to read, write and speak English fluently, having taken a dose of a single psychedelic that produced moderate to strong psychoactive effects, and affirming that after taking the substance they had an experience to which they attributed gaining psychological insight). After completing informed consent, respondents completed the survey questions. The study was approved by the Johns Hopkins University School of Medicine Institutional Review Board. No incentives were given for participation in this study.

Study recruitment flow

Recruitment occurred from June 2018–February 2019. A total of 15,470 people clicked the advertisement link, and 3177 consented and began the survey. Of these, 996 respondents were excluded because they reported using more than one substance in their psychedelic experience. Of the remaining 2181, 398 were excluded because they indicated using a psychedelic other than Lysergic acid diethylamide (LSD) or psilocybin (other psychedelics were excluded in order to more precisely examine insight effects among the two most often used psychedelic drugs), and 122 were excluded because their qualitative responses revealed potentially poor or inconsistent responding (e.g. reporting more than one substance used in the experience in a text response item but reporting only one substance used in a different item). Thus, the final sample was comprised of 1661 respondents.

Study measures

Demographics

The survey included questions about the age, education, sex, race, country/region of residence, income, and relationship status of each respondent.

Identification of single most insightful experience after taking psilocybin or LSD

For this survey, respondents were instructed to think back on the single most insightful experience they had after taking psilocybin or LSD and to complete a series of questionnaires (see below) based on that experience.

Acute insight experiences

We created the PIQ to assess the intensity with which respondents experienced a broad range of acute insight (e.g. gained an awareness into emotions, behaviors, beliefs, memories, or relationships) after taking a classic psychedelic. Respondents were asked to rate the intensity (on a six-point scale: from 0=“No; not at all” to 5=“Extremely (more than ever before in my life)”) with which they experienced each of 28 insight experiences at any point during their session. See Table 1 for items and item means and standard deviations. See the Supplementary Material for final version of the PIQ, wording of instructions to respondents and response options, and a description of scoring.

Psychedelic experience

Respondents were asked to report which psychedelic substance they had used and the subjective strength of dose (i.e. low, moderate, moderately high, high) that they consumed.

Acute mystical experiences

The MEQ was used to assess the degree to which subjective mystical-type phenomena occurred during the reported psychedelic experience (Barrett et al., 2015; MacLean et al., 2012). Respondents were asked to rate the intensity of the identified experience (on a six-point scale from 0="None; not at all" to 5="Extreme (more than ever before in my life)") with which they experienced each of 30 items during their session. Similar to prior research (Davis et al., 2018, 2019, 2020), we calculated a mean score of all items on the questionnaire. Internal consistency of the total scale was excellent (alpha=0.95).

Acute challenging experiences

The CEQ was included to measure the degree to which challenging experiences occurred during the reported psychedelic experience (Barrett et al., 2016). Respondents were asked to rate the intensity of the identified experience (using a six-point scale from 0="None; not at all" to 5="Extreme (more than ever before in my life)") with which they experienced each of 26 challenging psychological or physical item descriptions during the psychedelic session. Similarly to prior research (Davis et al., 2018, 2019, 2020), we calculated a total mean score to measure the overall intensity of challenging experiences during the session. Internal consistency of the total scale was excellent (Cronbach’s alpha=0.95).

Session impacts scale

The insight-specific subscale from the SIS (Elliott and Wexler, 1994) was developed to measure overall general insight that might occur during a specific period of time. In this study the scale was used to assess overall general insight that occurred during the reported psychedelic experience. The SIS-Insight subscale is a five-item self-report measure of the experienced impacts of psychotherapy sessions and measures the specific contents of clients' reactions to these sessions (e.g. “I now have new insight about myself or have understood new things about me;” “I now have new insight about another person or have understood something new about someone else”). Respondents were asked to rate all items using a five-point scale from 0="Not at all" to 4="Very much." Internal consistency of the total scale was good (Cronbach’s alpha=0.84).

Changes in psychological flexibility

The Acceptance and Action Questionnaire II (AAQII; Bond et al., 2011) was developed to measure psychological inflexibility. In the present study the AAQII was used to examine the degree of changes in psychological flexibility that respondents experienced from before to after the identified psychedelic experience. The AAQII is a seven-item measure and respondents were asked to respond to each item on a scale from 1=“Never true” to 7=“Always true.” Lower scores on this measure are indicative of greater psychological flexibility, however, for this analysis we reversed the scores so that higher scores are indicative of greater psychological flexibility to aid in interpretation of our findings. Internal consistency was good for the psychological flexibility before measure (alpha=0.91) and the psychological flexibility after measure (alpha=0.88). Changes in psychological flexibility were examined by calculating an overall mean change score. To calculate this score, we subtracted the mean psychological flexibility before score from the mean psychological flexibility after score (thus, the higher the change score the greater the increase in psychological flexibility). This mean change score was then used in analyses.

Changes in well-being and satisfaction with life

The Satisfaction with Life Scale (SWLS; Pavot and Diener, 2008) was used to examine the degree of changes in well-being and satisfaction with life that respondents experienced from before to after the identified psychedelic experience. The SWLS is a five-item measure and respondents were asked to respond to each item on a scale from −3="strongly disagree" to 3="strongly agree." Higher scores are indicative of greater well-being and satisfaction with life. Internal consistency reliability was good for the SWLS before measure (alpha=0.86) and the SWLS after measure (alpha=0.82). Changes in well-being and satisfaction with life were examined by calculating an overall mean change score. To calculate this score, we subtracted the mean SWLS score before the experience from the mean SWLS score after the experience (thus, the greater the score the greater the increase in well-being and life satisfaction). This mean change score was then used in analyses.

Results

Respondent characteristics

Sample demographics are provided in Table 2. The sample was comprised primarily of young adult (M age=22.9, standard deviation (SD)=8.5), Caucasian/White (83%), non-Hispanic (91%), men (72%), from the USA (66%). Approximately one-half (50%) of the sample reported income greater than or equal to US$50,000, and over one-half (58%) reported a trade/vocational school or college degree.

Table 2.

Sample demographics and comparison between exploratory and confirmatory strata.

	Stratum					Total sample
	Exploratory	Confirmatory
n	652	759	t	df	p	1661
Age			0.596	1395	0.551
Mean	22.7	22.9				22.9
Standard deviation	8.4	8.8	χ2	df	p	8.5
Education			0.794	3	0.851
Less than college degree	45.3%	43.1%				41.7%
Associate or technical/trade degree	12.4%	13.2%				13.8%
Bachelor’s degree	25.9%	26.2%				26.0%
Master’s or doctoral degree	16.4%	17.5%				18.5%
Sex			0.853	2	0.653
Male	77.1%	75.2%				72.0%
Female	22.7%	24.5%				26.6%
Intersex	0%	0%				0.1%
Other	0%	0%				0.2%
Prefer not to answer	0.1%	0.3%				1.1%
Race			0.253	3	0.969
Caucasian/White	92.5%	92.2%				83.4%
African American/Black	0%	0%				0.7%
Asian	1.1%	0.9%				3.0%
Native Hawaiian/Pacific Islander	0%	0%				0.1%
Native American	0%	0%				0.7%
Mixed race	4.8%	4.9%				7.1%
Other	1.7%	2.0%				4.9%
Psychedelic substance			0.030	2	0.985
Psilocybin	42.8%	42.3%				43.6%
LSD	57.2%	57.6%				56.4%
Subjective dose strength			0.048	7	0.9062
Low	4.7%	4.9%				6.3%
Moderate	41.9%	42.0%				40.3%
Moderately high	34.5%	34.7%				33.8%
High	18.9%	18.4%				19.5%

Matched exploratory and confirmatory analysis strata

The sample of 1661 responses was stratified into two sub-samples that were matched on age, sex, education, race, dose level, and frequency of each psychedelic substance (psilocybin, LSD). Stratification was carried out with the strata function of the sampling package (Tille and Matei, 2012) in R (R Core Team, 2012), using simple random sampling without replacement. Before stratification, frequencies were calculated for each cell in the factorial model assumed by the stratification variables to verify that there were no cells with fewer than two observations. A total of 250 observations were removed in this procedure, yielding a total scale analysis sample of 1411 (exploratory stratum n=652, confirmatory stratum n=759). After stratification, demographic data were compared between strata using a two-sample t-test for age and chi-squared independence tests for all other demographic variables. Demographic data for the strata are presented in the left-hand portion of Table 2. No variables differed significantly between strata.

Scale development, replication, and reliability

The exploratory stratum was analyzed using principal component analysis with varimax rotation. Parallel analysis indicated that two factors should be extracted. This was confirmed by visual inspection of the scree plot. Any item that loaded less than 0.4 on either component or equal to or more than 0.4 on both components was excluded to yield a simple factor structure. This led to initial exclusion of four items (Table 1; items in italics). Principal component analysis was repeated, and one additional item was removed due to loading of greater than or equal to 0.4 on both components (marked with a note in Table 1). A final principal component analysis yielded a simple component structure. Standardized Cronbach’s alpha, item-total correlations, and average inter-item correlations were then assessed to determine internal consistency of the scale items.

Parallel analysis on the remaining items for each component identified a single principal component for each component. Cronbach’s alpha for component 1 (α=0.93, 95% confidence interval (CI): 0.92–0.94) and component 2 (α=0.85, 95% CI: 0.84–0.87), corrected item-total correlations for component 1 (0.52⩽r⩽0.78) and component 2 (0.48⩽r⩽0.71), and average inter-item correlation for component 1 (r=0.48) and component 2 (r=0.37) indicated high internal consistency between and among the remaining items. No items numerically increased Cronbach’s alpha or average inter-item correlation by more than 0.01 when removed, and thus all remaining items were retained.

The final component structure from the exploratory stratum was then validated using confirmatory factor analysis of data from the confirmatory stratum. Factor loadings for all factors were allowed to vary on their primary factor and were set to 0 for the alternate factor. Fit indices for this model (Root Mean Square Error of Approximation (RMSEA)=0.052 (90% CI=0.045–0.058), Standardised Root Mean Residual (SRMR)=0.027, Comparative Fit Index (CFI)=0.973) indicated acceptable model fit (Hu and Bentler, 1999).

Analysis of individual item means (and SDs) revealed no statistical differences between the psilocybin and LSD subgroups (data available upon request). We named this measure the PIQ and component 1 was named the Avoidance and Maladaptive Patterns Insights (AMP) subscale and component 2 was named the Goals and Adaptive Patterns Insights (GAP) subscale. The AMP and GAP subscales are strongly correlated (r=0.62, p<0.001), and each subscale is strongly correlated to the mean of all items on the PIQ (AMP and PIQ: r=0.96, p<0.001; GAP and PIQ: r=0.82, p<0.001). The final questionnaire with scoring is provided in the Supplementary Material.

Construct validity

The construct validity of the PIQ was explored using the entire sample (n=1661). Data were first assessed for linearity using a scatter plot and an F-test and found to be linear. Next, weak-to-moderate correlations were found between mean PIQ, and AMP and GAP subscale, scores, and mystical-type effects as measured by mean MEQ scores (PIQ: r=0.38, p<0.001; AMP: r=0.27, p<0.001; GAP: r=0.50, p<0.001) and weak correlations between mean PIQ and AMP subscale scores and challenging effects as measured by mean CEQ scores (PIQ: r=0.14, p<0.001; AMP: r=0.21, p<0.001; GAP: r=−0.02, not significant (NS)), thus suggesting that the PIQ was assessing a domain of psychedelic experience not measured by the MEQ or CEQ. Also supporting construct validity, a strong correlation was found between mean PIQ, and AMP and GAP subscale, scores and insight measured by mean SIS scores (PIQ: r=0.69, p<0.001; AMP: r=0.64, p<0.001; GAP: r=0.59, p<0.001). The observation that the PIQ, its subscales, and SIS were strongly correlated, albeit to a modest degree, suggests that the PIQ and SIS measures are related constructs but that the PIQ measures aspects of insight not accounted for by the type of general insights measures by the SIS.

Predictive validity based on retrospective reports of changes in psychological flexibility and well-being/life satisfaction

As an assessment of the possible predictive validity of the PIQ, correlations were examined between the mean PIQ, and AMP and GAP subscale, scores (PIQ range 0–5, M=3.08, SD=1.03; AMP range 0–5, M=2.90, SD=1.22; GAP range 0–5, M=3.36, SD=1.01), and retrospective reports of changes in psychological flexibility (mean AAQII range −4.14–6.00; M=1.55, SD=1.46) and life satisfaction (mean SWLS range −3.00–4.00; M=1.15, SD=1.05). A strong correlation was found between the mean PIQ, and the AMP and GAP subscale, scores, and mean changes in psychological flexibility (PIQ and AAQII: r=0.56, p<0.001; AMP and AAQII: r=0.55, p<0.001; GAP and AAQII: r=0.41, p<0.001) and a moderate correlation was found between the mean PIQ, and the AMP and GAP subscale, scores, and mean changes in general well-being and life satisfaction (PIQ and SWLS: r=0.41, p<0.001; AMP and SWLS: r=0.38, p<0.001; GAP and SWLS: r=0.36, p<0.001).

Incremental validity

As an assessment of the incremental validity of the PIQ scale, two stepwise linear regressions were conducted to examine whether the PIQ contributed unique variance in predicting changes in psychological flexibility (AAQII) and changes in well-being and life satisfaction (SWLS) beyond mystical experiences (MEQ) and challenging experiences (CEQ). In the first regression, predicting mean changes in psychological flexibility, the first model was significant (MEQ: β=0.25, p<0.001; CEQ: β=0.03, p=0.225), F(2,1658)=56.59, p<0.001, adjusted R²=0.06. The second model, adding in mean scores from the PIQ, was also significant (MEQ: β=0.05, p=0.033; CEQ: β=−0.05, p=0.022; PIQ: β=0.55, p<0.001), F change (1,1657)=601.52, p<0.001, adjusted R²=0.31. In the second regression predicting mean changes in well-being and life satisfaction, the first model was significant (MEQ: β=0.29, p<0.001; CEQ: β=0.02, p=0.481), F(2,1658)=78.50, p<0.001, adjusted R²=0.09. The second model, adding in mean scores from the PIQ, was also significant (MEQ: β=0.16, p<0.001; CEQ: β=−0.03, p=0.150; PIQ: β=0.35, p<0.001), F change (1,1657)=209.21, p<0.001, adjusted R²=0.19.

Two stepwise linear regressions were then conducted to examine whether the AMP and GAP subscales of the PIQ contributed unique variance in predicting changes in psychological flexibility (AAQII) and changes in well-being and life satisfaction (SWLS). The first model was significant (MEQ: β=0.25, p<0.001; CEQ: β=0.03, p=0.225), F(2,1658)=56.59, p<0.001, adjusted R²=0.06, when predicting mean changes in psychological flexibility. The second model, adding in mean scores from the AMP and GAP subscales, was also significant (MEQ: β=0.09, p<0.001; CEQ: β=−0.08, p<0.001; AMP: β=0.52, p<0.001; GAP: β=0.04, p=0.147), F change (2,1656)=316.78, p<0.001, adjusted R²=0.31. In the second regression predicting mean changes in well-being and life satisfaction, the first model was significant (MEQ: β=0.29, p<0.001; CEQ: β=0.02, p=0.481), F(2,1658)=78.50, p<0.001, adjusted R²=0.09. The second model, adding in mean scores from the AMP and GAP subscales, was also significant (MEQ: β=0.17, p<0.001; CEQ: β=−0.04, p=0.104; AMP: β=0.27, p<0.001; GAP: β=0.10, p=0.001), F change (2,1656)=104.55, p<0.001, adjusted R²=0.19.

Discussion

The present study describes the development and initial psychometric examination of a novel scale to measure acute psychological insight that may occur in the context of a psychedelic experience. The resulting self-report measure was found to have two subscales measuring (a) avoidance and maladaptive patterns insights (AMP) and (b) goals and adaptive patterns insights (GAP), both of which demonstrated a high level of internal consistency as did the overall PIQ scale. Construct validity was supported by strong correlations with the PIQ scale and the SIS, as well as with both PIQ subscales and the SIS. Also supporting construct validity were weak-to-moderate correlations between the AMP subscale and MEQ and CEQ, and a moderate correlation with the PIQ scale (and the GAP subscale) and the MEQ. Overall, PIQ, and AMP and GAP, scores were moderately-to-strongly correlated with increases in psychological flexibility and well-being/life satisfaction, suggesting predictive validity. A weak correlation between the mean scores on the PIQ and CEQ, as well as the AMP subscale and CEQ, and no correlation between the mean GAP subscale and CEQ scores suggests that, despite any difficulties associated with insight experiences, these experiences were not necessarily interpreted as being substantially challenging. Lastly, incremental validity was established showing that mean PIQ and AMP subscale scores predict unique variance in changes in psychological flexibility, and mean PIQ, and mean AMP and GAP, scores predict changes in well-being and life satisfaction beyond previously established measures of acute psychedelic effects (MEQ and CEQ).

The PIQ has the potential to extend our understanding of the psychological mechanisms that may account for therapeutic effects of psychedelics. Mean PIQ scores had a moderate correlation with MEQ scores suggesting that the PIQ may capture experiential qualities that are distinct from those that fall under the category of mystical-type experiences. Though much attention has been paid to the contribution of mystical-type experiences to positive outcomes of psychedelic experiences (Bogenschutz et al., 2015; Griffiths et al., 2006, 2008, 2011, 2016; MacLean et al., 2011), positive outcomes are possible in the absence of such effects (Roseman et al., 2018). Indeed, the PIQ explores in detail a more readily describable facet of acute subjective psychedelic drug effects (i.e. psychological insight). Given that the PIQ scale predicts retrospective reports of increases in psychological flexibility and improvement in life satisfaction and well-being beyond what is predicted by the MEQ and CEQ scales, this suggests that the PIQ may reflect insights that are therapeutic in content. This finding indicates that it may also prove to be a valuable clinical tool to examine how phenomena that are otherwise difficult to describe may produce tangible shifts in one’s mental health and assumptive worldview.

Our findings were generally consistent with other literature on insight as it relates to psychedelic drug effects. Insight experiences have previously been found to correlate with positive outcomes after psychedelic use in prospective laboratory assessment in depressed patients (Roseman et al., 2018), as well as in retrospective survey data from those with history of alcohol use disorder (Garcia-Romeu et al., 2019). Compared to Garcia-Romeu et al. (2019) we found a different correlation between the PIQ and the MEQ (0.38 vs 0.51). The smaller correlation between the PIQ scale and the MEQ in this sample may have been a result of differences in the assessment tool (single-item assessment vs multi-item PIQ), the study sample, or differences in psychedelic drugs reported in the experience (psilocybin/LSD vs any psychedelic). Although the content of the Emotional Breakthrough Inventory (Roseman et al., 2019) appears to share some phenomenology related to psychological insights, Roseman et al. (2018) did not provide sufficient detail of data to compare to our findings. Furthermore, we did not include the EBI in the current study because it had not been published when we collected data, thus future research will be needed to explore the relationship between these measures. Nevertheless, the PIQ is a more comprehensive measure compared to the insight items used in prior studies, which may be an advantage in that it could capture more nuances within the insight experience and thus provide more specificity with regard to measuring this acute effect.

There are a number of challenges in understanding the present results in the context of the existing literature on insight in traditional psychotherapy. Apart from varying definitions of insight, much of the research in this area has to do with trait insightfulness rather than changes in insight secondary to treatment (i.e. acute insight experiences in the context of a discrete event) (Gibbons et al., 2007). Additionally, prior studies that assessed change in insight have used a variety of assessment tools and have assessed different psychotherapeutic interventions in different clinical populations (Gibbons et al., 2007). However, the literature does reveal a few applicable studies that are largely consistent with our findings. For example, Kivlighan et al. (2000) found that self-reported insight, as defined by conscious awareness of thoughts and feelings contributing to distress, increased over the course of 20 psychoanalytic psychotherapy sessions. Insight was correlated with positive outcomes with a “lead-lag” relationship (i.e. decreases in target complaints were generally preceded by and were proportional to increases in insight). This is congruent with our findings in that insight, as an aspect of the acute psychedelic experience, likely preceded any self-reported retrospective changes in well-being. Kivlighan et al. (2000) also used a definition of insight that resembled the two items that loaded most strongly onto the first subscale of the PIQ (i.e. awareness and discovery of feelings that were previously avoided). Similarly, Grande et al. (2003) found that patients with psychosomatic, “neurotic,” and personality disorders who gained insight over the course of psychotherapy, as measured by a clinician rater, had improved ability to cope by the end of treatment. As Mallinckrodt (1994) suggested, insight may be an evolving process. This point was further demonstrated by O’Connor et al. (1994) who found that insight followed a “high-low-high” pattern over a 16-week course of psychotherapy. Given the cross-sectional design of the present study, we were not able to evaluate whether there were any changes in reported insight over time. Future research should consider this in a study with longitudinal methodology.

This cross-sectional study has several limitations. Responses were obtained through retrospective report via an anonymous Web-survey. Responses may thus have been influenced by access to the Internet, differences in exposure to online recruitment information, recall bias, differences in temperament of those who are more likely to respond to web surveys, or mood at time of filling out the questionnaire. Additionally, advertisements used for recruiting specifically targeted individuals who reported insight experiences secondary to psychedelic use, which may have skewed the result to favor a stronger correlation between insight experiences and well-being. The development of the scale also lacked independent ratings or validation of content validity, which could have increased confidence in the breadth of the items included in the preliminary and final questionnaires. The study design also lacked inclusion of questions related to the context (e.g. set and setting) in which the psychedelic was ingested, which could have been associated with acute and enduring effects. Consistent with prior studies that had similar recruitment procedures (Barret et al., 2016; Davis et al., 2018), respondents consisted primarily of young white men. Thus, results may not be more widely generalizable to other demographic groups. Mental health history was also not assessed, which limits the application of these data in clinical populations. The psychological value of this scale needs further examination in prospective studies with more diverse samples in order to overcome the limitations to generalizability noted above.

This study demonstrates several positive elements of reliability and validity of the PIQ in assessing acute psychological insight, a less-well characterized aspect of psychedelic drug effects. Our findings suggest that psychological insight may represent a distinct facet of the psychedelic experience that is associated with long-term positive outcomes. Prospective measurement of acute insight effects and subsequent changes in mood, behaviors, and well-being in clinical and laboratory settings would be necessary to strengthen interpretations about the relationship between this construct and other measures of acute psychedelic effects as well as the predictive validity of this measure. Evaluation of the attribution of insight as a function of time since the psychedelic experience should also be evaluated. Finally, the PIQ may also be useful in assessment of insight effects of traditional psychotherapy or other interventions and provide an additional basis for comparison.