Researching unexplained phenomena: empirical-statistical validity and reliability of the Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2)

Abstract

Anomalous phenomena are unexplained occurrences, such as paranormal experiences, that challenge the ontological bases of current scientific knowledge and are considered scientifically impossible. Problematically, some scientific research yields significant statistical results in favor of the existence of telepathy, precognition, mind-matter interaction, and mediumship. The current study presents and statistically justifies the Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2), a new psychological instrument to measure and detect the main psychological explanations for anomalous experiences. A nonprobabilistic sample of 3,224 subjects without a psychiatric history were recruited from the general population of Spain. Exploratory factor analysis (EFA) was used to examine the internal structure of the MMSI-2's 174 items. Direct oblimin and promax oblique rotations were applied as criteria for axis rotation. Cronbach's alpha coefficients and their ordinal transformation were also calculated, and gender-differentiated scales for the raw MMSI-2 scale scores were developed. The first-order factorial solution yielded a total of 16 factors that explained 92.84% of the variance. Of these, 10 corresponded to the psychological variables cited in the background literature, four classified the anomalous phenomena according to their sensory mode, and two represented prototype control scales for this class of psychometric inventory. The higher-order EFA grouped the MMSI-2 scales into four macrofactors that together explained 97.737% of the variance. Satisfactory reliability rates were obtained (alphas>0.8). The full version of the MMSI-2 with 174 items is a valid and reliable psychometric instrument for evaluating anomalous phenomena and the theoretically concomitant psychological variables. Similarly, the scaling of scores can be used in psychological assessment as a screening tool to identify clinically suspected psychological variables.

Psychology; Psi Phenomena; Anomalous Experiences; Hallucination; Parapsychology; Schizotypy; Paranormal Belief

1. Introduction

Some experiences reported by patients in clinical interviews contradict the ontological bases of current scientific knowledge and are considered “unexplained” by psychology and psychiatry (e.g., Bobrow, 2003). “Paranormal” experiences are such cases, which are formally referred to as anomalous phenomena (e.g., French and Stone, 2014). The current study introduces a new psychometric instrument to detect and assess possible psychological explanations for experiences of anomalous phenomena, called the Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2).

1.1. Theoretical background

Scientific research of anomalous phenomena is complex. The main problem is that the challenge raised regarding anomalous phenomena is also based on previously published scientific evidence (e.g., Bem, 2011; Bem et al., 2016; Mossbridge et al., 2012). This means that it is not just an ideological and epistemological debate (e.g., Carter, 2012). The most serious problem can be observed in the fact that some scientific research yields significant results in favor of the existence of these alleged anomalous phenomena (e.g., Beischel et al., 2015; Kelly and Arcangel, 2011; Schwartz and Russek, 2001) and contradicts conventional scientific knowledge related to the psychology of perception, sensation and cognition (e.g., Álvarez, 2007; Bunge, 2013; Reber and Alcock, 2019). This is an example of ‘psi’ phenomena, which include precognition, telepathy, mediumship and anomalous mind-matter interaction (see Jinks, 2019). These behaviors are also classified as beliefs in the paranormal because they are considered impossible phenomena according to the scientific canon (e.g., Irwin, 2009). Although there are reasons to deny the scientific validity of these investigations that support the existence of ‘psi’ phenomena (e.g., O’Keeffe and Wiseman, 2005; Reber and Alcock, 2019; Wagenmakers et al., 2011), the fact that studies with significant results exist also requires that such research be replicated through the use of the scientific method (e.g., Popper, 2008; Storm and Ertel, 2001; Storm et al., 2013).

En the field of psychological assessment, anomalous behaviors in which the patient believes they can read other people's minds (e.g., telepathy) or feel the presence of dead beings (e.g., mediumship) are behaviors whose clinical value could be both pathological and nonpathological (see Irwin, 2009). Psychological models justifying ‘psi’ phenomena and other anomalous phenomena can be summarized as follows: (1) the continuum model of psychoses, which justifies ‘psi’ phenomena as hallucinatory and delirious symptoms (e.g., Johns and van Os, 2001; Stefanis et al., 2002; van Os et al., 2009); (2) the semiotic model of perception (see Ey et al., 1980), which explains these phenomena as perceptual errors (e.g., cognitive biases or perceptual deformations) (e.g., Barberia et al., 2018; El-Mallakh and Walker, 2010); and (3) the phenomenological model, which defines these behaviors as cognitive and verbal representations based on the subject's systems of meanings (e.g., Font, 2016; French and Stone, 2014; Irwin, 1993, 2000, 2003, 2009). Unlike the other models, the phenomenological model does not assume that the etiology of these behaviors is necessarily related to the pathological or symptomatic, nor does it accept that they are caused by errors or mistakes made by the individual (see Irwin et al., 2013; Irwin and Watt, 2007). As some research suggests, systems of meaning and cognitive structures define the way of perceiving, thinking about, and interpreting stimuli in the environment (e.g., Fishbein and Ajzen, 1975). Therefore, the phenomenological model assumes that the etiology of these abnormalities can be observed in the patient's psychological profile, which can justify why a person thinks and acts in a certain way (e.g., Groth-Marnat, 2009; Jaspers, 1993).

The most relevant research takes into account, as predictive factors, certain subclinical personality traits, usually based on paranoid, narcissistic, histrionic and schizotypic attributes, that are positively correlated with anomalous experiences (e.g., López-Rodrigo et al., 1996; Roe and Morgan, 2002; Simmonds-Moore et al., 2019). There are also numerous papers that identify variables influencing anomalous phenomena with very high effects−in statistical terms of effect size (see Cohen, 1988)−related to substance use or substance abuse (e.g., Luke, 2012; Sideli et al., 2019; Wilkins et al., 2012). Other variables with significant results are alternating states of exhaustion/anxiety (e.g., Roe and Bell, 2016; Simmonds-Moore, 2009; Williams et al., 2007) and thrill-seeking (e.g., Gow et al., 2004; Smith et al., 2009). Another extensively studied variable is that of traumatic experiences in childhood, which are prevalent among subjects who report anomalous phenomena (e.g., Lawrence et al., 1995; Lynn and Rhue, 1988; Parra, 2019).

Some studies focus on simulation and fraud behaviors (see Álvarez, 2007; Leonard and Williams, 2019; Stieger and Hergovich, 2013). Taking as a reference the semiotics of perception, these works examine the psychological biases that would justify the invention of false memories or elements that would trigger lying as the principal mediator (e.g., French, 2003; Wilson and French, 2006). One of the biases that has been corroborated is the Barnum effect (e.g., O’Keeffe and Wiseman, 2005). According to Shermer (2011), this effect is observed in overly general statements that seek to validate the anomalous phenomenon, causing the subject to easily identify with the elements presented and accept them as true. Based on the contributions of Boyce and Geller (2002), this effect can be considered verbal conduct that incites deception and should be measured as a control variable. Measurement of this factor in psychometric instruments is not common; however, its inclusion seems necessary to prevent not only the bias derived from the Barnum effect but also deception and simulation behaviors (see Tombaugh, 2011). Conventional clinical evaluation tests have chosen to measure only unconscious lying as a result of image manipulation (e.g., in response to the social desirability or negative presentation of a behavior) (e.g., Ben-Porath and Tellegen, 2019; Millon, 1994; Morey, 2011). However, the control of only variables associated with image manipulation is not sufficient to evaluate lying (see Cardona, 2002; Vrij et al., 2019). On their own, are not effective indicators for the detection of deception (see Fernández-Ballesteros, 2011). In reality, the assessment of lying requires the incorporation of new measures that respond to its complexity and focus more specifically within the area of simulation and fraud (e.g., MacNeil and Soper, 2019).

Other works point to variables that are less consistent but have equally significant positive correlations (French and Stone, 2014). These variables are creativity, intuition, extraversion and dissociative disorders (e.g., Czekóová et al., 2018; Rabeyron et al., 2018; Thalbourne and Haraldsson, 1980). However, these variables present unstable statistical behavior because other research has not demonstrated significant correlations (e.g., Maraldi, 2019; Swami et al., 2011). For example, Francis et al. (2010) found no association between extraversion and anomalous phenomena. A more recent study comparing two groups−one comprising those who did not believe in the paranormal or had not had anomalous experiences and another comprising subjects who considered themselves mediums−noted that levels of dissociation were higher for the group of nonbelievers (e.g., Vencio et al., 2018). Similarly, another publication found positive correlations between paranormal beliefs and critical thinking (e.g., Musch and Ehrenberg, 2002). In reality, both analytical and critical thinking traditionally constitute attributes that are antagonistic to belief in the paranormal (e.g., Hergovich and Arendasy, 2005). It does not seem sufficiently rigorous to explain these differences and contradictions as methodological errors or statistical artifacts (Irwin, 2009).

More behavioral approaches focus on the analysis of models related to processes of suggestion (e.g., Gibson and Heap, 1991). This approach has two aspects: on the one hand, some professionals understand suggestion as a process of alteration of consciousness that can be varied and manipulated through hypnosis techniques (e.g., Hambleton, 2008). On the other hand, another very different approach investigates suggestion as a psychological predisposition or trait that describes the emotional lability of a subject as a result of environmental influences (e.g., Hefferline et al., 1972). Along these lines, suggestibility is the degree to which a subject tends to change−presumably automatically−the typology and intensity of their emotional reactions based on the effects produced by environmental stimuli (e.g., Linton and Sheehan, 1994). This approach describes three types of suggestibility: (1) interrogative suggestibility, (2) primary suggestibility and (3) secondary suggestibility. The first of these refers to the degree of emotional lability induced in the subject exclusively by stimuli derived from social interaction (e.g., Gudjonsson, 1984, 2003). The issue lies with−and this is what determines interrogative suggestibility−how much social persuasion is needed to generate changes in the emotional reactions of the interlocutor subject. High suggestibility is observed when there are low levels of persuasion and high levels of emotional lability (e.g., Polczyk, 2005). Primary suggestibility, in contrast, refers to ideomotor and psychobiological markers that predict the degree of emotional lability. This is in contrast to secondary suggestibility, which refers to the levels of vivid imagination needed to predict such emotional lability (e.g., Eysenck, 2017). Studies that relate levels of suggestion to anomalous experiences take into account altered states of consciousness and secondary suggestibility (e.g., Eysenck and Sargent, 1982). Regarding altered states of consciousness, it should be noted that, with higher levels of alteration, there are more perceived anomalous experiences (e.g., Luke, 2012; Maij et al., 2017; Moreira-Almeida and Lotufo-Neto, 2017). Likewise, the greater the secondary suggestibility, the greater the propensity to develop anomalous phenomena (e.g., Eysenck, 2017; Terhune and Smith, 2006; Wiseman et al., 2003). There are also works with positive correlations that address primary suggestibility and interrogative suggestibility, but the consistency of their results is questionable (e.g., Brugger and Mohr, 2008; Haraldsson, 1985; Hergovich, 2003). Along these lines, it appears that secondary suggestibility and altered states of consciousness are the most frequently researched types of suggestibility and hence the best predictors of anomalous phenomena (Eysenck and Sargent, 1982).

As mentioned at the outset, the identification of variables concomitant with anomalous phenomena faces a major challenge: in some cases, the existing scientific literature is not entirely clear as to which variables correlate with anomalous phenomena and belief in the paranormal (e.g., Houran and Lange, 2004; Houran et al., 2019). Some of the trends highlighted in the preceding paragraphs have been replicated, with different results, and measurement instruments that effectively assess the correlated factors have not been confirmed in the scientific literature.

Another problem is that of which psychological variables to evaluate and how to measure the correlation between anomalous phenomena and these variables (e.g., Cameron, 2016; Lawrence, 2016). In fact, no conventional clinical questionnaires have been prepared or validated to relate previously known psychological variables to perceived anomalous phenomena (e.g., Pasricha, 2011). One of the criticisms of the instruments most commonly used in clinical diagnosis is that the items examine only pathological symptoms and do not express more attenuated indicators of the evaluated disorders (e.g., Hueso, 2011; Shiah et al., 2014). This suggests that these instruments are valid for samples of patients with an underlying psychopathology, but although they present normative scales for the general nonclinical population, the content of the items does not change categorically and remains qualitatively pathological (e.g., Butcher et al., 1995; Williams and Lally, 2017). Another very obvious difficulty is that the most widely recognized questionnaires in clinical practice conceive of anomalous experiences as exclusively psychopathological symptoms and do not allow a quantitative analysis beyond their pathological condition (e.g., Irwin, 1993, 2009). It should be noted that nonclinical questionnaires exist that do allow the examination of anomalous perceptions and belief in the paranormal at the psychometric level (e.g., Bell et al., 2006; Mason and Claridge, 2006; Stefanis et al., 2002), but they do not take into account possible concomitant psychological variables that allow clinical, psychological and forensic decisions to be made regarding the etiology of the perceived anomalous phenomena (e.g., Irwin, 2009). Practically speaking, if the evidence published does not guarantee the formulation of a conceptually sound explanatory theory, there is a need to examine and replicate – at least from an exploratory standpoint (see Gallagher et al., 1994) − the methodological bases of scientific precedents (e.g., Utts, 2018).

1.2. Research objectives

The interests of this research can be summarized by two questions. First, what behavioral variables relate to, explain, and allow us to understand anomalous phenomena? Second, are these variables operative enough to validate a new psychometric test?

Accordingly, the study's objective is to propose and develop an empirical-statistical tool to identify, evaluate, and measure causes that could explain experiences of anomalous phenomena. The study thus created, developed, and examined the validity and reliability of the Multivariable Multiaxial Suggestibility Inventory-2 (MMSI-2), an innovative psychometric instrument that examines anomalous phenomena based on various causal factors cited in the scientific literature.

2. Materials & methods

2.1. Human ethics

Participants gave their written consent to use their anonymous data for statistical purposes. All of them were over 18 years old and voluntarily collaborated without receiving any financial compensation. The procedures were carried out in compliance with the institutional regulations of the university and the Spanish Government Data Protection Law 15/1999. The Committee of Ethical Guarantees of Ramon Llull University reviewed, favorably evaluated and approved this research. Similarly, all procedures adhere to the Helsinki Declaration of 1975, revised in 2013.

2.2. Participants

This study used a nonprobabilistic convenience sample and was conducted between 2013 and 2019 (N = 3,224). The sample comprised both men (49.5%) and women (50.5%), all adults (>18; mean = 34.64; SD = 14.791) without a reported psychiatric history (i.e., no previous psychiatric diagnosis and, therefore, no officially recognized mental disorder).

Following the sampling representativity criteria proposed by Muñiz (2003), the subjects came from three different Spanish communities. The groups were: (1) The Community of Madrid (N = 1,102; with mean ages = 34.32; SD = 14.416), (2) The Autonomous Community of Catalonia (N = 1,338; with mean ages = 35.46; SD = 15.549), and (3) The Community of Castilla-La Mancha (N = 784; with mean ages = 33.70; SD = 13.902). Two additional sociodemographic variables were recorded to further characterize the sample. Socioeducational level was evaluated based on the standards proposed in Spain's National Institute of Statistics and was classified into five levels: (1) no schooling (0.2%), (2) elementary education (2%), (3) compulsory secondary education or basic vocational training (14.5%), (4) baccalaureate or higher vocational training (40.4%), and (5) university or higher education (40.8%).

Each participant was consulted to determine their self-reported belief in the existence of paranormal phenomena based on three ordinal categories: 0 = ‘I do not believe at all’ (29.1%_Total, 32.8%_Madrid, 44.2%_Catalonia, 23%_{Castilla-La Mancha}); 1 = ‘I question it’ (36.1%_Total, 34.9%_Madrid, 38.2%_Catalonia, 26.9%_{Castilla-La Mancha}); and 2 = ‘I believe completely’ (34.8%_Total, 34.6%_Madrid, 42.7%_Catalonia, 22.7%_{Castilla-La Mancha}). All subjects voluntarily agreed to participate in the study and signed informed consent on paper or digitally.

2.3. Procedure

Figure 1 summarizes all phases involved in the psychometric and methodological development of the MMSI-2.

Figure 1.

Flowchart summarizing the phases in development and creation of the MMSI-2. The contents available in the manuscript are highlighted in green, while blue contents are available under prior contact with the author and orange contents are methodological and logistical decisions.

The first phase took place in 2012. A research project was initiated that sought to quantitatively examine perceived anomalous phenomena by relating them to possible psychological history variables. During the first year of research, the necessary bibliographic sources were consulted, and a theoretical framework was designed to inform the drafting of the items.

The second phase occurred in 2013. An initial bank of items (N = 223) was created based on the scientific literature cited above and the behavioral indicators associated with each variable found in the literature. References to possible clinical behaviors were taken from the DSM-IV-TR (American Psychiatric Association, 2002) and the DSM-V (American Psychiatric Association, 2013). The items were written exclusively by the author of the manuscript. The items consisted of affirmative and negative phrases that addressed anomalous phenomena (e.g., ‘I have been able to sense the thoughts of other people’) and psychological attributes correlated with anomalous behaviors, including personality traits, clinical trends, anxiety states, and cognitive biases (e.g., ‘I allow my emotions to affect my thoughts’). After a review of form and content by the research team at the time, the 223 items were sent to 22 professional psychologists with different specializations (including a specialization in methodology and research). The qualitative evaluation method used was that proposed by Hambleton (1980). The procedure involved assessing the degree of fit between the content of each item and the intended object of study. The judges were asked to specify the rational quality of the fit for each item using a graduated scale from 0 to 5 (where 0 = ‘the contents of the item do not conform to the variable they intend to measure’ and 5 = ‘the contents of the item are fully in line with the construct’). Note that the MMSI is a multiaxial test because it evaluates multiple psychological constructs. This means that during the content validity process, a single construct did not exist for all items. Each item was compared to the construct to which it referred. The constructs that were initially specified were (1) perceptual alterations, (2) histrionism, (3) schizotypy, (4) paranoia, (5) narcissism, (6) cognitive biases/Barnum effect, (7) social desirability, (8) substance use, (9) anxiety states, (10) suggestibility, (11) predisposition toward fraud, (12) thrill-seeking and (13) the participant's level of collaboration with the study. Childhood trauma, creativity, intuition, extraversion, and dissociative disorders were omitted as elements of the MMSI-2 because of the low statistical consistency of their results when tested among nonclinical samples. Although these constructs were based on the scientific literature cited in the previous section, no previous theoretical model was defined given the inconsistencies of some of the published evidence.

Items that yielded an average value equal to or greater than three points on the scale were included in the final version, while items with a lower average score were omitted from the final version of the test. Of the 223 items, 49 were eliminated, leaving a total of 174. Once the items were drafted and screened, a decision was made regarding how the responses should be coded to facilitate the valid and reliable quantification of scores. Following the recommendations of Kline (1999), a Likert-type scale with a range of response options from 1 to 5 was selected. This system allowed subjects to indicate their level of agreement according to each statement, with 1 = strongly disagree, 2 = disagree, 3 = agree, 4 = very much agree and 5 = strongly agree. Of the 174 items, 139 were positive statements, and 35 were negative statements. For the items consisting of negative statements, responses were reverse coded: 1 = strongly agree, 2 = very much agree, 3 = somewhat agree, 4 = disagree, and 5 = strongly disagree. As Price and Mueller (1986) suggest, when a psychometric test is excessively long, it is appropriate to alternate between the two types of items to avoid biases related to acquiescence, neutrality and systematic denial. Thus, the distribution of the 174 items that comprised the experimental version of the test was not entirely random. Rather, two criteria were applied: (1) affirmative and negative items were alternated; and (2) items expressing the level of collaboration with the evaluation were placed at the end of the questionnaire. The items intended to measure the participant's level of collaboration expressed impossible or absurd content that, under normative or nonpathological conditions, would force the subject to be completely at odds with what the sentence says. An example is item 153 in this version: ‘Red Riding Hood is a real person.’ If the participant has collaborated with the assessment and does not present any psychiatric pathology, they should not agree with item 153. Consequently, responses should fluctuate between values of 1 and 2 or be equal to 1. This criterion was included based on the statistical contributions of Guildford (1954), who originally noted the presence of a progressive decrease in levels of attention as a length of tests increases. This alteration is due to fatigue resulting from the length of the questionnaire, which promotes comprehension errors and random responses (e.g., Schmitt and Stults, 1985). Both errors and random responses could impair the validity of responses to the MMSI-2. One way to identify presumably random responses is by measuring the participant's level of collaboration (e.g., Morey, 2011). As suggested by Butcher et al. (2019), the level of collaboration measured for this purpose assumes that errors in comprehension or random responses will be observed for the later items on the questionnaire and for the items occurring early in the measurement. For this reason, items similar to item 153 were located starting at item 52.

The third phase of the research took place in January and March of 2014. During this period, the test application materials and informed consent forms were prepared, and the sociodemographic variables to be recorded to ensure the heterogeneity and representativeness of the sample were chosen. Based on the contents of the items and the suggestions of the team of experts, it was concluded that the MMSI, despite having a relationship with clinical evaluation, would not serve a diagnostic/psychopathological purpose (other, more suitable instruments already exist for this) and hence, the decision was made to apply it only to subjects without a reported psychiatric history. Collaboration agreements were signed with the psychological centers and companies involved in the collection of the sample (see acknowledgments). The evaluation materials were designed in both pencil-and-paper and digital formats. The format used was left to the discretion of the collaborating groups that were to use the materials.

The fourth phase of research took place between 2014 and 2016. During this phase, data collection began, and an initial matrix was prepared and refined based on the participants’ responses. Cases were disregarded if the responses contained missing values or were unclear or had excessive strikethroughs or corrections. A first pilot analysis was conducted in November 2016 to determine the heterogeneity of the sample responses. Coefficients of variation were mainly used to examine the responses and the sociodemographic variables recorded. The results showed no sample biases.

Finally, the fifth phase of research was carried out in 2017 and 2019. In this part of the procedure, the sampling, drafting and debugging of the final data matrices was completed, and the analysis of the collected data began.

2.4. Instruments

The instrument used was the 174-item version of the MMSI that remained after the 223 original items were refined; this version was labeled the MMSI-2. The MMSI-2 examines 16 first-order factors: Inconsistencies (K), Lies (L), Fraud (F), Simulation (Si), Neurasthenia (Nt), Substance Use (Cs), Suggestibility (Su), Thrill-Seeking (Be), Histrionism (Hi), Schizotypy (Ez), Paranoia (Pa), Narcissism (Na), Anomalous Visual/Auditory Phenomena (Pva), Anomalous Tactile Phenomena (Pt), Anomalous Olfactory Phenomena (Po) and Anomalous Cenesthetic Phenomena (Pc). The responses are coded using a scale of 1–5, on which the participant must indicate his or her level of agreement with what is stated in each item. It should be kept in mind that some items are scored in reverse; these reverse-scored items are those marked with an asterisk in the factorial solution presented in Tables 1, 2, 3, 4, 5, 6, and 7. The MMSI-2 also contains four higher-order factors: Clinical Personality Tendencies (CPT), Anomalous Perceived Phenomena (APP), Incoherent Manipulations (IMA), and Altered States of Consciousness (ASC). The calculation of the macrofactors is obtained from the sum of the direct scores of the corresponding first-order factors for each higher-order factor. The higher the scores for each attribute, the greater the frequency and intensity of those characteristics in the participant.

Table 1.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	1	2	3
	Pva	Si	L
Pva 39	0.653
Pva 47	0.651
Pva 148	0.648
Pva 111	0.646
Pva 25	0.645
Pva 1	0.642
Pva 130	0.641
Pva 96	0.634
Pva 117	0.632
Pva 84	0.629
Pva 143	0.593
Si 67		0.895
Si 144		0.886
Si 61		0.885
Si 14		0.878
Si 38		0.858
Si 114		0.858
L 172∗			-0.946
L 42∗			-0.922
L 93∗			-0.912
L 164∗			-0.908
L 36∗			-0.904
L 106∗			-0.904
L 63			0.903
L 21∗			-0.903
L 136			0.899
L 11∗			-0.897
L 168∗			-0.894
L 160∗			-0.892
Eigenvalue	46.472	28.778	22.043

Note: This table has several extensions included in the following tables. Factor loadings under 0.4 were eliminated (N = 3,224). Pva= Anomalous Visual/Auditory Phenomena; Si= Simulation; L= Lies. ∗Items are scored in reverse.

Table 2.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	3	4	5
	L	Nt	Hi
L 131	0.892
L 78	0.891
L 29	0.882
L 7	0.879
L 15	0.876
L 86	0.874
L 150∗	-0.870
L 70	0.869
L 119	0.860
L 163∗	-0.819
L 49∗	-0.810
Nt 108		0.925
Nt 74		0.925
Nt 76		0.924
Nt 115∗		-0.923
Nt 100		0.920
Nt 161		0.919
Nt 118∗		-0.918
Nt 43		0.914
Nt 166∗		-0.908
Nt 5		0.900
Nt 123		0.898
Nt 155∗		-0.892
Nt 56		0.880
Nt 165		0.876
Nt 50		0.869
Hi 122			0.932
Hi 158			0.929
Hi 37			0.926
Eigenvalue	22.043	14.819	10.622

Note: Factor loadings under 0.4 were eliminated (N = 3,224). L= Lies; Nt= Neurasthenia; Hi= Histrionism. ∗Items are scored in reverse.

Table 3.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	5	6	7
	Hi	K	F
Hi 8	0.924
Hi 94	0.920
Hi 73	0.917
Hi 169	0.915
Hi 44	0.912
Hi 171	0.897
Hi 88	0.896
Hi 121	0.888
Hi 137	0.881
Hi 129	0.874
K 132		0.895
K 153		0.892
K 75		0.875
K 154∗		-0.871
K 52∗		-0.862
K 99		0.858
K 128∗		-0.855
K 127		0.850
K 140∗		-0.834
K 87∗		-0.832
K 147		0.795
K 62∗		-0.670
F 101			0.872
F 53			0.868
F 141			0.867
F 107			0.867
F 157∗			-0.867
F 23∗			-0.866
Eigenvalue	10.622	7.857	6.415

Note: Factor loadings under 0.4 were eliminated (N = 3,224). Hi= Histrionism; K= Inconsistencies; F= Fraud. ∗Items are scored in reverse.

Table 4.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	7	8	9
	F	Cs	Su
F 133∗	-0.866
F 146	0.866
F 9∗	-0.865
F 72∗	-0.864
F 30∗	-0.864
F 81∗	-0.863
F 170	0.862
F 120	0.860
F 71∗	-0.858
F 6	0.856
F 167∗	-0.848
F 98∗	-0.844
F 174∗	-0.841
F 112∗	-0.779
Cs 32		0.781
Cs 57		0.745
Cs 3		0.735
Cs 46		0.689
Cs 90		0.678
Cs 24		0.667
Cs 65		0.442
Su 4			0.988
Su 22			0.985
Su 41			0.982
Su 109			0.980
Su 80			0.975
Su 95			0.955
Su 51			0.946
Eigenvalue	6.415	5.586	4.681

Note: Factor loadings under 0.4 were eliminated (N = 3,224). F= Fraud; Cs= Substance Use; Su= Suggestibility. ∗Items are scored in reverse.

Table 5.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	10	11	12
	Pa	Pt	Po
Pa 55	0.910
Pa 110	0.908
Pa 138	0.906
Pa 45	0.905
Pa 64	0.899
Pa 10	0.893
Pa 26	0.884
Pa 116	0.879
Pa 40	0.875
Pa 142	0.865
Pt 18		0.873
Pt 85		0.856
Pt 13		0.852
Pt 152		0.851
Pt 34		0.838
Pt 134		0.833
Pt 54		0.811
Po 16			0.845
Po 58			0.843
Po 31			0.835
Po 48			0.832
Po 79			0.818
Po 77			0.818
Po 89			0.805
Eigenvalue	3.639	2.973	2.294

Note: Factor loadings under 0.4 were eliminated (N = 3,224). Pa= Paranoia; Pt= Anomalous Tactile Phenomena; Po= Anomalous Olfactory Phenomena.

Table 6.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	13	14
	Na	Pc
Na 12	0.744
Na 162	0.741
Na 20	0.740
Na 145	0.739
Na 126	0.738
Na 159	0.738
Na 149	0.737
Na 104	0.734
Na 27	0.731
Na 68	0.730
Na 60	0.729
Na 103	0.727
Pc 91		0.740
Pc 66		0.739
Pc 33		0.730
Pc 97		0.724
Pc 173		0.713
Pc 59		0.712
Pc 83		0.704
Pc 2		0.693
Pc 105		0.664
Eigenvalue	1.845	1.413

Note: Factor loadings under 0.4 were eliminated (N = 3,224). Na= Narcissism; Pc= Anomalous Cenesthetic Phenomena.

Table 7.

First-order exploratory factor analysis with oblimin rotation.

Items	Extracted factors
	15	16
	Be	Ez
Be 92	0.914
Be 124	0.728
Be 82	0.722
Be 17	0.563
Ez 28		0.679
Ez 135		0.678
Ez 102		0.677
Ez 69		0.677
Ez 35		0.675
Ez 113		0.673
Ez 151		0.672
Ez 156		0.672
Ez 19		0.671
Ez 139		0.666
Ez 125		0.665
Eigenvalue	1.156	0.961

Note: Factor loadings under 0.4 were eliminated (N = 3,224). Be= Thrill-Seeking; Ez= Schizotypy.

2.5. Data analysis

The statistical design of this research was multivariable and was based on the development of both a first-order and higher-order exploratory factor analysis (henceforth EFA).

The first-order EFA was applied to the matrix of correlations among the 174 test items. The extraction of the factors was performed using the unweighted least squares procedure, which is considered the most robust method because it does not require prior estimation of the commonalities of the items (see Harman and Jones, 1966). To optimize the factorial solution, direct oblimin oblique rotation was used (with δ = 0). This decision was based on the theoretical background that shows intercorrelations between the different factors that were extracted (see French and Stone, 2014). According to Pedhazur and Schmelkin (2001), in the social sciences, and especially psychometrics, oblique rotations are recommended because they assume interdependence among latent factors. Moreover, absolute independence among the extracted factors themselves is not an insurable assumption (e.g., Abad et al., 2015). Given the logic of oblique rotations, in this type of solution, unlike with orthogonal rotations, it is not advisable to provide the explained variance for each extracted first-order factor since each explained variance would be overlapping (and hence biased) by the intercorrelations among the factors (e.g., Gorsuch, 1983). Consequently, we noted only the corresponding original eigenvalues for each factor and the total explained variance of the EFA, which does not consist of the summation of the explained variances of each factor. The total number of factors was established based on the parallel analysis method rather than the classic Guttman-Kaiser method, which is less accurate for the retention of factors (e.g., Reise et al., 2000). Following the recommendations of Mulaik (2018), the saturation matrix offered and analyzed in this report corresponds to the pattern matrix, which is much easier to interpret than the structure matrix. Likewise, this matrix shows the ordered saturations greater than or equal to 0.4. Following Thompson (2004), if an item had a saturation of >0.4 on two or more factors, it was removed from the test. The theoretical classification of the extracted factors was carried out based on the recommendations of Borsboom et al. (2004), who proposed the analysis of the contents of the items grouped within the same factor as a criterion. Subsequently, according to the common conceptual characteristics of each group of items, the corresponding labels for each factor were decided. For these analyses, the statistical programs JAMOVI® and MPLUS 5.2 were used, which allow the calculation of the matrix of polychoric correlations and the use of the parallel analysis method.

For the application of the second-order EFA, the correlation matrix for the primary factors extracted in the previous EFA was analyzed. On this occasion, the correlation matrix was not polychoric, since the scores for each factor represent quantitative interval values. Thus, the linear correlations matrix was determined for the 16 variables. The criterion for the extraction of the new factors was the same as that used in the first-order EFA. However, as Gorsuch (1983) states, the rotation of the axes was carried out using the promax method (κ = 1). This rotation initially combines orthogonal rotations to complete the application of oblique rotations of the axes (see Martínez-Arias et al., 2006). Regarding second-order factors, there is a possibility that they are less correlated with each other, resulting in more independent behavior compared with the primary factors. This does not mean that the second-order factors are completely independent of each other, and therefore, it would not make sense a priori to apply a purely orthogonal rotation. In this EFA, the Guttman-Kaiser criterion was used to determine the number of second-order factors to retain. Taking into account the logic of O'Connor (2000), the parallel analysis method was rejected because the eigenvalue of the first factor excluded by the classical method (which was factor 5) was substantially removed from 1 (λ₅ = 0.161). This indicates that the factors extracted from factor number 4 were irrelevant due to their low variability (see also Mulaik, 2018). Therefore, in this context, it would not make sense to apply parallel analysis to determine whether it was necessary to include another factor. For the analysis of the saturations and the theoretical categorization of the secondary factors, the same procedures were used as in the first-order EFA. For the latter EFA, SPSS 25 was used.

Regarding the reliability of the test, for the 16 first-order factors, the ordinal transformation of Cronbach's alpha coefficient was chosen based on the contributions of McDonald (1999):

$$\alpha =\frac{n}{n-1}\left[\frac{n{\left(\overline{\lambda }\right)}^2-{\overline{\lambda }}^2}{n{\left(\overline{\lambda }\right)}^2+\left({\mu }^2\right)}\right]$$ [1]

where

$\overline{\lambda }$ is the arithmetic mean of the factorial loads,

${\overline{\lambda }}^2$ is the square arithmetic mean of the factorial loads,

and

${\mu }^2$ is the arithmetic mean of the single variance.

Cronbach's alpha coefficient was used to calculate the reliability of the macrofactors since the scores of the 16 factors are quantitative. For these calculations, Microsoft Excel spreadsheets and the program SPSS 25 were used.

Finally, gender-differentiated general scales were created based on the standard derived scores (PT or simply T) and the sample percentiles (abbreviated as Pcs).

3. Results

3.1. First-order exploratory factor analysis

Prior to the application of the EFA, there was a need to check whether the items were sufficiently correlated with each other. For this purpose, the Kaiser-Meyer-Olkin (KMO) sample adequacy test was used. Bartlett's test of sphericity was not applied based on the transformation of the chi square from the determinant of the polychoric correlation matrix because this statistic is highly sensitive to sample size (e.g., Ruiz, 2000). The KMO index yielded a favorable result regarding the use of the EFA technique (KMO = 0.941). The results of the factorial solution by means of direct oblimin rotation are presented in Tables 1, 2, 3, 4, 5, 6, and 7. The sedimentation graph is also presented, with the simulated average eigenvalues for 100 random samples (representing the parallel analysis method) (See Figure 2). As shown, crossing the two curves retained 16 primary factors. To illustrate and more easily indicate the crossing point, the graph is presented only for the first 30 factors (which were the most significant for this statistical decision).

Figure 2.

Scree-plot of parallel analysis of the first-order EFA.

The solution extracted a total of 16 factors that together explained 92.84% of the variance. Factor 1 consisted of 11 items that expressed visual and auditory perceptual alterations. This factor was classified as Anomalous Visual/Auditory Phenomena (Pva) and obtained an eigenvalue of 46.472. Factor 2 included 6 items that reflected tendencies related to the Barnum effect. It obtained an eigenvalue of 28.778 and was classified as Simulation (Si). Factor 3 included 23 items that coincide with behaviors typical of social desirability. It yielded an eigenvalue equal to 22.043 and was classified as Lies (L). Factor 4 described behaviors associated with states of anxiety and acute fatigue. It included 15 items that demonstrated an eigenvalue equal to 14.819 and was classified as Neurasthenia (Nt). Factor 5 was classified as Histrionism (Hi) as it included 13 items whose behaviors may be associated with attenuated symptoms typical of histrionic personality disorder. It had an eigenvalue of 10.622. Factor 6 contained 12 items and demonstrated an eigenvalue of 7.857. These items had two types of content in common. On the one hand, some items described beliefs that are present in individuals without prior psychopathological diagnosis (e.g., item K-154: ‘I believe I deserve to be respected’). On the other hand, there were items that contained absurd and impossible content (e.g., Item K-153: ‘Red Riding Hood is a real person’) and were closely related to the participant's degree of collaboration and the identification of potential inconsistencies in each subject's responses. For this reason, Factor 6 was classified as Inconsistencies (K). Factor 7 had an eigenvalue of 6.415 and included 20 items. It described behaviors that indicate tendencies involving low morals, manipulation, and deviousness (e.g., Item F-9: ‘If my best friend committed fraud, I would not report it’). Thus, it was classified as Fraud (F). Factor 8 included seven items related to substance use and abuse. It had an eigenvalue of 5.586 and was categorized as Substance Use (Cs). Factor 9 showed an eigenvalue of 4.681 and comprised seven behaviors associated with emotional lability and permeability resulting from environmental influences. Thus, this variable was classified as Suggestibility (Su). Factor 10 included 10 items related to distrust, skepticism, and certain attenuated symptoms of paranoid personality disorder. It had an eigenvalue of 3.639 and was classified as Paranoia (Pa). Factor 11 included 7 items describing perceptual distortions of a tactile nature. It obtained an eigenvalue of 2.973 and was classified as Anomalous Tactile Phenomena (Pt). Factor 12 had an eigenvalue of 2.294 and included 7 items related to perceptual distortions of an olfactory nature. For this reason, it was classified as Anomalous Olfactory Phenomena (Po). Factor 13 comprised 12 items expressing attenuated symptoms characteristic of narcissistic personality disorder. It had an eigenvalue of 1.845 and was categorized as Narcissism (Na). Factor 14 comprised 9 items that allude to hallucinatory experiences related to depersonalization and derealization. It had an eigenvalue of 1.413 and was classified as Anomalous Cenesthetic Phenomena (Pc). Factor 15 was classified as Thrill-Seeking (Be) and included just four items with an eigenvalue of 1.156. The final first-order factor included 11 items associated with social withdrawal, magical thinking and isolation. These traits are characteristic of schizotypic personality disorder and were also expressed in a subtle or attenuated manner; hence, this factor was classified as Schizotypy (Ez). Its eigenvalue was 0.961.

The descriptive statistics for this first EFA are shown in Table 8, which also includes the descriptive statistics for the higher order factors and the alpha reliability coefficients, which will be discussed in the following paragraphs.

Table 8.

Descriptive statistics of MMSI-2 scales and alpha coefficients (N = 3,224).

F	Minimum and Maximum direct scores	M	SD	Asymmetry (Error = 0.043)	Kurtosis (Error = 0.086)	Alphas
K	12–60	18.13	5.758	3.350	16.129	0.973∗
L	23–115	48.64	20.365	1.289	1.007	0.994∗
F	20–100	42.76	18.407	1.130	0.603	0.993∗
Si	6–30	16.05	4.202	0.577	0.577	0.996∗
Nt	15–75	33.34	13.755	1.097	0.397	0.997∗
Cs	7–35	13.95	5.659	1.435	2.023	0.878∗
Su	7–35	18.57	5.517	0.124	-0.157	0.995∗
Be	4–20	11.64	3.366	0.224	-0.435	0.865∗
Pva	11–55	20.01	8.714	1.679	2.691	0.996∗
Pt	7–35	14.87	5.915	1.420	1.908	0.988∗
Po	7–35	15.68	5.813	1.082	0.863	0.996∗
Pc	9–45	19.09	7.390	1.276	1.431	0.994∗
Hi	13–65	38.22	8.661	0.179	0.434	0.993∗
Ez	11–55	32.00	7.805	0.194	0.347	0.997∗
Pa	10–50	30.05	8.609	-0.318	-0.180	0.991∗
Na	12–60	36.53	8.965	-0.248	0.093	0.997∗
IMA	61–305	125.63	48.172	1.309	1.468	0.870∗∗
APP	34–170	69.65	27.743	1.397	1.764	0.988∗∗
ASC	22–110	47.29	19.358	1.186	0.810	0.819∗∗
CPT	57–285	167.00	42.481	-0.048	-0.075	0.979∗∗

Note: F= Factors; M = Means; SD = Standard Deviations; ∗ = McDonald's alpha; ∗∗ = Cronbach's alpha; CPT = Clinical Personality Tendencies; APP = Anomalous Perceived Phenomena; IMA = Incoherent Manipulations; ASC = Altered States of Consciousness.

3.2. Second-order exploratory factor analysis

Once the 16 primary factors were defined, a second-order EFA was applied to empirically justify the macrofactors. A favorable KMO was obtained for the use of a higher-order EFA (KMO = 0.837). This analysis also did not apply Bartlett's sphericity test for the same reason that it was not used in the first-order EFA. Tables 9 and 10 show the correlation matrix for the 16 primary factors on which the adequacy of EFA is based. Table 11 shows the factor loads of the second-order EFA with the axes rotated. The descriptive statistics for the macrofactors are provided in Table 8.

Table 9.

Lineal correlations between MMSI-2 scales.

Items	K	L	F	Si	Nt	Cs	Su	Be
K	-
L	0.897	-
F	0.889	0.998	-
Si	0.888	0.962	0.970	-
Nt	-0.313	-0.319	-0.320	-0.353	-
Cs	-0.288	-0.282	-0.283	-0.310	0.986	-
Su	-0.600	-0.649	-0.653	-0.692	0.541	0.466	-
Be	-0.603	-0.655	-0.661	-0.710	0.511	0.433	0.991	-
Pva	0.181	0.185	0.185	0.187	-0.085	-0.078	-0.193	-0.187
Pt	0.112	0.121	0.121	0.121	-0.047	-0.044	-0.114	-0.109
Po	0.079	0.089	0.090	0.089	0.003	0.006	-0.063	-0.059
Pc	0.111	0.118	0.118	0.120	-0.026	-0.022	-0.103	-0.098
Hi	-0.629	-0.656	-0.658	-0.712	0.497	0.423	0.950	0.956
Ez	-0.628	-0.657	-0.661	-0.715	0.493	0.420	0.950	0.957
Pa	-0.630	-0.671	-0.671	-0.705	0.529	0.466	0.954	0.949
Na	-0.643	-0.669	-0.670	-0.710	0.511	0.445	0.955	0.955

Table 10.

Lineal correlations between MMSI-2 scales.

Items	Pva	Pt	Po	Pc	Hi	Ez	Pa	Na
K
L
F
Si
Nt
Cs
Su
Be
Pva	-
Pt	0.990	-
Po	0.985	0.990	-
Pc	0.993	0.994	0.996	-
Hi	-0.196	-0.115	-0.065	-0.104	-
Ez	-0.198	-0.118	-0.067	-0.107	0.998	-
Pa	-0.206	-0.124	-0.070	-0.110	0.980	0.982	-
Na	-0.210	-0.127	-0.074	-0.114	0.991	0.992	0.994	-

Table 11.

Second-order exploratory factor analysis.

Items	Extracted factors
	1	2	3	4
	CPT	APP	IMA	ASC
Ez	0.924
Hi	0.923
Na	0.918
Pa	0.905
Be	0.899
Su	0.894
Pc		0.998
Po		0.997
Pt		0.993
Pva		0.987
L			0.915
F			0.913
Si			0.861
K			0.813
Cs				0.961
Nt				0.927
Eigenvalue	8.879	3.857	1.714	1.188

Note: Factor loadings under 0.4 were eliminated (N = 3,224).

The Guttman-Kaiser method retained four factors that in total explained 97.737% of the model's variance. The first factor had an eigenvalue of 8.879, included the Na, Pa, Ez, Hi, Su, and Be scales, and was classified as Clinical Personality Tendencies (CPT). The second factor had an eigenvalue of 3.857, included the perceptual scales Po, Pc, Pt, and Pva, and was classified as Anomalous Perceived Phenomena (APP). The third factor had an eigenvalue of 1.714, included the Si, F, L, and K scales, and was classified as Incoherent Manipulations (IMA). The final factor had an eigenvalue of 1.188, included the Cs and Nt scales, and was classified as Altered States of Consciousness (ASC).

3.3. Reliability and scaling of direct scores

The reliability rates of the primary factors and macrofactors are shown in Table 8. According to the classification by George and Mallery (2003), all of them were satisfactory and excellent (>0.8). These results suggested that no item or scale should be deleted from the test to optimize its internal consistency.

Following the recommendations of Muñiz (2003), scaling of the MMSI-2 was carried out based on two normative groups differentiated by gender. The percentiles (Pc) and standard derived scores (or simply T scores) were used. Tables 12, 13, 14, 15, and 16 present the standardized scales with corresponding direct scores for both PT and Pc.

Table 12.

T Scores and Percentiles of MMSI-2 scales (men).

Pc	Scales								T
	K	L	F	Si	Nt	Cs	Su	Be
99	42–60	110–115	96–100	28–30	72–75	33–35	33–35	20	73
98	37–41	106–109	93–95	27	69–71	31–32	31–32	19	71
97	33–36	102–105	89–92	25–26	67–68	30	29–30	18	69
96	30–32	98–101	86–88	-	65–66	28–29	28	-	68
95	29	94–97	83–85	23–24	64	27	-	-	66
90	22–28	81–93	71–82	21–22	55–63	22–26	25–27	16–17	63
85	21	71–80	63–70	20	49–54	19–21	24	15	60
80	20	63–70	57–62	19	45–48	17–18	23	-	58
75	19	58–62	52–56	18	41–44	16	22	14	57
70	-	52–57	48–51	-	38–40	15	-	13	56
65	18	49–51	45–47	17	34–37	-	21	-	54
60	-	46–48	42–44	-	32–33	14	20	12	53
55	17	44–45	40–41	16	30–31	13	19	-	51
50	-	41–43	36–39	-	28–29	-	-	-	50
45	-	39–40	34–35	15	27	12	18	11	49
40	16	38	33	-	26	-	17	-	47
35	-	37	31–32	14	25	11	-	10	46
30	-	35–36	30	-	24	-	16	-	44
25	15	34	28–29	13	23	10	15	9	43
20	-	33	27	12	22	9	13–14	-	42
15	14	31–32	25–26	-	21	-	12	8	40
10	13	28–30	24	11	19–20	8	10–11	7	37
5	-	27	23	10	18	7	9	6	34
4	12	26	22	-	-	-	-	-	32
3	-	25	-	9	17	-	8	-	31
2	-	24	21	-	-	-	-	-	29
1	-	23	20	6–8	15–16	-	7	4–5	27
N	1,596	1,596	1,596	1,596	1,596	1,596	1,596	1,596	N
Mean	18.15	48.61	42.73	16.05	33.41	13.97	18.57	11.64	Mean
SD	5.80	20.32	18.36	4.19	13.78	5.67	5.53	3.37	SD

Note: Pc= Percentiles; T= T Scores; N= sample; M= Means; SD= Standard Deviations.

Table 13.

T Scores and Percentiles of MMSI-2 scales (men).

Pc	Scales								T
	Pva	Pt	Po	Pc	Hi	Ez	Pa	Na
99	51–55	34–35	33–35	43–45	62–65	53–55	49–50	58–60	73
98	47–50	32–33	31–32	41–42	59–61	50–52	47–48	55–57	71
97	44–46	31	30	38–40	56–58	49	46	53–54	69
96	41–43	30	29	37	54–55	47–48	45	52	68
95	39–40	29	28	35–36	53	46	44	51	66
90	33–38	24–28	24–27	30–34	49–52	42–45	40–43	47–50	63
85	28–32	20–23	21–23	26–29	47–48	39–41	37–39	45–46	60
80	25–27	18–19	20	24–25	45–46	38	36	43–44	58
75	23–24	17	18–19	22–23	43–44	36–37	35	42	57
70	21–22	16	17	21	42	35	34	41	56
65	20	15	16	20	41	34	-	40	54
60	19	14	-	19	40	33	33	39	53
55	18	-	15	18	39	-	-	38	51
50	17	-	14	17	38	32	32	36–37	50
45	-	13	-	16	37	31	31	-	49
40	16	-	13	-	36	30	29–30	35	47
35	15	12	-	15	35	29	28	34	46
30	-	-	12	-	34	28	26–27	32–33	44
25	14	11	-	14	32–33	27	23–25	30–31	43
20	13	10	11	13	31	25–26	21–22	28–29	42
15	-	9	10	12	29–30	23–24	18–20	25–27	40
10	12	-	9	11	25–28	20–22	15–17	22–24	37
5	-	8	8	10	24	19	14	20–21	34
4	11	-	-	-	23	18	13	19	32
3	-	7	-	-	21–22	17	12	17–18	31
2	-	-	-	9	18–20	15–16	11	15–16	29
1	-	-	7	-	13–17	11–14	10	12–14	27
N	1,596	1,596	1,596	1,596	1,596	1,596	1,596	1,596	N
Mean	19.96	14.84	15.65	19.05	38.22	32.00	30.05	36.53	Mean
SD	8.65	5.88	5.78	7.35	8.66	7.81	8.61	8.97	SD

Note: Pc= Percentiles; T= T Scores; N= sample; M= Means; SD= Standard Deviations.

Table 14.

T Scores and Percentiles of MMSI-2 scales (women).

Pc	Scales								T
	K	L	F	Si	Nt	Cs	Su	Be
99	50–60	110–115	97–100	29–30	72–75	33–35	33–35	20	73
98	39–49	106–109	93–96	27–28	69–71	31–32	31–32	19	71
97	33–38	102–105	89–92	25–26	67–68	29–30	29–30	18	69
96	30–32	98–101	86–88	-	65–66	28	28	-	68
95	29	94–97	83–85	24	64	27	-	-	66
90	22–28	81–93	71–82	22–23	55–63	22–26	25–27	16–17	63
85	21	71–80	63–70	20–21	49–54	19–21	24	15	60
80	20	63–70	57–62	19	45–48	17–18	23	-	58
75	19	58–62	52–56	-	41–44	16	22	14	57
70	-	53–57	48–51	18	37–40	15	-	13	56
65	18	49–52	45–47	17	34–36	-	21	-	54
60	-	46–48	42–44	-	32–33	14	20	12	53
55	17	44–45	40–41	16	30–31	13	19	-	51
50	-	41–43	36–39	-	28–29	-	-	-	50
45	-	39–40	34–35	15	27	12	18	11	49
40	16	38	33	-	26	-	17	-	47
35	-	37	31–32	14	25	11	-	10	46
30	-	35–36	30	-	24	-	16	-	44
25	15	34	28–29	13	23	10	15	9	43
20	-	32–33	27	12	22	9	13–14	-	42
15	14	31	25–26	-	21	-	12	8	40
10	13	28–30	24	11	19–20	8	10–11	7	37
5	12	27	23	10	18	7	9	6	34
4	-	26	22	-	-	-	-	-	32
3	-	25	-	9	17	-	8	-	31
2	-	24	21	-	-	-	-	-	29
1	-	23	20	6–8	15–16	6	7	4–5	27
N	1,628	1,628	1,628	1,628	1,628	1,628	1,628	1,628	N
Mean	18.20	48.67	42.79	16.06	33.28	13.92	18.56	11.64	Mean
SD	5.98	20.42	18.46	4.22	13.74	5.65	5.51	3.37	SD

Note: Pc= Percentiles; T= T Scores; N= sample; M= Means; SD= Standard Deviations.

Table 15.

T Scores and Percentiles of MMSI-2 scales (women).

Pc	Scales								T
	Pva	Pt	Po	Pc	Hi	Ez	Pa	Na
99	52–55	34–35	34–35	43–45	62–65	53–55	49–50	58–60	73
98	47–51	33	31–33	41–42	59–61	50–52	47–48	55–57	71
97	44–46	31–32	30	39–40	56–58	49	46	54	69
96	42–43	30	29	37–38	54–55	47–48	45	52–53	68
95	39–41	29	28	35–36	53	46	-	51	66
90	33–38	24–28	24–27	30–34	49–52	42–45	40–44	47–50	63
85	28–32	20–23	22–23	26–29	47–48	39–41	37–39	45–46	60
80	25–27	18–19	20–21	24–25	45–46	38	36	43–44	58
75	23–24	17	18–19	22–23	44	36–37	35	42	57
70	21–22	16	17	21	42–43	35	34	41	56
65	20	15	16	20	41	34	-	40	54
60	19	14	-	19	40	33	33	39	53
55	18	-	15	18	39	-	-	38	51
50	17	-	14	17	38	32	32	36–37	50
45	-	13	-	-	37	31	31	-	49
40	16	-	13	16	36	30	29–30	35	47
35	15	12	-	15	35	29	28	34	46
30	-	-	12	-	34	28	26–27	32–33	44
25	14	11	-	14	32–33	27	23–25	30–31	43
20	13	10	11	13	31	25–26	20–22	28–29	42
15	-	-	10	11–12	28–30	23–24	18–19	25–27	40
10	12	9	9	-	25–27	20–22	15–17	22–24	37
5	-	8	8	10	24	19	14	20–21	34
4	11	-	-	-	23	18	13	19	32
3	-	7	-	-	21–22	17	12	17–18	31
2	-	-	-	-	19–20	15–16	11	15–16	29
1	-	-	7	9	13–18	11–14	10	12–14	27
N	1,628	1,628	1,628	1,628	1,628	1,628	1,628	1,628	N
Mean	20.06	14.90	15.71	19.13	38.21	31.99	30.04	36.53	Mean
SD	8.78	5.95	5.84	7.43	8.66	7.81	8.61	8.97	SD

Note: Pc= Percentiles; T= T Scores; N= sample; M= Means; SD= Standard Deviations.

Table 16.

T Scores and Percentiles of MMSI-2 scales (men and women).

Pc	Second-order scales								T
	Men				Women
	IMA	APP	ASC	CPT	IMA	APP	ASC	CPT
99	276–305	161–170	105–110	265–285	286–305	163–170	105–110	265–285	73
98	263–275	151–160	100–104	260–264	266–285	152–162	100–104	260–264	71
97	249–262	143–150	97–99	254–259	249–265	144–151	96–99	254–259	69
96	239–248	137–142	93–96	250–253	239–248	138–143	93–95	250–253	68
95	229–238	131–136	91–92	248–249	230–238	131–137	91–92	242–249	66
90	195–228	111–130	77–90	223–247	196–229	111–130	77–90	222–241	63
85	175–194	95–110	68–76	207–222	175–195	96–110	68–76	207–221	60
80	159–174	87–94	62–67	199–206	159–174	87–95	62–67	199–206	58
75	147–158	80–86	57–61	193–198	148–158	80–86	57–61	193–198	57
70	137–146	75–79	53–56	184–192	138–147	75–79	52–56	184–192	56
65	129–136	71–74	49–52	183	129–137	71–74	49–51	183	54
60	123–128	68–70	46–48	179–182	123–128	68–70	46–48	179–182	53
55	117–122	65–67	43–45	173–178	117–122	65–67	43–45	173–178	51
50	108–116	61–64	40–42	169–172	108–116	62–64	40–42	169–172	50
45	103–107	59–60	39	160–168	103–107	59–61	39	160–168	49
40	100–102	56–58	37–38	154–159	100–102	56–58	37–38	154–159	47
35	97–99	55	36	148–153	97–99	55	36	148–153	46
30	92–96	52–54	34–35	141–147	92–96	52–54	34–35	141–147	44
25	88–91	48–51	32–33	132–140	88–91	48–51	32–33	132–140	43
20	85–87	45–47	31	123–131	84–87	46–47	31	122–131	42
15	79–84	42–44	29–30	110–122	79–83	43–45	29–30	110–121	40
10	74–78	39–41	26–28	95–109	73–78	39–42	26–28	95–109	37
5	71–73	38	25	89–94	71–72	38	25	89–94	34
4	69–70	37	-	84–88	69–70	37	-	84–88	32
3	67–68	36	24	78–83	67–68	35–36	24	78–83	31
2	65–66	35	-	71–77	65–66	-	-	68–77	29
1	61–64	34	22–23	57–70	61–64	34	22–23	57–67	27
N	1,596	1,596	1,596	1,596	1,628	1,628	1,628	1,628	N
Mean	125.53	69.49	47.38	167.02	125.72	69.81	47.20	166.98	Mean
SD	47.99	27.57	19.39	42.49	48.37	27.92	19.33	42.49	SD

Note: Pc= Percentiles; T= T Scores; N= sample; M= Means; SD= Standard Deviations.

4. Discussion

The initial objective of this research was to examine the psychometric properties of the MMSI-2. A secondary objective was to analyze the factorial covariation among indicators that measure anomalous phenomena and indicators that evaluate the different psychological attributes that are presumably concomitant with these anomalous phenomena. In general, the factorial solutions and the reliability coefficients obtained suggest that the MMSI-2 is a valid and reliable instrument for the multiaxial evaluation of anomalous phenomena.

4.1. Analysis of results

First, it must be kept in mind that the MMSI-2 is a multiaxial instrument because it incorporates different evaluation constructs. The factorial solution offered by the first-order EFA (see Tables 1, 2, 3, 4, 5, 6, and 7) is compatible with the psychological variables previously observed in the published scientific literature. As can be observed in the conceptual framework, of the 15 psychological variables that were initially identified in the theoretical evidence (e.g., paranoia, narcissism, histrionism, schizotypy, substance abuse, exhaustion/anxiety, thrill-seeking, childhood trauma, simulation, fraud, creativity, intuition, extraversion, dissociative disorders and suggestibility), the first-order EFA allowed up to 10 to be identified. It should be recalled that of the 16 primary factors, four are anomalous phenomena (Pva, Pt, Po and Pc), and the K and L scales are included as prototypical variables of this type of psychometric inventory (e.g., Millon, 1994; Morey, 2011). As explained in the procedure section of this study, the remaining variables based on theory (childhood trauma, creativity, intuition, extraversion and dissociative disorders) had already been dismissed from the 174-item version. This compatibility is indicates that the items on the MMSI-2 were developed correctly and constitute valid empirical indicators for measuring the constructs included in the 16 factors/scales of the test. Similarly, regarding factorial loads, it can be observed that the reverse-scored items had negative saturations, a fact that also confirms the goodness of the measurement of each indicator/item.

As a second observation, it should be noted that the first-order EFA with factors that coincide with the constructs from the literature does not examine the relationship of the 10 psychological variables plus the K and L scales with the anomalous perceived phenomena. The results of the first-order EFA may prove the exploratory validity of the MMSI-2 measures and analyze the relationships between the items; however, they do not serve to examine the possible relationships among the extracted factors. If such an intercorrelation does exist between the scales, its effects should be observable in the second-order factor solution. Specifically, it would be expected that the secondary APP would distribute the primary factors without isolating the perceptual scales from other factors. The fact that the solution yielded the secondary factor and that its variables showed no significant saturation in any other macrofactor suggests that the Pva, Pt, Po and Pc scales may present behavior that is independent of the other scales. Consequently, this fact calls into question the relationship of anomalous phenomena with the other psychological variables. It should be noted that this observation does not imply that the macrofactor APP is independent of the other higher-order factors given that the rotation used in the solution was oblique and influenced the loadings of the primary factors (and not the second-order factors). However, the independence of these four scales can also be checked by analyzing the correlation matrix in Tables 9 and 10. The correlations of the variables Pva, Pt, Po and Pc presented values close to 0 when they were related to other psychological variables. If these scales were related to the other variables, they should have had correlations other than 0. These nuances do not disprove the theoretical background that defends the relationship between anomalous phenomena and the highlighted psychological variables. However, they do call these theories into question and suggest, as indicated by French and Stone (2014), that anomalous behaviors related to ‘psi’ phenomena and parapsychology do not have an obvious psychological explanation if they go beyond the psychopathological. The question that arises from this observation confronts the following thought: depending on how anomalous phenomena are evaluated, their covariant behavior may be more or less independent with respect to the other variables. It should be kept in mind that the MMSI-2 examines anomalous phenomena by relating them to ‘psi’ phenomena and parapsychological experiences. By way of speculation, one might wonder what would happen with the second-order EFAs or the correlation matrix in Tables 9 and 10 if anomalous phenomena were evaluated exclusively as psychotic hallucinatory symptoms. It would then be possible for such intercorrelations to vary, yielding different results.

Along these same lines, the CPT scale includes the Pa, Na, Hi and Ez scales as subclinical features but also adds Su and Be, which had very high factorial loads. Specifically, the term “personality” was added to the CPT scale because it is understood that suggestibility and thrill-seeking are also stable psychological traits (e.g., Irwin et al., 2013). Therefore, according to the theoretical framework, the suggestibility that the MMSI examines should be related to secondary suggestibility (and not to altered states of consciousness) as it is concomitant with other subclinical personality traits (e.g., Smith et al., 2009). This observation is also supported by the content of the items pertaining to Su, which conceptually coincide with the understanding of suggestibility as a personality tendency (e.g., Eysenck, 2017; Hefferline et al., 1972). This differs from the macrofactor ASC, which probably does not measure traits, but rather measures dynamic psychological states (e.g., Groth-Marnat, 2009; Hambleton, 2008). According to the theory, both Nt and Cs are common characteristics that are observable during alterations of consciousness (e.g., Alvarado, 1998; Jinks, 2019). However, the latter secondary factor also suggests that the anxiety evaluated by MMSI-2 is not a stable psychological trait but a situational anxious state.

The macrofactor IMA demonstrated a grouping of variables consistent with what was expected based on the theory (e.g., Álvarez, 2007; French and Stone, 2014; Wilson and French, 2006). All the scales included in the IMA are variables that can be interpreted as manipulations of the answers based on deception. Unlike what was expected, while theoretically social desirability (L) is a common tendency observable in personality measures (e.g., Eysenck, 2017; Groth-Marnat, 2009), this test involves a scale that does not demonstrate high saturation in the CPT factor. The correlation matrix (see Tables 9 and 10) supports this idea and suggests that this scale also has independent behavior. In fact, Fernández-Ballesteros (2011) pointed out that social desirability is a factor that provides a very low explained variance in personality inventories, a statement that does not seem so inconsistent with the low correlations obtained in the MMSI-2. The same behavior can be extrapolated to the K, F, and Si scales. Additionally, taking into account the difficulties present in the analysis of fraudulent behavior (e.g., MacNeil and Soper, 2019), the macrofactor IMA invites us to think about the extent to which the MMSI-2 scales could be useful in lie detection, especially in forensic assessments. The EFA applied in this sample, and specifically its eigenvalues, indicates that this is a group of scales and items that provides more statistical information than other constructs (see Tables 1, 2, 3, 4, 5, 6, and 7; e.g., λ_Si = 28.778>λ_Ez = 0.961). The reason why these scales have so much weight in the MMSI-2 (according to their eigenvalues and factorial charges) is unknown. However, this statistical evidence could be used and tested in other areas of evaluation where these variables have been evaluated with little reliability (e.g., Cardona, 2002; Vrij et al., 2019).

Third, it should be recalled that the MMSI-2 was created as a tool with which to compare psychological hypotheses that seek to explain anomalous phenomena and test the ‘psi’ hypothesis. To do this, it is appropriate to interpret the scales created and analyze the scope they offer in psychological evaluation. Direct scores may be useful in future statistical research; however, if the assessment or comparison of hypotheses is applied individually (e.g., through the development of psychological profiles), then it is necessary to determine the position the patient occupies within a normative group (see Kline, 1999; Muñiz, 2003). At the same time, following Martínez-Arias et al. (2006), in this context, the scales also make it possible to define the empirical thresholds that determine the extent to which an evaluated subject develops behaviors occurring at a frequency and intensity that is becoming clinically suspect. These cut-off points are normative and determined according to the transformation scale used. According to the psychometric contributions of other authors (e.g., Ben-Porath and Tellegen, 2019; Morey, 2011), in the case of the MMSI-2, the use of T scores instead of percentiles is recommended. Percentiles have been shown to facilitate statistical research but have shown less benefit for the analysis of the patient/client's psychometric profile. PT have an average value of 50 points, and their standard deviations have a value of 10 points (e.g., Abad et al., 2015). The following cut-off points are recommended for both men and women: PT ≥ 50 and PT ≥ 60. The significance of the scores is questionable when they exceed the average value (PT = 50) and clear when a PT ≥ 60 is obtained. Due to the eigenvalues and asymmetric-positive distribution of scores in the K, F, Cs and IMA scales, the use of more restrictive cut-off points (PT ≥ 40) is recommended. This suggestion is based on the fact that the behaviors described in the items of these scales are dysfunctional and are inconsistent with psychological normality. However, this proposal should be validated in subsequent studies, and hence, caution is recommended. The value of all of these cut-off scores and these scales is observed when a subject scores high on both the scales that examine abnormal phenomena and one of the other psychological variables. The possible debate raised by the MMSI-2 involves whether significant scores for psychological variables could explain significant values on the APP scale. This debate will probably be clarified when the factors are analyzed in subsequent confirmatory studies, as recommended by Gorsuch (1983).

To specify the behaviors assessed in each factor or scale, Tables 17 and 18 summarize and classify the attributes measured in each dimension.

Table 17.

Contents and behaviors assessed by each scale in the MMSI-2 (scales 1–8).

Scales	Content assessed
K	Random answers Psychopathological risks	Understanding of items Degree of cooperation with the interview
L	Presence of lies Defensive behaviors Moralistic behaviors	Credulity Unmotivated decisions
F	Psychological games Deliberate lies Manipulation Frivolous behaviors	Machiavellianism Be bad thought Be a whistle
Si	Ambiguous answers Confusing answers	Lack of responsibility Barnum effect
Nt	Emotional instability Intermittent tiredness Anxiety	Lack of energy Somatic behaviors
Cs	Toxic consumption Drug intake	Self-medication Attention disorders
Su	Emotional lability Fearful behaviors Emotional intensity	Difficulty understanding feelings Sensitivity
Be	Tendency toward morbidity Overstimulation Sympathy for the exotic	Curiosity Fantasy trend

Note: K= Inconsistencies; L= Lies; F= Fraud; Si= Simulation; Nt= Neurasthenia; Cs= Substance Use; Su= Suggestibility; Be= Thrill-Seeking.

Table 18.

Contents and behaviors assessed by each scale in the MMSI-2 (scales 9–16).

Scales	Content assessed
Pva	Hearing voices of deceased beings Seeing deceased beings (ghosts) Seeing supernatural beings	Seeing strange shadows Hearing unidentified noises Hearing music of unknown origin
Pt	Feeling unexplained chills Feeling touch without anyone else present	Perceiving the presence of others who are not physically there Feeling of pressure in different parts of the body
Po	Change in the quality of odors Sensing unexplained odors	Perceiving the scent of a deceased being Perceiving smells at impossible distances
Pc	Déjà vu experiences Strange sensations in everyday places	Inability to recognize familiar places Recognizing to unfamiliar places Sudden changes in the size of body parts
Hi	Seductive behaviors Eccentric behaviors Affective conflicts	Tendency to exaggerate Fear of feeling alone
Ez	Superstitious behaviors Difficulties in socialization	Tendency to isolate Tendency toward apathy Magical thinking
Pa	Social mistrust Persecutory anxieties Irrational interpretations	Difficulties in making commitments Feelings of betrayal
Na	Self-referral ideas Difficulties in integrating criticism	Impatience Need for differentiation Search for ambitions

Note: Pva= Anomalous Visual/Auditory Phenomena; Pt= Anomalous Tactile Phenomena; Po= Anomalous Olfactory Phenomena; Pc= Anomalous Cenesthetic Phenomena; Hi= Histrionism; Ez= Schizotypy; Pa= Paranoia; Na= Narcissism.

EFAs are a good start and constitute a very useful empirical basis for assigning validity to a psychometric instrument (see Mulaik, 2018), however, the extracted factors must be subsequently replicated with a confirmatory factor analysis (henceforth CFA). While EFAs attribute validity to measures, CFAs also confirm and validate the structural model that justifies the conceptual bases of the instrument (e.g., Ruiz, 2000). Indeed, as with Houran et al. (2019), the main problem encountered in this research was the lack of previously validated theoretical models. As mentioned in the previous sections, although instruments exist that examine anomalous perceptions among nonclinical samples (e.g., Bell et al., 2006; Mason and Claridge, 2006; Stefanis et al., 2002), consistent statistical results are not provided when anomalous phenomena are related to other psychological variables outside the psychopathological framework (see Irwin, 2009; Parker, 2006). Thus, from an empirical perspective and taking into account sample size, EFA was chosen as the most suitable mathematical design for this type of scenario (see Kline, 1999; Mulaik, 2018). In fact, this limitation emphasizes that the MMSI-2 also requires a valid theoretical foundation that verifies the relationships among the constructs it evaluates. However, it is not possible to determine a theoretical model without consistent prior evidence, and this psychometric work based on EFAs is thought to be able to provide such evidence.

Another drawback is seen in the trait vs. state aspect of some of the scales in the MMSI-2. It is unclear whether the macrofactor CPT includes variables of a trait type and the ASC dimension includes factors of a state type. To verify the state vs. trait conditions in the respective scales, a test-retest design of repeated samples must be applied to simultaneously analyze the reliability of differences between the scores (e.g., Abad et al., 2015). These reliability indices would be complementary to internal consistency and would not alter the obtained alpha coefficients, which were excellent (e.g., George and Mallery, 2003).

A very obvious limitation also lies in the interpretation of scores using PTs. Scaling in this type of test is essential, but it seems necessary to validate the cut-off points using designs that analyze the sensitivity and specificity of the MMSI-2. These designs can be based on ROC curves and logistic regression, although an external classifier would need to be determined to allow comparison between the MMSI forecasts and classifications that were assumed as criteria. It is true that the scales offer T scores, which are useful in psychometric evaluation to facilitate decision-making in response to the degree of significance of the scores. However, individual T scores do not allow us to resolve the dilemma derived from research into ‘psi’ phenomena with significant results. If a subject obtained significant scores on the APP scales and high values for the other psychological variables, it would not be possible to confirm that the anomalous phenomena were produced by high scores on those psychological variables. However, this hypothetical profile would provide sufficient grounds to suspect that psychological scales are predictors of perceived anomalous phenomena. This limitation means that new research and contrasts are needed to examine the variation in PTs on the APP scales based on the possible effects of scores obtained for the other variables.

Given these limitations and the scientific literature cited, future lines of research should address three key points: (1) the conceptual understanding of observable differences between anomalous experiences understood as attenuated psychotic hallucinations and anomalous phenomena evaluated as anomalous experiences related to ‘psi’ phenomena; (2) the statistical analysis of new psychometric properties of the MMSI-2 that identify and confirm the factorial solution presented in this report; and (3) the goodness of cut-off points that would allow evaluative decisions (though not diagnostic ones) to be made (see also Jabbari et al., 2018; Lappalainen, 2019; Van Zeebroeck, 2019).

4.2. Conclusions

This study presents three main findings. First, the study obtained 12 empirical markers (the 12 scales of the MMSI-2) for identifying, examining, and measuring possible causes of perceived anomalous phenomena. Four specific markers were also obtained for evaluating abnormal experiences. Second, the study demonstrated that it is possible to define and establish an empirical-statistical model for evaluating anomalous phenomena. This model should allow for examination of perceptual anomalies to determine whether they are the result of hallucination, biases, deliberate fraud, or behaviors without a psychological-psychiatric explanation. Third, it is also concluded that it is necessary to review this factor model and validate it through a CFA and structural equation models.

Ultimately, the MMSI-2 is accepted as a valid and reliable psychometric instrument for evaluating anomalous phenomena and the theoretically concomitant psychological variables. The 174 items and the 20 psychometric scales of the MMSI-2 can be used in future studies aimed at psychological profile analysis and statistical research to confirm the predictive relationship between different psychological variables and anomalous phenomena.

Declarations

Author contribution statement

A. Escolà-Gascón: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.