MMPI-2-RF Validity Scales Add Utility for Predicting Treatment Engagement During Partial Psychiatric Hospitalizations

Abstract

Partial psychiatric hospitalizations are resource-intensive clinical services designed to stabilize patients in the short term, prevent inpatient hospitalizations, and encourage long-term recovery. Typically, providers base their referral decisions on categorical diagnoses and subjective impressions of patient distress without closely considering the evidence for reporting biases. The present study followed veterans (n = 430) participating in partial psychiatric hospitalization services. We evaluated the extent to which clinical diagnoses at intake predicted treatment variables and changes in later mental health care utilization. Using hierarchical linear regressions with bootstrap confidence intervals, Minnesota Multiphasic Personality Inventory–2–Restructured Form content-based validity scales demonstrated incremental utility for predicting patient outcomes beyond intake diagnoses. Elevated Fp-r (“Infrequent Psychopathology Responses”) scores independently predicted an increased number of times arriving late for partial hospitalization programming, self-report of worse current functioning at intake, and a relative increase in mental health care encounters in the 12 months following discharge. Low K-r (“Adjustment Validity”) scores independently predicted self-report of worse current functioning at both intake and later discharge from partial hospitalization. Thus, indicators of severe psychopathology overreporting as well as the unlikely disavowal of emotional adjustment (i.e., high Fp-r, low K-r) predicted engagement with health care services and self-presentations of symptoms over and above the diagnostic impressions from referring providers. We discuss how indicators of content-based invalid responding on the Minnesota Multiphasic Personality Inventory–2–Restructured Form have real-world value for understanding patient behavior and shaping clinical interventions among vulnerable populations.

When patients report a psychiatric crisis, providers in large health care systems have several intervention choices including inpatient hospitalization. Concerns about rising costs and patient satisfaction have led to a resurgence of interest in “partial” psychiatric hospitalizations as an alternative for moderate-to-severe symptom presentations (Schene, 2004). These short-term day treatment programs keep patients embedded in their current lives while providing interventions within a therapeutic group milieu environment (Rosie, 1987). This elevated level of care gives patients a chance to learn new emotion regulation and interpersonal skills, gain crisis problem-solving support, receive additional diagnostic and treatment planning clarification, and achieve symptom stabilization to prevent more involved and costly inpatient hospitalizations (Forgeard et al., 2018; Leung et al., 2009). The existing research suggests similar efficacy between full and partial psychiatric hospitalizations for reducing acute psychiatric distress, but higher patient and family satisfaction and greater social functioning in the months following discharge from the latter (Horvitz-Lennon et al., 2001; Kallert et al., 2007). Given the resource-intensive nature of partial hospitalizations, there is a need for increased understanding of the clinical characteristics of people most able to benefit from the day treatment milieu format. Closer examination of patient engagement and clinical outcomes in the context of partial hospitalizations may reveal malleable behaviors and attitudes associated with better treatment responses.

Diagnostic complexity is a consistent predictor of partial hospitalization outcomes. For example, comorbid depression is associated with worse overall functioning and smaller relative improvements after participating in day treatments for eating disorders (Fewell et al., 2017; Hayes et al., 2019). Furthermore, greater clinical need as indicated by psychotic disorders, comorbid traits of personality disorders, and recent discharges from inpatient hospitalizations all predict poorer partial hospitalization treatment outcomes (Beard et al., 2016; Larivière et al., 2010; Zeeck et al., 2016). The effect of partial hospitalizations may also depend on extra-diagnostic factors beyond symptom complexity such as motivation level, hedonic capacity, and perceived social support during treatment (Fasoli et al., 2010; Zeeck et al., 2009). In addition, optimistic treatment outcome expectations by patients predict enhanced partial hospitalization responses (Beard et al., 2016). Therefore, it is likely that individual differences outside of just clinical diagnoses (e.g., patient self-appraisals, attitudes, and beliefs about current disability) may shape treatment responses.

The credibility of patients’ communication with providers about their current experiences and symptoms may be associated with treatment outcomes. Most mental health care relies on self-disclosures. Accurate verbal portrayal of current functioning from patients to providers is important for determining which intervention targets are most pressing. This is particularly true during acute crises when clinicians must triage problems and quickly provide therapeutic benefit, sometimes with limited time and resources. When patients inaccurately convey the scope or magnitude of their symptoms relative to what might be concluded using outside evidence, interventions can become inefficient and haphazard. Critical problems can be missed or inadvertently minimized, which may delay the administration of needed treatments. Noncredible reporting need not be deliberate (e.g., malingering). For some patients, noncredible communication about distress and impairment can be driven by subtle factors such as incentives and contextual demands outside of immediate awareness (Frueh et al., 2003). Regardless of the exact cause, inaccurate or distorted symptom portrayals may signal the presence of psychological, interpersonal, and institutional processes with the potential to interfere with constructive participation in treatments. The extent to which veterans’ inaccurate reporting may predict behavioral engagement and mental health service utilization after discharge from partial hospitalization milieus is unknown. Instruments that assess distorted self-perceptions about functioning or reduced ability to accurately communicate about symptoms may help identify people experiencing barriers to full and active engagement within elevated care settings.

Two of the most widely used measures of personality in general clinical settings and U.S. Veterans Affairs (VA) medical centers are the Minnesota Multiphasic Personality Inventory−2 (MMPI-2; Camara et al., 2000) and the MMPI–2–Restructured Form (MMPI-2-RF; Ingram et al., 2020). Both instruments contain scales for identifying two types of invalid responses: non-content-based responding (e.g., random endorsements) and content-based responding (e.g., noncredible over- or under-reporting of symptoms; Ben-Porath, 2012). The original and revised forms of these scales have been repeatedly validated for high-stakes forensic applications (Arbisi et al., 2006; Sharf et al., 2017). Despite the presumed utility for informing pretreatment assessments, there are relatively few research studies evaluating the MMPI-2-RF as a tool for predicting treatment engagement (Anestis et al., 2015; Arbisi et al., 2022; Patel & Suhr, 2020; Tylicki et al., 2019). Validity scales could be uniquely valuable in this regard. Psychological assessments are sometimes conceptualized as microcosms of a patient’s broader style of engagement within a health care system (Anestis et al., 2015). Validity scales administered during psychological assessments are sensitive to a variety of potentially treatment-relevant behavioral processes that would be undetectable using symptom scales alone: compliance with directions and instructions, readiness to share openly about one’s experiences, and willingness to conduct thoughtful discriminations between personally relevant and irrelevant symptoms. MMPI-2-RF validity scales, although not designed or originally intended to predict behavioral phenomena in clinical settings, may be underutilized tools for assessing patients’ ability to participate fully in a range of treatment modalities.

Despite the potential for clinical benefit in case conceptualizations, there is some evidence that MMPI-2 validity scales are being ignored in VA settings when findings about noncredibility conflict with interview-based diagnostic impressions (Arbisi et al., 2004). A lack of full consideration of validity scales is noteworthy in VA health care systems given the contextual pressures on veterans to seek or maintain financial compensation for psychiatric conditions connected with military service (Frueh et al., 2007). The financial compensation can be substantial (up to the equivalent of full-time employment) and has been shown to lead to meaningful improvements in the quality of life among veterans (Murdoch et al., 2011). Furthermore, service connection disability status can unlock no out-of-pocket cost VA medical and psychiatric care. Thus, veterans interact with VA under significant secondary gain pressures. Institutional influences like these have been found to reinforce noncredible reporting of psychopathology even after veterans are already connected with ongoing clinical care (Frueh et al., 2003, 2007). The compensation benefits process may be reinforcing implicit attitudes about the necessity to amplify symptom reporting to obtain (or maintain access to) treatment. More routine considerations of pretreatment response validity in VA settings may save time and resources by highlighting opportunities to modify clinical care in ways that are responsive to these issues (e.g., collaborative feedback, additional assessment, delaying treatment).

The presence of multiple validity scales is an important feature of the MMPI-2-RF for assessing noncredible reporting. Not all content-based validity scales are the same. These scales are calibrated to assess a variety of domains in which invalid responding can occur (Sweet et al., 2021). Furthermore, access to multiple scales gives clinicians and researchers the ability to shift interpretive focus to the validity domains with the greatest relevance and research basis given their population of interest. For example, F-r (“Infrequent Responses”) detects responses uncommon in the general population, Fs (“Infrequent Somatic Responses”) assesses somatic content rarely endorsed by medical patients seeking treatment for physical conditions, and FBS-r (“Symptom Validity Scale”) detects noncredible somatic and cognitive endorsements that can occur in civil litigation contexts (Ben-Porath, 2012).

Three content-based validity scales on the MMPI-2-RF are particularly relevant for characterizing noncredible reporting during psychiatric hospitalizations (Arbisi & Ben-Porath, 1995; Ben-Porath, 2012). These measures quantify patterns of endorsements that are atypical among individuals with genuine psychiatric conditions. Fp-r (“Infrequent Psychopathology Responses”) was specifically designed and validated for the assessment of overreporting in settings with high rates of psychiatric disorders. Meta-analytic review of Fp-r suggests this scale is highly effective at discriminating feigned responding from genuine psychopathology (Sharf et al., 2017) and outperforms other overreporting scales in the identification of feigned emotional disorders (Burchett & Bagby, 2022). Items from Fp-r are rarely endorsed by individuals with serious psychopathology, including psychiatric inpatients (Glassmire et al., 2017). Underreporting, or a minimization of current dysfunction, is also of heightened relevance for providers in hospitalization settings. When the level of care is being evaluated, it is vital for providers to have access to indicators about concealment of symptoms or minimization of safety risks. On the MMPI-2-RF, L-r (“Uncommon Virtues”) assesses uncommon endorsements of positive moral attributes and activities consistent with an overly virtuous self-presentation; K-r (“Adjustment Validity”) assesses endorsement patterns demonstrated by individuals with frank psychopathology who also deny current dysfunction. L-r and K-r are both predictive of substantive reductions in psychopathology endorsements across the MMPI-2-RF (Khazem, Anestis, et al., 2021). L-r and K-r are especially relevant as indicators of underreporting about suicidal ideation and other related risk factors, which are frequently primary intervention targets during psychiatric hospitalizations (Khazem, Rufino, et al., 2021).

Although some have questioned the utility of assessing response validity in applied settings (McGrath et al., 2010; although see Morey, 2012; Rohling et al., 2011, for counterpoints), it is often assumed that willingness and capability to accurately report symptoms can contribute to more targeted assessments and collaborative therapeutic relationships (Finn, 2007; Patel & Suhr, 2020). To examine the impact of content-based validity on treatment and mental health care utilization, we completed a longitudinal observational study among veterans enrolled in a partial psychiatric hospitalization program. Using intake psychiatric diagnoses from referring VA providers, we aimed to predict engagement with partial hospitalization services, therapeutic group experiences within the milieu, self-report of current functioning, and frequency of outpatient mental health service encounters after discharge. Importantly, we included the MMPI-2-RF content-based validity scales of Fp-r, L-r, and K-r as simultaneous predictors using hierarchical regressions. Thus, we sought to demonstrate the degree to which indicators of over- or under-reporting with heightened relevance for psychiatric hospitalizations have incremental utility for predicting treatment variables beyond the diagnostic clinical impressions of referral sources. We hypothesized Fp-r would be independently associated with less consistent behavioral engagement during the partial hospitalization, greater endorsements of symptomatology at discharge, and more frequent mental health encounters with VA providers following discharge (Anestis et al., 2015; Arbisi et al., 2022). We hypothesized that elevations on L-r and K-r would be independently associated with fewer endorsements of symptomatology (i.e., negative association; Khazem, Anestis, et al., 2021; Khazem, Rufino, et al., 2021). These analyses may clarify the extent to which noncredible reporting about functioning and current symptomatology may be affecting treatment engagement and recovery in real-world clinical settings. Hierarchical models also address practical questions about whether MMPI-2-RF validity scales add clinical value to the prediction of patient outcomes over and above the intake information typically available to partial hospitalization providers.

Method

Participants and Procedure

The study was archival and included veterans enrolled in the partial psychiatric hospitalization program at a large VA medical center in the United States between 2014 and 2015. VA clinical staff administered the MMPI-2-RF to veterans shortly after their admission to the program. All self-report instruments were administered as part of the intake procedure to inform clinical care during the veterans’ participation in the program. Veterans received feedback regarding the MMPI-2-RF from licensed psychologists and acknowledged understanding that their results would be used clinically to clarify psychiatric diagnoses over the course of their partial hospitalization. The archival study was approved by the institutional review board of this VA medical center. In total, 465 veterans completed the MMPI-2-RF and other study measures. This study was not preregistered.

The final study sample included 430 veterans (92.5%) who produced MMPI-2-RF profiles that did not evidence significant response inconsistency or nonresponsiveness (see Measures section below). Demographics and frequencies of Diagnostic and Statistical Manual of Mental Disorders-5 (American Psychiatric Association, 2013) clinical diagnoses at admission are listed in Table 1. The majority of veterans identified as male and White of European ancestry. Veterans were 46 years old on average with an age range of 21–82 years. In total, 90.3% of veterans described completing at least a high school degree or equivalent level of education. Most veterans reported current unemployment. Anxiety, depression, trauma related, and substance use disorders were the most observed psychiatric diagnoses at admission (see Measures section below).

Table 1.

Demographics, Clinical Diagnoses, and Response Validity Scores

Variable	n	%	M	SD	Skewness	Kurtosis
Male	370	86.0
Age			46.0	13.5	0.13	−0.99
Nonwhite racial minority	69	16.0
Married or partnered	174	40.5
Education (years)			13.4	1.8	0.93	0.66
Current employment	133	30.9
Service connection (%)			43.7	34.9	0.14	−1.39
Anxiety Dx	131	30.5
Depression Dx	294	68.4
Trauma Dx	120	27.9
Substance Dx	198	46.0
Personality disorder Dx	50	11.6
Thought disorder Dx	60	14.0
F-r (raw)			11.6	6.2	0.31	−0.41
Fp-r (raw)			3.1	2.3	1.24	1.94
L-r (raw)			3.1	2.0	0.58	0.18
K-r (raw)			4.2	2.6	0.69	0.10
F-r (T score)			91.9	23.0
Fp-r (T score)			68.8	19.1
L-r (T score)			52.6	9.8
K-r (T score)			38.9	9.0

Note. Dx = psychiatric diagnosis upon intake; F-r = Infrequent Responses scale; Fp-r = Infrequent Psychopathology Responses scale; L-r = Uncommon Virtues scale; K-r = Adjustment Validity scale.

Measures

Thirty-five veterans were excluded from the study sample based on 15 or more missing MMPI-2-RF item responses (Cannot Say), Variable Response Inconsistency–revised T scores of 80 or greater, or True Response Inconsistency–revised T scores of 80 or greater. In keeping with the study aims, veterans were not excluded based on content-based validity scales assessing over- and under-reporting (Ben-Porath, 2012). We observed higher rates of overreporting in the current treatment-seeking study sample than would be expected within the general population, as evidenced by mean T scores falling well above 50 on MMPI-2-RF overreporting scales. One hundred two veterans (23.7%) produced invalid testing protocols based on F-r T scores of 120. This level of infrequent responding is uncommon even among individuals with genuine, severe psychopathology (Ben-Porath, 2012). Thirty-eight veterans (8.8%) produced invalid testing protocols based on Fp-r T scores greater than 100 indicative of a considerably greater frequency of symptom endorsements rarely described by individuals with genuine, severe psychopathology (Figure 1). Ninety veterans (20.9%) produced Fp-r T scores greater than 80 likely indicating overreporting. When combining the most stringent overreporting cutoffs for F-r (T score = 120) and Fp-r (T score > 100), 110 veterans (25.6%) met criteria for invalid responding. In comparison, only seven veterans (1.6%) produced L-r T scores greater than 80 indicative of extremely favorable self-presentations. No veterans presented themselves as unusually well-adjusted based on K-r T scores greater than 70. In summary, this evidence collectively indicates a greater than expected rate of overreporting among a substantial subset of study participants. We were interested in whether this elevated frequency of noncredible reporting has clinical consequences.

Figure 1. MMPI-2-RF Content-Based Validity Scales.

Note. Histograms depicting frequencies of responses on content-based validity scales. MMPI-2-RF = Minnesota Multiphasic Personality Inventory–2–Restructured Form; Fp-r = Infrequent Psychopathology Responses scale; L-r = Uncommon Virtues scale; K-r = Adjustment Validity scale.

Veterans completed the Outcome Questionnaire (OQ)–45.2 (Lambert et al., 1996) of self-reported functioning and psychotherapy treatment progress contemporaneously with the MMPI-2-RF (n = 393; Table 2). The measure has a total score (higher scores = greater dysfunction) and subscale scores in three domains: Symptom Distress for experiences of depression and anxiety, Interpersonal Relations for conflict and alienation, and Social Role for workplace and home life difficulties. At discharge, the OQ-45.2 was readministered to willing participants (n = 289). Furthermore, veterans completed the Group Climate Questionnaire (GCQ)-S measure of the group milieu experience (n = 300; MacKenzie, 1983). Subscales include Engagement assessing a positive working group atmosphere, Conflict assessing perceived hostility within the group, and Avoidance assessing perceived abdication of personal responsibility by other group members within the milieu. VA staff documented the length of treatment (days), premature withdrawal, number of no-show days, and number of times arriving late for partial psychiatric hospitalization programming for each veteran.

Table 2.

Study Measures

Variable	M	SD	Minimum	Maximum	Skewness	Kurtosis
Length of treatment (days)	20.4	5.8	0.0	41.0	−0.90	2.29
Times late	0.39	0.88	0.00	5.00	2.84	8.85
No-shows (days)	0.53	1.06	0.00	7.00	2.77	9.02
GCQ-Engagement	4.2	0.9	1.4	6.0	−0.36	0.03
GCQ-Conflict	1.0	0.8	0.0	4.3	1.21	1.89
GCQ-Avoidance	2.9	1.0	0.0	6.0	−0.06	0.24
OQ-45.2 total: Intake	94.5	23.3	28.0	152.0	−0.18	0.08
OQ-45.2 total: Discharge	75.5	25.4	15.0	154.0	0.11	−0.24
OQ-45.2 SD: Intake	56.4	14.7	15.0	92.0	−0.25	0.11
OQ-45.2 SD: Discharge	43.6	15.6	5.0	91.0	0.13	0.00
OQ-45.2 IR: Intake	22.1	6.8	0.0	39.0	−0.17	−0.08
OQ-45.2 IR: Discharge	18.7	7.2	1.0	37.0	0.03	−0.50
OQ-45.2 SR: Intake	16.0	5.8	2.0	31.0	−0.07	−0.37
OQ-45.2 SR: Discharge	13.2	5.4	1.0	28.0	0.32	−0.51
MH encounters preadmission	41.8	56.4	1.0	420.0	3.02	11.20
MH encounters postdischarge	65.8	75.4	1.0	576.0	2.45	8.51

Note. GCQ = Group Climate Questionnaire-S; OQ-45.2 = Outcome Questionnaire–45.2; SD = Symptom Distress; IR = Interpersonal Relations; SR = Social Role; MH = mental health clinical service line.

Information was extracted from the medical record regarding diagnoses and treatment engagement using a standardized protocol and a modified record review form (Arbisi et al., 2008). For the purpose of analysis, the following clinical diagnoses were aggregated using a similar grouping scheme as represented by the Diagnostic and Statistical Manual of Mental Disorders-5: anxiety disorder, panic disorder, and agoraphobia into anxiety disorder; major depressive disorder, persistent depressive disorder, and unspecified depressive disorder into depressive disorder; full threshold and subthreshold (posttraumatic stress disorder at least clinically significant intrusive reexperiencing or avoidance symptoms) into trauma-related disorder; alcohol use disorder, tobacco use disorder, and other substance use disorder into substance use disorder; Cluster A personality disorders, traits of Cluster A personality disorder, Cluster B personality disorders, traits of cluster B personality disorders, and unspecified personality disorder as personality disorder; Bipolar I and Bipolar II disorders as bipolar disorder; and schizophrenia, schizoaffective disorder, schizotypal personality disorder, and other psychotic disorder as psychotic disorder. Schizotypal personality disorder was only counted as a psychotic disorder and not included in the personality disorder category. Due to the low frequencies at intake, we further combined this grouping with bipolar disorder into a thought disorder category for analysis. Only three individuals had a diagnosis of obsessive-compulsive disorder at intake and this variable was not considered further. In addition, administrative records were accessed for the number of billable outpatient mental health encounters (both phone and in-person) with VA providers in the 12 months following discharge and the 12 months prior to intake into the partial psychiatric hospitalization program.

Analysis

During initial measure development, F-r scale items were selected based on infrequent endorsements within the MMPI-2-RF normative sample (Ben-Porath, 2012). However, this also makes F-r sensitive to pronounced emotional distress and severe psychopathology, which are uncommon in the general population. This method of scale construction has contributed to challenges in interpreting F(-r) in applied clinical settings where severe psychopathology is common and expected. For example, genuine emotional distress measured within full or partial psychiatric hospitalizations often leads to elevations on F-r. Test developers responded by creating a separate overreporting index tailored for severe psychopathology comprised of items rarely endorsed by individuals with severe psychopathology. These items are included on the Fp-r scale (Ben-Porath, 2012).

Test administrators are recommended to consider Fp-r when contextualizing emotional distress detected on other MMPI-2-RF scales such as F-r (Ben-Porath, 2012). To demonstrate this confound of emotional distress inherent with F-r, we computed bivariate Pearson correlations between F-r, Fp-r, L-r, K-r and the Restructured Clinical Demoralization scale (RCd) using the study sample (n = 430 veterans). Associations were substantially stronger between RCd and F-r (r = .744, p < .001) than between RCd and Fp-r (r = .417, p < .001) or RCd and L-r (r = −.313, p < .001). Correlations between RCd and K-r were in an intermediate range (r = −.627, p < .001). Given the nature of our partial hospitalization study sample with the expected high incidence rate of severe psychopathology, we used Fp-r as our primary measure of overreporting to reduce confounds of emotional distress. Thus, the examination of Fp-r represents a more stringent test of potential associations between overreporting and outcomes in this sample.

In the present study, our primary outcome measures included engagement with the partial hospitalization milieu experience (length of treatment, no-shows, times late, and premature withdrawal from programming [yes/no]), GCQ-S self-report of milieu group dynamics, OQ-45.2 self-report of symptomatology at intake and discharge, and number of treatment encounters with mental health providers following discharge. We note that MMPI-2-RF validity scales are not designed to be used clinically to assess the severity of current symptomatology. We planned to examine associations between Fp-r, L-r, K-r, and the OQ-45.2 in part to demonstrate that these validity scales are performing as expected in terms of associations with extra-MMPI-2-RF symptom measures.

Predictors were evaluated for significance using percentile bootstrap methodology with 95% confidence intervals (Mooney et al., 1993; Wood, 2005) and repeated resampling (2,000 times per model; consistent random seed number across all models; SPSS 25). This created nonparametric effect distributions representative of the observed data. To evaluate cross-sectional associations, a series of hierarchical linear and logistic regressions were performed. In Step 1, all binary categorical diagnostic variables were entered simultaneously as predictors. In Step 2, Fp-r, L-r, and K-r raw scores were entered simultaneously as predictors. In this way, we could model the extent to which measures of over- and under-reporting independently predict study measures over and above the clinical impressions provided by referring clinicians at intake. We reported unstandardized (b) and standardized (β) coefficients. Collinearity among the predictors was evaluated using the variance inflation factor for linear regressions; no observed variance inflation factor value was above 1.54. Heteroscedasticity was inspected using the residual plots. Premature withdrawal, a binary variable, was predicted instead using logistic regressions. For the OQ-45.2 and 12-month mental health treatment encounter variables, we also modeled longitudinal change by entering the baseline/intake variables as covariates during Step 1. We characterized the overall magnitude of change across all veterans using repeated measures t tests. Repeated measures effects were evaluated similarly with percentile bootstrap methodology using 95% confidence intervals.

For interested readers, we repeated the main study analyses using F-r as a predictor in the Supplemental Materials. These analyses come with the caveat that F-r elevations in this sample likely reflect both overreporting as well as genuine emotional distress connected with psychopathology. In addition, we repeated the main study analyses in the Supplemental Materials using a categorical overreporting invalidity variable based on F-r = 120 and Fp-r ≥ 100 T score cut scores. This conservative cutoff simulates how these invalidity scales are often used in clinical practice.

Study data and materials are available from the senior author upon request for findings verification.

Results

Milieu Engagement and Experiences

Veterans with a trauma-related disorder, b = −1.95, 95% CI_boot [−3.26, −.61], β = −.151, or a thought disorder, b = −3.14, 95% CI_boot [−5.09, −1.26], β = −.188, experienced shorter treatment durations during their partial psychiatric hospitalization. Similarly, veterans with a trauma-related disorder, b = .796, 95% CI_boot [.118, 1.489], OR = 2.22, or a thought disorder, b = .796, 95% CI_boot [.179, 1.749], OR = 2.53, were more likely to withdraw prematurely from treatment (Table 3). Veterans with an anxiety disorder were more frequently late to partial hospitalization programming, b = .324, 95% CI_boot [.111, .555], β = .169, and had more no-show days, b = .259, 95% CI_boot [.015, 521], β = .113. All other categorical diagnosis confidence intervals included zero when predicting treatment duration, premature withdrawal, number of late arrivals, or number of no-show days. In Step 2, Fp-r predicted being late more frequently to partial hospitalization programming over and above the intake categorical diagnoses, b = .047, 95% CI_boot [.005, 092], β = .123. All other content-based validity scale confidence intervals included zero.

Table 3.

Summary of Main Manuscript Hierarchical Linear and Logistic Regression Results

Dependent variable	Step 1 (categorical Dxs)	Step 2 (validity scales)
Length of treatment (days)	Trauma related (−), thought disorder (−)
Premature withdrawal	Trauma related (+), thought disorder (+)
Times late	Anxiety (+)	Fp-r (+)
No-shows (days)	Anxiety (+)
GCQ-Engagement	Trauma related (+)
GCQ-Conflict	Depression (+)
GCQ-Avoidance
OQ-45.2 total: Intake	Trauma related (+), depression (+)	Fp-r (+), K-r (−)
OQ-45.2 total: Discharge		K-r (−)
OQ-45.2 total: Longitudinal change
OQ-45.2 SD: Intake	Trauma related (+), depression (+)	Fp-r (+), K-r (−)
OQ-45.2 SD: Discharge		K-r (−)
OQ-45.2 SD: Longitudinal change
OQ-45.2 IR: Intake	Depression (+)	L-r (−), K-r (−)
OQ-45.2 IR: Discharge		K-r (−)
OQ-45.2 IR: Longitudinal change
OQ-45.2 SR: Intake		K-r (−)
OQ-45.2 SR: Discharge		K-r (−)
OQ-45.2 SR: Longitudinal change
MH encounters postdischarge	Thought disorder (+), substance use (+)	Fp-r (+)
MH encounters longitudinal change	Thought disorder (+)	Fp-r (+)

Note. Direction of the observed significant effects (CI_boot 95%) is noted in parentheses. Dxs = diagnoses; OQ-45.2 = Outcome Questionnaire–45.2; SD = Symptom Distress; IR = Interpersonal Relations; SR = Social Role; GCQ = Group Climate Questionnaire-S; MH = mental health clinical service line; Fp-r = Infrequent Psychopathology Responses scale; L-r = Uncommon Virtues scale; K-r = Adjustment Validity scale; CI = confidence interval.

Veterans with a trauma-related disorder reported experiencing a more positive milieu working atmosphere on GCQ-Engage, b = .232, 95% CI_boot [.024, .446], β = .120. Veterans with a depressive disorder described elevated tension and anger within the milieu on GCQ-Conflict, b = .243, 95% CI_boot [.033, .472], β = .132 (Table 3). All other categorical diagnosis confidence intervals included zero in Step 1, and all content-based validity scale confidence intervals included zero in Step 2 when predicting GCQ scores.

Cross-Sectional Symptom Presentations and Treatment Responses

With respect to longitudinal changes in functioning (i.e., intake to discharge), veterans reported large relative improvements in overall functioning based on the OQ-45.2 total score, M_difference = 19.45, 95% CI_boot [17.02, 21.96], Cohen’s d = .91 (Table 3). The improvements were most pronounced within the OQ-45 subdomain of Symptom Distress, M_difference = 12.93, 95% CI_boot [11.37, 14.54], Cohen’s d = .95. Improvements within the subdomains of Interpersonal Relations, M_difference = 3.66, 95% CI_boot [2.95, 4.36], Cohen’s d = .59, and Social Role, M_difference = 2.86, 95% CI_boot [2.30, 3.46], Cohen’s d = .57, were medium in magnitude.

At intake, a trauma-related disorder predicted worse OQ-45.2 total, b = 6.65, 95% CI_boot [1.05, 12.48], β = .128, and Symptom Distress, b = 5.09, 95% CI_boot [1.95, 8.58], β = .155, cross-sectional scores (Table 3). Veterans with a depressive disorder endorsed worse OQ-45.2 total, b = 6.09, 95% CI_boot [.35, 11.99], β = .120, Symptom Distress, b = 3.99, 95% CI_boot [.34, 7.84], β = .125, and Interpersonal Relations, b = 1.83, 95% CI_boot [.31, 3.39], β = .125, scores at intake as well. All other categorical diagnosis confidence intervals included zero when predicting OQ-45.2 functioning at intake. In Step 2, high Fp-r scores independently predicted worse OQ-45.2 total, b = 1.52, 95% CI_boot [.54, 2.61], β = .152, and Symptom Distress scores during intake, b = 1.07, 95% CI_boot [.45, 1.72], β = .169. Furthermore, low K-r scores independently predicted worse OQ-45.2 total, b = −3.06, 95% CI_boot [−3.88, −2.18], β = −.344, Symptom Distress, b = −1.88, 95% CI_boot [−2.42, −1.34], β = −.336, Interpersonal Relations, b = −.633, 95% CI_boot [−.893, −.346], β = −.246, and Social Role scores at intake, b = −.545, 95% CI_boot [−.785, −.312], β = −.248. Finally, low L-r scores independently predicted worse OQ-45.2 Interpersonal Relations scores at intake, b = −.429, 95% CI_boot [−.831, −.052], β = −.127. All other content-based validity scale confidence intervals included zero in Step 2.

At discharge, all categorical diagnosis confidence intervals included zero when predicting current cross-sectional OQ-45.2 functioning (Table 3). In Step 2, low K-r independently predicted worse OQ-45.2 total, b = −2.50, 95% CI_boot [−3.74, −1.27], β = −.262, Symptom Distress, b = −1.64, 95% CI_boot [−2.40, −.88], β = −.280, Interpersonal Relations, b = −.526, 95% CI_boot [−.856, −.160], β = −.195, and Social Role scores at discharge, b = −.341, 95% CI_boot [−.625, −.074], β = −.167. All other content-based validity scale confidence intervals included zero in Step 2.

Using linear regressions with baseline/intake scores as covariates, all categorical diagnosis confidence intervals included zero when predicting longitudinal change in OQ-45.2 functioning in Step 1 (Table 3). In Step 2, all content-based validity scale confidence intervals also included zero. Thus, no study predictors were associated with longitudinal changes in self-presentations of functioning difficulties.

Postpartial Hospitalization Treatment Engagement

In the 12 months prior to enrollment in the partial psychiatric hospitalization program, study veterans experienced on average 41.8 (SD = 56.4) documented encounters with VA mental health providers. During the subsequent 12-month period following discharge, veterans experienced an increase in documented VA mental health encounters (M = 65.6, SD = 75.3; M_difference = −23.7, 95% CI_boot [−30.7, −16.9]). This increase, although clinically meaningful, was small in statistical effect size (Cohen’s d = .32).

Veterans with diagnoses of thought disorder, b = 38.52, 95% CI_boot [6.88, 74.97], β = .177, and substance use disorder, b = 21.24, 95% CI_boot [5.97, 35.54], β = .141, experienced more frequent VA mental health encounters following discharge (Table 3). All other categorical diagnosis confidence intervals included zero. In Step 2, higher Fp-r scores independently predicted more frequent mental health encounters following discharge, b = 3.40, 95% CI_boot [.44, 6.53], β = .106. All other content-based validity scale confidence intervals included zero in Step 2.

When longitudinal change was modeled, veterans with thought disorder diagnoses at intake experienced an increase in relative mental health encounters following discharge, b = 38.76, 95% CI_boot [7.50, 76.20], β = .178 (Table 3). All other categorical diagnosis confidence intervals contained zero for predicting changes in mental health encounter frequency. In Step 2, higher Fp-r scores independently predicted a longitudinal increase in mental health encounters, b = 3.33, 95% CI_boot [.53, 6.21], β = .103. All other content-based validity scale confidence intervals included zero in Step 2.

Discussion

Among a cohort of veterans participating in partial psychiatric hospitalizations, categorical Diagnostic and Statistical Manual of Mental Disorders-5 diagnoses at intake predicted a number of variables related to treatment experience and engagement: trauma-related disorders predicted shorter partial hospitalizations, premature withdrawal from treatment, more positive perceptions of the group milieu experience, and worse personal functioning; depressive disorders predicted greater perceived conflict within the milieu and worse functioning; anxiety disorders predicted being late more often for treatment programming and increased number of no-show days; substance use disorders predicted more frequent VA mental health encounters following discharge; and thought disorders predicted shorter partial hospitalizations, premature withdrawal from treatment, and longitudinal increases in VA mental health encounters following discharge. When entered simultaneously as predictors during Step 2 of the regression models, MMPI-2-RF content-based validity scales were also associated with several key treatment variables. Elevated endorsements on Fp-r and fewer endorsements on K-r predicted worse self-presentations of functioning on nearly all OQ-45.2 measures independent of the categorical diagnoses. In addition, higher Fp-r scores predicted more late arrivals for treatment programming and longitudinal increases in VA mental health encounters following discharge. Thus, content-based validity scales on the MMPI-2-RF added unique information for predicting patient engagement during and after the partial psychiatric hospitalizations over and above the diagnostic clinical assessments offered by referring providers.

Across all study veterans, the increases in mental health care utilization 12 months postdischarge are consistent with the intended purpose of partial hospitalizations: to intervene during an emerging crisis and facilitate longer term access to outpatient services. The association between more mental health encounters and thought and substance use disorders may indicate a greater clinical need among these individuals (Fasoli et al., 2010). Endorsements on Fp-r, items not frequently endorsed in acute psychiatric care settings, also predicted longitudinal increases in mental health encounters independent of intake diagnoses. Furthermore, Fp-r endorsements were associated with late arrivals to treatment programming. In other studies, Fp-r has the largest effect sizes in differentiating between individuals instructed to feign psychiatric conditions and psychiatric patients responding accurately to the MMPI-2-RF (Ingram & Ternes, 2016; Sharf et al., 2017). As such, the association between higher scores on Fp-r and increased posttreatment encounters could indicate mixed motivation about the recovery process or personal barriers toward full utilization of clinical services. Horner et al. (2014) reported similar findings among veterans participating in routine neuropsychological evaluations. Veterans who demonstrated evidence of performance-based invalidity (i.e., effort testing) experienced more emergency department visits as well as more frequent and longer durations of inpatient hospitalizations in the 12 months following assessment. Taken together, there are at least two possibilities: (a) invalid responding during both formal and informal evaluations obscures the diagnostic picture, which leads to less targeted treatments and thereby less robust long-term intervention responses, and/or (b) ambivalent engagement with the assessment process predicts a broader pattern of reduced collaborative engagement with treatment providers and the broader mental health care system. Fp-r appears to add value beyond clinical diagnosis in identifying individuals who may utilize specialty care in greater amounts after completing a partial psychiatric hospitalization program.

Both Fp-r and K-r predicted self-report of impaired functioning on the OQ-45.2 even after covarying for intake diagnoses. K-r was the strongest (negative) predictor of self-reported dysfunction at intake based on effect size. Also, K-r independently predicted discharge scores whereas Fp-r did not. This scale is sensitive to defensiveness and underreporting as well as unexpectedly high levels of emotional adjustment in nonclinical setting (Ben-Porath, 2012). It is noteworthy that no current interpretive guidelines exist for the disavowal of emotional adjustment based on low K-r scores in spite of a T-score measurement range that extends below 30 (Ben-Porath, 2012). The associations between Fp-r and K-r scores and worse functioning during partial hospitalization calls to mind the F-K index from the original MMPI and MMPI-2 instruments (i.e., Gough, 1950; “dissimulation index”), which was used to identify “fake bad” and “fake good” profiles. In some studies, the F-K index demonstrated equivalent or marginally superior performance in detecting feigned psychiatric conditions such as posttraumatic stress disorder than the Fp scale alone (Nijdam-Jones et al., 2020). Future studies could explore whether the item pools from Fp-r and K-r on the MMPI-2-RF can be combined to increase measurement precision in the prediction of overreporting. It is unknown whether the combined item set adds incrementally to the scales from which the items were drawn. We also observed only a single independent association between L-r and self-report of current functioning on Interpersonal Relations, which further highlights some of the unique predictive properties of Fp-r and K-r together for this population. L-r may not have sufficient measurement resolution at the low end to meaningfully detect underendorsements similarly as K-r.

Neither intake clinical diagnosis nor content-based validity scores (Fp-r, K-r, L-r) predicted longitudinal changes in self-reported OQ-45.2 functioning following completion of the partial hospitalization treatment. Therefore, veterans experienced similar short-term benefits from participating in the group therapy milieu environment, which occurred regardless of their primary clinical complaints or hospitalization-relevant symptom validity biases. However, the study was limited by a lack of a clinical control group. The comparability of these partial hospitalization effects relative to a waitlist cohort without intensive clinical services is unknown. The study also lacked follow-up measurements of functioning using the OQ-45.2 in the months and years following discharge from the partial program. It is possible that these medium-to-large changes in distress and role functioning are differentially maintained depending on the primary clinical complaints or overreporting patterns. Extended longitudinal follow-ups would help clarify whether associations between Fp-r and treatment engagement are also associated with sustained differences in functioning and symptomatology within outpatient settings after discharge.

More generally speaking, this observational study offers a window into the consequences of overreporting in specialty clinics and hospital systems. Overreporting can have pervasive influences on many aspects of patient–provider interactions. Indeed, the largest statistical effect sizes in this study were for the validity measures and not for clinical diagnoses at intake. Routine consideration of content-based validity scales in case conceptualizations can add insight for treatment planning when psychosocial issues are intersecting with psychopathology in complex ways. Clinical indications of content-based invalidity should elicit an appreciation for the potential interpersonal and systemic reinforcement structures currently in place acting as barriers for full engagement with health care (Finn, 2007). This could include direct intervention to address the content-based invalidity. There is a need for controlled trials exploring the efficacy of various intervention feedback strategies based on psychological testing data (e.g., Finn & Tonsager, 1992), especially with respect to addressing overreporting and under-reporting in clinical settings (Potik et al., 2012). One promising approach includes the use of therapeutic assessment techniques to help patients conceptualize their reporting styles in the broader context of their overall clinical complaints, distress, personal motivations, and history of interactions with both providers and other persons (Finn, 2007). The observed associations for over-reporting in the present study heighten the urgency for evidenced-based intervention protocols to address reporting biases in clinical settings.

There are several limitations. Since the completion of data collection for this study, the updated MMPI-3 instrument was released. The MMPI-3 features some item updates to the validity scales, but correlations with the validity scales from the MMPI-2-RF among clinical and forensic samples are high (r ≥ 84 for Fp[-r] and r ≥ .87 for K[-r]; Ben-Porath & Tellegen, 2020). We would expect similar associations if our study veterans had simultaneously completed the MMPI-3 with the MMPI-2-RF, but we lack empirical results to support this. The study was not preregistered; although we predicted associations for Fp-r across multiple outcome measures, the analyses using multiple outcome domains were exploratory in nature. Several subtle effects including Fp-r scores predicting more late arrivals for treatment programming and longitudinal increases in VA mental health encounters following discharge would not survive strict corrections for multiple comparisons across all enacted study models. Validity scale scores do not speak to the exact motivations behind those endorsements, which could span a wide gamut of extrinsic and intrinsic domains (Dandachi-FitzGerald et al., 2022). Researchers are increasingly recognizing additional possibilities beyond overt malingering as motivational factors. This could include an emotionally driven “cry for help” involving indiscriminate endorsement of psychopathology (Young, 2019). There is a need for additional research on not only the consequences of overreporting using validity scales but also individual differences in the motivational states driving patients to engage in overreporting on validity scales. Finally, we also acknowledge the value of study replications within other VA and non-VA settings. This is important given the clinical implications of potentially using validity scales in unconventional ways to predict treatment engagement.

In conclusion, partial psychiatric hospitalizations remain an important tool for providing more intensive care to individuals in acute crises. Reducing barriers to full participation in this type of treatment is of vital importance for reducing human suffering. For the current investigation, we conducted a longitudinal observational study of veterans and observed multiple associations with intake clinical diagnoses and partial hospitalization treatment variables. Furthermore, indicators of overreporting on the MMPI-2-RF predicted relevant treatment measures over and above the categorical diagnoses offered by referring clinicians. Rather than being binary indicators of test interpretability only, content-based validity scales may have utility for understanding the ways patients present themselves when interfacing with clinical providers. Content-based validity indicators (i.e., high Fp-r, low K-r) were uniquely associated with current and future engagement with mental health care services and maladaptive self-presentations of symptoms beyond what could be explained by pretreatment diagnostic impressions. As such, clinicians should be increasingly encouraged to consider content-based validity scales as an additional source of information in support of clinical decision making and case conceptualizations.

Public Significance Statement.

The present study examined changes in functioning, individual experiences, and behaviors reported by veterans participating in partial psychiatric hospitalizations. MMPI-2-RF validity scales uniquely predicted treatment engagement and symptom endorsement, both during the partial hospitalizations and in the 12 months postdischarge. There is an added benefit in using measures of overreporting to understand patient behavior and outcomes in clinical settings.