Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2022 Apr 1;17(4):e0266352. doi: 10.1371/journal.pone.0266352

The relationship between psychological characteristics of patients and their utilization of psychiatric inpatient treatment: A cross-sectional study, using machine learning

Sou Bouy Lo 1,2, Christian G Huber 1, Andrea Meyer 2, Stefan Weinmann 1,3, Regula Luethi 1, Frieder Dechent 1, Stefan Borgwardt 1,4, Roselind Lieb 2, Undine E Lang 1, Julian Moeller 1,2,¤,*
Editor: Giuseppe Carrà5
PMCID: PMC8975161  PMID: 35363806

Abstract

High utilizers (HU) are patients with an above-average use of psychiatric inpatient treatment. A precise characterization of this patient group is important when tailoring specific treatment approaches for them. While the current literature reports evidence of sociodemographic, and socio-clinical characteristics of HU, knowledge regarding their psychological characteristics is sparse. This study aimed to investigate the association between patients’ psychological characteristics and their utilization of psychiatric inpatient treatment. Patients from the University Psychiatric Clinics (UPK) Basel diagnosed with schizophrenia spectrum or bipolar affective disorders participated in a survey at the end of their inpatient treatment stay. The survey included assessments of psychological characteristics such as quality of life, self-esteem, self-stigma, subjective experience and meaning of psychoses, insight into the disease, and patients’ utilization of psychiatric inpatient treatment in the last 30 months. The outcome variables were two indicators of utilization of psychiatric inpatient treatment, viz. “utilization pattern” (defined as HU vs. Non-HU [NHU]) and “length of stay” (number of inpatient treatment days in the last 30 months). Statistical analyses included multiple regression models, the least absolute shrinkage and selection operator (lasso) method, and the random forest model. We included 112 inpatients, of which 50 were classified as HU and 62 as NHU. The low performance of all statistical models used after cross-validation suggests that none of the estimated psychological variables showed predictive accuracy and hence clinical relevance regarding these two outcomes. Results indicate no link between psychological characteristics and inpatient treatment utilization in patients diagnosed with schizophrenia spectrum or bipolar affective disorders. Thus, in this study, the examined psychological variables do not seem to play an important role in patients’ use of psychiatric inpatient treatment; this highlights the need for additional research to further examine underlying mechanisms of high utilization of psychiatric inpatient treatment.

Introduction

High utilizers (HU), also known as heavy or frequent users or repeaters, are a group of patients with an above-average use of psychiatric inpatient treatment. Although only approximately 10% to 30% of all inpatients are HU, these clients’ use of the healthcare system’s resources is well above average [1–3]. However, they seldom receive treatment tailored to their specific characteristics [4].

Prior studies have investigated the high utilization phenomenon to identify characteristics associated with high utilization. Identifying robust characteristics of HU patients is important when tailoring specific treatment approaches for them, to optimize treatment and improve their health and wellbeing [5, 6].

To investigate the high utilization phenomenon, prior studies focused on sociodemographic and socio-clinical patient characteristics and used variables such as length of hospital stay in days, and/or number of hospital admissions within a specific time window as indicators of patients’ utilization of psychiatric inpatient treatment. Some of these studies used a cutoff value for these variables to classify patients into HU and Non-High Utilizer (NHU) [7–10]. Studies showed that patients with a high utilization pattern are younger [8, 11–14] than those with low utilization of psychiatric inpatient treatment. Contradictory results were found regarding the sex of HU and NHU patients. Several studies reported a higher prevalence of men among inpatients with high utilization [2, 15, 16], whereas other studies showed a predominance of women [1, 8, 11, 17, 18].

HU have more often been diagnosed with schizophrenia, schizoaffective disorders, and bipolar affective disorders compared with NHU, indicating that the high utilization phenomenon may be specifically relevant to this patient group [3–5, 11–13, 19–22]. Moreover, HU patients showed higher comorbidity rates with personality disorders and substance abuse disorders than NHU patients [2, 3, 6, 9, 10, 23, 24]. However, one study found no differences in diagnoses between HU and NHU inpatients [25].

Homelessness was also associated with high utilization [9, 14, 22, 24]; moreover, a study reported that 68% of the HU patients had problems with living conditions [26]. The lack of an alternative source of psychosocial care such as access to sheltered housing for HU has been reported previously [3, 10, 27]. The majority of all HU patients reported difficulties in their social/interpersonal relationships [14, 18, 23, 24, 26, 28]. Patients with high utilization reported only four contacts in their social networks on an average, which often consisted of family members and medical staff [18, 26]. Furthermore, HU patients showed higher pharmacological non-adherence, more suicidal tendencies [2, 10, 11, 29], higher personal burden [6] and greater severity of psychopathology [13, 25, 28] compared with NHU inpatients. Moreover, poor insight in psychosis was associated with higher utilization of psychiatric inpatient treatment [30, 31].

While the current literature reports evidence on the demographic and socio-clinical characteristics of high utilization described above, the role of psychological characteristics of patients resulting in high utilization remains unclear. For instance, associations between patients’ psychological characteristics and their utilization of psychiatric inpatient treatment have rarely been examined. This is a research gap as the inclusion of further patient characteristics in high utilizer research may increase the understanding of the underlying mechanisms of high utilization in the context of psychiatric inpatient treatment, potentially informing clinical interventions in the future.

In this context, quality of life, self-esteem, self-stigma, subjective experience and meaning of psychoses, and insight into the disease are promising candidates to examine associations with utilization of psychiatric inpatient treatment in patients with schizophrenia spectrum or bipolar affective disorders because they varied depending on the clinical course of these mental disorders in prior studies [32–37]. High severity levels of a mental disorder can generally be considered as an indicator of high utilization [13, 25, 28]. Therefore, we hypothesized an association between the mentioned psychological characteristics and patients’ use of psychiatric inpatient treatment, and investigated it through an exploratory study.

Aim of study

This study aimed to examine associations between psychological characteristics and utilization of psychiatric inpatient treatment in patients with schizophrenia spectrum or bipolar affective disorders from the UPK Basel. Psychological characteristics included measures of quality of life, self-esteem, self-stigma, subjective experience and meaning of psychoses, and insight into the disease.

Materials and methods

Ethical approval

The Ethics Committee Northwest and Central Switzerland (EKNZ) (EKNZ BASEC 2017–02203) provided ethical approval for this study. All participants gave written informed consent after receiving information on the study. Protocols of the present study adhere to the principles expressed in the Declaration of Helsinki regarding research with human participants.

General framework

Data were obtained from the research project “Characteristics of High Utilization in the Psychiatric University Hospital (UPK) Basel,” conducted from April 2018 to May 2019 at the UPK Basel in Switzerland. The UPK provides psychiatric treatment for inpatients and outpatients, covering a catchment area of approximately 190,000 people in and around the city of Basel.

Study population and procedure

We recruited patients from three psychiatric-inpatient wards belonging to the Center for Psychotic Disorders (ZPE) of the Department of Adult Psychiatry UPK Basel. The three wards have an inpatient treatment capacity of 54 beds, and provide diagnosis-specific psychiatric and psychotherapeutic treatment for patients with schizophrenia spectrum disorder (ICD-10: F20-F29) or bipolar affective disorder (ICD-10: F31.1, F31.2). Admission to the ZPE may either be voluntary or involuntary. In the corresponding study period, a trained psychologist responsible for the study’s recruitment assessed all patients admitted to one of the three study wards regarding inclusion and exclusion criteria. Inclusion criteria included good knowledge of the German language, an age range between 18 and 65 years, and a diagnosis of schizophrenia spectrum disorder or bipolar affective disorder. Patients who were unable to consent or those with criminal convictions, dementia, psychiatric diseases resulting from organic causes, or a primary diagnosis of substance abuse disorder were excluded. Subsequently, we asked eligible patients at the end of their inpatient treatment stay, i.e., when the psychiatrist in charge of the case announces that discharge management will be initiated for the respective patient, to participate in this study. After receiving information about the project, the participants signed the informed consent and completed the assessment within an average duration of 50 minutes. The psychologist responsible for the study was present during the entire assessment and aided the patients with queries related to the paper-and-pencil questionnaires if necessary. Patients admitted several times to the ZPE during the study period could only participate once in this study.

Assessments

Assessments included five paper-and-pencil questionnaires on patient psychological characteristics and interviews on utilization of psychiatric inpatient treatment, psychopathology, and sociodemographic information.

Quality of life

The World Health Organization Quality of Life (WHOQOL)-BREF measures quality of life through self-report, and includes four domains: physical health, psychological health, social relationships, and environment. All items were rated on a five-point Likert scale. Domain scores were converted to a 0–100 spectrum, with higher scores indicating a higher quality of life. The WHOQOL-BREF is a validated and well-established questionnaire. Cronbach’s alpha for the German translation indicated good internal consistency within the range of 0.76 to 0.88 [38, 39].

Self-esteem

The Rosenberg Self-Esteem Scale (RSE) assesses self-esteem with 10 items on a five-point Likert scale (from 1 = strongly disagree to 5 = strongly agree) [40]. A higher rating indicates a higher level of self-esteem. The total score was calculated for all items. The German adaptation of the self-report questionnaire revealed satisfactory psychometric properties, while the Cronbach’s alpha was 0.88 [41].

Self-stigma

To detect the reflection of the patients’ internalized self-stigma, the Self-Stigma of Mental Illness Scale-Short Form (SSMIS-SF) was used [42]. The short-form of the self-report questionnaire assesses four constructs rated on a nine-point Likert scale: awareness, agreement, application and hurts-self. Awareness measures patients’ consciousness about common stereotypes about people with mental illnesses. Agreement assesses the degree of belief that the stereotypes are accurate. Application indicates the extent to which patients internalize the stereotypes and apply these to themselves. Consequently, the degree of their decreased self-esteem and self-efficacy is assessed by hurts-self. The constructs were formed by summing up the corresponding items. The English original SSMIS-SF provides good internal consistency ranging between 0.69 and 0.87 [43].

Subjective experience and meaning of psychoses

Aspects of meaning and experiencing the psychosis were measured using self-report with the Subjective Sense in Psychosis questionnaire (SUSE). The questionnaire was adapted using the framework of a trialogue research project [44, 45]. The shortened version consists of 29 items, through five subscales, measuring the past (biographical integration), positive and negative present experiences (symptoms positive and symptoms negative), and positive and negative expectations of the future (positive consequences and negative consequences). The answers are rated on a four-point Likert scale. Cronbach’s alpha of the English original questionnaire demonstrated good internal consistency of 0.73 to 0.86. The test-retest reliability ranged from 0.66 to 0.84 [46, 47].

Insight

Insight into the disease was assessed using Becks Cognitive Insight Scale (BCIS) [48]. This self-report questionnaire assesses patients’ cognitive processes and limitations. The sum of nine items captures the subscale self-reflectiveness and six items represent self-certainty. Answers were rated on a four-point Likert scale (from 0 = strongly not agree to 3 = strongly agree). The English original scale provides good internal consistency, with a Cronbach’s alpha level of 0.60 for self-certainty and 0.68 for self-reflectiveness. The test-retest reliability showed an adequate correlation of 0.60 [49].

Utilization of psychiatric inpatient treatment

We assessed retrospective information regarding patients’ mental health care utilization during the last 30 months before the current admission using the Client Sociodemographic and Service Receipt Inventory (CSSRI; [50, 51]). To improve the data quality, inpatient treatment information was cross-checked with electronic records of our patients. We used two different indicators to operationalize patients’ mental health care utilization pattern. First, we calculated the dichotomous variable “utilization pattern” by classifying patients into a HU group (HU; ≥180 inpatient treatment days during 30 months and/or ≥3 number of admissions over 18 months) or a NHU group (NHU; <180 inpatient treatment days during the 30 months and/or <3 number of admissions over 18 months). Second, we calculated the continuous variable “length of stay,” which indicates the number of inpatient treatment days in the last 30 months for each patient. We used a dichotomous and a continuous indicator of patients’ inpatient utilization pattern as preliminary literature revealed no consistent standard in this regard [7–10].

Psychopathology

We assessed the severity of patients’ psychopathology using the German form of the Brief Psychiatric Rating Scale (BPRS). The assessment consists of 18 items, which capture different symptoms of psychopathology (e.g., somatic complaints, feelings of guilt, depressed mood, hostility, and grandiosity), and was rated by the attending psychiatrist. The items were evaluated on a seven-point Likert scale (from 1 = not present to 7 = extremely severe). The sum score consists of all items; higher scores indicate a higher degree of psychopathology. The BPRS is one of the most widely used and validated scales in clinical research and provides good psychometric properties. The retest reliability was approximately 0.7, the internal consistency ranged between 0.75 and 0.79, and the inter-rater reliability ranged between 0.56 and 0.87 [52, 53].

Sociodemographic information

We obtained standard sociodemographic information at the beginning of the assessment.

Statistical analyses

To report descriptive statistics, we chose the first quartile, median, and third quartile for continuous variables as well as absolute numbers and percentages for discrete variables.

In the first set of analysis, we estimated the association between psychological characteristics of the patients and their utilization of psychiatric inpatient treatment through two separate multiple logistic and linear regression models. The dichotomous variable “utilization pattern” (HU vs. NHU) and the continuous variable “length of stay” (number of inpatient treatment days in the last 30 months before the current stay) were outcome variables. Physical health, psychological health, social relationships, and environment (four subscales of the quality of life questionnaire; [38, 39]); self-esteem [40], awareness, agreement, application, and hurts-self (five subscales of the self-stigma questionnaire; [42]); biographical integration, symptoms positive, symptoms negative, positive consequences, and negative consequences (five subscales of the subjective sense in psychosis questionnaire; [44, 45]); as well as self-reflectiveness and self-certainty (subscales of the insight into the disease questionnaire; [48]) were the predictors in both regression models. To reduce risk of confounding, we adjusted these analyses for several a priori defined potential confounders including sex [2, 8, 11, 18], age [2, 11, 13, 14], and psychopathology [13, 25, 28]. Multicollinearity among the predictors and covariates was moderate, with variance inflation factors ranging between 1.3 and 5.2.

Predictive accuracy is the ability of a statistical model to fit well not only with the (training) data it was developed with but also with corresponding new, previously unseen (test) data. Multiple regression models often suffer from overfitting, and therefore have low predictive accuracy [54], especially if the number of predictors is high relative to the number of participants (linear model) or the number of cases (logistic model), as is the case in our study. To avoid model overfit, we ran two additional models that use machine learning algorithms—the least absolute shrinkage and selection operator (lasso) model, and the random forest model. Both models contained the same predictors, outcomes, and adjustments described above. The lasso model uses a variable selection procedure, in which regression coefficients are deliberately shrunk by implying a penalty term to the likelihood function when fitting the model. Consequently, the lasso models are somewhat more biased than those obtained from multiple regression models but less variable, i.e., they exhibit increased predictive accuracy [55]. Thus, when replicating the current study, predictors whose coefficients have not been shrunk to zero from the lasso are likely to be predictive. The random forest model belongs to the family of decision tree models, which characteristically generate a large number of bootstrapped decision trees, based on random samples of variables. The model has been shown to exhibit high predictive accuracy [56]. Since our two outcome variables were continuous (length of stay) or dichotomous (HU vs. NHU), both the lasso and random forest models were based on a multiple logistic and linear regression model, respectively. To determine the predictive accuracy of the models, we conducted repeated (n = 10) 10-fold cross-validation. We used the root mean square error (RMSE, i.e. the square root of the variance of the residuals) and the R-squared (R2) as the measures of model accuracy for continuous and dichotomous outcomes, respectively.

The outcome–“length of stay”–and the predictors–“application” (self-stigma questionnaire), “hurts-self” (self-stigma questionnaire), and “negative consequences” (subjective sense in psychosis questionnaire)–were all log transformed to approximate normality and homoscedasticity.

All models were run with R, version 3.3.0, including the R package glmnet for the lasso model [57]. A significance threshold of p<0.05 was set for all univariate analyses.

Results

After screening 533 patients admitted to the ZPE over the recruitment period, we excluded 421 patients as they did not meet inclusion criteria, declined to participate, or because participation was not indicated or feasible. Finally, 112 patients participated and were included in the analyses (see Flowchart Fig 1).

Fig 1. Flowchart of study participants.

Fig 1

Open in a new tab

Participants spent a median of 71 days, on an average, in psychiatric inpatient care over the past 30 months before the current admission. We classified 50 of them as HU and 62 as NHU. Demographic and clinical characteristics of participants are summarized in Table 1.

Table 1. Demographic, clinical, and psychological characteristics of High Utilizer and Non-High Utilizer patients and in the total sample a.

Characteristic Utilization Total
High Utilizer (n = 50) Non-High Utilizer (n = 62) (n = 112)
Sexb
    Female 22.00 (44.00%) 28.00 (45.16%) 50.00 (44.64%)
    Male 28.00 (56.00%) 34.00 (54.84%) 62.00 (55.36%)
Age 42.00 (32.75; 50.25) 38.00 (28.00; 50.25) 40.00 (30.00; 50.00)
Length of stay in days in the last 30 months 144.50 (94.50; 224.75) 35.50 (16.75; 63.25) 71.50 (29.00; 145.25)
Primary diagnosis
    Bipolar disorder 12.00 (24.00%) 14.00 (22.58%) 26.00 (23.21%)
    Schizophrenia spectrum disorder 38.00 (76.00%) 48.00 (77.42%) 86.00 (76.79%)
Psychopathology 44.50 (36.00; 50.00) 40.00 (33.75; 47.00) 43.00 (35.00; 48.00)
Quality of life
    Physical health 71.43 (56.25; 78.57) 67.86 (55.36; 75.00) 67.86 (57.14; 78.57)
    Psychological health 68.75 (53.13; 83.33) 66.67 (54.17; 75.00) 66.67 (54.17; 79.17)
    Social relationships 66.67 (50.00; 75.00) 66.67 (50.00; 83.33) 66.67 (50.00; 75.00)
    Environment 68.75 (56.25; 78.13) 71.88 (59.38; 81.25) 68.75 (56.25; 81.25)
Self-esteem 37.00 (31.00; 44.25) 38.00 (34.50; 45.00) 38.00 (33.00; 45.00)
Self-stigma
    Awareness 30.00 (24.00; 37.25) 28.00 (23.00; 33.00) 29.00 (23.00; 35.00)
    Agreement 18.50 (11.50; 23.00) 17.00 (11.50; 22.00) 17.00 (12.00; 23.00)
    Apply 12.00 (5.00; 18.25) 11.00 (8.00; 16.50) 11.00 (7.00; 17.00)
    Hurts-self 9.00 (5.00; 20.00) 7.00 (5.00; 13.00) 8.00 (5.00; 15.00)
Subjective experience and meaning of psychoses
    Biographical integration 3.40 (2.60; 3.85) 3.20 (2.60; 3.80) 3.40 (2.60; 3.80)
    Symptoms positive 3.00 (1.75; 3.43) 2.75 (2.00; 3.37) 2.80 (1.80; 3.40)
    Symptoms negative 2.79 (1.97; 3.50) 2.75 (2.06; 3.35) 2.75 (2.00; 3.38)
    Positive consequences 3.40 (2.35; 3.80) 3.40 (2.90; 3.90) 3.40 (2.80; 3.80)
    Negative consequences 1.83 (1.50; 2.38) 1.83 (1.50; 2.42) 1.83 (1.50; 2.37)
Insight into the disease
    Self-reflectiveness 15.00 (11.00; 18.75) 15.00 (11.00; 19.00) 15.00 (11.00; 19.00)
    Self-certainty 9.50 (7.25; 12.00) 9.00 (7.00; 11.00) 9.00 (7.00; 11.00)
Open in a new tab

Note. Values based on row data.

a If not otherwise specified, median (25th percentile; 75th percentile) is reported

b Number (percent) is reported

See the correlation matrix of all relevant variables presented in S1 Table.

Multiple regression models

Coefficients based on the multiple logistic and the multiple linear regression models are depicted in Table 2. The multiple logistic regression model reported a significant association between “utilization pattern” (HU vs. NHU) and psychological health (quality of life), awareness (self-stigma), and hurts self (self-stigma) in our sample. HU patients reported higher psychological health, awareness and hurts-self compared with NHU patients. Model performance for the multiple logistic regression model based on the area under curve (AUC) was 0.79 and strongly decreased to 0.59 after cross-validation (an AUC of 0.50 denotes no predictive accuracy at all, i.e., pure guessing).

Table 2. Estimated independent effects of psychological characteristics.

Variable Utilization pattern (High Utilizer vs. Non-High-Utilizer) Length of stay (number of inpatient treatment days)
Multiple logistic regression model Lasso RF Multiple linear regression model Lasso RF
β Std. Error z-value p β VIV β Std. Error t-value p β VIV
(n = 112) (n = 112)
Sex -0.27 0.27 -1.00 0.32 -0.27 0.00 -0.10 0.11 -0.91 0.37 -0.09 0.00
Age 0.26 0.27 0.99 0.32 0.26 14.96 0.01 0.11 0.11 0.91 0.03 20.49
Psychopathology 0.58 0.27 2.19 0.03 * 0.57 70.78 0.17 0.11 1.55 0.13 0.19 82.59
Quality of life
    Physical health 0.20 0.34 0.60 0.55 0.20 43.41 -0.08 0.14 -0.55 0.58 0.00 58.90
    Psychological health 1.26 0.52 2.41 0.02 * 1.23 67.62 0.45 0.21 2.11 0.04 * 0.00 84.41
    Social relationships -0.12 0.33 -0.37 0.71 -0.12 26.83 -0.10 0.14 -0.76 0.45 0.00 62.58
    Environment -0.12 0.30 -0.41 0.68 -0.12 51.35 -0.05 0.13 -0.44 0.66 0.00 69.77
Self-esteem -0.77 0.45 -1.71 0.09 -0.76 58.73 -0.21 0.18 -1.13 0.26 0.00 75.13
Self-stigma
    Awareness 0.53 0.27 1.99 0.05 * 0.52 100.00 -0.03 0.11 -0.26 0.80 0.00 100.00
    Agreement -0.23 0.27 -0.85 0.40 -0.23 46.95 0.13 0.11 1.15 0.25 0.00 89.68
    Application -0.62 0.38 -1.62 0.11 -0.60 63.57 -0.25 0.15 -1.73 0.09 0.00 77.84
    Hurts-self 1.05 0.44 2.40 0.02 * 1.03 93.17 0.44 0.17 2.57 0.01 * 0.00 77.35
Subjective experience and meaning of psychoses
    Biographical integration 0.23 0.26 0.86 0.39 0.22 36.42 -0.03 0.11 -0.23 0.82 0.00 58.26
    Symptoms positive 0.61 0.39 1.58 0.11 0.59 64.55 0.14 0.15 0.96 0.34 0.00 84.20
    Symptoms negative 0.23 0.32 0.70 0.49 0.21 60.39 -0.09 0.13 -0.67 0.51 0.00 74.82
    Positive consequences -0.48 0.33 -1.44 0.15 -0.47 56.20 -0.02 0.13 -0.13 0.90 0.00 63.04
    Negative consequences -0.18 0.32 -0.55 0.58 -0.17 40.51 0.02 0.13 0.16 0.88 0.00 71.68
Insight into the disease
    Self-reflectiveness -0.08 0.29 -0.27 0.79 -0.08 60.31 -0.13 0.12 -1.04 0.30 0.00 81.75
    Self-certainty 0.06 0.26 0.24 0.81 0.06 46.12 0.08 0.11 0.75 0.45 0.00 68.53
Open in a new tab

Note. Values based on transformed data.

Lasso = Least absolute shrinkage and selection operator; RF = Random forest; VIV = Variable importance values; p = Significance value. Variable importance values are scaled to lie between 0 and 100.

Results of the multiple linear regression models revealed significant associations between the continuous variable “length of stay” and psychological health (quality of life) and hurts-self (self-stigma); the higher patients’ utilization of psychiatric inpatient treatment, the higher were their scores on psychological health and hurts-self. Model performance for the multiple linear regression model based on the root mean square error (RMSE) and the variance explained (R2) was 0.88 and 0.22, respectively, and again strongly deteriorated to 1.07 (RMSE) and .08 (R2) after cross-validation.

Lasso models

Coefficients based on the lasso models are depicted in Table 2. Coefficients for the dichotomous outcome “utilization pattern” were comparable with those based on an ordinary logistic regression model since the lasso led to the highest predictive accuracy if all predictors were included (and thus none set to 0). However, model performance after cross-validation was low (AUC = 0.58) and not better than that for the cross-validated multiple logistic regression model reported above.

Coefficients based on the lasso model of the outcome “length of stay” were set to 0 for all predictors. They were higher than 0 for only the covariates that were deliberately not shrunk. Thus, none of the predictors based on this model showed predictive accuracy. Model performance after cross-validation (RMSE = 0.99, R2 = 0.10) was slightly better than that of the cross-validated multiple linear regression model. This was, however, due to the three covariates alone since the lasso excluded all predictors.

In sum, the cross validated lasso models suggest that none of the psychological variables tested in this study were able to predict either of the two indicators of patients’ utilization of psychiatric inpatient treatment.

Random forest models

Variable importance values for the two outcomes “utilization pattern” and “length of stay” are shown in Table 2. The most important variables were awareness (self-stigma), and hurts-self (self-stigma) for “utilization pattern” and psychopathology, psychological health (quality of life), awareness (self-stigma), agreement (self-stigma), symptoms positive (subjective experience and meaning of psychoses), and self-reflectiveness (insight into the disease) for length of stay. However, these results must be seen against the background of the low model performance of the random forest model for both outcomes (utilization pattern: AUC = 0.60; length of stay: RMSE = 0.98, R2 = 0.11).

Discussion

This study examined the association between specific psychological characteristics and the use of psychiatric inpatient treatment in patients diagnosed with schizophrenia spectrum or bipolar affective disorders from the UPK Basel. While the multiple regression models reported significant associations between specific psychological variables and patients’ utilization of psychiatric inpatient treatment, the low model performance of the cross validated multiple regression models combined with the findings of the lasso and the random forest models indicated that none of the psychological variables were actually able to predict patients’ utilization of psychiatric inpatient treatment. Hence, our findings do not suggest a link between patients’ psychological characteristics estimated in this study and their utilization of psychiatric inpatient treatment. As the values of the respective measures of accuracy (AUC, RMSE, and R2) were insufficient in all the models, this suggests that other patient characteristics–which were not captured in this study–were relevant for utilization of psychiatric inpatient treatment.

To date, prior studies primarily investigated whether sociodemographic and clinical characteristics such as employment, housing or education status (e.g., [9, 22, 23, 26]), social network [18, 26], pharmacological adherence [2, 10, 11, 29], diagnostic information [3, 5, 19, 22], or insight into the disease [31] were related to patients’ use of psychiatric inpatient treatment. In contrast, this study examined patients’ quality of life, self-esteem, self-stigma, and subjective experience and meaning of psychoses. Thus, this study broadens prior findings by focusing on psychological variables to examine associations between patient characteristics and their utilization of psychiatric inpatient treatment.

In this study, the findings of the cross-validated multiple regression models and the lasso and random forest models indicated that none of the estimated psychological patients’ variables showed predictive accuracy regarding patients’ utilization of psychiatric inpatient treatment. However, this does not indicate that these psychological patient characteristics are not clinically relevant to patients with schizophrenia spectrum or bipolar affective disorders. In contrast, there is a growing body of evidence suggesting that self-stigma, self-esteem, and insight into the disease varies among these patients compared to other patient groups. Moreover, these characteristics should be specifically addressed in the psychotherapeutic treatment of patients with psychoses (e.g., [31, 58–62]). In contrast to our findings, prior studies have reported a link between a decreased insight into disease and a higher utilization of psychiatric inpatient treatment in patients with psychosis [30, 31]. However, the findings of this study do not suggest a link between the estimated psychological variables including insight into disease and utilization of psychiatric inpatient treatment. One potential explanation for this observation might be that the examined psychological characteristics of the patients do not vary depending on the use of psychiatric inpatient treatment in patients diagnosed with schizophrenia spectrum or bipolar affective disorders. In this case, there is no evidence that these characteristics should be addressed differently in the psychotherapeutic treatment of HU compared to NHU patients diagnosed with schizophrenia spectrum or bipolar affective disorders. However, this explanation brings up other questions to the fore: whether it is not rather the previously reported factors (e.g., problems in the areas of housing/social network or services that are not tailored to the needs of the patients) that are linked to patients’ utilization of psychiatric inpatient treatment, or whether there are other psychological patient characteristics relevant in this context but not examined in this study. Other possible explanations for our observations could be that the methodological aspects in this study ensured that no link between the psychological characteristics of patients and their utilization of psychiatric inpatient treatment was revealed. Potential methodological aspects that can be considered in this context include the comparatively small sample size, potential presence of further confounders that were not included in the analyses, and a potential bias in the estimation of the psychological patient characteristics. It could be, for example, that the self-report questionnaires used in this study did not reliably estimate the patients’ psychological characteristics due to their acute psychosis during inpatient treatment. Hence, future studies should be performed in order to either replicate or correct the findings of this study in larger samples. To reduce the risk of a bias, these studies may include further potential confounders such as drug adherence, social support, or substance use in their research design and may investigate patients’ psychological characteristics among other less severe psychopathological conditions during outpatient treatment.

Identification of robust psychological variables linked to utilization of psychiatric inpatient treatment in future studies may enhance the understanding of the underlying mechanisms of high utilization and, in turn, foster tailoring of psychotherapeutic interventions to the specific characteristics of HU patients. Describing specific patient groups and tailoring psychiatric and psychotherapeutic interventions to their specific characteristics were discussed as important factors in the implementation of a patient-centered psychiatry with open doors and least restrictive treatment [63–67].

This study has some limitations. First, the sample size is rather small and a substantial proportion of screened patients declined participation. Hence, the findings should be replicated with a larger sample. Second, we measured patients’ psychological characteristics in the presence of a researcher; thus, the occurrence of the “interviewer effect” is possible [68]. Accordingly, patients may have trivialized statements, which may have potentially led to underestimating respective variable levels. Third, in this cross-sectional observational study, we entered patients’ psychological variables as predictors and “utilization pattern” and “length of stay” as outcome variables into our statistical models. One reason for this was that quality of life, self-esteem, self-stigma, and insight into the disease revealed a relatively high stability over time in prior research [69–73]. These psychological variables may precede and cause patients’ behaviors related to their use of psychiatric inpatient treatment. Another reason was that prior observational studies investigating utilization of psychiatric inpatient treatment usually defined indicators of utilization as outcomes based on the assumption that utilization relates to an outcome concept, which can be influenced by therapeutic interventions (e.g., [5]). In future studies, the use of prospective study designs could clarify the question about whether variables estimating patients’ utilization of psychiatric inpatient treatment should be defined as predictors or outcomes. Fourth, another potential limitation is that we cannot rule out the presence of selection bias [74]. As participation was voluntary and inclusion and exclusion criteria were applied, our study sample may possess specific characteristics compared to the respective total patient population, which may limit the generalizability of our findings. Due to these limitations, it is necessary to replicate our findings through further studies before clinical implications for practice can be drawn. Despite these limitations, this study has important strengths. We applied methodologically rigorous "state of the art" statistics to estimate associations between patients’ psychological characteristics and their utilization of psychiatric inpatient treatment. These included the lasso model for variable section combined with cross-validation to assess the predictive accuracy of the investigated models based on test data rather than training data. The lasso method considers the family-wise error rate and is, therefore, less likely to cause chance effects [75–77]. Moreover, cross-validation is associated with a higher predictive accuracy, especially when the number of predictors considered is large, relative to the sample size [54]. In this study, using only multiple regression models without cross-validation would have led to several significant associations between psychological variables and utilization, which would likely disappear when replicated. Another strength of the study is the application of two different indicators for utilization of inpatient treatment: the dichotomous classification HU versus NHU, and the continuous variable number of inpatient treatment days during the last 30 months. As prior studies criticized the heterogenous definition of “high utilization,” the use of different parameters seemed more appropriate for the statistical analyses of our data [10]. Furthermore, this study examined the high utilization phenomenon in patients diagnosed with schizophrenia spectrum or bipolar affective disorders. High utilization occurs more often in this group of patients compared to patient groups with other primary diagnoses (e.g., [13, 19–22]); thus, we studied the high utilization phenomenon in a particularly relevant population.

In conclusion, our findings do not suggest a link between the estimated psychological characteristics and utilization of psychiatric inpatient treatment in patients with schizophrenia spectrum or bipolar affective disorders. The insufficient understanding of the mechanisms linking specific patients to high utilization, the potentially high personal burden of HU patients and their relatives, as well as the objective to tailor treatment concepts to their specific characteristics highlights the need for greater understanding in this field.

Supporting information

S1 Table. Correlation matrix containing bivariate correlations between all relevant variables.

(DOCX)

Click here for additional data file. (24.3KB, docx)

Acknowledgments

The authors want to thank Lukas Imfeld for his help with the extraction of anonymized data and Melanie Nuoffer for her help with the preparation of the tables and literature research.

Data Availability

The ethical approval for this study does not include that the anonymized data may be made freely available in public repositories. Therefore, the raw data supporting the conclusions of this manuscript were not uploaded for free access, but will be made available by the Center for Psychotic Disorders of the University Psychiatric Clinics Basel (secretariat email: ZPE@upk.ch), without undue reservation, to any qualified researcher.

Funding Statement

JM and SW received funding from the Universitäre Psychiatrische Kliniken (UPK) Basel Forschungsfonds, project no. 25036. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Hadley TR, Culhane DP, McGurrin MC. Identifying and tracking “heavy users” of acute psychiatric inpatient services. Adm Policy Ment Health. 1992;19: 279–290. [Google Scholar]
  • 2.Kent S, Fogarty M, Yellowlees P. A review of studies of heavy users of psychiatric services. Psychiatr Serv. 1995;46(12): 1247–1253. doi: 10.1176/ps.46.12.1247 [DOI] [PubMed] [Google Scholar]
  • 3.Spiessl H, Hübner-Liebermann B, Binder H, Cording C. „Heavy Users” in einer psychiatrischen Klinik—Eine Kohortenstudie mit 1811 Patienten über fünf Jahre. Psychiatr Prax. 2002;29: 350–354. doi: 10.1055/s-2002-34659 [DOI] [PubMed] [Google Scholar]
  • 4.Golay P, Morandi S, Conus P, Bonsack C. Identifying patterns in psychiatric hospital stays with statistical methods: towards a typology of post-deinstitutionalization hospitalization trajectories. Soc Psychiatry Psychiatr Epidemiol. 2019;54: 1411–1417. doi: 10.1007/s00127-019-01717-7 [DOI] [PubMed] [Google Scholar]
  • 5.Kromka W, Simpson S. A narrative review of predictors of adult mental health emergency department return visits and interventions to reduce repeated use. J Emerg Med. 2019;57: 671–682. doi: 10.1016/j.jemermed.2019.08.005 [DOI] [PubMed] [Google Scholar]
  • 6.Roick C, Gärtner A, Heider D, Dietrich S, Angermeyer MC. Heavy use of psychiatric inpatient care from the perspective of the patients affected. Int J Soc Psychiatry. 2006;52: 432–446. doi: 10.1177/0020764006066824 [DOI] [PubMed] [Google Scholar]
  • 7.Broadbent E, Kydd R, Sanders D, Vanderpyl J. Unmet needs and treatment seeking in high users of mental health Services: role of illness perceptions. Aust N Z J Psychiatry. 2008;42: 147–153. doi: 10.1080/00048670701787503 [DOI] [PubMed] [Google Scholar]
  • 8.Kent S, Fogarty M, Yellowlees P. Heavy utilization of inpatient and outpatient services in a public mental health service. Psychiatr Serv. 1995;46(12): 1254–1257. doi: 10.1176/ps.46.12.1254 [DOI] [PubMed] [Google Scholar]
  • 9.Lindamer LA, Liu L, Sommerfeld DH, Folsom DP, Hawthorne W, Garcia P, et al. Predisposing, enabling, and need factors associated with high service use in a public mental health system. Adm Policy Ment Health. 2012;39: 200–209. doi: 10.1007/s10488-011-0350-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Roick C, Gärtner A, Heider D, Angermeyer MC. Heavy user psychiatrischer Versorgungsdienste. Psychiatr Prax. 2002;29: 334–342. doi: 10.1055/s-2002-34658 [DOI] [PubMed] [Google Scholar]
  • 11.Junghan UM, Brenner HD. Heavy use of acute in-patient psychiatric services: the challenge to translate a utilization pattern into service provision. Acta Psychiatr Scand Suppl. 2006;113: 24–32. doi: 10.1111/j.1600-0447.2005.00713.x [DOI] [PubMed] [Google Scholar]
  • 12.Katschnig H, Strassmayr C, Endel F, Berger M, Zauner G, Kalseth J, et al. Using national electronic health care registries for comparing the risk of psychiatric re-hospitalisation in six European countries: opportunities and limitations. Health Policy. 2019;123: 1028–1035. doi: 10.1016/j.healthpol.2019.07.006 [DOI] [PubMed] [Google Scholar]
  • 13.Lay B, Lauber C, Rössler W. Prediction of in-patient use in first-admitted patients with psychosis. Eur Psychiatry. 2006;21: 401–409. doi: 10.1016/j.eurpsy.2005.12.004 [DOI] [PubMed] [Google Scholar]
  • 14.Morlino M, Calento A, Schiavone V, Santone G, Picardi A, de Girolamo G, et al. Use of psychiatric inpatient services by heavy users: findings from a national survey in Italy. Eur Psychiatry. 2011;26: 252–259. doi: 10.1016/j.eurpsy.2010.11.005 [DOI] [PubMed] [Google Scholar]
  • 15.Casper ES, Pastva G. Admission histories, patterns, and subgroups of the heavy users of a state psychiatric hospital. Psychiatr Q. 1990;61: 121–134. doi: 10.1007/BF01064912 [DOI] [PubMed] [Google Scholar]
  • 16.Holohean EJ, Pulice RT, Donahue SA. Utilization of acute inpatient psychiatric services: “heavy users” in New York State. Adm Policy Ment Health. 1991;18: 173–181. [Google Scholar]
  • 17.Hadley TR, McGurrin MC, Pulice RT, Holohean EJ. Using fiscal data to identify heavy service users. Psychiatr Q. 1990;61: 41–48. doi: 10.1007/BF01065163 [DOI] [PubMed] [Google Scholar]
  • 18.Kent S, Yellowlees P. The relationship between social factors and frequent use of psychiatric services. Aust N Z J Psychiatry. 1995;29: 403–408. doi: 10.3109/00048679509064947 [DOI] [PubMed] [Google Scholar]
  • 19.Evans LJ, Harris V, Newman L, Beck A. Rapid and frequent psychiatric readmissions: associated factors. Int J Psychiatry Clin Pract. 2017;21: 271–276. doi: 10.1080/13651501.2017.1324037 [DOI] [PubMed] [Google Scholar]
  • 20.Jeppesen RM, Christensen T, Vestergaard CH. Changes in the utilization of psychiatric hospital facilities in Denmark by patients diagnosed with schizophrenia from 1970 through 2012: the advent of ‘revolving door’ patients. Acta Psychiatr Scand Suppl. 2016;133: 419–425. doi: 10.1111/acps.12549 [DOI] [PubMed] [Google Scholar]
  • 21.Kapur K, Young AS, Murata D. Risk adjustment for high utilizers of public mental health care. J Ment Health Policy Econ. 2000;3: 129–137. doi: 10.1002/mhp.85 [DOI] [PubMed] [Google Scholar]
  • 22.Lorine K, Goenjian H, Kim S, Steinberg AM, Schmidt K, Goenjian AK. Risk factors associated with psychiatric readmission. J Nerv Ment Dis. 2015;203: 425. doi: 10.1097/NMD.0000000000000305 [DOI] [PubMed] [Google Scholar]
  • 23.Di Lorenzo R, Sagona M, Landi G, Martire L, Piemonte C, Del Giovane C. The revolving door phenomenon in an italian acute psychiatric ward: a 5-year retrospective analysis of the potential risk factors. J Nerv Ment Dis. 2016;204: 686–692. doi: 10.1097/NMD.0000000000000540 [DOI] [PubMed] [Google Scholar]
  • 24.Pasic J, Russo J, Roy-Byrne P. High utilizers of psychiatric emergency services. Psychiatr Serv. 2005;56: 678–684. doi: 10.1176/appi.ps.56.6.678 [DOI] [PubMed] [Google Scholar]
  • 25.Lucas B, Harrison-Read P, Tyrer P, Ray J, Shipley K, Hickman M, et al. Costs and characteristics of heavy inpatient service users in outer London. Int J Soc Psychiatry. 2001;47: 63–74. doi: 10.1177/002076400104700106 [DOI] [PubMed] [Google Scholar]
  • 26.Kent S, Yellowlees P. Psychiatric and social reasons for frequent rehospitalization. Hosp Community Psychiatry. 1994;45: 347–350. doi: 10.1176/ps.45.4.347 [DOI] [PubMed] [Google Scholar]
  • 27.Stulz N, Bielinski D, Junghan UM, Hepp U. Stationärer Heavy Use und die Inanspruchnahme institutioneller ambulanter Dienste in einem Schweizerischen Versorgungssystem. Psychiatr Prax. 2012;39: 332–338. doi: 10.1055/s-0032-1305231 [DOI] [PubMed] [Google Scholar]
  • 28.Roick C, Heider D, Kilian R, Matschinger H, Toumi M, Angermeyer MC. Factors contributing to frequent use of psychiatric inpatient services by schizophrenia patients. Soc Psychiatry Psychiatr Epidemiol. 2004;39: 744–751. doi: 10.1007/s00127-004-0807-8 [DOI] [PubMed] [Google Scholar]
  • 29.Kennedy P, Hird F. Description and evaluation of a short-stay admission ward. Br J Psychiatry. 1980;136: 205–215. doi: 10.1192/bjp.136.3.205 [DOI] [PubMed] [Google Scholar]
  • 30.Kemp RA, Lambert TJR. Insight in schizophrenia and its relationship to psychopathology. Schizophr Res. 1995;18: 21–28. doi: 10.1016/0920-9964(95)00018-6 [DOI] [PubMed] [Google Scholar]
  • 31.Ramu N, Kolliakou A, Sanyal J, Patel R, Stewart R. Recorded poor insight as a predictor of service use outcomes: cohort study of patients with first-episode psychosis in a large mental healthcare database. BMJ Open. 2019;9: e028929. doi: 10.1136/bmjopen-2019-028929 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Gumley A, Karatzias A, Power K, Reilly J, McNay L, O’Grady M. Early intervention for relapse in schizophrenia: impact of cognitive behavioural therapy on negative beliefs about psychosis and self-esteem. Br J Clin Psychol. 2006;45: 247–260. doi: 10.1348/014466505X49925 [DOI] [PubMed] [Google Scholar]
  • 33.Lysaker PH, Roe D, Ringer J, Gilmore EM, Yanos PT. Change in self-stigma among persons with schizophrenia enrolled in rehabilitation: associations with self-esteem and positive and emotional discomfort symptoms. Psychol Serv. 2012;9: 240–247. doi: 10.1037/a0027740 [DOI] [PubMed] [Google Scholar]
  • 34.Ritsner M, Gibel A, Ratner Y. Determinants of changes in perceived quality of life in the course of schizophrenia. Qual Life Res. 2006;15: 515–526. doi: 10.1007/s11136-005-2808-9 [DOI] [PubMed] [Google Scholar]
  • 35.Moritz S, Mahlke CI, Westermann S, Ruppelt F, Lysaker PH, Bock T, et al. Embracing psychosis: a cognitive insight intervention improves personal narratives and meaning-making in patients with schizophrenia. Schizophr Bull. 2018;44: 307–316. doi: 10.1093/schbul/sbx072 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Carroll A, Fattah S, Clyde Z, Coffey I, Owens DGC, Johnstone EC. Correlates of insight and insight change in schizophrenia. Schizophr Res. 1999;35: 247–253. doi: 10.1016/s0920-9964(98)00142-x [DOI] [PubMed] [Google Scholar]
  • 37.Weiler MA, Fleisher MH, McArthur-Campbell D. Insight and symptom change in schizophrenia and other disorders. Schizophr Res. 2000;45: 29–36. doi: 10.1016/s0920-9964(99)00215-7 [DOI] [PubMed] [Google Scholar]
  • 38.Skevington SM, Lotfy M, O’Connell KA, WHOQOL Group. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res. 2004;13: 299–310. doi: 10.1023/B:QURE.0000018486.91360.00 [DOI] [PubMed] [Google Scholar]
  • 39.Group WHOQOL. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychol Med. 1998;28: 551–558. doi: 10.1017/s0033291798006667 [DOI] [PubMed] [Google Scholar]
  • 40.Rosenberg M. Rosenberg self-esteem scale (RSE). Accept Commit Ther Meas Package. 1965;61: 52. [Google Scholar]
  • 41.Roth M, Decker O, Herzberg PY, Brähler E. Dimensionality and norms of the Rosenberg self-esteem scale in a German general population sample. Eur J Psychol Assess. 2008;24: 190–197. doi: 10.1027/1015-5759.24.3.190 [DOI] [Google Scholar]
  • 42.Corrigan PW, Watson AC, Barr L. The self-stigma of mental illness: implications for self-esteem and self-efficacy. J Soc Clin Psychol. 2006;25: 875–884. doi: 10.1521/jscp.2006.25.8.875 [DOI] [Google Scholar]
  • 43.Corrigan PW, Michaels PJ, Vega E, Gause M, Watson AC, Rüsch N. Self-stigma of mental illness scale—short form: reliability and validity. Psychiatry Res. 2012;199: 65–69. doi: 10.1016/j.psychres.2012.04.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Klapheck K, Nordmeyer S, Cronjäger H, Naber D, Bock T. Subjective experience and meaning of psychoses: the German subjective sense in psychosis questionnaire (SUSE). Psychol Med. 2012;42: 61–71. doi: 10.1017/S0033291711001103 [DOI] [PubMed] [Google Scholar]
  • 45.Bock T, Priebe S. Psychosis seminars: an unconventional approach. Psychiatr Serv. 2005;56: 1441–1443. doi: 10.1176/appi.ps.56.11.1441 [DOI] [PubMed] [Google Scholar]
  • 46.Bock T, Brysinski T, Klapheck K, Bening U, Lenz A, Naber D. On subjective meaning of psychoses. Construction, validation and first application of a new questionnaire—the SuSi-Project (Hamburg). Psychiatr Prax. 2010;37: 285–291. doi: 10.1055/s-0030-1248424 [DOI] [PubMed] [Google Scholar]
  • 47.Klapheck K, Lincoln TM, Bock T. Meaning of psychoses as perceived by patients, their relatives and clinicians. Psychiatry Res. 2014;215: 760–765. doi: 10.1016/j.psychres.2014.01.017 [DOI] [PubMed] [Google Scholar]
  • 48.Beck AT, Baruch E, Balter JM, Steer RA, Warman DM. A new instrument for measuring insight: the Beck cognitive insight scale. Schizophr Res. 2004;68: 319–329. doi: 10.1016/S0920-9964(03)00189-0 [DOI] [PubMed] [Google Scholar]
  • 49.Riggs SE, Grant PM, Perivoliotis D, Beck AT. Assessment of cognitive insight: a qualitative review. Schizophr Bull. 2012;38: 338–350. doi: 10.1093/schbul/sbq085 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Chisholm D, Knapp MR, Knudsen HC, Amaddeo F, Gaite L, van Wijngaarden B. Client socio-demographic and service receipt inventory—European version: development of an instrument for international research. EPSILON Study 5. European psychiatric services: inputs linked to outcome domains and needs. Br J Psychiatry Suppl. 2000;(39): 28–33. doi: 10.1192/bjp.177.39.s28 [DOI] [PubMed] [Google Scholar]
  • 51.Roick C, Kilian R, Matschinger H, Bernert S, Mory C, Angermeyer MC. German adaptation of the client sociodemographic and service receipt inventory—an instrument for the cost of mental health care. Psychiatr Prax. 2001;28 Suppl 2: 84–90. doi: 10.1055/s-2001-17790 [DOI] [PubMed] [Google Scholar]
  • 52.Mass R, Burmeister J, Krausz M. Dimensionale Struktur der deutschen Version der Brief Psychiatric Rating Scale (BPRS). Nervenarzt. 1997;68: 239–244. doi: 10.1007/s001150050119 [DOI] [PubMed] [Google Scholar]
  • 53.Overall JE, Gorham DR. The brief psychiatric rating scale. Psychol Rep. 1962;10: 799–812. [Google Scholar]
  • 54.Harrell FE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Cham: Springer International Publishing; 2015. doi: 10.1007/978-3-319-19425-7 [DOI] [Google Scholar]
  • 55.Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. New York, NY: Springer New York; 2009. doi: 10.1007/978-0-387-84858-7 [DOI] [Google Scholar]
  • 56.Breiman L. Random Forest. Mach Learn. 2001;45: 5–32. doi: 10.1023/A:1010933404324 [DOI] [Google Scholar]
  • 57.Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33: 1–22. doi: 10.18637/jss.v033.i01 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Amador X. I am not sick, I don’t need help!: how to help someone with mental illness accept treatment. Vida Press; 2000. [Google Scholar]
  • 59.Collett N, Pugh K, Waite F, Freeman D. Negative cognitions about the self in patients with persecutory delusions: an empirical study of self-compassion, self-stigma, schematic beliefs, self-esteem, fear of madness, and suicidal ideation. Psychiatry Res. 2016;239: 79–84. doi: 10.1016/j.psychres.2016.02.043 [DOI] [PubMed] [Google Scholar]
  • 60.Mittal D, Sullivan G, Chekuri L, Allee E, Corrigan PW. Empirical studies of self-stigma reduction strategies: a critical review of the literature. Psychiatr Serv. 2012;63: 974–981. doi: 10.1176/appi.ps.201100459 [DOI] [PubMed] [Google Scholar]
  • 61.Moritz S, Andreou C, Schneider BC, Wittekind CE, Menon M, Balzan RP, et al. Sowing the seeds of doubt: a narrative review on metacognitive training in schizophrenia. Clin Psychol Rev. 2014;34: 358–366. doi: 10.1016/j.cpr.2014.04.004 [DOI] [PubMed] [Google Scholar]
  • 62.Moritz S, Woodward TS. Metacognitive training in schizophrenia: from basic research to knowledge translation and intervention. Curr Opin Psychiatry. 2007;20: 619–625. doi: 10.1097/YCO.0b013e3282f0b8ed [DOI] [PubMed] [Google Scholar]
  • 63.Arnold BD, Moeller J, Hochstrasser L, Schneeberger AR, Borgwardt S, Lang UE, et al. Compulsory admission to psychiatric wards—who is admitted, and who appeals against admission? Front Psychiatry. 2019;10: 544. doi: 10.3389/fpsyt.2019.00544 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Hochstrasser L, Voulgaris A, Möller J, Zimmermann T, Steinauer R, Borgwardt S, et al. Reduced frequency of cases with seclusion is associated with “opening the doors” of a psychiatric intensive care unit. Front Psychiatry. 2018;9: 57. doi: 10.3389/fpsyt.2018.00057 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Hochstrasser L, Fröhlich D, Schneeberger AR, Borgwardt S, Lang UE, Stieglitz R-D, et al. Long-term reduction of seclusion and forced medication on a hospital-wide level: implementation of an open-door policy over 6 years. Eur Psychiatry. 2018;48: 51–57. doi: 10.1016/j.eurpsy.2017.09.008 [DOI] [PubMed] [Google Scholar]
  • 66.Lang U. Innovative Psychiatrie mit offenen Türen: Deeskalation und Partizipation in der Akutpsychiatrie. Berlin: Springer; 2013. [Google Scholar]
  • 67.Steinauer R, Krückl JS, Moeller J, Vogel M, Wiesbeck GA, Walter M, et al. Opening the doors of a substance use disorder ward—benefits and challenges from a consumer perspective. Front Psychiatry. 2020;11: 580885. doi: 10.3389/fpsyt.2020.580885 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.West BT, Blom AG. Explaining interviewer effects: a research synthesis. J Surv Stat Methodol. 2016;5(2): 175–211. doi: 10.1093/jssam/smw024 [DOI] [Google Scholar]
  • 69.Atkinson T. The stability and validity of quality of life measures. Soc Indic Res. 1982;10: 113–132. doi: 10.1007/BF00302506 [DOI] [Google Scholar]
  • 70.Kuster F, Orth U. The long-term stability of self-esteem: its time-dependent decay and nonzero asymptote. Pers Soc Psychol Bull. 2013;39: 677–690. doi: 10.1177/0146167213480189 [DOI] [PubMed] [Google Scholar]
  • 71.Lysaker PH, Tunze C, Yanos PT, Roe D, Ringer J, Rand K. Relationships between stereotyped beliefs about mental illness, discrimination experiences, and distressed mood over 1 year among persons with schizophrenia enrolled in rehabilitation. Soc Psychiatry Psychiatr Epidemiol. 2012;47: 849–855. doi: 10.1007/s00127-011-0396-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.McEvoy JP, Apperson LJ, Appelbaum PS, Ortlip P, Brecosky J, Hammill K, et al. Insight in schizophrenia. Its relationship to acute psychopathology. J Nerv Ment Dis. 1989;177: 43–47. doi: 10.1097/00005053-198901000-00007 [DOI] [PubMed] [Google Scholar]
  • 73.McGorry PD, McConville SB. Insight in psychosis: an elusive target. Compr Psychiatry. 1999;40: 131–142. doi: 10.1016/s0010-440x(99)90117-7 [DOI] [PubMed] [Google Scholar]
  • 74.Heckman JJ. Sample selection bias as a specification error. Econometrica. 1979;47: 153–161. doi: 10.2307/1912352 [DOI] [Google Scholar]
  • 75.Kukreja SL, Löfberg J, Brenner MJ. A least absolute shrinkage and selection operator (LASSO) for nonlinear system identification. IFAC Proc Vol. 2006;39: 814–819. doi: 10.3182/20060329-3-AU-2901.00128 [DOI] [Google Scholar]
  • 76.Schneider A, Hommel G, Blettner M. Linear regression analysis. Dtsch Arztebl Int. 2010;107(44): 776–782. doi: 10.3238/arztebl.2010.0776 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Series B Stat Methodol. 1996;58: 267–288. doi: 10.1111/j.2517-6161.1996.tb02080.x [DOI] [Google Scholar]

Decision Letter 0

Giuseppe Carrà

14 Oct 2021

PONE-D-21-06078The relationship between psychological characteristics of patients and their utilization of psychiatric inpatient treatment: A cross-sectional study, using machine learningPLOS ONE

Dear Dr. Moeller,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 28 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Giuseppe Carrà, PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section. 

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This study aimed to explore the psychological characteristics for high utilizers of psychiatric inpatient treatment. The authors enrolled 112 inpatients and classified into 50 as high utilizers and 62 as non-high utilizers. The measured psychological factors included quality of life, self-esteem, self-stigma, subjective experience and meaning of psychosis, and insight. The authors also measured the severity of illness using Brief Psychiatric Rating Scale. The authors used multiple regression model and LASSO model to assess the association between psychological factors and high utilization. The authors found there is no link between psychological characteristics and high utilization.

I think there are several concerns in the methodology.

Firstly, I think the sample size is small. In addition, there are many unmeasured factors, including drug compliance, social support, and substance use. These factors might lead to a null finding due to limited statistic power. Given these limitation, I think the study result is inconclusive.

Secondly, there might be collinearity between these variables. The authors should examine the variance inflation factor. LASSO could be the primary analysis because it might be better than multiple regression model in deal the collinearity problem.

Finally, the authors mentioned the used cross-validation to examine the performance and overfit of model. However, the authors did not mention how they conduct cross-validation.

Reviewer #2: The investigated topic seems both important and practically useful. I found that the authors provided a comprehensive description of the psychological characteristics chosen and that the process for recruiting and interviewing patients seemed sensible. However, I do not feel that training a multiple regression model and LASSO model are sufficient evidence that these psychological characteristics are not predictive of utilization pattern and length of stay. I would have liked to see the authors generate a correlation matrix prior to performing regression to perform the most basic analysis of whether certain features could be predictive.

Both models' performance after cross-validation was poor, nearing random, meaning that based on these models, the authors could not conclude that any of the variables were predictive for the two outcomes. However, there are many predictive methods in machine learning that also offer straightforward methods of extracting feature importance. The authors may be interested in reviewing survey papers such as "Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19" (Subudhi 2021) "Patient Length of Stay and Mortality Prediction: a survey" (Awad 2017) which describe many methods for prediction of clinical outcomes. For example, in a high performing predictive model of ICU admission, the authors of Subudhi 2021 performed a SHAP analysis on a trained random forest model. SHAP derives from Data Shapley, a method of attributing feature importance to to explain machine learning models. It would be interesting to see the authors of this paper attempt to train a more complex (and hopefully better performing) model and then analyze the psychological features based on a more performant model.

The topic of the paper has clinical significance and the dataset collected by the authors is both novel and useful. However, stronger statistical analysis and the investigation of further machine learning methods would make the paper more compelling towards its conclusion.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Isabel Chien

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Apr 1;17(4):e0266352. doi: 10.1371/journal.pone.0266352.r002

Author response to Decision Letter 0

25 Feb 2022

Responses to Reviewer#1:

This study aimed to explore the psychological characteristics for high utilizers of psychiatric inpatient treatment. The authors enrolled 112 inpatients and classified into 50 as high utilizers and 62 as non-high utilizers. The measured psychological factors included quality of life, self-esteem, self-stigma, subjective experience and meaning of psychosis, and insight. The authors also measured the severity of illness using Brief Psychiatric Rating Scale. The authors used multiple regression model and LASSO model to assess the association between psychological factors and high utilization. The authors found there is no link between psychological characteristics and high utilization.

I think there are several concerns in the methodology.

Firstly, I think the sample size is small. In addition, there are many unmeasured factors, including drug compliance, social support, and substance use. These factors might lead to a null finding due to limited statistic power. Given these limitation, I think the study result is inconclusive.

Response: Thank you very much for this helpful comment. For the current study, we recruited subjects from a very vulnerable population – psychotic patients in acute psychiatric care – where it is challenging to conduct studies. Since psychological characteristics have been largely neglected in high-utilizer research and we have applied rigorous statistical analyses, we consider the presented findings as clinically relevant despite the study’s limitations. We hope that the findings of our study will stimulate further research in this field. However, we fully agree with the reviewer that the sample size of this study is rather small with 112 patients included in the analyses. We also agree with the reviewer that we cannot rule out that there may be other potential confounders that we have not been able to consider in the present analyses. Therefore, we have now discussed these potential limitations in relation to our findings in more detail and adapted the revised manuscript accordingly.

Discussion: “Other possible explanations for our observations could be that the methodological aspects in this study ensured that no link between the psychological characteristics of patients and their utilization of psychiatric inpatient treatment was revealed. Potential methodological aspects that can be considered in this context include the comparatively small sample size, potential presence of further confounders that were not included in the analyses, and a potential bias in the estimation of the psychological patient characteristics.”

Discussion: “Hence, future studies should be performed in order to either replicate or correct the findings of this study in larger samples. To reduce the risk of a bias, these studies may include further potential confounders such as drug adherence, social support, or substance use in their research design and …”

Discussion: “Due to these limitations, it is necessary to replicate our findings through further studies before clinical implications for practice can be drawn.”

Secondly, there might be collinearity between these variables. The authors should examine the variance inflation factor.

Response: We thank the reviewer for raising the issue of collinearity and fully agree that the variance inflation factor should be reported. In our study, multicollinearity among the predictors and covariates was only moderate with variance inflation factors ranging between 1.3 and 5.2. A variance inflation factor of >10 is usually used as an indicator of multicollinearity. Since our values are clearly below this threshold, we have refrained from a detailed discussion in the manuscript and have included the sentence below in the revised manuscript.

Statistical analyses: “Multicollinearity among the predictors and covariates was moderate, with variance inflation factors ranging between 1.3 and 5.2.”

LASSO could be the primary analysis because it might be better than multiple regression model in deal the collinearity problem.

Response: We thank the reviewer for his comment, and we fully agree that the machine learning models are the main analyses of the study. However, we had chosen to report the regression models first, as much of the research in the field to date has focused solely on the use of regression models. By using this order of analysis and the resulting "story" we want to point out the importance of rigorous statistical analysis in the context of high utilizer research. We aim to indicate that we might have reached significantly different conclusions in this study if we had focused primarily on the significant associations between utilization pattern and self-stigma of the reported multiple regression models. We would therefore appreciate it very much if the reviewer agrees that we can retain the current order of the statistical analyses in the revised manuscript.

Finally, the authors mentioned the used cross-validation to examine the performance and overfit of model. However, the authors did not mention how they conduct cross-validation.

Response: We thank the reviewer for this comment and have revised the manuscript accordingly.

Statistical analyses: “To determine the predictive accuracy of the models, we conducted repeated (n = 10) 10-fold cross-validation. We used the root mean square error (RMSE, i.e. the square root of the variance of the residuals) and the R-squared (R2) as the measures of model accuracy for continuous and dichotomous outcomes, respectively.”

Responses to Reviewer#2:

The investigated topic seems both important and practically useful. I found that the authors provided a comprehensive description of the psychological characteristics chosen and that the process for recruiting and interviewing patients seemed sensible.

Response: We thank the reviewer for this comment and appreciate his positive feedback on our manuscript regarding the investigated topic, variable selection, and recruitment process.

However, I do not feel that training a multiple regression model and LASSO model are sufficient evidence that these psychological characteristics are not predictive of utilization pattern and length of stay.

Response: We thank the reviewer for this feedback and the specific suggestions, described below in the review, on how we can further improve the rigor of our statistical analyses. To improve the manuscript, we performed additional statistical analyses as suggested by the reviewer and ran a random forest model and included its findings in the revised manuscript. We reported these additional analyses in detail below in our point-by-point response. On the other hand, we discussed in more detail that we cannot completely rule out that methodological aspects were also responsible for the fact that we did not find a link between the psychological patient characteristics and their utilization of psychiatric inpatient treatment. Please see in this regard our point-by-point response to Reviewer 1.

I would have liked to see the authors generate a correlation matrix prior to performing regression to perform the most basic analysis of whether certain features could be predictive.

Response: We agree with the reviewer that a correlation matrix can increase the information content of the manuscript. Accordingly, we included a correlation matrix containing the bivariate correlations between all relevant variables as supporting information in the revised manuscript (supporting table S1, retrievable via hyperlink). We appended the table S1 also below our point-by-point responses.

Results: “See the correlation matrix of all relevant variables presented in supporting information S1 Table .”

Both models' performance after cross-validation was poor, nearing random, meaning that based on these models, the authors could not conclude that any of the variables were predictive for the two outcomes. However, there are many predictive methods in machine learning that also offer straightforward methods of extracting feature importance. The authors may be interested in reviewing survey papers such as "Comparing machine learning algorithms for predicting ICU admission and mortality in COVID-19" (Subudhi 2021) "Patient Length of Stay and Mortality Prediction: a survey" (Awad 2017) which describe many methods for prediction of clinical outcomes. For example, in a high performing predictive model of ICU admission, the authors of Subudhi 2021 performed a SHAP analysis on a trained random forest model. SHAP derives from Data Shapley, a method of attributing feature importance to explain machine learning models. It would be interesting to see the authors of this paper attempt to train a more complex (and hopefully better performing) model and then analyze the psychological features based on a more performant model. The topic of the paper has clinical significance and the dataset collected by the authors is both novel and useful. However, stronger statistical analysis and the investigation of further machine learning methods would make the paper more compelling towards its conclusion.

Response: We thank the reviewer very much for this comment and for the further literature relating to predictive methods in machine learning. Following the suggestion of the reviewer, we have calculated a second machine learning model, namely a trained random forest model calculating variable importance values for the two outcomes “utilization pattern” and “length of stay”. Due to the low model performance of the random forest model for both outcomes (utilization pattern: AUC=0.60; length of stay: [RMSE=0.98, R2=0.11], the conclusions based on the findings have remained largely the same. We included the trained random forest model as an additional statistical analysis in the manuscript and revised the manuscript accordingly.

Statistical analyses: “To avoid model overfit, we ran two additional models that use machine learning algorithms⎯the least absolute shrinkage and selection operator (lasso) model, and the random forest model. Both models contained the same predictors, outcomes, and adjustments described above.”

Statistical analyses: “The random forest model belongs to the family of decision tree models, which characteristically generate a large number of bootstrapped decision trees, based on random samples of variables. The model has been shown to exhibit high predictive accuracy (Breiman, 2001).”

Breiman L. Random forests. Mach Learn. 2001;45(1):5-32. doi:10.1023/A:1010933404324

Results: “Variable importance values for the two outcomes “utilization pattern” and “length of stay” are shown in Table 2. The most important variables were awareness (self-stigma), and hurts-self (self-stigma) for “utilization pattern” and psychopathology, psychological health (quality of life), awareness (self-stigma), agreement (self-stigma), symptoms positive (subjective experience and meaning of psychoses), and self-reflectiveness (insight into the disease) for length of stay. However, these results must be seen against the background of the low model performance of the random forest model for both outcomes (utilization pattern: AUC=0.60; length of stay: RMSE=0.98, R2=0.11).”

Discussion: “As the values of the respective measures of accuracy (AUC, RMSE, and R2) were insufficient in all the models, this suggests that other patient characteristics – which were not captured in this study – were relevant for utilization of psychiatric inpatient treatment.”

Response: Moreover, we also added several minor changes when we revised the manuscript to include the new findings of the random forest model. For further information, please see our Revised Manuscript with Track Changes. We have included the specific variable importance values of the random forest model in Table 2 of the revised manuscript.

Attachment

Submitted filename: Response to Reviewers_HU140122.docx

Click here for additional data file. (47KB, docx)

Decision Letter 1

Giuseppe Carrà

21 Mar 2022

The relationship between psychological characteristics of patients and their utilization of psychiatric inpatient treatment: A cross-sectional study, using machine learning

PONE-D-21-06078R1

Dear Dr. Moeller,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Giuseppe Carrà, PhD

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have addressed all my concerns well. I have no further comments on the revised manuscripts.

Reviewer #2: I find that the authors have addressed comments brought up in both reviews appropriately. Again I reiterate that the problem setting seems interesting and significant and although the authors present a negative finding, this work serves as a good initiation into this particular research question. There is potential for more sophisticated machine learning methods to improve the predictive performance and interpretability of the presented models; hopefully these can be explored in follow-up research.

*Note to the authors: This reviewer is a "she," in future it would be good practice to maintain gender neutrality when referencing reviewers.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Isabel Chien

Acceptance letter

Giuseppe Carrà

25 Mar 2022

PONE-D-21-06078R1

The relationship between psychological characteristics of patients and their utilization of psychiatric inpatient treatment: A cross-sectional study, using machine learning

Dear Dr. Moeller:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Giuseppe Carrà

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Table. Correlation matrix containing bivariate correlations between all relevant variables.

    (DOCX)

    Click here for additional data file. (24.3KB, docx)
    Attachment

    Submitted filename: Response to Reviewers_HU140122.docx

    Click here for additional data file. (47KB, docx)

    Data Availability Statement

    The ethical approval for this study does not include that the anonymized data may be made freely available in public repositories. Therefore, the raw data supporting the conclusions of this manuscript were not uploaded for free access, but will be made available by the Center for Psychotic Disorders of the University Psychiatric Clinics Basel (secretariat email: ZPE@upk.ch), without undue reservation, to any qualified researcher.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES