Development and Validation of Hospital Mental Health Screen to Detect Psychiatric Morbidity in Medically Ill Patients in India

Abstract

Background:

Psychiatric morbidities often go unnoticed in medically ill patients. It is essential to screen patients with medical morbidity so that they can be referred to psychiatrists for early interventions in general hospitals in India. There is a potential lacuna in terms of the availability of a scale that can aptly identify psychiatric symptoms in medically ill patients beyond depression or anxiety, especially in low-resource settings like India.

Aim:

The aim was to detect psychiatric morbidity in medically ill patients in India.

Methodology:

Items were generated using deductive and inductive approaches. Item-Content Validity Index (I-CVI) and Scale-Content Validity Index/Universal Agreement (S-CVI/UA) were computed by involving eight subject matter specialists. The tool was circulated to 397 medically ill patients for computing The exploratory factor analysis (EFA). Domain-wise reliability using Cronbach’s alpha was calculated for six factors. The concurrent criterion validity of the Hospital Mental Health Screen (HMHS) tool was calculated by the receiver operating curve (ROC) against the gold standard of any psychiatric morbidity diagnosed by two psychiatrists in 397 medically ill patients. We used IBM SPSS version 23.

Results:

Initially, 34 items were generated. At the I-CVI threshold of 79%, seven items were discarded. The S-CVI/UA of the scale was 85.1%. The Kaiser–Meier–Olkin Measure of Sampling Adequacy (KMO MSA) was found to be 0.916. At a factor loading threshold of 0.4 and an eigenvalue above 1, a six-factor structure was extracted using principal component analysis and varimax rotation. Domain-wise reliability was computed, which was between 0.657 and 0.840. The final tool consisted of 27 Likert items (0 = never to 4 = always). Using the ROC curve at the 19.5 threshold, 91.4% of the positive outcomes were correctly classified and 9.5% of the adverse outcomes were expected to be incorrectly identified by the HMHS screening tool.

Conclusion:

HMHS is a valid and reliable tool with good screening properties, designed especially for the Indian setting. This scale can assist in identifying psychiatric morbidity in medically ill patients in low-resource settings. There is further scope for performing confirmatory factor analysis (CFA) to reinforce the factor structure of HMHS.

Key Messages

• Psychiatric morbidities often go unnoticed in medically ill patients due to the non-availability of a standard screening tool in low-resource settings like India.

• The Hospital Mental Health Screen (HMHS) is a valid and reliable tool with good screening properties that can identify psychiatric symptoms beyond depression or anxiety.

• The HMHS can be used as a screening tool in India’s primary healthcare and general hospital settings.

Identifying mental health issues among medically ill patients is gaining attention as the influence of psychiatric disorders is widely recognized among chronic medical conditions. The COVID-19 pandemic fueled this. As per the 2022 report of the World Health Organization (WHO) on mental health, a steep rise of 25% in mental morbidity for common conditions such as depression and anxiety was observed in the first year of the COVID-19 pandemic. ¹ It also stated that mental disorder is one of the most crucial contributors to years lived with disability (YLDs) as worldwide it is accountable for one in every six YLDs. ² In India, for early detection and treatment of mental morbidity, the Government of India (GOI) launched the National Mental Health Programme (NMHP) in 1982 and the District Mental Health Programme (DMHP) in 1996. ³ Despite all the efforts to improve mental health, the estimated burden in India is still alarming, with 2,443 disability-adjusted life years (DALYs) per 10,000 population and a financial loss of approximately US$ 1.03 trillion. ⁴

Psychiatric disorders are highly prevalent in patients with medical illnesses. In psoriasis, for example, approximately one-third of patients experience psychiatric comorbidity, while the data also suggests that it can be as high as 62%. ⁵ Similarly, in patients with respiratory illnesses, the psychiatric morbidity can be as high as 58%, ⁶ with panic and other anxiety disorders being highly prevalent (34%). ⁷ Medically ill persons who have comorbid mental conditions showed a significantly decreased overall physical and psychosocial quality of life compared to persons without mental disorders. ⁸ Data suggests that depression symptoms are prominent in cardiac patients, especially when patients encounter acute cardiac events or cardiac surgery. The situation gets worse when such conditions get aggravated by mood disorders in such hospitalized patients, leading to premature deaths. ⁹ There is a scarcity of assessment tools available for identifying mental health issues in patients with medical illnesses. The available tools such as the General Hospital Questionnaire (GHQ), ¹⁰ Patient Health Questionnaire (PHQ), ¹¹ and Hospital Anxiety Depression Scale (HADS) ¹² are suitable for screening depressive and anxiety disorders and have also been validated in non-Western populations, but they have limitations in terms of not being able to screen other psychological aspects routinely encountered by subjects with medical illnesses such as apprehension of illness, worries about cost of treatment, overall distress, coping ability, and few psychiatric disorders such as substance use, obsessive compulsive disorder (OCD), and psychosis. ¹³ The available tools have done reasonably well in screening depressive and anxiety symptoms predominantly. On the flip side, some are expensive and non-accommodative due to cultural differences. Therefore, the translation of these scales for the Indian population has limitations.

Consultation liaison psychiatry (CLP) in India has seen a developing platform after refocusing mental health issues from asylum psychiatry to general hospital psychiatry. In this context, there is an extreme need to develop an assessment tool to detect psychiatric morbidity beyond depression and anxiety symptoms in medically ill patients in India. It is essential here to understand that screening tools for psychiatric morbidity in the general population cannot be used for those with medical illnesses, as these have items that have common features of medical illnesses, such as bodily symptoms, fatigue, loss of concentration, and disturbance in biological functions. The tool for screening psychiatric morbidity should not have such items, as these would lead to falsely enhanced scores. ¹⁴ Therefore, such a tool could include the domains related to stress, coping, depression, anxiety, spirituality, burden of treatment, psychosis, and substance use and should also be applicable in general hospitals and primary care. Developing and validating such a tool in India is a novel approach. It would reduce the physician’s burden of identifying mental health issues and referring them to appropriate specialists if required. The present study aims to fill this knowledge and research gap to detect psychiatric morbidity in medically ill patients in India.

Aim

The study aims to detect psychiatric morbidity in medically ill patients in India.

Methodology

Ethical Approval

The institute’s ethics committee approved the proposal for the present study (IHEC-LOP/2020/IM0252). The participant information sheet (PIS) and informed consent form (ICF) were designed in Hindi and English. Participants who gave written consent were recruited for the study.

Data Collection Procedure and Eligibility Criteria for Participants

Data was collected between December 2020 and January 2022. Patients aged 18 years or older with established medical diagnoses by a physician or specialist were invited to participate in the study. Patients requiring palliative care or those with mental retardation were excluded. Finally, 397 patients with varied medical illnesses as diagnosed by physicians at the hospital OPD consented to participate in the study. Two psychiatrists also interacted with these 397 patients and dichotomously classified the patients into two categories, with or without the presence of any psychiatric morbidities.

Steps Involved in Tool Development and Validation

For the development and validation of the tool, numerous steps based on the standard practice were followed (Figure 1). The steps are described in detail in the following subsections.

Figure 1. Steps Followed for Developing and Validating HMHS.

Step 1: Item Generation

As part of the deductive approach, a thorough literature review was conducted by five psychiatrists. Existing scales for detecting psychiatric illnesses and psychological issues among medically ill subjects were examined and reviewed, including GHQ, ¹⁵ PHQ, ¹¹ and the NIMHANS screening tool ¹⁶ for detecting psychological distress. Following this, the required items were selected cautiously with the addition of crucial psychiatric disorder domains that were commonly encountered, including depression, anxiety, psychosis, hyperactive state, OCD, substance use disorders, adjustment disorders, distress, coping, and perceived stress. Two experts (one psychiatrist and one physician) proficient in both Hindi and English performed forward and backward translation of the tool independently. A third expert resolved the queries and mismatches in the translations. Equivalence theory was followed for the translational process.

Considering the intense involvement of psychiatrists in the deductive approach of the tool development process, we included five medical specialists with face-to-face interaction to get rich insights from medical doctors. After this, an inductive approach was followed by circulating the tool to 20 medically ill patients to get feedback from prospective scale users through exhaustive in-depth interviews with 20 prospective scale users (medically ill patients), and the emerging themes were considered. This feedback helped us in fine-tuning the items.

Step 2: Content Validity by Expert Evaluation

The tool was then circulated to three experts (two psychiatrists and one psychologist) for computing the Item Content Validity Index (I-CVI) and Scale Content Validity Index/ Universal Agreement (S-CVI/UA) to strengthen the scientific rationality further. The experts were requested to rate each item in no face-to-face interaction on a 1–4 Likert scale as “1 = not relevant,” “2 = item needs revision,” “3 = relevant but needs minor revision,” or “4 = very relevant.” For each item, the I-CVI was calculated by summing the number of experts scoring 3 or above for relevancy and dividing the number by the total number of experts who participated, that is, eight. Subsequently, the items with an I-CVI of less than 79% were discarded. All items with an I-CVI above 79% were considered “appropriate.” Following this, the S-CVI/UA was calculated. It was computed by dividing the number of items by the I-CVI of one with the total number of items.

Step 3: Exploratory Factor Analysis

The exploratory factor analysis (EFA) was conducted to understand the latent constructs. The scale was finalized after experts circulated CVI to 397 medically ill patients. The scale had items on a 5-point Likert scale (0 = never, 1 = rarely, 2 = sometimes, 3 = mostly, 4 = always). Bartlett’s test and the Kaiser–Meier–Olkin Measure of Sampling Adequacy (KMO MSA) were done to check whether the data was suitable for factor analysis. A scree plot was created, and factors were described based on varimax rotation. For better concurrences between physician diagnosis and self-report diagnosis, the tool was designed to be self-rated, as self-report tools are found to complement physician understanding of patient symptom description. ¹⁷

Step 4: Reliability Using Cronbach’s Alpha

The tool’s internal consistency was measured by calculating Cronbach’s α. The item deletion method was used, and the overall and domain-wise values were computed.

Step 5: Concurrent Validity Using ROC

The concurrent criterion validity of the HMHS screening tool was calculated by the receiver operating curve (ROC) against the gold standard, which in the present study was the presence or absence of any psychiatric morbidity as diagnosed by two psychiatrists in 397 medically ill patients.

Statistical Analysis

Analysis was conducted using IBM SPSS version 23. The respondents’ descriptive statistics were calculated as the mean and standard deviation for continuous variables and frequency and percentages for categorical variables.

The KMO MSA was used to understand the data’s suitability for factor analysis. Principal component analysis (PCA) was done with varimax rotation, and a scree plot was generated. The factor loading threshold was set at 0.40. Reliability statistics were calculated domain-wise and for the overall scale with Cronbach’s α.

To understand the sensitivity and specificity of the HMHS screening tool, the ROC was computed using SPSS 23. For this purpose, the psychiatric diagnosis was coded as a dichotomous (present/absent) variable, and the score on the HMHS scale was calculated as a continuous variable. Each item on the scale was scored between 0 and 4 (0 for never to 4 for always). The final score was the summation of values for 27 items. Thus, the range of the score was between 0 and 108. The mean and SD for the scores were also computed from the 397 responses. We hypothesized that a lower score is predictive of a negative outcome or absence of psychiatric morbidity, and a high score is predictive of a positive outcome or presence of psychiatric morbidity in medically ill patients.

Results

The median age of the respondents was 42 years (IQR: 31–54 years). Sociodemographic details and general information are outlined in Table 1.

Table 1.

Sociodemographic Details of the Respondents (N = 397).

Category	Sub-category	Frequency (%)
Gender	Male	238 (59.9)
	Female	159 (40.1)
Residence	Rural	221 (55.7)
	Urban	176 (44.3)
Psychiatric diagnosis	Absent	222 (55.9)
	Present	175 (44.1)
Ward	Medicine	206 (51.9)
	Surgery	146 (36.8)
	Ophthalmology	20 (5)
	Pulmonary medicine	12 (3)
	ENT	10 (2.5)
	Orthopedics	3 (0.8)

Step 1: Item Generation

Initially, 34 items were generated via a deductive approach involving a literature review and an inductive approach involving circulating the tool to 20 medically ill patients. The 34 items were then given to eight experts for computation of content validity.

Step 2: Content Validity by Expert Evaluation

Seven items had I-CVIs below 79%. These seven items were discarded as with experts more than five, the I-CVI should be greater than or equal to 79%. Thus, after calculating the I-CVI, there were 27 items. The S-CVI/UA of the scale was calculated for these 27 items. Out of the 27 items, 23 had an I-CVI of 1. By dividing 23 by 27, the S-CVI/UA of the scale was calculated. It was above 80% (85.1%).

Step 3: Exploratory Factor Analysis

After computing the content validity, the 27 items were factor-analyzed. The 397 respondents rated these items on a 5-point Likert scale. The KMO MSA was found to be 0.916. Bartlett’s test of sphericity was significant with a value of 0.000 (chi-square of 4246.690 and df of 351), depicting that the data could be factor-analyzed.

We used PCA with varimax rotation. The factor loading threshold was set at 0.4. Six categories were identified based on the eigenvalue above 1 and factor loading of 0.40. A scree plot was also generated (Figure 2). The factors explained 62.244% of the variance. The distribution of items based on the six factors is described in Table 2.

Figure 2. Scree Plot.

Table 2.

Distribution of Items (Original Hindi Items Translated Here for Readers) Under Six Factors After Varimax Rotation.

Items	F1	F2	F3	F4	F5	F6
मैं अपने दिन का अधिकांश समय निष्फल कामों में गुज़ार देता/देती हूँ।	0.760
मुझे उन चीजों में मजा नहीं आता जिनमें पहले आता था।	0.665
मैं बेमतलब चीजों के बारे में बहुत ज्यादा सोचता/सोचती हूँ, जिन्हें मैं रोक नहीं सकता/सकती।	0.650
मैं अपने काम में ध्यान नहीं लगा पाता/पाती हूँ।	0.643
मुझे नींद आने में समस्या होती है।	0.614
मैं ज्यादा देर तक इंतजार नहीं कर सकता/सकती और अधीर हो जाता/जाती हूँ।	0.535
मैं अपने स्वास्थ्य/मित्रों वगैरह से जुड़ी दैनिक समस्याओं को सुलझा नहीं पा रहा/रही हूँ।	0.446
मुझे डर है कि मेरी बीमारी वापस आ जाएगी।		0.743
मैं अपने इलाज/शल्य क्रिया (सर्जरी) के खर्च को लेकर बहुत चिंतित हूँ।		0.730
मुझे तनाव में अक्सर दर्द होता है।		0.668
मेरी बीमारी मेरे परिवार के लिए एक बोझ है।		0.621
मुझे बिना किसी कारण के बेहद घबराहट होती है।		0.507
मैं कुछ ऐसी चीजें/ऐसे लोग देख सकता/सकती हूँ जो कोई और नहीं देख सकता।			0.833
मुझे लगता है कि लोग मुझे नुकसान पहुंचाना चाहते हैं।			0.559
मुझे इतनी बेचैनी होती है कि मैं देर तक एक जगह पर नहीं बैठ सकता/सकती।			0.494
मुझे लगता नहीं कि मेरा कोई भविष्य है।			0.445
मुझे सार्वजनिक जगहों पर बहुत बेचैनी/असुविधा महसूस होती है, जैसे कि कोई मुझे देख रहा हो।			0.408
मैं शराब/धूम्रपान/अन्य नशीले पदार्थों के सेवन की वजह से अपनी जिम्मेदारियां (अपना और अपने परिवार का ख्याल रखना) नहीं निभा पा रहा/रही हूँ।				0.788
मैं शराब/ड्रग/धूम्रपान के सेवन से अपने जीवन की समस्याओं का समाधान कर सकता/सकती हूँ।				0.746
मैं अपने शराब/धूम्रपान/अन्य नशीले पदार्थों के इस्तेमाल को काबू में नहीं कर पा रहा/रही हूँ।				0.739
मैं कुछ चीजें बार बार करता/करती हूँ जिन्हें मैं रोक नहीं सकता/सकती, जैसे कि बार बार हाथ धोना, बार बार देखना कि ताला बंद है कि नहीं।					0.728
मुझे अपने बारे में या मुझसे बात करते हुए लोग सुनाई देते हैं, जब कोई आस पास मौजूद भी नहीं होता।					0.483
मैं अपने जीवन की समस्याओं को आध्यात्मिक/धार्मिक तरीके से हल कर सकता/सकती हूँ।					0.468
मुझे बिना किसी वजह के अपने दिल की धड़कन तेज महसूस होती है, मेरे हाथ कांपने लगते हैं, और पसीना भी आने लगता है।					0.420
मुझे अपनी जिंदगी ख़त्म करने का मन होता है।						0.746
मुझे अपने ऊपर बिना किसी वजह के गुस्सा आता है।						0.456
मेरे व्यक्तिगत, सामाजिक, या व्यवसायिक जीवन में बहुत लड़ाई होती हैं।						0.428

Step 4: Reliability Using Cronbach’s Alpha

Factor-wise reliability was computed for each category. The details of the same are described in Table 3.

Table 3.

Cronbach’s Alpha for Each Factor.

Category	Number of Items	Cronbach’s Alpha
Factor 1	7	0.840
Factor 2	5	0.806
Factor 3	5	0.744
Factor 4	3	0.702
Factor 5	4	0.657
Factor 6	3	0.673

Reliability was found to be high in factor one, which has seven items (00.840), and in factor two, which has five items (00.806). In factor three, which has five items, and factor four, which has three items, reliability is found to be good, with values of 00.744 and 00.702, respectively. It is in the moderately acceptable range for factors five and six, which have four and three items, with values of 00.657 and 00.673, respectively.

Step 5: Concurrent Validity Using ROC

The mean and SD score for the HMHS scale on 27 items was 21.05 ± 16.94. Psychiatrists dichotomously categorized the patients as medically ill patients with the presence or absence of psychiatric morbidity. The test variable for the ROC curve was the HMHS score of 27 items, and the psychiatric diagnosis given by psychiatrists was set as the state variable. The ROC curve was generated to understand visually the trade-off between a true positive rate (TPR) and one minus false positive rate (FPR). At a threshold score of 19.5, the sensitivity was found to be 0.914. Thus, at a cut-off score of 19.5, 91.4% of the positive outcomes are expected to be correctly classified and identified by the HMHS screening tool (true positive rate). The score of (1 − specificity) was found to be 0.095 at a threshold of 19.5, depicting that 9.5% of the adverse outcomes are incorrectly identified by the HMHS screening tool (false positive rate). Figure 3 depicts the ROC in detail.

Figure 3. ROC Curve.

The ROC curve is toward the top extreme left, indicating the instrument has good screening properties. Out of 397 patients who were medically ill, 181 had an HMHS score of more than 19.5. Thus, based on HMHS, psychiatric morbidity was present in 45.5% of medically ill patients. Supplementary Table 4 shows the area under the curve and related information. The area under the curve was high, with a value of 0.966 and a standard error of 0.009 (asymptotic significance of 0.000). The 95% CI was 0.949–0.983. AUC was greater than 0.9, indicating that the HMHS is an excellent test with desirable sensitivity and specificity (90.5%) characteristics and superior discrimination.

If data on patients screened negative for questions on depression but positive for questions on suicidality is available, we can look for findings.

Discussion

Our findings confirm that the HMHS is a valid and reliable tool with 27 items. It has good screening properties and is customized for Indian settings. It is a unique and novel scale that can assist in detecting psychiatric morbidity in medically ill patients. It can also be used in primary healthcare and general hospital settings. Being self-rated, this tool can facilitate the work of busy physicians in timely identifying psychiatric morbidity in medically ill patients and referring them to befitting psychiatrists. In a comparable study, a computer-assisted semi-structured tool was validated in India for comprehending psychiatric morbidity in primary care settings. That tool had good diagnostic properties (sensitivity 1.00 and specificity 0.94) but had the limitations of a small sample size of 82. ¹⁸

Furthermore, in the present study, the I-CVI threshold was set at 0.79 and S-CVI/UA at 80%, which ensured that the instrument had an appropriate pool of items.^19-21 In addition, EFA in the present study was conducted on 397 participants (approximately 14 for each item), which suggests a good sample size for validation as enumerated in numerous studies.^22,23 The reliability of items 1, 2, 3, and 4 was suggestive of good internal consistency (Cronbach’s α values were above 0.7); however, for factors 5 and 6, it was in the moderately acceptable range (Cronbach’s α values were between 0.65 and 0.7). ²⁴

Various studies have been conducted so far that have shown the importance of assessing psychiatric morbidity in medically ill patients. A recent study used an application named GMHAT for the assessment of depression and anxiety in diabetes patients; though useful, it may not screen for the other major psychiatric morbidities as in HMHS. Further, HMHS is a screening tool that essentially excludes somatic symptoms (overlapping physical and mental symptoms), enhancing its utility in adults as well as adolescent populations. There is an increasing trend in recognizing the co-occurrence of mental disorders with chronic medical illnesses such as diabetes mellitus, cardio-metabolic diseases etc.; therefore, for better integration of physical and mental health in primary healthcare, a new screening tool is going to be helpful. A study conducted in the Netherlands concluded that 45% of medically ill patients whose illnesses were of a medical or surgical nature had some form of psychiatric disorders. ²⁵ This was comparable to our study, which showed around 45.5%. Though several studies have highlighted the importance of understanding psychiatric morbidity in medically ill patients,^26,27 to the best of our knowledge and experience, this is a first-of-its-kind study in an Indian setting, which was conducted on a larger sample of patients. We attempted to develop a valid and reliable scale that exhibited good screening properties for screening psychiatric morbidity even in a low-resource setting. Furthermore, HMHS has a screener for psychosis, which is an absolute requirement for general health and primary health settings aligning with the model of integrated healthcare. The HMHS, in our view, supports the objectives of sustainable development goals, which are about good health and well-being and industry, innovation, and infrastructure. A majority of low- and middle-income countries would have similar healthcare infrastructure and a possibly higher prevalence of psychiatric morbidities in medically ill patients. Therefore, more research is expected on HMHS for wider acceptability in addition to the standard tools mentioned in the literature.

Limitations

The initial items generated should ideally be twice the final scale; however, our initial items were 34 and the final items were 27. We could not perform a CFA due to logistic constraints.

Conclusion

Keeping in view the prevailing psychiatric commodities in medically ill patients, it is essential to think of a customized scale focusing mainly on screening psychiatric disorders and timely referring the patients to psychiatrists so that their befitting treatment can be initiated. The HMHS can be used as a screening tool in India’s primary healthcare and general hospital settings, considering its good screening properties and satisfactory validity and reliability features. There is also a potential scope for conducting CFA so that the developed factor structure can be further reinforced. The tool can also be reintroduced in illiterate populations through pictorial representations similar to the community mental health education and detection (CMED) tool.