A Way to Increase the Sensitivity and Specificity of the Hamilton Depression and Anxiety Scales

Abstract

Objective:

The Hamilton Depression Rating Scale (HDRS-17) and the Hamilton Anxiety Rating Scale (HARS-14) have been acknowledged as gold standards in evaluating the severity of depression and anxiety. The specificity and sensitivity of these scales in predicting somatic complaints of depression and anxiety are issues in both clinical and research areas. The present study proposes a new model to enhance the sensitivity and specificity of HDRS-17 and HARS-14 for predicting symptoms of insomnia, inappetence, and loss of libido in psychiatric patients.

Methods:

This study included 1507 patients diagnosed withbipolar disorder, depression, panic disorder, obsessive-compulsive disorder, and generalized anxiety disorder. The HDRS-17 and the HARS-14 were utilized as predictive scales for the prediction of patients’ sleep, appetite, and libido. The sensitivity and specificity were computed using the receiver operating characteristic (ROC). Logistic regression was performed to enhance the predictive values. The predictive value of the logistic regression model was not satisfactory, and a conversion table was therefore designed for each symptom-diagnosis subgroup. The new joint ROC model was then used to recalculate the sensitivity and specificity of the 2 scales for each symptom-diagnosis subgroup. The outcome is a prediction table, presented for use by clinicians.

Results:

It was observed that the new statistical model, the joint ROC, increased the sensitivity and specificity of the HDRS-17 and the HARS-14.

Conclusion

: Based on the results of the evaluations with the HDRS and the HARS, the joint ROC method was developed to better predict the presence of symptoms.

Introduction

The Hamilton Anxiety Rating Scale (HARS, also termed HAM-A) and the Hamilton Depression Rating Scale (HDRS, also termed HAM-D) have been the most widely used, clinician-rated, semi-structured measurements in psychiatric practice and research. Originally presented by Max R Hamilton in 1959 (HARS)¹ and 1960 (HDRS),² they have been preferred to measure the severity of symptoms of depression and anxiety in patients diagnosed with various psychiatric problems.

The HARS consists of 14 items pertaining to somatic and psychological symptoms, including anxious mood, depressed mood, tension, insomnia, somatic symptoms, problems in the intellectual, sensory, cardiovascular, respiratory gastrointestinal, genitourinary, or autonomic systems, and the behavior observed at interview (fidgety, restless, etc.). Each item is scored on a scale from 0 (not present) to 4 (very severe), with a total score range of 0-56. A total score <17 represents mild anxiety, 18-24 indicates mild to moderate anxiety, and <25-30 indicates moderate to severe anxiety.³

Similarly, the original version of the HDRS consists of 17 elements to measure the severity of depressed mood, feelings of guilt, suicide, insomnia, capability of work and activities, retardation in speech and thought, agitation, anxiety (psychological and somatic), gastrointestinal, genital or general symptoms, hypochondriasis, loss of weight, and insight about the illness. More than half of the items are rated between 0 and 4, as in the HARS. The rest are scored from 0 to 2, except for weight loss (0-3). Patients with a score of 0-7 are considered normal or in remission period. A total score > 20 is considered to indicate at least moderate to severe depression.⁴

Both the HARS and the HDRS have been accepted as gold standards in psychiatric practice for 40 years, due to their psychometric properties. Being applied to numerous patients with various psychiatric diagnoses, their psychometric properties vary depending on the version used in the study. A meta-analysis revealed that the HDRS-17 is a valid and sensitive index; however, its structured versions should be preferred.⁵ After a structured interview guide for the Hamilton Depression and Anxiety Scale (SIGH-D and SIGH-A) was released, the reliability of the scales increased.^6,7 Even though they are accepted as gold standards in measuring symptomatology of depression and anxiety, they have some problems for which solutions have been attempted with revisions and modifications.

Unlike the HDRS, the main problem with the HARS is that it consists of a group of symptoms rather than a single symptom. For instance, symptoms related with difficulty in falling asleep, broken sleep, unsatisfying sleep and fatigue on waking, dreams, nightmares, and night terrors are classified under the category of insomnia in the HARS. Additionally, a third of the items in the HARS were found to be related with depression. Finally, the inclusion of depression-related items in the HARS makes it difficult to interpret anxiolytic treatment interventions which also used in depression.⁸

Likewise, the HDRS scale has some limitations that create problems with standardization. Although Hamilton stated that the last 4 items added to the 17-item scale (HDRS-17) should not be counted in total scores, the HDRS-21 is frequently reported in many studies.⁹ Further, issues related with scoring guidelines and criteria have become problematic for HDRS-17. While some items are scored between 0 and 2 points, the others take between 0 and 4 points, which in turn affect the sum total scores. These shortcomings raised concerns in some researchers, who then tried to develop item changes and revised scoring methods for HDRS.¹⁰ Factor analysis applied to the original version of HDRS-17 revealed that the depression severity scale is not a unidimensional scale, and consisted of factors between 2 and 8. Researchers claimed that the multidimensionality of the HDRS restricts measuring the severity of symptoms pertaining to depression.¹¹ When physical illnesses are treated with medications, for example, sedative hypnotics used for insomnia and anxiety, there will be a possible decrease in HDRS factors relevant with somatic complaints, which makes it difficult to discriminate the effects of psychiatric treatment from physical treatment.¹²

Both scales include the core physical symptoms of depression—insomnia, lack of sexual desire, and appetite. Sleep disturbance is a causal and risk factor in various psychiatric disorders. Therefore, treatment interventions for sleep problems are also crucial in ameliorating and preventing psychiatric symptoms.¹³ Sexual dysfunction is also a common problem in depression. Prior to SSRI prescription, approximately 40-70% of patients with MDD report sexual problems.¹⁴ Loss of appetite is a characteristic of depression.¹⁵

Indeed, these symptoms are not intrinsic to depression. Lowered sexual desire is a common problem in bipolar disorder, panic disorder, obsessive-compulsive disorder, and anxiety disorders.^14,15 Further, the prescription of SSRI and SNRI medications may lead to loss of sexual desire for some patients even after treatment.¹⁶ As for loss of appetite, the appetite hormones are sensitive to acute and chronic stress, and thus, appetite may be increased or decreased in patients with anxiety.¹⁷ Although patients with unipolar depression more frequently suffer from inappetence and insomnia than bipolar patients, patients with bipolar disorder tend to report inappetence and insomnia together.¹⁸ Patients with OCD and panic disorder comorbid with depression may suffer from appetite and sleep problems.^19-21

Given that there is a standardization problem for the HDRS and HARS scales both clinically and statistically, we aimed to present a new statistical model that can enhance the predictive values of the 2 rating scales (HDRS and HARS) of symptomatology for insomnia, inappetence and loss of libido in numerous psychiatric patients.

Methods

Subjects

This retrospective study was confirmed by the Ethics Committee of Uskudar University Non-Interventional Research Ethics Board (Approval No: 61351342/January 2021-38). The participants consisted of individuals who consulted a private psychiatric practice in Istanbul, Turkey. They were all diagnosed according to the Structural Clinical Interview for DSM-5 by the same psychiatrist.²² Totally, 1507 patients diagnosed with one of the following mental disorders were examined: bipolar disorder, depression, panic disorder, obsessive-compulsive disorder, and generalized anxiety disorder (Table 1). The inclusion criteria for these patients were as follows:

Table 1.

Number of Patients Based on Diagnosis

Diagnosis	N	%
Bipolar disorder	193	12.8
Depression	530	35.2
Panic disorder	156	10.4
OCD	259	17.2
GAD	369	24.5
Total	1507	100.0

OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

1. First interview between April 2016 and September 2020;

2. Diagnosis at first interview;

3. A 17-item HDRS-17 and a 14-item HARS-14 measurement at the first interview; and

4. Drug-free for at least more than 3 weeks.

Statistical Analysis

All statistical analyses were performed with Statistical Package for the Social Sciences (SPSS) version 25.0 (IBM SPSS Corp.; Armonk, NY, USA).

Step 1—Estimating Cut-off Points for HDRS and HARS: Thevalidity values, the accuracy of the gold standard scales, were investigated. The gold standard is the best suitable test to detect a specific symptom or diagnosis. Sensitivity and specificity are the 2 determinants of validity. In the clinical context, while sensitivity refers to the ability of a test to detect a diseased person, specificity refers to the ability of the test to truly discriminate a healthy person. Accuracy is the ability of a test to accurately detect the healthy and diseased people in proportion to the total diagnoses. They are best represented in a 2×2 table (Table 2). The predictive values of a test are also determined by comparing the test results with gold standards. The negative predictive value is the proportion of true negatives determined by both the gold standard and the test, to the total test negatives, including the false results. Similarly, the positive predictive value of a test is the proportion of true positives according to both the gold standard and the test, to the total negative results of the test, including the false results.²³

Table 2.

2 × 2 Table for Sensitivity, Specificity, Negative Predictive Value, and Positive Predictive Values of a Test

	Test Negative (No)	Test Positive (Yes)
Gold standard disease absent (No)	True negatives (TN)a	False positive (FP)b	Total absence of disease a+b
Gold standard disease present (No)	False negative (FN)c	True positives (TP)d	Total presence of disease c+d
Total	Total test negatives a+c	Total test positives b+d	Total diagnosis a+b+c+d

Sensitivity(d/d+c) indicates the proportion of true positives (d) to total presence of disease (c+d). Specificity (a/a+b) refers to the proportion of true negatives (a) to the total number of patients diagnosed with absence of disease (a+b). Accuracy ((a+d)/(a+b+c+d)) is the proportion of true positives and true negatives to total diagnoses. Negative predictive value is the proportion of true negatives determined by both the gold standard and the test (a) to the total test negatives including the false results (a+c). The positive predictive value is the proportion of true positives detected by both the gold standard and the test (d) to the total positive results of the test including the false results (b+d).

The performance of the diagnostic test was also measured by the Receiver Operating Characteristic (ROC) curve. The ROC curve is the plot of the true positive ratio (sensitivity) versus the false positive ratio (1 − specificity) across varying cut-offs in the unit square. The performance of the diagnostic test manifests in the ROC curve as the test curve stretching to the top of the y-axis, thus moving away from the diagonal line of sensitivity and the 1 − positivity ratio, that is, the chance factor of diagnosis.²³ The area under curve (AUC) is the total area under the ROC curve. The area takes a value ranging from 0 to 1, in which 0 represents an ineffective test while 1 represents a perfect test. It is considered an effective way to summarize the overall diagnostic accuracy of the test.²⁴

The HDRS-17 and the HARS-14 were utilized as predictive scales to predict a decrease in patients’ sleep, appetite, and libido. A symptom-diagnosis match was made via the ROC curve. The AUC values were checked for the interpretation of predictive results of the HDRS and HARS scales. Cut-off values were determined by the Youden index.

Step 2—Logistic Regression Analysis: Logistic regression analysis was then performed to enhance the predictive values determined in step 1. The HDRS and HARS were taken as predictors.

Step 3—Conversion Table and joint ROC model: The predictive value of the logistic regression model was unsatisfactory, and therefore a conversion table (Table 3) was designed, considering symptom presence, AUC, and cut-off values. A major index and a minor index for each symptom-diagnosis subgroup was determined based on the higher AUC values observed in step 1. To illustrate, the AUC value of HARS is higher than for HDRS for sleep symptoms in patients with bipolar disorders. Thus, HARS becomes the major index and HDRS becomes the minor index for the sleep-bipolar subgroup in the conversion table. Conversion values are produced ad hoc. In the conversion table, two-thirds and one-third of the value are given for the major index and minor index, respectively. The results do not change even if the values were changed ±3/4 for the major index and ±1/4 for minor index.

Table 3.

Conversion Table Produced by Combining the 2 ROC Curves

Row	Gold standard	Major Index	Minor Index	Conversion
1a	0	0	0	−1.00
2b	0	0	1	−0.67
3c	0	1	0	−0.33
4d	0	1	1	0.01
5d	1	0	0	−0.01
6c	1	0	1	0.33
7b	1	1	0	0.67
8a	1	1	1	1.00

The rows represent all the possible prediction possibilities determined by the gold standard, major index, and minor index. The row numbers of the conversion matrix indicate:

^aExact match:

First-Row: Symptom is absent. The number of patients for whom the absence of symptoms was predicted by both the major index and minor index. Conversion value: −1.

Eighth-Row: Symptoms are present. The number of patients for whom the presence of symptoms was predicted by both the major index and minor index. Conversion value: 1.

^bMajor match:

Second-Row: Symptom is absent. The number of patients for whom the absence of symptoms was predicted by the major index but not by the minor index. Conversion value: −0.67.

Seven-Row: Symptom is present. The number of patients for whom the presence of symptoms was predicted by the major index but not by the minor index. Conversion value: 0.67.

^cMinor Match:

Third-Row: Symptom is absent. The number of patients for whom the absence of symptoms was predicted by the minor index but not by the major index. Conversion value: −0.33.

Sixth-Row: Symptom is present. The number of patients for whom the absence of symptoms was predicted by the minor index but not by the major index. Conversion value: 0.33.

^dContradiction:

Fourth-Row: Symptom is absent. The number of patients for whom the absence of symptoms was not predicted by the major index and the minor index. Conversion value: 0.01.

Fifth-Row: Symptom is present. The number of patients for whom the presence of symptoms was not predicted by the major index and the minor index. Conversion value: −0.01.

An example of the calculation for conversion values

A minor match for the third row of which the conversion value is 0.33: A bipolar disorder patient without insomnia (Gold standard: 0). The major index for sleep-bipolar disorder is HARS (see Table 2), which does not predict +0.67, while the minor index HDRS predicts −0.33. The total conversion value becomes +0.67 – 0.33 = 0.33

After all the symptom-diagnosis pairs are reanalyzed by the new joint ROC model, the sensitivity and specificity values of the 2 scales together are recalculated for all the symptom-diagnosis subgroups.

Step 4 Logistic regression with joint ROC data

The sensitivity and specificity values calculated in step 3 are added to the logistic regression model. Next, 2 estimated P values are obtained. The first value represents the probability of presence of the related symptom in the patient with the relevant diagnosis. The second value indicates the probability of absence of the related symptom with the relevant diagnosis. It should be indicated that these values are independent from each other.

Step 1 Estimating the cut-off points for HDRS and HARS

The HDRS-17 and HARS-14 were utilized as predictive scales to predict the decrease in patients’ sleep, appetite, and libido. A symptom-diagnosis match was made via the ROC curve. AUC values were taken as predictive values. The cut-off values were determined by the Youden index (Table 4).

Table 4.

AUC and Cut-Off Values of HDRS and HARS for Insomnia, Inappetence, and Loss of Libido

Symptom-Diagnosis	HDRS		HARS		N
	AUC	Cut-Off	AUC	Cut-Off
Insomnia
Bipolar disorder	0.730	6.5	0.781	10.5	173
Depression	0.757	21.5	0.792	26.5	467
Panic disorder	0.642	8.5	0.702	20.5	139
OCD	0.736	9.5	0.781	15.5	241
GAD	0.728	6.5	0.734	17.5	341
Inappetence
Bipolar disorder	0.752	12.5	0.788	14.5	163
Depression	0.765	18.5	0.742	21.5	497
Panic	0.675	13.5	0.656	13.5	152
OCD	0.768	15.5	0.767	19.5	245
GAD	0.754	8.5	0.734	16.5	341
Loss of libido
Bipolar disorder	0.783	14.5	0.733	16.5	169
Depression	0.699	22.5	0.695	25.5	527
Panic	0.626	5.5	0.610	13.5	155
OCD	0.765	7.5	0.729	19.5	257
GAD	0.700	7.5	0.674	22.5	363

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; AUC, area under curve; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Step 2 Logistic Regression Analysis

As is seen in Table 2, the predictive values of HDRS and HARS alone were insufficient for some symptom-diagnosis pairs, for example, panic disorder-sleep, appetite, and libido. Logistic regression analysis was performed by taking the HDRS and HARS as predictors. Although there was an improvement observed in specificity and sensitivity values, they were unable to reach a sufficient point (Table 5a and b).

Table 5a.

Number of Patients Predicted by HDRS, HARS and Logistic Regression for Sleep, Appetite, and Libido Symptoms Based on Diagnosis

Diagnosis-Symptom	HDRS		HARS		Log-Reg
	No	Yes	No	Yes	No	Yes
Insomnia
Bipolar disorder
No	55	26	57	24	59	22
Yes	28	64	22	70	30	62
Depression
No	91	29	98	22	46	74
Yes	128	219	131	216	27	320
Panic disorder
No	44	26	42	28	43	27
Yes	24	45	20	49	25	44
OCD
No	96	46	103	39	113	28
Yes	31	69	28	72	46	54
GAD
No	101	74	122	53	131	44
Yes	34	132	55	111	65	101
Inappetence
Bipolar disorder
No	70	26	66	30	79	17
Yes	22	45	16	51	27	40
Depression
No	123	97	115	105	140	80
Yes	46	231	48	229	70	207
Panic disorder
No	78	24	34	68	91	11
Yes	23	27	4	46	35	15
OCD
No	147	28	137	38	164	11
Yes	24	46	24	46	41	29
GAD
No	144	95	134	105	217	22
Yes	19	83	22	80	63	39
Loss of libido
Bipolar disorder
No	76	14	71	19	69	21
Yes	29	50	25	54	27	52
Depression
No	147	68	133	82	100	115
Yes	119	193	102	210	65	247
Panic disorder
No	42	59	33	68	94	7
Yes	11	43	7	47	43	11
OCD
No	101	70	131	40	150	21
Yes	14	72	34	52	56	30
GAD
No	135	121	191	65	241	15
Yes	21	86	54	53	79	28

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Step 3 Conversion table and joint ROC model

When the joint ROC model was applied to all symptom-diagnosis subgroups, the sensitivity and specificity values showed a considerable increase for insomnia (Table 6a and b), inappetence (Table 7a and b), and loss of libido (Table 8a and b).

Table 5b.

Sensitivity and Specificity Values of HDRS and HARS Separately and Together in Logistic Regression

Symptom-Diagnosis	HDRS			HARS			Log-Reg
	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)
Insomnia
Bipolar disorder	67.9	69.6	68.8	70.4	76.1	73.4	72.8	67.4	69.9
Depression	75.8	63.1	66.4	81.7	62.2	67.2	38.3	92.2	78.4
Panic disorder	62.9	65.2	64.0	60.0	71.0	65.5	61.4	63.8	62.6
OCD	67.6	69.0	68.2	72.5	72.0	72.3	80.1	54.0	69.3
GAD	57.7	79.5	68.3	69.7	66.9	68.3	74.9	60.8	68.0
Inappetence
Bipolar disorder	72.9	67.2	70.6	68.8	76.1	71.8	82.3	59.7	73.0
Depression	55.9	83.4	71.2	52.3	82.7	69.2	63.6	74.7	69.8
Panic disorder	76.5	54.0	69.1	33.3	92.0	52.6	89.2	30.0	69.7
OCD	84.0	65.7	78.8	78.3	65.7	74.7	93.7	41.4	78.8
GAD	60.3	81.4	66.6	56.1	78.4	62.8	90.8	38.2	75.1
Loss of libido
Bipolar disorder	84.4	63.3	74.6	78.9	68.4	74.0	76.7	65.8	71.6
Depression	68.4	61.9	64.5	61.9	67.3	65.1	46.5	79.2	65.8
Panic disorder	41.6	79.6	54.8	32.7	87.0	51.6	93.1	20.4	67.7
OCD	59.1	83.7	67.3	76.6	60.5	71.2	87.7	34.9	70.0
GAD	52.7	80.4	60.9	74.6	49.5	67.2	94.1	26.2	74.1

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; SPE, Specificity; SEN, Sensitivity; ACU, Accuracy; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Table 6a.

Number of Patients Whose Insomnia Symptom is Predicted by the Joint ROC Model

Diagnosis-Symptom	HDRS		HARS		Log-Reg
	No	Yes	No	Yes	No	Yes
Insomnia
Bipolar disorder
No	55	26	57	24	59	22
Yes	28	64	22	70	30	62
Depression
No	91	29	98	22	46	74
Yes	128	219	131	216	27	320
Panic disorder
No	44	26	42	28	43	27
Yes	24	45	20	49	25	44
OCD
No	96	46	103	39	113	28
Yes	31	69	28	72	46	54
GAD
No	101	74	122	53	131	44
Yes	34	132	55	111	65	101

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Table 6b.

Sensitivity and Specificity Values of HDRS and HARS by the Joint ROC Model for Insomnia

Symptom-Diagnosis	HDRS			HARS			Log-Reg			Joint ROC
	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)
Insomnia
Bipolar disorder	67.9	69.6	68.8	70.4	76.1	73.4	72.8	67.4	69.9	100.0	82.6	90.8
Depression	75.8	63.1	66.4	81.7	62.2	67.2	38.3	92.2	78.4	87.5	100.0	96.8
Panic disorder	62.9	65.2	64.0	60.0	71.0	65.5	61.4	63.8	62.6	100.0	81.2	90.6
OCD	67.6	69.0	68.2	72.5	72.0	72.3	80.1	54.0	69.3	100.0	79.0	91.3
GAD	57.7	79.5	68.3	69.7	66.9	68.3	74.9	60.8	68.0	100.0	84.3	92.4

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; SPE, Specificity; SEN, Sensitivity; ACU, Accuracy; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Step 4 Logistic Regression with joint ROC data

The last 3 tables (Table 9a, b, and c) were designed for psychiatrists to decide whether patients with both HDRS and HARS scores would develop insomnia, loss of appetite and loss of libido. For each diagnosis (bipolar disorder, depression, panic disorder, OCD, and GAD) and symptom (insomnia, inappetence, and loss of libido), cut-off values for HDRS and HARS, obtained by the ROC and AUC method in step 1, were provided. Under the HDRS and HARS columns, scores below the cut-off values represent the absence of symptom (No), while scores above the cut-off values represent the presence of symptoms (Yes).

Table 7a.

Number of Patients Whose Inappetence Symptom is Predicted by the Joint ROC Model

Diagnosis-Symptom	HDRS		HARS		Log-Reg
	No	Yes	No	Yes	No	Yes
Inappetence
Bipolar disorder
No	70	26	66	30	79	17
Yes	22	45	16	51	27	40
Depression
No	123	97	115	105	140	80
Yes	46	231	48	229	70	207
Panic disorder
No	78	24	34	68	91	11
Yes	23	27	4	46	35	15
OCD
No	147	28	137	38	164	11
Yes	24	46	24	46	41	29
GAD
No	144	95	134	105	217	22
Yes	19	83	22	80	63	39

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

The final probability column in the table indicates 2 P values obtained by adding the outputs of the conversion matrix calculated in step 3 to the logistic regression. The first value represents the probability at which the patient would develop the relevant symptom depending on their HDRS and HARS scores being under or above the cut-off values. The second value represents the probability of not developing the symptom under the same conditions. The P values range from 0 to 1. Zero represents the absolute absence of the symptom, while one represents the absolute presence of the symptom. N/A indicates that the P values could not be calculated since there is no patient in the related symptom-diagnosis group providing HDRS and HARS scores under the relevant cut-off values.

To clarify the commentary of table, the following example can be observed. For bipolar disorder diagnosis, the probability of developing insomnia can be found as follows: the cut-off value determined in step 1 for HARS is 10.5 and for HDRS is 6.5. The scores of HARS <10.5 and of HDRS <6.5 indicate the absence of insomnia, stated as “No” in the table. Similarly, the scores of HARS> 10.5 and of HDRS> 6.5 indicate the presence of insomnia, stated as “Yes”. Accordingly, if we look at the end of the row presenting the absence of insomnia for bipolar disorder, it can be observed that the probabilities of developing and not developing insomnia are 0.42425 and 0.99993, respectively. Thus, we can conclude that a BD patient with HARS and HDRS scores below 10.5 and 6.5 is more likely than not to develop insomnia.

Another condition, for bipolar disorder: the presence of insomnia is detected by HDRS but not by HARS. Then, according to the joint ROC model, the probability of developing insomnia is 0.94793, and for not developing insomnia is 0.99840. Therefore, a clinician can say to the patient that the probability of not developing insomnia is slightly higher than for developing insomnia. Unlike the former model, which assumes the sum of 2 probabilities as 1, our new model proposes 2 different probabilities independent from each other. Accordingly, a clinician should make the predictions by considering the 2 different probabilities separately.

Discussion

HDRS and HARS are semi-gold standards in measuring the severity of symptoms related with depression and anxiety. Nonetheless, they have some statistical shortcomings that need to be addressed. The main step is the logistic regression used for sensitivity and specificity analyses. The reason why logistic regression fails to increase sensitivity and specificity is that logistic regression aims to explain one result, that is, the presence or absence of a symptom, by considering the 2 variables, HDRS and HARS. Another point is that different AUC values do not provide a practical meaning for clinicians, despite being convenient indices of diagnostic tests.²⁵ The present study proposed that the sensitivity and specificity of the combination of HDRS-17 and HARS-14 through the joint ROC model provide more accurate information to psychiatric practitioners regarding the developing of insomnia, inappetence, and loss of libido in psychiatric patients as it increased sensitivity and specificity values for each symptom-diagnosis subgroup.

One of the strengths of the present study is the examination of patients. All the patients were examined by the same physician. All the clinical diagnoses were made at the first interview. Besides, all the patients were drug-free for at least 3 weeks. Finally, the new joint ROC model was developed from a considerable sample size.

The limitations of the study could be that the groups are not homogenous with respect to diagnosis. We aimed to reach a moderate sample size for each diagnostic group. Since there was a scarcity of patients with other clinical diagnoses such as schizophrenia, attention deficit disorder, post-traumatic stress disorder, and addiction disorders, we decided to study the sample for 5 diagnoses. Future studies may attempt to apply the method to the patients with unstudied diagnoses.

Another limitation could be the restrictions of the self-report technique for HDRS and HARS. Although these scales are clinician-administered semi-structured scales, the clinicians evaluate patients’ verbal statements as responses. Nevertheless, the somatic symptoms of sleep, appetite, and libido are less likely to be affected by poor insight compared with anxious and depressive thoughts.

The main advantage of the joint ROC model is its ability to show the probability at which patients can (or not) develop the symptom, While HDRS and HARS manifest the presence (or absence) of the symptom, our model enriches the prediction by proposing the probabilities of developing (or not developing) the specific symptom. Another advantage of this model is its increased sensitivity and specificity values compared with those of HDRS and HARS per se. In previous studies, some shorter versions of HDRS have been developed to accurately measure the severity of appetite, insomnia in depression.⁹ It can be a better method to use a predictive model to decide the severity of symptoms rather than making modifications.

Although the HDRS and HAMD are not used in diagnosis, they have clinical importance in identifying the severity of symptoms and effects of treatments. Insomnia, inappetence, and loss of libido are significant symptoms to be checked for developing the appropriate treatment strategy. According to a WHO report, insomnia and sleep problems are one of the most reliable biomarkers of depression.¹⁵ As for appetite, it is important to detect metabolic syndrome in psychiatric disorders.²⁶ Finally, libido is checked in follow-up treatments, since antidepressant medications lead to post-SSRI sexual dysfunction.²⁷

Conclusion

The present study aimed to enhance the predictive values of the HDRS and HARS with respect to the 3 symptoms: insomnia, inappetence and loss of libido. The logistic regression model could not lead to expected improvement in the sensitivity and specificity, since the model tries to explain 1 result with 2 variables. Therefore, a new statistical model, the joint ROC, was developed. The model defines the 2 scales as major and minor scales according to the sensitivity and specificity values calculated by the McNemar’s table. The results show that the new model increased the sensitivity and specificity of the 2 scales by up to 90-100%. A practical table is also provided indicating the probability of a patient with the HDRS and HARS scores would develop the related symptom or not. The model can/should also be applied to other psychiatric patients. Given that the studied symptoms are important elements in the treatment of psychiatric disorders, it is proposed that the joint ROC model could considerably aid in clinical areas as well.

Table 7b.

Sensitivity and Specificity Values of HDRS and HARS by the Joint ROC Model for Inappetence

Diagnosis	HDRS			HARS			Log-Reg			Joint ROC
	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)
Inappetence
Bipolar disorder	72.9	67.2	70.6	68.8	76.1	71.8	82.3	59.7	73.0	100.0	77.6	90.8
Depression	55.9	83.4	71.2	52.3	82.7	69.2	63.6	74.7	69.8	100.0	91.7	95.4
Panic disorder	76.5	54.0	69.1	33.3	92.0	52.6	89.2	30.0	69.7	100.0	92.0	97.4
OCD	84.0	65.7	78.8	78.3	65.7	74.7	93.7	41.4	78.8	100.0	72.9	92.2
GAD	60.3	81.4	66.6	56.1	78.4	62.8	90.8	38.2	75.1	100.0	88.2	96.5

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; SPE, specificity; SEN, sensitivity; ACU, accuracy; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Table 8a11.

Number of Patients Whose Loss of Libido Problem is Predicted by the Joint ROC Model

Diagnosis-Symptom	HDRS		HARS		Log-Reg
	No	Yes	No	Yes	No	Yes
Loss of libido
Bipolar disorder
No	76	14	71	19	69	21
Yes	29	50	25	54	27	52
Depression
No	147	68	133	82	100	115
Yes	119	193	102	210	65	247
Panic disorder
No	42	59	33	68	94	7
Yes	11	43	7	47	43	11
OCD
No	101	70	131	40	150	21
Yes	14	72	34	52	56	30
GAD
No	135	121	191	65	241	15
Yes	21	86	54	53	79	28

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Table 8b.

Sensitivity and Specificity Values of HDRS and HARS by the Joint ROC Model for Loss of Libido

Symptom-Diagnosis	HDRS			HARS			Log-Reg			Joint ROC
	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)	SPE (%)	SEN (%)	ACU (%)
Loss of libido
Bipolar disorder	84.4	63.3	74.6	78.9	68.4	74	76.7	65.8	71.6	88.9	100	94.1
Depression	68.4	61.9	64.5	61.9	67.3	65.1	46.5	79.2	65.8	76.7	100	90.5
Panic disorder	41.6	79.6	54.8	32.7	87	51.6	93.1	20.4	67.7	100	88.9	96.1
OCD	59.1	83.7	67.3	76.6	60.5	71.2	87.7	34.9	70	100	83.7	94.6
GAD	52.7	80.4	60.9	74.6	49.5	67.2	94.1	26.2	74.1	100	83.2	95

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; Log-Reg, logistic regression; SPE, specificity; SEN, sensitivity; ACU, accuracy; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

Table 9a.

Estimated P Values of Insomnia for Each Diagnosis Based on the Joint ROC Method

Diagnosis	Cut-Off Values		P*
Bipolar disorder	HARS > 10.5	HAMD > 6.5	Insomnia
			Yes	No
	No	No	0.42425	0.99993
	No	Yes	0.94793	0.9984
	Yes	No	0.03471	0.00244
	Yes	Yes	0.99989	0.5296
Depression	HARS > 26.5	HAMD > 21.5	Insomnia
			Yes	No
	No	No	0.00003	0.15242
	No	Yes	0.09219	0.00004
	Yes	No	0.99744	0.99836
	Yes	Yes	1	0.1231
Panic Disorder	HARS > 20.5	HAMD > 8.5	Insomnia
			Yes	No
	No	No	0.00002	0.65149
	No	Yes	0.0006	0.05072
	Yes	No	0.01846	0.00169
	Yes	Yes	0.99995	0.60313
OCD	HARS > 15.5	HAMD > 9.5	Insomnia
			Yes	Yes
	No	No	0.37442	0.99994
	No	Yes	0.93548	0.99865
	Yes	No	0.02889	0.00311
	Yes	Yes	0.99986	0.58118
GAD	HARS > 17.5	HAMD > 6.5	Insomnia
			Yes	Yes
	No	No	0	0.64682
	No	Yes	0.96946	0.99977
	Yes	No	0.99999	0.59107
	Yes	Yes	0.99939	0.98824

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder.

^*Values produced by the conversion table were added to logistic regression. Next, 2 P values were obtained. The first value represents the probability at which the patient would develop the relevant symptom, depending on their HDRS and HARS scores being under or above the cut-off values. The second value represents the probability of not developing the symptom under the same condition. The P values range from 0 to 1. Zero represents the absolute absence of the symptom, while one represents the absolute presence of the symptom.

Table 9b.

Estimated P Values of Inappetence for Each Diagnosis Based on the Joint ROC Model

Diagnosis	Cut-Off Values		P *
Bipolar Disorder	HARS > 14.5	HAMD > 12.5	Inappetence
			Yes	No
	No	No	.37642	.99986
	No	Yes	.00238	.08408
	Yes	No	.03808	.00550
	Yes	Yes	.41672	.00033
Depression	HAMD > 18.5	HARS > 21.5	Inappetence
			Yes	No
	No	No	0	.79214
	No	Yes	.96075	.99996
	Yes	No	.00365	.00050
	Yes	Yes	.99999	.74530
Panic Disorder	HAMD > 13.5	HARS > 13.5	Inappetence
			Yes	No
	No	No	0	.88697
	No	Yes	0	.01155
	Yes	No	N/A	N/A
	Yes	yes	1	.84303
OCD	HAMD > 15.5	HARS > 19.5	Inappetence
			Yes	Yes
	No	No	.00009	.56086
	No	Yes	.00194	.05263
	Yes	No	.99736	.96072
	Yes	Yes	.4845	.00013
GAD	HAMD > 8.5	HARS > 16.5	Inappetence
			Yes	Yes
	No	No	0	.88538
	No	Yes	.99875	.99801
	Yes	No	.00003	.08434
	Yes	Yes	.14351	.00002

Table 9c.

Estimated P Values of Loss of Libido for Each Diagnosis Based on the Joint ROC Model

Diagnosis	Cut-Off Values		P *
Bipolar Disorder	HDRS > 14.5	HARS > 16.5	Loss of libido
			Yes	No
	No	No	.00008	.33227
	No	Yes	.06744	.00047
	Yes	No	.00234	.01437
	Yes	Yes	.99998	.28832
Depression	HDRS > 22.5	HARS > 25.5	Loss of libido
			Yes	No
	No	No	.00005	.41646
	No	Yes	.00161	.02032
	Yes	No	.99933	.95233
	Yes	Yes	.99998	.36739
Panic Disorder	HDRS > 5.5	HARS > 13.5	Loss of Libido
			Yes	No
	No	No	0	.90912
	No	Yes	.00001	.06511
	Yes	No	.00093	.00056
	Yes	Yes	1	.88221
OCD	HDRS > 7.5	HARS > 19.5	Loss of libido
	No	No	.00002	.75957
	No	Yes	.0005	N/A
	Yes	No	.01306	.00410
	Yes	Yes	.2781	.00016
GAD	HDRS > 7.5	HARS > 22.5	Loss of libido
			Yes	No
	No	No	0	.80188
	No	Yes	.91466	.99982
	Yes	No	.00704	.00239
	Yes	Yes	.99994	.76467

HDRS, Hamilton Depression Rating Scale-17; HARS, Hamilton Anxiety Rating Scale; OCD, Obsessive-Compulsive Disorder; GAD, Generalized Anxiety Disorder; N/A, not available.

^*Values categorized by the joint ROC model were added to logistic regression. Next, 2 P values were obtained. The first value represents the probability at which the patient would develop the relevant symptom, depending on their HDRS and HARS scores being under or above the cut-off values. The second value represents the probability of not developing the symptom, under the same conditions.

Funding Statement

The authors declared that this study has received no financial support.