Measuring mental health of the Dutch population: a comparison of the GHQ-12 and the MHI-5

Abstract

Backgroud

The objective is to compare the performance of the MHI-5 and GHQ-12, both measures of general mental health. Therefore, we studied the relationship of the GHQ-12 and MHI-5 with sociodemographic characteristics, self-reported visits to general practice and mental health care, and with diagnoses made by the general practitioner.

Methods

Data were used from the Second Dutch National Survey of General Practice, which was carried out in 104 practices. This study combines data from a representative sample of the Dutch population with data from general practice.

Results

The agreement between the GHQ-12 and MHI-5 is only moderate. Both instruments are however similarly associated with demographic characteristics (except age), self-reported health care use, and psychological and social diagnoses in general practice.

Conclusions

The performance of the MHI-5 and GHQ-12 in terms of predicting mental health problems and related help seeking behaviour is similar. An advantage of the MHI-5 is that it has been widely used, not only in surveys of mental health, but also in surveys of general health and quality of life, and it is shorter. A disadvantage of the MHI-5 is that there is no cut-off point. We recommend a study to establish a valid, internationally comparable cut-off point.

Introduction

Mental health is an important indicator for the health status of a population. Mental health can be measured by the different psychiatric diagnoses, such as major depression, anxiety disorders and schizophrenia. Diagnostic measurement instruments as the Composite International Diagnostic Interview (CIDI) or Schedules for Clinical Assessment in Neuropsychiatry (SCAN) are developed for this purpose. On the other hand, instruments like the General Health Questionnaire (GHQ) [1], Affect Balance Scale (ABS) [2], Symptom Checklist (SCL) [3], or the Patient Health Questionnaire (PHQ) [4], are developed to measure general mental health using a short questionnaire. These instruments aim at getting a general picture of the mental health status of an individual or a population, for example to identify risk groups or monitor changes over time. The Mental Health Inventory (MHI-5), one of the subscales of the Short Form-36 (SF-36), is a widely used instrument to measure quality of life [5]. It is even recommended to be used by a European framework [6]. However, the use of this instrument in the field of mental health is relatively modest. The psychometric properties of the MHI-5 are very similar to those of the GHQ-12 [7].

The purpose of this paper is to compare the results of the MHI-5 with the GHQ-12 in the general Dutch population and in general practice. Therefore, we studied the following research questions:

1. What is the relationship between the GHQ-12 and MHI-5, and how are both instruments related to sociodemographic characteristics?

2. What is the relationship of the GHQ-12 and MHI-5 with self-reported visits to general practitice and mental health care, and with diagnoses made by the general practitioner?

Methods

Sample and procedure

Data come from the Second Dutch National Survey of General Practice (DNSGP-2), a study that combines information from a representative sample of the general population with data from general practice. The DNSGP-2 was carried out in 104 general practices in the Netherlands, consisting of 195 General Practitioners (GPs) [8]. The participating general practices were a good representation of all Dutch practices according to region, urbanicity and location in deprived areas. The only exception was that the participating practices were slightly less often practices that are run by one practitioner (solo-practice). The 104 practices employ 195 physicians. These physicians are a good representation of all Dutch practitioners according to age and gender. The general practices list nearly 400,000 patients. The GPs registered all contacts with patients during one year in their practice computer (contact registration). In this contact registration, the morbidity of the patients, presented to the GP, was coded using the International Classification of Primary Care (ICPC) [9]. These data collections took place between April 2000 and January 2002.

A random sample of approximately five percent of all listed patients was asked to participate in a personal face-to-face health interview. These interviews were carried out during one year, with a random fourth part per three months to avoid seasonal patterns in morbidity. The response rate was 64.5% (12,699 participants). The sociodemographic characteristics of the participants of the study were highly comparable to those of the total Dutch population, although men, younger age groups and immigrants were slightly underrepresented.

For the purpose of this paper, only the 9,687 participants aged 18 years or older were included. Eight practices had incomplete data concerning the contact registration. Eventually 96 practices (7,539 people) were used for the analysis concerning the diagnoses made in general practice.

The data from the GP diagnoses and from the health interview were linked to each other, with a unique anonymous patient identifier.

Materials

Mental health was measured with the General Health Questionnaire (GHQ-12) and the Mental Health Inventory (MHI-5). The GHQ-12, a 12-item questionnaire, is used to identify people likely to have mental problems, and who therefore may need health care. Cut-off point ≥ 2 was used [1]. The MHI-5 measures general mental health [5] and is part of the Short Form Health Survey (SF-36), a questionnaire for measuring health-related quality of life [10]. The MHI-5 has a score of 0 to 100, where a score of 100 represents optimal mental health. The mean scores as well as the prevalence of mental problems were reported. Because no formal cut-off point is determined, we used the cut-off score for the MHI-5 (≤ 72) that generated the most comparable prevalence of mental problems as the GHQ-12 cut-off point ≥ 2.

Besides these mental health indicators, the health interview contained questions about sociodemographic characteristics: sex, age, educational attainment, and marital status. Educational attainment was recoded into 4 levels: primary education, lower secondary education, higher secondary education, or higher vocational education (college) and university.

Health care utilization was measured in the health interview by asking the participants whether they consulted their GP in the preceding two months (regardless of reasons for visit), and if they had had any consultation with mental health care (psychologist, psychiatrist, psychotherapist, social worker, regional institute for ambulant mental welfare, clinic for alcohol and drug abuse, or any other mental health care professional) in the preceding 12 months. Besides this information from the health interview, diagnoses were also ascertained in the contact registration in the general practice. In this registration the diagnosis was coded in ICPC chapters. Because social problems are strongly related to mental problems, we used both the psychological (P) and social (Z) chapter [9]. This represents whether the GP diagnosed a psychological or social problem or not. We used the ICPC-codes concerning symptoms/complaints and those concerning diseases/ diagnoses.

Method of analysis

Analyses were done using SPSS, version 11. Independent variables were sex, age, educational attainment, and marital status. Descriptive analyses to detect differences in mental problems between the sociodemographic groups were performed using Pearson's chi-square. Multivariate analysis was performed using binary logistic regression analyses. This analysis was done to report the relationship between sociodemographic variables and the mental health indicators, as well as between the mental health indicators, care utilisation, and ICPC diagnoses. This relationship is expressed in Odds Ratio's. The Odds Ratio's were analysed while adjusting for the independent variables.

Cohen's Kappa was calculated to measure the agreement between the GHQ-12 and MHI-5 scores at the level of individuals. A value of 1 indicates a perfect agreement, 1–0.8 almost perfect, 0.6–0.8 is substantial, 0.4–0.6 is moderate, 0.4–0.2 is fair, and 0.2–0 is slight to poor, and a value of 0 indicates that agreement is no better than chance [11].

Results

Agreement between GHQ-12 and MHI-5

The agreement between the GHQ-12 and MHI-5 was only modest. The correlation between GHQ-12 (score 0–12) and MHI-5 (score 0–100) is 0.64. Kappa statistics between GHQ ≥ 2 and MHI ≤ 72 was 0.49, indicating moderate agreement. Thirteen percent of the participants were likely to have mental problems both according to the GHQ-12 and the MHI-5, 70% had no mental problems according to both instruments. Thus, the remaining 17% had mental problems according to one instrument and no problems according to the other instrument.

Relationship with demographic characteristics

Almost 23% of the study populations had a GHQ-12 score above the cut-off point (≥ 2) (Table 1). These participants were therefore more likely to suffer from a mental disorder than those scoring below the cut-off point. The mean MHI-5 score of the population was 80. Using the cut-off point of ≤ 72 gave the most comparable prevalence of mental problems for the MHI-5 as for the GHQ-12, that is 21%.

Table 1.

Prevalence of probable mental problems according to the GHQ-12 and MHI-5 in relation to sociodemographic characteristics.

N = 9,687	GHQ-12 ≥ 2		MHI-5	MHI-5 ≤ 72
	%	OR (95% CI)	Mean (SD)	%	OR (95% CI)
Total	22.8	-	80 (16)	20.6	-
Sex
Male (ref)	19.2	1.00	83 (15)	14.6	1.00
Female	25.7	1.40 (1.26–1.55)	78 (17)	25.5	1.81 (1.62–2.02)
Age
18–24 (ref)	25.0	1.00	80 (6)	17.7	1.00
25–44	25.6	1.22 (0.99–1.51)	81 (15)	19.4	1.46 (1.15–1.85)
45–64	22.0	1.00 (0.79–1.25)	81 (16)	19.5	1.38 (1.07–1.78)
65–74	16.6	0.61 (0.47–0.81)	79 (17)	22.2	1.27 (0.96–1.70)
75+	21.1	0.70 (0.52–0.94)	76 (17)	30.9	1.55 (1.14–2.10)
Education
Primary	25.2	1.34 (1.12–1.60)	75 (19)	32.4	2.28 (1.90–2.75)
Lower	22.3	1.01 (0.88–1.16)	80 (16)	21.8	1.46 (1.25–1.71)
Secondary	21.1	0.87 (0.75–1.00)	82 (15)	16.7	1.08 (0.92–1.27)
Higher (ref)	23.5	1.00	82 (14)	16.0	1.00
Marital Status
Married (ref)	20.0	1.00	82 (15)	17.3	1.00
Unmarried	27.7	1.50 (1.30–1.73)	79 (16)	21.8	1.64 (1.41–1.92)
Divorced	34.5	2.02 (1.65–2.47)	73 (19)	35.6	2.59 (2.11–3.18)
Widowed	25.8	1.58 (1.28–1.95)	72 (19)	37.3	1.96 (1.62–2.39)

OR = Odds Ratio, adjusted for confounders; SD = Standard Deviation

The distribution of the outcomes on the GHQ-12 and MHI-5 scores was similar for most sociodemographic variables: women, those with a lower level of education, and divorced participants had most probable mental problems. Notable is that the relationships seem more pronounced for the MHI-5 than for the GHQ-12 (see for example educational attainment). Scores on the two instruments were differently related with age. According to the GHQ-12 scores, the oldest age groups (65–74 and 75+) probably had least mental problems, whereas the MHI-5 indicated that the oldest group (75+) had most problems.

Relationship with health care

Participants with probable mental problems according to the GHQ-12 and the MHI-5, consulted the GP about twice as much, and mental health care about five to seven times as much compared to the ones who had no mental problems (Table 2). The GHQ-12 predicted a visit to the general practitioner slightly better than the MHI-5, whereas the MHI-5 was a better predictor of a visit to mental health care. These results are adjusted for sociodemographic characteristics.

Table 2.

Self-report of consulting the general practitioner (preceding two months) and mental health care (preceding 12 months) in relation to mental health.

	General practitioner		Mental health care
N = 9,687	%	OR *	%	OR *
Total	41.9	-	6.2	-
GHQ-12 < 2	38.0	1.00	3.6	1.00
GHQ-12 ≥ 2	55.1	2.06 (1.86–2.28)	15.7	4.51 (3.78–5.38)
MHI-5 > 72	38.4	1.00	3.4	1.00
MHI-5 ≤ 72	55.5	1.79 (1.61–1.98)	17.8	6.50 (5.41–7.81)

* OR = Odds Ratio, adjusted for sociodemographic variables

The GP diagnosed almost 16% of his of her total practice population with psychological symptoms or diseases and almost 4% with social symptoms (Table 3). The relationship of these diagnoses with the GHQ-12 and MHI-5 was not significantly different: the GP diagnosed about 30% of the participants with an unfavourable GHQ-12 or MHI-5 score as having a psychological problem, and 8% as having a social problem. The most common psychological diagnoses made by the GP were depression or depressed feeling (3.5 + 0.9 = 4.4%), sleeplessness (4.2%), and anxiety or anxious feeling (2.5 + 1.2 = 3.7%) (Table 4). Social problems were much more uncommon; the most frequent social problem was associated with the relationship with the partner (0.8%). The mean number of diagnoses was 0.45 in the total study population, and about 1 in those who were likely to have a mental problem according to the GHQ-12 or MHI-5.

Table 3.

Getting a psychological or social diagnosis from the General Practitioner in relation to mental health.

	Psychological diagnosis		Social diagnosis
N = 7,593	%	OR *	%	OR *
Total	15.6	-	3.6	-
GHQ-12 < 2	11.5	1.00	2.2	1.00
GHQ-12 ≥ 2	28.2	3.02 (2.64–3.46)	8.0	3.68 (2.85–4.74)
MHI-5 > 72	11.2	1.00	2.4	1.00
MHI-5 ≤ 72	30.6	3.21 (2.80–3.69)	7.7	2.96 (2.29–3.84)

* OR = Odds Ratio, adjusted for sociodemographic variables

Table 4.

The most important psychological and social diagnoses of the total study population and of the population with probable mental problems according to the GHQ-12 or MHI-5.

N = 7,593	Total study population %	GHQ-12 ≥ 2 %	MHI-5 < 70 %
1. Sleeplessness/ sleep disorder otherwise	4.2	7.1	7.8
2. Depression	3.5	7.1	11.0
3. Anxious or nervous feeling	2.5	5.1	5.3
3. Neurasthenia/ surmenage	1.4	2.9	2.5
4. Anxiety disorder/ anxiety state	1.2	2.2	2.7
5. Down/ depressed feeling	0.9	2.0	2.6
Any psychological or social diagnosis	18.2	33.4	35.4
Mean number of psychological or social diagnoses	0.45	0.91	1.06

Discussion

In our study, we used two different instruments to measure mental health in the population: the GHQ-12 and the MHI-5. The GHQ-12 is used in many other studies on mental health [12-14]. Additionally, we used the MHI-5, because this instrument, being one of the dimensions of the SF-36, is increasingly used in both clinical and population studies [7,15,16]. Both the MHI-5 and the GHQ-12 revealed that, as reported in earlier studies, women, those with lower educational levels, and unmarried participants had more problems compared to men, those with higher education and married participants [12-16].

The relationship between age and mental health was and remains inconclusive. Interestingly, we found a different relationship with age for the two instruments of mental health in the same population. According to the GHQ-12, the older age groups reported least problems, whereas the MHI-5 found them to report most problems. McCabe and colleagues found no correlation between age and the score of MHI-5, whereas the score of the GHQ-12 was highest among the older age groups [7]. A relatively recent review answered the question whether old age reduces the risk of anxiety and depression, with a cautious yes [17]. Although not very convincing, this review found some evidence that ageing is associated with an intrinsic reduction in susceptibility to anxiety and depression. This conclusion seems more in line with our findings concerning the GHQ-12 than the MHI-5.

The difference in distribution of age groups and mental health found in the GHQ-12 and MHI-5 suggests that both instruments measure different aspects of mental health. This is also indicated by the relatively modest agreement we found between the two instruments. Weinstein and colleagues confirmed this finding in their comparison of the GHQ and MHI [18]. They found that the MHI performed significantly better than the GHQ in detecting affective disorders. Possibly, the MHI-5 detects more affective disorders than the GHQ-12, and this may have resulted in a modest agreement and in different findings regarding the age distribution.

Those with probable mental problems reported a visit to their GP twice as often compared to those without mental problems. They reported to visit mental health care about five to seven times as often. The GHQ-12 is a better predictor of seeking help through the GP, whereas the MHI-5 predicts a visit to mental health care better. These differences are, however, not very large. Moreover, the relationship of both instruments of mental health with the diagnosis of the GP is more or less similar.

The fact that not all people with an unfavourable score on the mental health indicators are also diagnosed by the GP as having a mental or social problem can not simply be interpreted as an unmet need for care. Several explanations exist why those with probable mental problems do not seek health care or, if they do, why the general practitioner does not diagnose their problems as psychological or social problems. For example, patients are often still reluctant to seek help for their mental problems [19,20]. Another explanation can be that not all GP-visits of patients with mental problems are related to their mental problems. Whether or not there is an unmet need for care for patients with mental problems should be subject for further study.

As relatively short instruments to measure mental health in the general population, both the GHQ-12 and the MHI-5 seem just as good. Both instruments are almost equally related to sociodemographic characteristics, self-reported visits to the general practitioner and mental health care, and to the diagnosis the general practitioner made. McCabe and colleagues already showed that not only the external validity, but also the internal consistency of both instruments is similar [7]. However, both instruments seem to measure different aspects of mental health. It is interesting to compare both instruments to diagnostic measurements instruments to shine some light on the relationship with different psychiatric disorders.

An advantage of the MHI-5 is that it is very short (only 5 items), and it is widely used, as yet especially in surveys regarding general health or quality of life. A disadvantage of the MHI-5, however, is that it is usually reported as a mean score. No formal cut-off point is established to distinguish those with probable mental problems from those without. Instead, the MHI-5 is a gliding scale from perfect mental health to poor mental health. For the purpose of our study, we used a cut-off point (≤ 72) that generated about the same prevalence of mental problems as the GHQ-12. Earlier work, comparing the scores on the MHI-5 with the CIDI-based psychiatric disorders depression and anxiety, revealed the same optimal cut-off point (highest area under the receiver operating curve) [21]. A similar German study, however, recommended 65 as the optimal cut-off point [15]. Apparently, there is no internationally comparable cut-off point. Therefore, we recommend more research into a valid, internationally comparable cut-off point for the MHI-5, based on an optimum of the specificity and sensitivity. This analysis should preferably be performed by comparing the MHI-5 scores with both population-based diagnoses (as for example based on the CIDI) and doctor based diagnoses. A cut-off point will improve the usefulness of the MHI-5 to a great extent, for example for screening mental health problems in primary care or for calculating a health expectancy based on mental health [21,22].

Authors' contributions

NH participated in the design of the study and statistical analyses and wrote the final manuscript. AG performed the statistical analyses and wrote the first draft of the paper. GW and PV participated in the design and coordination of the Second Dutch National Study of General Practice and participated in the design of the study. All authors read and approved the final manuscript.