Mental health diagnosis by nurses using the Global Mental Health Assessment Tool: a validity and feasibility study

Abstract

Background

The Global Mental Health Assessment Tool — Primary Care Version (GMHAT/PC) has been developed to assist health professionals to make a quick and comprehensive standardised mental health assessment. It has proved to be a reliable and valid tool in a previous study involving GPs. Its use by other health professionals may help in detecting and managing mental disorders in primary care and general health settings.

Aim

To assess the feasibility of using a computer-assisted diagnostic interview by nurses and to examine the level of agreement between the GMHAT/PC diagnosis and psychiatrists' clinical diagnosis.

Design of study

Cross-sectional validation study.

Setting

Primary care, general healthcare (cardiac rehabilitation clinic), and community mental healthcare settings.

Method

A total of 215 patients between the ages of 16 and 75 years were assessed by nurses and psychiatrists in various settings: primary care centre (n = 54), cardiac rehabilitation centre (n = 98), and community mental health clinic (n = 63). The time taken for the interview, and feedback from patients and interviewers were indicators of feasibility, and the kappa coefficient (κ), sensitivity, and specificity of the GMHAT/PC diagnosis were measures of validity.

Results

Mean duration of interview was under 15 minutes. The agreement between nurses' GMHAT/PC interview-based diagnosis and psychiatrists' International Classification of Diseases (ICD)–10 criteria-based clinical diagnosis was 80% (κ = 0.76, sensitivity = 0.84, specificity = 0.92).

Conclusion

The GMHAT/PC can assist nurses to make accurate mental health assessment and diagnosis in various healthcare settings and it is acceptable to patients.

INTRODUCTION

Mental health problems are one of the leading causes of disability in the world.¹ Early and accurate detection of mental health problems, followed by an appropriate treatment and management plan, may help to reduce the global burden on health and social care systems caused by mental disorders.

A vast majority of people with mental disorders, including those with severe mental illness, view primary care as the cornerstone of their healthcare system.² In the modern NHS, GPs and other primary care workers are expected to identify and assess the mental health needs of their patients, and to manage common mental disorders in primary care as outlined under standards 2 and 3 of the National Service Framework for Mental Health.³

How this fits in

Accurate detection and proper management of mental disorders in primary care needs a great deal of attention in the UK and worldwide. The Global Mental Health Assessment Tool — Primary Care Version, a computerised clinical tool, can help primary care professionals to make a quick and accurate mental health assessment and diagnosis. The pathways of care as a part of the programme can guide nurses and GPs in providing evidence-based treatments for their patients.

GPs, with their limited time in the primary care clinics (surgeries), and in some instances with only limited training and experience, are often criticised for failing to recognise a sizeable number of those suffering from mental disorders or to treat them adequately if they do.^4,5 However, the impact of GPs' detecting depression in patients seen in their practices seems to have little effect on the patients' outcome.⁶ More recent studies suggest that GPs are getting better at diagnosing and treating more severe mental disorders rather than milder forms.^7,8

The Department of Health introduced 500 new ‘gateway workers’ and another 1000 new ‘graduate primary care mental health workers’ trained in proven brief therapies to help GPs look after people with common mental health problems.⁹ Any such additional resource is a welcome move, but only an efficient use of existing and new resources will make any demonstrable impact on mental health services in primary care.¹⁰ Early findings suggest that primary care workers provide a range of skills valued by patients and primary care teams,¹¹ but other studies failed to show a positive impact of new workers in managing common mental disorders in primary care.^12,13

An independent policy review in the UK reported gaps in implementing the National Service Framework, particularly with reference to primary care.¹⁴ The problems that GPs and other primary care workers have with patients with mental illness are related to knowing what questions to ask; and how to make diagnosis from the symptoms elicited. Whereas the type of questions can be fairly easily learned, making a diagnosis is a skill requiring experience and is probably more difficult than is supposed by psychiatrists.

The current authors believe that a semi-structured mental health assessment process using the Global Mental Health Assessment Tool — Primary Care Version (GMHAT/PC) can help practitioners to know which questions to ask and how to diagnose with the help of a computer-assisted interview format. An earlier research report¹⁵ demonstrated the feasibility of using this method in primary care. Generally, patients who received the assessment by GMHAT/PC said they found it helpful as it covered more aspects of their mental health than the usual consultation.

As the GMHAT/PC covers a wide range of mental disorders, including psychoses and organic disorders, it should prove useful in early and accurate detection. The value of early detection and intervention, particularly in psychotic disorders, is well documented.¹⁶

The format of the GMHAT/PC is simple to administer, as questions appearing on the screen cover only one area at any time. The interviewer is expected to have some background experience of assessing mental health problems, but will require little training to use the schedule. For those who have no previous experience of mental health assessments, a short course or training package will be necessary.

A description of the GMHAT/PC is outlined in the earlier research report,¹⁵ which highlights its reliability and validity (including a comparison with the Hospital Anxiety and Depression Scale) as well as its usefulness in primary care.

The first window on the computer screen displays information about the patient and programme administration, followed by an instruction page that gives details of how to use the tool and rate the symptoms. The following screens consist of questions on the mental state symptoms or problems regarding: worries; anxiety and panic attacks; concentration; depressed mood, including suicidal risk; sleep; appetite; eating disorders; hypochondriasis; obsessions and compulsions; phobia; mania/hypomania; psychotic symptoms; disorientation; memory impairment; alcohol misuse; drug misuse; personality problems; and stressors.

The questions proceed in clinical order along a tree-branch structure. For each of the major clinical disorders, there are key screening questions with cutoff points which assist in shortening the interview. The interviewer may record a clinical diagnosis in the next section.

A summary report of symptoms, their scores, and a GMHAT/PC diagnosis is produced in a printable form. The main diagnosis by computer is derived using a hierarchical model and designed around the International Classification of Diseases (ICD)–10.¹⁷ The diagnostic programme takes account of the severity of symptoms (moderate to severe). It also generates alternative or additional diagnoses based on the presence of symptoms of other disorders. In addition, it includes an assessment of risk of self-harm. The programme also contains management guidelines for these disorders that practitioners can use to guide their management strategies.

METHOD

The GMHAT/PC has been developed mainly for use in primary care and general health settings. For its validation in this study, participants were needed with a range of mental health problems. For that reason the study was conducted in three settings: primary care — a large GP surgery; general health — a cardiac rehabilitation centre; and mental health — a community mental health centre. These settings were located in the north west of England and North Wales.

At least 50 patients were interviewed in each setting by one or more nurses using the GMHAT/PC. Two psychiatrists (each with a background of 7 years' psychiatry training) assessed each patient for their independent ICD-10-based clinical diagnosis.

Ethical approval was obtained, and all patients were informed about the purpose of the study with a written leaflet. Only those who gave valid consent participated in the study.

In the primary care setting, a large practice in Wirral (Wallasey) with five GPs and three practice nurses was chosen. For a period of 3 months, during the study, all consecutive patients who the GPs considered to be in need of a mental health assessment were referred to the practice nurses' weekly clinic. All consecutive patients who were in the community mental health centre for a period of 3 months and in the cardiac rehabilitation centre for a period of 6 months were included in the study.

The interviewers (nurses) had a brief training session to familiarise themselves with the GMHAT/PC before interviewing patients.

A nurse seconded from the general hospital liaison service at the community mental health centre, a practice nurse of the surgery in the primary care setting, and a nurse practitioner at the cardiac rehabilitation centre interviewed the study patients using the GMHAT/PC.

The psychiatrists, who were unaware of the GMHAT/PC ratings and diagnosis, interviewed each patient using ICD-10 diagnostic criteria, and recorded an independent clinical diagnosis.

Demographic data, the computer-generated (GMHAT/PC) diagnosis, and the psychiatrist's ICD-10-based clinical diagnosis were recorded on the database.

Statistics

The kappa coefficient (κ) was used to determine levels of agreement between GMHAT/PC diagnoses by nurses, and the psychiatrists' diagnoses (gold standard). Sensitivity and specificity analysis was then used to determine how the GMHAT/PC-assisted nurse interviews could identify cases with and without mental illness as determined by the psychiatrists. The χ² statistic was used to test for homogeneity in the levels of overall disagreement between each of the psychiatrists and the corresponding nurses.

RESULTS

There were 215 participants in the study: 122 (57%) male and 93 (43%) female. There was no significance difference between groups by sex with regard to age. However, a higher proportion of females (56%) than males (35%) were diagnosed as having a mental illness, as determined by a psychiatrist. Overall mean time to administer the GMHAT/PC was 14.60 minutes. Table 1 gives a summary of demographics.

Table 1.

Sample demographics (n = 215).

	Males (n = 122, 57%)	Females (n = 93, 43%)
Diagnosis of mental illness (by psychiatrist):an (%)	43 (35)	52 (56)
Age in years:b mean (SD), range	53.12 (15.78), 20–87	49.20 (16.08), 19–85
Duration of interview in minutes:b mean (SD), range	14.01 (7.17), 4–42	15.38 (8.43), 2–51

^aStatistically significant difference between males and females (χ² test, P = 0.002).

^bNo significant difference between males and females (t test). SD = standard deviation.

Validity of the GMHAT/PC

Full data set

There was a good level of agreement between nurses' (GMHAT/PC) diagnosis and psychiatrists' (clinical) diagnosis of any mental illness (κ = 0.76, 95% confidence interval [CI] = 0.68 to 0.84). There was good sensitivity (0.84) and specificity (0.92), with nurses correctly identifying 80 out of the 95 participants diagnosed with mental illness, and 110 out of 120 of those without. The level of agreement for anxiety and stress-related disorders was also good (κ = 0.65). However, because of the small number of participants identified by the psychiatrists as having a diagnosis of such disorders, the 95% CI for κ was wide ranging (0.49 to 0.81). This suggests that the level of agreement could range from moderate to very good.

Sensitivity was 0.72, with nurses correctly identifying 18 of the 25 participants diagnosed with anxiety and stress-related disorders. Specificity was 0.95, with the nurses correctly identifying 181 of the 190 participants not suffering from anxiety and stress-related disorders. The level of agreement for depression showed κ = 0.78 and 95% CI = 0.68 to 0.89. Sensitivity was 0.73, and specificity 0.98, with the nurses correctly identifying 40 of the 55 participants diagnosed by the psychiatrists as having depression, and 157 out of 160 of those without. Levels of agreement are summarised in Table 2.

Table 2.

Levels of agreement between nurses and psychiatrists.

	κ (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)
Mental illness	0.76 (0.68 to 0.84)	0.84 (0.77 to 0.91)	0.92 (0.87 to 0.97)
Depression	0.78 (0.68 to 0.89)	0.73 (0.61 to 0.85)	0.98 (0.96 to 1.00)
Anxiety/stress-related disorders	0.65 (0.49 to 0.81)	0.72 (0.54 to 0.90)	0.92 (0.95 to 0.98)
Psychosis	0.92 (0.71 to 1.00)	0.86 (0.62 to 1.00)	1.00
Other	0.69 (0.45 to 0.92)	1.00	0.97 (0.95 to 0.99)

Positive predictive accuracy: 80/90 = 0.89, 95% CI, 0.83 to 0.95. Negative predictive accuracy: 110/125 = 0.88, 95% CI 0.82 to 0.94.

Psychiatrists

The κ coefficient was used to determine the level of agreement between the GMHAT/PC-assisted nurse's interview and psychiatrists' diagnoses. Sensitivity and specificity values were calculated. The κ coefficients (κ = 0.66, 95% CI = 0.52 to 0.80 and κ = 0.82, 95% CI = 0.68 to 0.96 respectively) suggest that there is a good level of agreement between the GMHAT/PC-based nurses' interview and the first psychiatrist, and a very good level of agreement between the nurses and the second psychiatrist. Sensitivity was higher for psychiatrist 1 (0.87 compared to 0.75 for psychiatrist 2). This is due, in part, to psychiatrist 2 having fewer cases of mental illness: 24 compared with 71. Consequently, although there was some heterogeneity between the two psychiatrists it was sufficiently small to allow combining of the two groups.

Sex

There was a higher level of agreement for males compared with female participants (κ = 0.80, 95% CI = 0.73 to 0.87 and κ = 0.70, 95% CI = 0.55 to 0.85 respectively). Sensitivity and specificity were also higher for males (0.86 and 0.94 respectively) compared with females (0.83 and 0.88 respectively). However, both groups had levels of sensitivity and specificity that were more than acceptable.

Results from the three sites

Data were divided into three groups (Table 3) depending on the type of unit: group 1 contained participants from the community mental health centre; group 2 had participants from the GP surgery; and group 3 participants were from the cardiac rehabilitation centre. Summary statistics showed the highest prevalence of mental illness in the GP site (72%); the community mental health centre had a prevalence of 51%, and the cardiac rehabilitation centre had the lowest prevalence (24%). The proportion of females to males varied across groups, ranging from 61% females in the GP site to 30% females in the cardiac rehabilitation centre site.

Table 3.

Summary statistics of the three sites (n = 215).

	Community mental health centre (n = 63, 29%)	GP surgery (n = 54, 72%)	Cardiac rehabilitation centre (n = 98, 46%)
Diagnosis of mental illness, n (%)	32 (51)	39 (72)	24 (24)
Sex,a % female	49	61	30
Age in years:b mean (SD), range	35.33 (10.58), 19–60	51.80 (13.56), 21–74	61.57 (10.96), 28–87
Time in minutes:c mean (SD), range	14.11 (8.20), 2–47	16.00 (7.20), 6–42	14.14 (7.73), 4–51
κ (95% CI)	0.65 (0.46 to 0.84)	0.60 (0.35 to 0.85)	0.82 (0.68 to 0.96)
Sensitivity (95% CI)	0.78 (0.64 to 0.90)	0.95 (0.89 to 1.00)	0.75 (0.56 to 0.92)
Specificity (95% CI)	0.87 (0.75 to 0.99)	0.60 (0.35 to 0.85)	1.00

^aStatistically significant difference between units (χ² P<0.001).

^bStatistically significant between units (ANOVA P<0.001).

^cNo statistically significant difference between units (ANOVA).

Although the level of agreement between nurses and psychiatrists for the community mental health centre site was good (κ = 0.65, 95% CI = 0.46 to 0.84), because of the relatively small numbers the true level of agreement ranges from moderate to very good. Nurses correctly identified 25 of the 32 participants with mental illness (sensitivity 0.78), and 27 of the 31 participants diagnosed without mental illness (specificity 0.87). There was a similar level of agreement for the GP site, with κ = 0.60, 95% CI = 0.35 to 0.85; again, because of the relatively small number of participants there was a wide confidence interval. Sensitivity was high (0.95), with the nurses correctly identifying 37 of the 39 participants identified with mental illness; however, specificity was lower (0.60), with the nurses correctly identifying 9 of the 15 participants with no mental illness.

Level of agreement was higher for the cardiac rehabilitation centre (κ = 0.82, 95% CI = 0.68 to 0.96), suggesting very good agreement. Specificity was 100%, with the nurse correctly identifying all the 74 participants diagnosed without mental illness, while sensitivity was 75% (18 out of 24 correctly identified).

Table 4 gives an overall diagnostic distribution of the participants.

Table 4.

Psychiatrists' and GMHAT/PC-assisted nurses' diagnosis cross-tabulation.

	Nurses' GMHAT-PC diagnosis
Psychiatrists' clinical diagnosis	No mental illness	Organic	Psychosis	Depression	Anxiety/stress-related disorders	Other	Total
No mental illness	110	1		1	6	2	120
Organic	1						1
Psychosis			6	1			7
Depression	8	1		40	3	3	55
Anxiety/stress-related disorders	5			1	18	1	25
Other	1					6	7
Total	125	2	6	43	27	12	215

GMHAT/PC = Global Mental Health Assessment Tool – Primary Care Version

Cases with disagreement

Of the eight patients with depression as diagnosed by the psychiatrist who had no primary GMHAT/PC diagnosis, three had other possible diagnoses of depression, a further two had another possible diagnosis of anxiety, and the remaining two had no other possible diagnosis. Of the five cases of anxiety-related disorders with no primary GMHAT/PC diagnosis, two had other possible diagnoses: one anxiety and one phobia.

Feasibility of the GMHAT/PC

None of the participants declined participation in the study, and none gave any negative feedback. When asked what they thought of the interview, most expressed satisfaction that the nurse covered all aspects of their mental health using the GMHAT/PC. The nurses who interviewed patients found the GMHAT/PC user friendly and asked whether they could continue using it in their routine practice.

DISCUSSION

Summary of main findings

The findings of this study are encouraging, and appear to support the view that other health professionals, particularly nurses, and possibly others with some training, can use the computer-assisted programme GMHAT/PC to make a valid assessment and diagnosis of mental disorder.

The mean duration of the interview of around 15 minutes makes it feasible in routine assessments in primary care and general health settings. The authors accept that GPs may not have enough time in their routine primary care clinics (surgeries), but they can utilise other health professionals in assessing the mental health of their patients using the GMHAT/PC. In some cases it took longer to complete the interview, mainly because the patients wanted to report their emotional problems and the nurses felt it was necessary to listen to them. Both the patients and nurses found the GMHAT/PC not only acceptable, but useful in making a quick, yet comprehensive, mental health assessment.

Strengths and limitations of the study

This study had a good overall sample size, with a varying degree of psychopathology in different healthcare settings. Clinical diagnoses were made by trained psychiatrists based on an independent clinical interview using ICD-10 criteria to compare with the GMHAT/PC diagnosis. The relatively small number of people with severe mental disorders in the study population limits the power of agreement between diagnoses.

It was necessary to include patients from mental health services in the study to include individuals at the severe end of mental disorders. As anticipated, there were very few such cases in the primary care and general health settings. Consecutive attenders were interviewed in the mental health service setting; some were in remission or partial remission of their mental illness. The primary care sample had a higher prevalence of mental disorders, as the participants who were deemed to have mental health problems were selected by the GPs.

Nurses who interviewed patients in the study reported that they were motivated to use computers in their future clinical assessments.

One of the main drawbacks of the study is inclusion of a limited number of cases in the primary care setting. These too were selected cases where the GPs felt that the patient needed further mental health assessment. It should be acknowledged that the validity of the GMHAT/PC diagnosis applies to this group of patients, and further research is required to include a larger sample, preferably at multiple primary care sites.

Comparison with existing literature

In the UK, various National Service Frameworks and policy guidelines have identified that mental health problems have significant comorbidity with chronic physical illness, and are a serious hindrance to good outcome.^3,18 Taking into account the New Ways of Working document,¹⁹ it is clear that mental health diagnoses are made primarily by GPs and health professionals other than psychiatrists. The present results have shown that the GMHAT/PC is a tool that gives general health professionals the ability to make mental health diagnoses, particularly for depression and anxiety.

A high mental health morbidity has a particularly adverse effect on general health and social wellbeing in the population of developing countries.²⁰ Given that the identification of mental health problems is vital for improving the outcome in many chronic illnesses, it is essential to improve recognition rates. Studies on guideline dissemination have failed to demonstrate effectiveness in changing clinician behaviour, whereas more structured implementation strategies have produced some favourable results.²¹

There are a number of case-finding tools available, but most are for depression and few are used routinely.²² A recent tool developed to screen and identify people at a high risk of mental health problems could be useful in the primary care setting.²³ As it is a clinical interview tool, the GMHAT/PC helps in diagnosis as well as in providing guidelines for how to help patients once mental health problems are identified. It should also add to the mental health assessment skills of primary care nurses.

Implications for future research and clinical practice

If incorporated in patient information systems, the GMHAT/PC can serve several functions. In addition to a standardised assessment, it provides measurement of symptoms and, if repeated over time, it could be used as a practical outcome measure. The GMHAT/PC depression and anxiety ratings compared well with the Hospital Anxiety and Depression scale in the authors' previous study.¹⁵

The authors hope to carry out further studies in the primary care setting, to evaluate the usefulness of the GMHAT/PC in routine care.

Following publication of the first research report in 2004,¹⁵ health professionals in different parts of the world have requested that the GMHAT/PC is translated into their languages. In developing countries this may help to provide basic mental health assessment to a large population by trained primary care workers. Management guidelines (care pathways) can be modified to suit the population.

So far the GMHAT/PC is being translated into Spanish, Dutch, German, Chinese, Hindi, Arabic, and Ghanaian. Further studies are in progress to assess its validity and usefulness in different cultures.

Funding body

The project was funded by the Evidence Based Centre (Research and Development), Cheshire and Wirral Partnership NHS Trust

Ethical approval

Cheshire Research Ethics Committee (National Research Ethics Service) (05/Q1506/2)

Competing interests

The authors have stated that there are none

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss