Experience of implementing new mental health indicators within information systems in six low- and middle-income countries

Abstract

Background

Successful scale-up of integrated primary mental healthcare requires routine monitoring of key programme performance indicators. A consensus set of mental health indicators has been proposed but evidence on their use in routine settings is lacking.

Aims

To assess the acceptability, feasibility, perceived costs and sustainability of implementing indicators relating to integrated mental health service coverage in six South Asian (India, Nepal) and sub-Saharan African countries (Ethiopia, Nigeria, South Africa, Uganda).

Method

A qualitative study using semi-structured key informant interviews (n = 128) was conducted. The ‘Performance of Routine Information Systems’ framework served as the basis for a coding framework covering three main categories related to the performance of new tools introduced to collect data on mental health indicators: (1) technical; (2) organisation; and (3) behavioural determinants.

Results

Most mental health indicators were deemed relevant and potentially useful for improving care, and therefore acceptable to end users. Exceptions were indicators on functionality, cost and severity. The simplicity of the data-capturing formats contributed to the feasibility of using forms to generate data on mental health indicators. Health workers reported increasing confidence in their capacity to record the mental health data and minimal additional cost to initiate mental health reporting. However, overstretched primary care staff and the time-consuming reporting process affected perceived sustainability.

Conclusions

Use of the newly developed, contextually appropriate mental health indicators in health facilities providing primary care services was seen largely to be feasible in the six Emerald countries, mainly because of the simplicity of the forms and continued support in the design and implementation stage. However, approaches to implementation of new forms generating data on mental health indicators need to be customised to the specific health system context of different countries. Further work is needed to identify ways to utilise mental health data to monitor and improve the quality of mental health services.

Declaration of interest

None.

Within the area of mental health, there is a worldwide initiative to expand access to care by integrating mental health into primary healthcare.¹ Scale-up of any global health programme requires routine monitoring of key indicators.² Member states of the World Health Organization (WHO) have committed to reporting and monitoring national-level indicators for implementation of the global Mental Health Action Plan, 2013–2020.³ However, most low- and middle-income countries (LMICs) do not yet have adequate mental health indicators to monitor their in-country programmes.^4,5

There is a pressing need to develop evidence-based mental health indicators for local programme monitoring and to understand ‘how’ data on these indicators can be collected in routine LMIC settings.⁶ The ‘how’ question can be addressed through assessment of implementation of procedures to collect data on key mental health indicators, with particular consideration of the acceptability to patients and contextual feasibility.⁷ Attending to the ‘how’ of implementation can tangibly improve mental health service monitoring and is crucial for the viability of ongoing efforts to scale-up mental health services in LMICs.⁸

Development of mental health indicators in Emerald programme

As part of the Emerald programme (Emerging Mental Health Systems in LMICs),⁹ we established a set of key indicators for mental health programme monitoring, through a Delphi process and through building consensus among a broad range of stakeholders across six LMICs: Ethiopia, India, Nepal, Nigeria, South Africa and Uganda.¹⁰ The final set of indicators covered mental health service utilisation for priority disorders, unmet needs of people with mental health problems, the quality of services provided and the associated financial risk to the person and their family.

The selected indicators allowed measurement of key dimensions of universal health coverage, including the proportion of the target population receiving appropriate mental healthcare at district level in the six Emerald countries. Implementation of mental health data collection forms at a primary care level was evaluated quantitatively to assess their utility and validity.¹¹ In this study, we present findings from a qualitative study aiming to explore the acceptability, sustainability, feasibility and perceived costs of implementing the new mental health data collection forms in the context of integrated primary mental healthcare services in the six Emerald countries. A pre-existing conceptual framework, the Performance of Routine Information System Management (PRISM) framework, was used to assess the performance of these indicators. The PRISM framework describes the inputs of health information systems as determinants affecting the process leading to better-quality health management information systems (HMISs).¹²

Method

Study design

A cross-country qualitative study was conducted with a framework approach. Semi-structured interviews were conducted with 128 key informants across the sites. A qualitative approach was used to achieve rich and detailed understanding of interviewees’ points of view.¹³

Settings

The study was carried out in each of the six Emerald LMICs where a district-level mental healthcare plan was being scaled up to integrate mental health into primary care and reduce the treatment gap for priority disorders. Integration of mental health within primary care in Ethiopia, India, Nepal, Uganda and South Africa was led by Programme for Improving Mental Health Care (PRIME),¹⁴ and by the EuropeAid programme in Nigeria. The district mental healthcare plans have been described previously;¹⁵ in brief, they included training of primary healthcare workers in the WHO's Mental Health Gap Action Programme¹⁶ or PC101 (in South Africa)¹⁷ for primary care workers, combined with community and health system interventions to support this task-sharing model of care. Once the district mental healthcare plans had been implemented and running for about 12 months, the new mental health indicators and forms (health facility pro forma available upon request) were introduced.

For this study, the term HMIS refers to a system of collecting, processing and analysing routine health data that already exists in the country's setting. At the primary care level in the six Emerald countries, the initial data collection component of the mental health information system is paper-based and managed by health workers (mostly nurses). However, the subsequent data compilation becomes electronic. At the district level and above, mental health data in India, Nepal, Nigeria and South Africa are compiled electronically. Ethiopia largely relies on paper forms; however, there are some instances where electronic HMISs have been piloted. Data collection in health facilities in all six countries is managed by health workers, most often nurses.

The final list of indicators, type of forms or registers used for data collection, and the focal person responsible for implementing the new forms in each of the six countries are described in Table 1. Before introducing the new procedures for collecting the indicators, strategies such as 2-day training courses for health workers/managers, demonstration sessions and monthly supervision visits were used. The new mental health indicators had already been implemented for 6–8 months before this qualitative study was conducted.

Table 1.

Mental health indicators and its implementation

Country	Tools capturing mental health indicators	Final list of indicators	Responsibility of data collection and data reporting
Ethiopia	Out-patient registration book	Service utilisation by disorder (psychosis, bipolar disorder, depression, alcohol use disorder, epilepsy, suicide attempt, other), severity, referral, essential medication stock-out	Mental health focal person in the health centre (general nurse or health officer)
South Africa	ROR, tick register/sheet. PC101 guides to screen patients, PRIME referral forms	Service utilisation by disorder (psychosis, bipolar disorder, depression, alcohol use disorder, epilepsy, suicide attempt, other), follow-up, referral	Healthcare providers complete, tick register and ROR and data is consolidated by the data-capturing personnel in the facility
Nepal	OPD register	Service utilisation by disorder (psychosis, depression, alcohol use disorders, suicidal attempt), severity, functioning, follow-ups, referrals, referred by, approximate time since the last appointment, payment for consultation and medical expenses, out-of-pocket costs	Health workers (prescribers) within the health posts
India	Screening register, case register, follow-up register, referral slips and smile cards	Service utilisation by disorder (psychosis, depression, alcohol use disorders, suicidal attempt, other), severity, referral, number of trained mental health professionals, medicines out of stock, readmissions	Nurses supervised by PRIME Case Managers for reporting
Uganda	Patient's medical form, patient registers	Service utilisation by disorder (psychosis, depression, alcohol use disorder, epilepsy, suicidal attempt, other), severity, referral, essential medication	Dedicated HMIS officer supervised by the facility manager
Nigeria	Patient's medical form, patient registers, OPD registers, summary forms	Service utilisation by disorder (psychosis, depression, alcohol use disorder, epilepsy, suicide attempt, other), severity, referral, essential medication stock-out, number of trained mental health professionals	Primary healthcare clinician; Clinic Records Officer; District (local government) Monitoring and Evaluation officer; with supervision from Emerald Programme Officer

ROR, Rationalization of Registers; PRIME, Programme for Improving Mental Health Care; OPD, Out-Patient Department; HMIS, health management information system.

Sampling

Participants for interviews were identified and recruited based on their roles and responsibilities within primary healthcare facilities. Interviews were conducted with key informants, including health facility staff responsible for collecting mental health data (nurses, HMIS officers, record officers), clinicians, programme managers, facility heads/managers, supervisors and case managers in the study districts (Table 2).

Table 2.

Study samples in each Emerald country site

	Health workers/health records staff	Health managers/programme managers/facility heads/medical officers	Supervisors/case managers	Total respondents
Ethiopia	6	5	0	11
India	10	9	7	26
Nepal	22	2	4	28
Nigeria	15	15	6	36
South Africa	8	6	0	14
Uganda	3	10	0	13
Total				128

Health managers and medical officers/clinicians from the PRIME scale-up facilities were approached separately. The health managers did not have any role in choosing the clinicians or vice versa. Those who consented were included in the interview. Interviews were kept confidential and anonymised.

Procedures and instruments

Data were collected in each of the six countries between February and August 2017. A semi-structured topic guide was developed in English and translated into the local languages where necessary (Ethiopia: Amharic; India: Hindi; Nepal: Nepali; English was used in Nigeria, South Africa and Uganda) for use during the interviews. Back translations of the topic guides were not carried out owing to time constraints. The researchers carrying out the interviews were based at the site offices and were mainly MS or PhD graduates in public health/health management, psychology or other related disciplines.

The topic guide was based on a subgroup of the key implementation outcomes identified by Proctor et al,⁷ namely acceptability, sustainability, feasibility and cost. Definitions for each of these implementation outcomes are depicted in Table 3. Previously developed monitoring and evaluation topic guides from the MIND ME project (https://www.mhinnovation.net/innovations/mind-me-africa) were also referred for the development of the topic guides.²

Table 3.

Definitions of implementation outcomes assessed in this study

Implementation outcomes – definitions by Proctor et al7
Acceptability: Perception among implementation stakeholders that a given treatment, service, practice or innovation is agreeable, palatable or satisfactory
Sustainability: The extent to which a newly implemented treatment is maintained or institutionalised within a service setting's ongoing and stable operation
Feasibility/utility: The extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting
Cost: The cost impact of an implementation effort

Ethical considerations

Organisational and ethical permissions from the appropriate in-country institutions, as well as cross-country approval from King's College London and the WHO Institutional Review Boards, were obtained before approaching participants in each country. All participants provided informed consent.

Data analysis

Individual semi-structured interviews were transcribed verbatim for the analysis. Translations to English were carried out for interviews conducted in local languages.

The data analysis was underpinned by thematic analysis principles.¹⁸ The process started with open coding, where initial descriptive codes were applied to the data. These initial codes were subsequently grouped into broader categories, reflecting emerging common themes and underpinning latent constructs (parent themes). At this stage of the analysis process it was noted that these parent themes corresponded with the input domains outlined in the PRISM conceptual framework.¹² At this point, a decision was made to use a framework approach to proceed with data analysis,¹⁹ with the PRISM framework inputs guiding subsequent analysis. These inputs, summarised as parent themes for this study, were categorised by the PRISM framework into technical, organisational/environmental and behavioural determinants. The PRISM framework also details elements within each of these inputs; for this study, these were considered as subthemes within the three parent themes (see Table 4 for an overview of the integrated framework).

Table 4.

Parent themes and subthemes (based on PRISM framework)¹² and Proctor et al's⁷ implementation outcomes

PRISM framework: input determinants and process description	Proctor et al's implementation outcomes
Input determinants	Perceived acceptability
Technical factors
Overall impression
Accuracy
Organisational factors	Perceived acceptability, feasibility, sustainability and cost
Governance and planning
Availability of resources
Training
Feasibility
Costs
Importance to HMIS for mental health
Supervision
Integration with national HMIS
Usability of these forms in future
Behavioural factors	Perceived acceptability
Level of knowledge
Competence and confidence levels for HMIS tasks
Motivation
Process description	Not applicable
(Mental health indicators and implementation – refer to Table 1)
Tools used for HMIS
Data collection
Data processing and data analysis
Use of information and feedback on HMIS to staff

PRISM, Performance of Routine Information System Management; HMIS, health management information system.

An analysis framework reflecting these parent themes and subthemes was circulated to country researchers (D.G., J.A., J.M., N.M., C.H., S.A.) by a simple spreadsheet. This spreadsheet was subsequently populated with data (author summaries, participant summaries and quotes) by the country researchers. Finally, these data were synthesised by the lead researcher (S.A.).

Results

We first report findings on the technical factors to influence implementation of the new mental health indicators. We then discuss the role of organisational/environmental factors, presenting similarities and differences between the processes in each country. Finally, we elaborate on the behavioural components that emerged as enabling or hindering the integration of mental health data collection into primary care in the six countries.

The following analyses were conducted at country level; analysed data were collated at cross-country level and are described here to compare the similarities and differences across countries. However, wherever necessary, cadre-specific responses are also highlighted in the section below.

Technical influences

Interviewees in all countries perceived that the new mental health forms led to generation of mental health data by making it easier to document a patient's records. Across countries, for many of the interviewees, this was the most significant achievement of the programme. One of the programme coordinators in India reported:

‘For the first time in 15 years we are getting some sort of monthly reports from districts and even from CHCs [community health centres]. The DMHP [District Mental Health Programme] is quite old in Sehore district and we have for the first time been able to build such data system.’ (ID-05, Madhya Pradesh, India).

Similarly, in Ethiopia, a mental health focal person described the importance of mental health indicators in his health centre:

‘We record on the register and follow up cases. For example, the guidelines state that the patients with epileptic seizures who take medications for 2 years should stop taking the medications if they do not show signs and symptoms of seizure and epilepsy anymore. So, to follow this up, it is necessary to record this on the register. In my opinion, in this regard the register is very good.’ (ID-01, Ethiopia).

Most interviewees in all six countries agreed that the new indicators were clear and easy to understand, and they experienced improved accuracy of their reporting over time, which was partly because of the familiarity with using the form as an integral part of their work. As per a respondent in South Africa:

‘The mental health referral form used in South Africa refers to a one-page form where nurses are expected to tick impression, diagnosis etc. Initially when the nurses first made use of the referral form, there were minor issues with completeness and accuracy of the form, e.g. nurses would tick “other” but would not provide a narrative. It has improved now.’ (ID-02, South Africa).

However, despite the simplicity and familiarity with the new mental health forms, some respondents in India, Uganda, Nepal and South Africa expressed concerns about the additional time spent on filling out the forms. For example, in Ethiopia, health workers highlighted that the low level of literacy in the rural population lengthened the data-recording time. In Nigeria, health workers suggested that the recording time varied and extended up to 20 min, again highlighting that this was often when the patient was illiterate. One respondent at a health post in Nepal elaborated how additional time for reporting mental health indicators was a major concern for them.

‘Mental health reporting takes time but we do not have proper time, we cannot manage time according to the situation because so many patients are coming to the health post with so many types of disease, and for different types of service so that we have difficulty to manage proper time to record the information in this register. That is our problem.’ (ID-11, Nepal).

Respondents’ views on the time burden varied with the kind of information the health workers collected. Financial indicators on cost of medicine and out-of-pocket expenditure were said to be particularly difficult to collect by most respondents across countries. Some respondents referred to the sensitivity of asking people to divulge information on financial indicators. In Ethiopia, infrequently used indicators such as alcohol use disorder were found to be less important, mainly because health centres are not a preferred point of contact for the management of such disorders. In Nepal and India, indicators on severity of illness and functional assessment were difficult to collect, as these indicators were perceived to be more time-consuming than others.

Respondents reflected on the iterations of the forms that occurred during the initial phase of implementation. On one hand, some mental health system indicators were dropped, but on the other hand, certain additions were made to the existing list of indicators. For example, indicators on comorbidities were added in Uganda, Nigeria and Ethiopia, and an indicator measuring ‘where patients are referred from’ was added in Nepal based on the requirements of their health facilities. An indicator relating to the rural/urban divide was added in Ethiopia because it was considered a key equity indicator by the Federal Ministry of Health. Inclusion of a ‘history taking’ indicator in the new mental health forms was recommended in South Africa because of its importance in diagnosing patients with mental disorders.

In some countries, health supervisors and managers indicated that using the new mental health forms had improved their monitoring competencies. For example, health managers in South Africa were able to disseminate the findings from the new mental health forms through internal meetings. Similarly, in Uganda, a clinical officer reported their plans to compile mental health data at the end of the month and reflect upon it in health facility staff meetings. In three countries (Ethiopia, India and Nepal), there was no reported evidence to support use of data in improving services. However, in Nigeria, respondents were optimistic about the usefulness of mental health data collected by these new forms. In Nigeria, a respondent mentioned:

‘After collating it per facility, you know that we can collate it monthly, we can collate it every three months, we can use it every 6 months, we need to know where the problem is, what the problem and where the problem is, so and we know how to address it, how we can fix it, then we know, ah! Then who are our main targets.’ (ID-02, Nigeria).

Correspondingly, in Uganda, a senior medical officer pointed out the importance of routine mental health data for organisational planning:

‘This information [from the Mental Health HMIS] will help us to plan well for patients with mental health problems in our hospital. Now we have a shortage of drugs and it is because the government is not really aware that these are conditions that are affecting its people.’ (ID-05, Uganda).

Overall, interviewees conveyed that an improvement in mental health reporting at the facility level would enable better programme monitoring. This was a motivation to continue using the indicators.

Organisational influences

Coordinating mechanisms within/across departments

A need to understand and account for coordination issues within/across departments was an active issue in the implementation of the new mental health forms, and was emphasised explicitly by four out of the six Emerald countries (Nepal, India, Ethiopia, South Africa). In Nepal, the non-involvement of district officials delayed implementation. As a health worker in Nepal pointed out:

‘[The] HMIS section focal person of the DPHO [district programme health officer] was not involved in our [implementation of Emerald forms] process, so it created difficulties in coordination. The DPHO are aware that they need to keep the record but no concrete mechanism/plan is in place to collect and store the record.’ (ID-07, Nepal).

Similarly, in India, unclear directives from the state health directorate delayed the allocation of mental health tasks, such as recording and counselling for mental health patients, to the existing nurses/health workers and created confusion. In South Africa, a lack of coordination between prescribers and non-prescribers made access to out-patient department registers difficult, leading to infrequent and incomplete reporting. Issues also arose from parallel reporting systems in countries such as Ethiopia and India. Nurses at the district-hospital level in India used the new forms for reporting for the National Health Mission but also continued reporting in parallel for the district metal health programme.

Resource demands in introducing mental health forms

Despite a strong sense of the importance of the new forms, the additional time taken to incorporate this change within routine practice, by overstretched health workers, was expressed by respondents in India, South Africa, Nepal and Uganda. Health workers collecting data mentioned that a cause of delayed reporting was linked to the type of illness, as people affected by certain mental disorders require longer consultation and reporting time. As described by a nurse in Uganda:

‘The biggest challenges I face to finish my records is, now that it is after a long explanation that some people may realize that they have a condition.’ (ID-01, Uganda).

Often, concerns about availability of space,²⁰ counsellors (Uganda) and specialists,²⁰ and the timely supply of essential psychotropic drugs (Ethiopia, India, Nepal, South Africa, Uganda) had an indirect effect on reporting. Correspondingly, procurement of forms, registers and other basic administrative issues delayed the reporting in two (South Africa, India) out of the six Emerald countries.

To strengthen the information systems for mental health, all countries except South Africa utilised additional in-service training of health workers. Further, training on mental health indicators of staff at higher organisational levels, such as within the Department of Health, were suggested in Uganda and Ethiopia.

In all six countries, the primary care facilities were being run by the government. Minimal or no additional cost was anticipated in the initiation of mental health reporting. Health workers in Uganda, Nepal, Nigeria, South Africa and India, however, anticipated additional printing costs. In Nepal, the additional human-resource costs of additional staff required for data reporting were mentioned. In Ethiopia, respondents did not consider the minimal additional cost for introducing mental health indicators to be prohibitive, but rather highlighted the importance of committing to sustain the scale-up initiative.

To create a more sustainable environment for mental health reporting, all countries suggested the need for supervision for quality assessment and for motivating non-specialist workers to collect mental health data at primary care facilities. Success of the implementation of the new data system was attributed to the supervision of health workers through Emerald review meetings in Uganda, case manager visits in India and regular review visits to complete out-patient department registers in Ethiopia.

Integration of mental health indicators within routine information systems

In relation to the adoption of mental health indicators within the pre-existing health information systems, all country respondents reported that integration was possible. The following enabling factors for integration were described: (a) the need to report on mental health data (all countries); (b) the simplicity of the forms (Nigeria, Uganda); (c) reducing duplication by embedding into previous reporting systems (India²⁰) and (d) the perception that integration would increase demand of mental health services (Nigeria).

At the time of data collection in Ethiopia, some mental health indicators (measuring prevalence and treatment rates for behavioural disorders, epilepsy and other mental disorders) were already included in the HMIS. However, more comprehensive inclusion of mental disorders (e.g. to separate psychosis and depression) was considered important by respondents in Ethiopia. Three countries either did not report on the process of integration (South Africa) or reported poor likelihood of complete integration (India, Nepal):

‘Yes, it will be hard to integrate everything. We now have a different register and we can know what the case, whom we should call is. But if all of these go into the compiled register, then we have to distinguish the cases. There is a different register from the Government of Nepal for tuberculosis, leprosy, so if the register of mental health is made that way, then it can happen but compiling it together might be difficult.’ (ID-05, Nepal).

Similar to Nepal, some respondents from India perceived partial integration to be feasible and others anticipated the need for alternative strategies to achieve district-, state- and national-level integration. For example, for district and other lower levels of the health system, training modules for management of information systems and combined training needs were reported to be prerequisites for adequate integration. Four out of six countries (India, Nepal, Ethiopia, South Africa) commented positively with regard to the usability of the new forms in the future. In Nepal and Ethiopia, health workers perceived that the new data system would be useful for monitoring individual patient cases. In India, respondents saw the new data system as providing some baseline information on the coverage of mental health services in the future.

Behavioural influences

The level of knowledge, competence, confidence and motivation of health workers who were implementing the health information systems were all seen to affect the likelihood of implementation. Measures such as on-the-job training of health workers (all countries) and brief pamphlets for health providers to prompt the intervention (India,²⁰) improved knowledge on mental health indicators and their implementation. In terms of competency, all countries reported self-sufficiency over the new forms, which over time resulted in forming habits to complete them. Two out of the six countries said they had a system of reporting even before actual service delivery was initiated. In South Africa, the confidence of healthcare providers increased with the development and availability of resources such as the PC101 guideline and referral forms. However, in Nepal and Uganda, health workers demanded incentives for the new role. In Nigeria, experience in implementing similar information systems for other programmes assisted in boosting confidence in implementing the new forms:

‘We are already used to routinely documenting patient records for other patients. For such [mental health] patients that just came to the hospital for the first time, we record …. [demographic data], their number is on it. So, when they come back, that small card helps us to fish out their main card. So basically, we have been very sure on how to complete the new forms.’ (ID-01, Nigeria).

Discussion

Overall findings

In this cross-country qualitative study conducted in two South Asian and four sub-Saharan African countries, we explored the experiences of front-line health workers in implementing new forms to generate data on mental health indicators for monitoring the scale-up of integrated mental health programmes in primary healthcare. We found that there were a number of barriers and facilitators that affected implementation of the new forms. Some of the facilitators and barriers overlapped across the studied countries, whereas others did not. Overall, the new indicators were found to be feasible in the primary care facilities.

Our results show that barriers to measuring new mental health indicators related to the time consumed in recording some indicators (particularly severity of illness and functionality), overstretched health workers, poor coordination within and across departments and poor service delivery (owing to lack of medication, space and counsellors), which indirectly affected data capture. On the other hand, simplicity of the forms, motivation and competence of health workers and, to an extent, perceived use of mental health indicators for monitoring and programme management, were reported as facilitators for better implementation outcomes. Implementation strategies such as training courses to assist initial use of new forms and supervision (using various methods) to ensure continued use were reported to be essential.

Various new indicators developed in the country sites were reported to have contributed to mental health service improvement, such as indicators measuring essential medication stock-out in Ethiopia, India, Uganda and Nigeria; approximate time since the last appointment in Nepal and number of trained mental health professionals in Nigeria and India (refer to Table 1).

Advancement from previous studies

The successful implementation of mental health indicators is dependent not only on the strength of evidence regarding the effectiveness of that indicator, but is equally a function of its acceptability, feasibility and sustainability.⁷ Studies such as that by Ndetei and Jenkins⁸ have identified the need for unconventional and innovative approaches to collect data on mental health indicators; for example, by utilising community health workers and primary and midcadre health workforce. Our study has gone a step further by exploring perspectives on the use of forms generating data on mental health indicators by health workers at a primary care level, where mental health services are being integrated. Few studies from high-income country contexts have reported evidence regarding the feasibility of implementing performance indicators for mental healthcare programmes,²¹ and fewer still in lower-income country settings.⁹ Previous evaluations of routine health information systems also do not provide insights on implementation outcomes^22,23 and do not cover the specific domain of mental health indicators.

Understanding acceptability, feasibility and sustainability of introducing new forms

In our study, across the six countries where the Emerald programme was implemented, mental health forms to capture new indicators were accepted because of their simplicity and general satisfaction with the content. Reported confidence and competence in completing new mental health forms by participants further underlined their acceptability. Therefore, the perceived acceptability of the new reporting system was high. Contextual considerations are necessary in implementation and evaluation of information systems.^20,24 Based on context, certain countries in our study tailored approaches by adding some indicators (on sociodemographics in Ethiopia, patient history in South Africa and patient referrals in Nepal) and omitting others (indicators on cost in Ethiopia, Uganda, Nigeria and Nepal, and severity in Nigeria and India).

As suggested from other studies and reports,^25,26 every health worker in our study also understood the need for mental health information generated from routine information systems. However, study participants reported little (Uganda, Nigeria, South Africa) to no (Ethiopia, India, Nepal) evidence on the use of information generated from the new forms. Despite being a potentially cost-effective source of valuable information, there is little evidence in the literature on the reported use of HMISs.²⁷ More studies are needed to investigate the use of information to inform local planning. The learning health system approach tries to do this and is being tested in Nepal and Ethiopia as part of the OPAL (Optimizing Provider Attitudes and competence in Learning mental health systems) project,²⁸ and (in Ethiopia) through the ASSET (health system strengthening in sub-Saharan Africa) project.²⁹

Repeated measures to understand acceptability and feasibility of information systems over time can assist in improving their use for patient care and facility management. Jordans et al measured utility of these mental health indicators by quantitatively analysing health records at two time points during the implementation phase.¹¹ Nesting different assessment methods over time can redefine barriers and refine implementation of data systems in mental health programmes.

The increased workload resulting from completing the new mental health forms presents another set of sustainability challenges, particularly when the same non-specialist staff are responsible for both task-shared mental health service delivery and completing patient records. For the system of mental health reporting to function, buy-in from management staff is crucial to ensure sustainability. Similar measures have been suggested for strengthening hospital-based mental health information systems in Ghana and South Africa.^6,30

Our study affirms the need for supervision and active facilitation for inception and normalisation of the new reporting process as well as the use of routine data for local planning. This data can be used for measuring utilisation patterns over time. Similarly, accuracy and overall quality of immunisation records was seen to have been enhanced through auditing and supervision.³¹

All participants from the six countries supported the idea of integration of mental health indicators with other routine indicators, with two (India, Nepal) suggesting partial integration. There is extensive evidence of integrating mental health into primary care, with the aim of strengthening mental health information systems.³² In a review by Ndetei and Jenkins, challenges and opportunities were identified in linking mental health data systems to other data systems and vice versa for better clinical and overall outcomes.⁸ However, there is no clear evidence on integrating mental health indicators within routine information systems. Therefore, further measures are needed to assess the feasibility of integrating all data systems at the primary care level on a large scale, to estimate their cost and other system implications and to evaluate whether integration improves data quality and usage at primary care level.

Study limitations

This study has several limitations. First, as this was a qualitative study, we are reporting on the perceptions of respondents with respect to the implementation of the new mental health forms. Nonetheless, the more in-depth understanding that was possible complements the more representative findings obtained from quantitative approaches.¹¹ Second, there may have been nested social desirability bias considering that respondents were usually being interviewed at their place of work. More objective approaches, including participant observation, could have reduced social desirability bias. Third, a cross-country researcher analysed a synthesised spreadsheet developed by country researchers. Although quality checks of external reviewing were put in place, some of the local nuances may not have been captured.

In conclusion, in this qualitative study exploring the use of new mental health indicators in primary care facilities across six LMICs, the views of respondents from the different countries were mixed. Barriers to implementation across settings were related to the time taken to complete indicators measuring the functionality and symptom severity of people diagnosed with mental disorders. However, the simplicity of the new data collection method, competence and motivation of health workers in completing the new forms, and the appreciation that the new system held value and utility, were factors supporting implementation of the new system. There is a pressing need to integrate mental health indicators into routine health information systems. Even so, further research is needed to examine the sustainability of this integration and to find ways to support the use of mental health service data to improve the reach and quality of care.

Funding

S.A. is supported by Emerald project, funded under the European Commission's Seventh Framework programmes. S.A. acknowledges the financial support from the Psychiatric Research Trust and the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King's College London NHS Foundation Trust. This study is supported by the European Union Seventh Framework Programme (FP7/2007–2013) Emerald project. G.T. is supported by the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South London and by the NIHR Applied Research Centre (ARC) at King’s College London NHS Foundation Trust, and the NIHR Applied Research and the NIHR Asset Global Health Unit award. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. G.T. receives support from the National Institute of Mental Health of the National Institutes of Health under award number R01MH100470 (Cobalt study). G.T. is supported by the UK Medical Research Council in relation the Emilia (MR/S001255/1) and Indigo Partnership (MR/R023697/1) awards. D.C. is a staff member of the World Health Organization. The authors alone are responsible for the views expressed in this publication and they do not necessarily represent the decisions, policy or views of the World Health Organization. M.S. is supported by the NIHR Global Health Research Unit for Neglected Tropical Diseases at the Brighton and Sussex Medical School. G.T., C.H., I.P. and C.L. are funded by the NIHR Global Health Research Unit on Health System Strengthening in Sub-Saharan Africa, King's College London (GHRU 16/136/54) using UK aid from the UK Government. The views expressed in this publication are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. C.H. additionally receives support from AMARI as part of the DELTAS Africa Initiative (DEL-15-01).