Feasibility and impact of data-driven learning within the suicide prevention action network of thirteen specialist mental healthcare institutions (SUPRANET Care) in the Netherlands: a study protocol

Abstract

Introduction

Improvement of the quality and safety of care is associated with lower suicide rates among mental healthcare patients. In The Netherlands, about 40% of all people that die by suicide is in specialist mental healthcare. Unfortunately, the degree of implementation of suicide prevention policies and best practices within Dutch mental healthcare services is variable. Sharing and comparing outcome and performance data in confidential networks of professionals working in different organisations can be effective in reducing practice variability within and across organisations and improving quality of care.

Methods and analysis

Using formats of professional networks to improve surgical care (Dutch Initiative for Clinical Auditing) and somatic intensive care (National Intensive Care Evaluation), 113 Suicide Prevention has taken the lead in the formation of a Suicide Prevention Action Network (SUPRANET Care), with at present 13 large Dutch specialist mental health institutions. Data on suicide, suicide attempts and their determinants as well as consumer care policies and practices are collected biannually, after consensus rounds in which key professionals define what data are relevant to collect, how it is operationalised, retrieved and will be analysed. To evaluate the impact of SUPRANET Care, standardised suicide rates will be calculated adjusted for confounding factors. Second, the extent to which suicide attempts are being registered will be analysed with the suicide attempt data. Finally, professionals’ knowledge, attitude and adherence to suicide prevention guidelines will be measured with an extended version of the Professionals In Training to STOP suicide survey.

Ethics and dissemination

This study has been approved by the Central Committee on Research Involving Human Subjects, The Netherlands. This study does not fall under the scope of the Medical Research Involving Human Subjects Act (WMO) or the General Data Protection Regulation as stated by the Dutch Data Protection Authority because data are collected on an aggregated level.

Strengths and limitations of this study:

• Sharing of data which are jointly chosen, operationalised, defined and registered.

• Analyses of standardised suicide rates, allowing for benchmark comparisons between and within organisations, and for monitoring changes in service provision.

• Providing biannual feedback to the participating institutions using feedback reports alongside guided improvement makes that mental healthcare organisations and practitioners can immediately use the results into their practice.

• Due to the aggregation of the collected patient and treatment data to protect the privacy of patients, it is not possible to decrypt personal patient information to follow patients in time.

Introduction

Mental health problems are important risk factors for suicide and suicidal behaviour.^1–3 Many patients with psychiatric disorders, like mood, anxiety and personality disorders, also suffer from suicidal ideation that may lead to self-harming behaviours or to suicide.^{4 5} This makes suicide prevention a core component and responsibility of healthcare services, in particular of those working in the field of behavioural and specialist mental health.⁶ In The Netherlands, about 40% of all people that die by suicide is in specialist mental healthcare.⁷

The implementation of guideline best practices appears to be of paramount importance in preventing suicide among patients in healthcare. A recent large-scale UK study showed that the implementation of service guideline recommendations significantly reduced the suicide rate with more than two suicides per 10 000 patient contacts.⁸ Kapur et al ⁹ demonstrated a 20%–30% reduction of suicide rates in mental health services in England associated with 16 specific service improvements and implementation of guideline recommendations. In 2012, the Dutch multidisciplinary guideline for the diagnosis and treatment of suicidal behaviour was published,¹⁰ but its uptake by the field is problematic with marked degrees of variation of suicide prevention policies and practices in mental healthcare institutions across the country.¹¹ To promote its implementation, a 1-day training programme was developed and tested with significant positive effects on professionals’ competences and attitude towards guideline best practices.¹² To date, the majority of specialist mental health workers have not partaken in this training. These observations illustrate that guideline implementation and quality improvement can be difficult.^{13 14}

Although suicide is a relatively common cause of death in the high-risk population of patients in specialist mental health,^{7 15} its population base rate is too low to assess the preventative impact of specific practices or routines within a single healthcare organisation. Suicide attempts that have a much higher incidence rate are considered a valuable proxy outcome measure to evaluate the effectiveness of prevention and intervention.¹⁶ Unfortunately, most mental health organisations in The Netherlands do not systematically register and analyse suicide attempts in their patient populations. To date, annual suicide numbers are collected on institutional and national levels for reporting purposes only.¹⁵ Due to confounders and lack of standardised registration, these absolute numbers are not useful to drive learning and improvement. As a result, the extent to which the (lack of) implementation of guideline recommendations affects suicide and suicide attempt rates among patients in Dutch specialist mental healthcare is unclear.

Given the growing concerns and waning acceptance of suicide as an outcome of mental health treatment among healthcare professionals and in Dutch society, guideline implementation has become a focal point of the national suicide prevention strategy.¹¹ Commissioned by the Ministry of Welfare, Health and Sports, 113 Suicide Prevention change agents monitored the degree of implementation of guideline-based policies in the largest 25 Dutch specialist mental healthcare organisations¹¹. This resulted in growing awareness of their responsibility and potential to enhance suicide prevention efforts. Bringing together leaders and key healthcare professionals within interested specialist mental health organisations, 113 proposed to form a Suicide Prevention Action Network in healthcare (SUPRANET Care). The SUPRANET Care programme is modelled after successful examples in Dutch somatic healthcare: The National Intensive Care Evaluation (NICE) project and the Dutch Initiative for Clinical Auditing (DICA) network that showed improved quality of care as a result of benchmark feedback based on joint registration of standardised process and outcome data. Within the NICE network,¹⁷more than 90 participant intensive care units of general hospitals across The Netherlands share, evaluate and use registered data to improve the quality of care¹⁸. DICA was founded with the objective to organise and support clinical audits by facilitating on legal, technical, methodological and logistic issues¹⁹. This has led to improved quality of care with reduced practice variance in the field of colorectal, pancreatic and cardiovascular surgery. Also, SUPRANET Care takes example after the successful implementation of treatment guidelines for anxiety disorders in The Netherlands. Van Dijk et al showed that a multilevel, multifaceted and systematical implementation strategy resulted in higher quality of care leading to earlier patient recovery compared with a treatment setting in which guidelines were passively disseminated. ²⁰

SUPRANET Care aims at improving quality and safety of care to enhance suicide prevention by: (1) collecting standardised process, practice and suicide (attempt) outcome data, (2) providing benchmark feedback reports to participating organisations, (3) identifying trends and promising preventative practices and (4) systematically implement these practices across the network. After due settlement of legal and logistic issues specifically pertaining to privacy and safety of the sharing of data, the SUPRANET Care Foundation was founded. The programme’s first data collection took place in 2017. This paper describes the activities of SUPRANET Care and the evaluation of its feasibility and impact.

Suicide prevention action network (SUPRANET Care)

SUPRANET Care is the confidential learning network of at present 13 specialist mental health institutions in The Netherlands that share the ambition to optimise suicide prevention. Legally, it is a non-profit foundation governed by a board that includes four senior psychiatrists (working as chief medical officers in participating organisations); a patient and family advocate; and two PhD-level quality improvement/implementation researchers and experts (the SUPRANET Care project leader and the chairman of the National Intensive Care Evaluation). Each participant organisation has signed a contract pertaining to the confidential exchange and analysis of the data and pays an annual 10 000 euro participation fee to the SUPRANET Care Foundation. The board of the SUPRANET Care Foundation established two workgroups, the Quality of Care Group and the Registration Group, in which professionals recruited from the participant organisations participate. The first group is consulted on the quality of care indicators relevant and feasible to use in daily practice. The second group determines what data are relevant and feasible to collect, and how the data variables are operationalised and retrieved.

Multifaceted improvement programme

A multifaceted benchmark and quality improvement programme is offered to each participant containing the following three elements: (1) Biannual feedback reports with benchmark information based on data collected from all participating organisations. The feedback reports are generated by an analysis and support team of 113 Suicide Prevention and sent to and discussed with local suicide prevention teams within the organisations. (2) Improvement modules supported and initiated by the SUPRANET Care board. The aim is to develop strong multidisciplinary teams that continuously promote and monitor suicide prevention activities within the organisations. Ultimate goal is to create useful quality indicators that guide these teams. The third element (3) concerns exchange meetings, leadership development, educational sessions and outreach visits by the national support team of 113 Suicide Prevention to help the institutions interpreting their feedback reports and to formulate and execute action plans for improvement.

Recruitment of SUPRANET Care participants

Participants were recruited by 113 Suicide Prevention using invitational conferences to inform candidates about the nature of the SUPRANET programme and the possibility (and necessity) of cocreating this programme. Participants can partake within the SUPRANET Care programme annually. At this moment, 13 Dutch mental healthcare institutions participate within the network. In order to be eligible, participants have to provide specialist care involving acute inpatient clinics, residential care, outpatient clinics, crisis resolution/home treatment care and partial hospitalisation for adults and elderly (18 and older). Next to specialist care, most (n=10) provide general basic mental healthcare (GB GGZ) to patients with mild or non-complex mental health problems. From January to June 2017, the 13 Dutch mental healthcare institutions participating in the SUPRANET Care programme provided care to more than 300 000 patients.

Data collection of SUPRANET care

SUPRANET Care collects data on suicide, suicide attempts and their determinants in a national registry, as well as consumer care policies and practices to provide meaningful feedback on successful approaches to prevent suicide in mental healthcare. Consensus rounds with key professionals recruited from the participating organisations resulted in the definition of a minimal data set consisting of data pertaining to all patients in treatment with respect to gender, age, Diagnostic and Statistical Manual of Mental Disorders, Fourth/Fifth Edition (DSM-IV/V) diagnosis, Global Assessment of Functioning (GAF) score, type of care, marital status, safety plan, waiting list duration, registration of a contact person, treatment duration, suicides and suicide attempts. Furthermore, organisational characteristics of participating institutions are collected including the number of psychiatric beds, total number of psychiatric admission days and absenteeism of staff. Each SUPRANET participant agreed to deliver the data on an aggregated level to the SUPRANET Care data analyst, who combines them in a national registry. Data are collected every 6 months.

Privacy

To protect the privacy of patients, data managers of participating mental health institutions aggregate the patient and treatment data. Using aggregated data, neither SUPRANET Care nor the data analyst is able to decrypt personal patient information. Hereby, it does not fall within the scope of the General Data Protection Regulation. Aggregated data and the results of statistical analyses will be reviewed by researchers of SUPRANET Care to ensure the anonymity of both patients and mental health institutions before publication. The SUPRANET Care data analyst works in a secure network environment and uses a central database to pool the data. On request, data will be made available for other research after approval of the SUPRANET Care board.

Evaluation of the feasibility and impact of SUPRANET Care

The purpose of the evaluation is to investigate the activities of SUPRANET Care by examining its feasibility and impact on suicide and suicide attempts.

This study aims to answer:

1. Is SUPRANET Care implemented as intended, in terms of:

   1. Is the multifaceted performance feedback provided and used as intended?
   2. Does SUPRANET Care facilitate the implementation of key guideline recommendations?
   3. Is it feasible to register reliable, unambiguous data on completed suicide and on suicide attempts, and on this basis, to generate meaningful feedback?
2. Does the implementation strategy of SUPRANET Care lead to:

   1. Reduced suicide rates in time compared with baseline?
   2. Increased registration of suicide attempts in time compared with baseline?
   3. Improved mutually shared professional knowledge, attitude and adherence to suicide prevention guidelines in time compared with baseline?

Materials and methods

Design

The outcomes to evaluate the feasibility and impact of SUPRANET Care are studied using an uncontrolled longitudinal prospective design. To determine whether the SUPRANET Care implementation approach affects the three outcome variables (standardised suicide mortality, registration of suicide attempts and professional knowledge), an implementation study will be performed using an interrupted time series analysis at three levels. Level 1 is a process evaluation: is the multifaceted feedback performed as intended? Level two is the measurement of the extent of implementation of the quality indicators. Finally, the third level is the effect over time of the intervention on the three outcome variables (standardised suicide mortality, registration of suicide attempts and professional knowledge).

Evaluation procedure of the feasibility and impact of SUPRANET Care

1. Is SUPRANET Care implemented as intended, in terms of:

a. Is the multifaceted performance feedback provided and used as intended?

In order to answer the first research questions, we will evaluate the multifaceted performance feedback. Annual interview rounds will be held with the local team members and at least three professionals per institution to determine (1) the extent to which the multifaceted feedback is performed as intended, (2) whether feedback reports provide meaningful information to professionals, (3) how feedback reports are used in practice for improvement actions and (4) which best practices arise. Data derived from the interviews on the process evaluation will be described and will contribute to the knowledge of successes and barriers of the implementation approach.

b. Does SUPRANET care facilitate the implementation of key guideline recommendations and better quality of suicide prevention in mental healthcare?

To answer this research question, we will validate and examine the implementation process of a core set of relevant and action-oriented quality indicators. In order to do this, standardisation of definitions and terminology is needed. By using a standard terminology and a data dictionary, all institutions know exactly what is meant and results are comparable and can be used for benchmarking. To achieve this, first, project leads select quality indicators for suicide prevention in mental health based on a literature search and the Dutch multidisciplinary guideline. Next, the selected quality indicators are discussed in a small group of mental healthcare professionals and suicide experts (the SUPRANET Care Quality of Care group). This discussion results in a basic set of relevant and action-oriented quality indicators. Finally, the Delphi method will be used to further achieve convergence of opinion among suicide experts, members of clients’ advisory boards, experts with experiences in suicidal behaviour and healthcare professionals to create common definitions and nomenclature.

After standardisation of language, at least five quality indicators for implementation are jointly chosen. Criteria for selection of quality indicators are relevance (it affects the number of suicides in the institution), action orientation (it can be influenced by the mental health institutions or professionals themselves) and feasibility (it is feasible to implement and monitor). At least 50 experts in the field of suicide prevention and staff members of each mental health institution will receive an online questionnaire for expert opinion. After the Delphi round, the prioritised indicators and definitions are proposed to the SUPRANET Care Quality of Care group and the SUPRANET Care board. After this, the selected quality indicators will be implemented with the feedback procedure as described above.

During the study period, the implementation process on each quality indicator will be measured and evaluated with the SUPRANET Care database. Prioritised quality indicators (eg, safety plan, waiting list) will be operationalised and included in the minimal data set. Results from the SUPRANET Care database will be used to transfer knowledge among mental healthcare institutions.

c. Is it feasible to register reliable, unambiguous data on completed suicide and on suicide attempts, and on this basis, to generate meaningful feedback?

To examine the feasibility of registering completed suicide and suicide attempt data, the extent of registration will be monitored biannually on five-point rating scale (0=mental health care institution does not register suicide (attempts); 5=mental health care institutions registers all suicide (attempts) of their patients). In addition, suicide and suicide attempt data will be monitored biannually in the SUPRANET Care database whereby changes can be investigated. To this end, standardisation of definitions and terminology of suicide and suicide attempt is of great importance. The SUPRANET Care registration group determines the definitions of suicide (attempt) for adoption by all SUPRANET Care mental health institutions.

2. Does the implementation strategy of SUPRANET Care lead to suicide safer mental healthcare institutions in terms of three outcome variables?

 a. Reduced suicide rates in time compared with baseline

Standardised suicide rates will be the primary outcome variable of this study. In order to analyse the effect of the SUPRANET Care programme on reducing suicide rates in mental healthcare institutions, all suicide cases will be defined and measured. A recent pilot across four SUPRANET Care institutions showed the feasibility of extracting these data from existing data registration systems and the ability to compute suicide rates adjusted for relevant confounding factors to make comparisons over time plausible.

In order to identify differences between institutions and within institutions over time, standardised suicide rates will be calculated biannually. Differences between and changes in the number of suicides could be attributed to differences in the patient population of institutions. To compare mortality data, absolute numbers of suicide have to be adjusted for confounders (eg, demographic, psychiatric severity factors) in order to be able to attribute differences in patient suicide rates to policy, service or staff-related factors of the institutions. Therefore, for each SUPRANET Care institution, suicide rates will be adjusted for confounding factors in the client population of each institution using indirect standardisation. This method is preferred when one or more confounding specific mortality rates are based on small numbers. ²¹ Adjustment for risk factors like gender, age and DSM-IV/V diagnosis will make comparison within and between institutions more reasonable, and thereby learning possible.

b. Increased registration of suicide attempts in time compared with baseline

The second outcome variable in this study is the extent to which suicide attempts are being registered. Currently, suicide attempts are hardly registered in Dutch mental health institutions. Monitoring and registration of suicide attempts may be one of the quality indicators improving the quality of care for suicidal patients as a suicide attempt is an important risk factor for completed suicide. ²² SUPRANET Care will encourage the registration of suicide attempts of patients in care. Changes in the extent to which suicide attempts of patients are registered will be analysed with the suicide attempt data that are monitored biannually in the national SUPRANET database. We hypothesise that SUPRANET Care will lead to increased registration of suicide attempts.

c. Improved mutually shared knowledge, attitude and adherence to suicide prevention guidelines in time compared with baseline

The third outcome variable is improved mutually shared professional knowledge, attitude and adherence to suicide prevention guidelines compared with baseline. In order to measure the outcome, an extended version of the PITSTOP suicide survey (Professionals In Training to STOP suicide) among crisis teams and ambulatory care teams in each participating mental health institution will be held to test (1) the shared knowledge of suicidal behaviour and suicide prevention, (2) the attitude of healthcare professionals towards suicidal patients and (3) adherence to the clinical practice guidelines.²³ This questionnaire will be conducted in crisis teams and ambulatory care teams at baseline (before the SUPRANET Care implementation approach) with annual repeated measurements after 1, 2 and 3 years. An improvement in shared knowledge and attitude of professionals and adherence to guidelines is expected. ²³

Statistical analysis

First, the implementation progress will be analysed. The first data collection is for the purpose of having the baseline measurement. Outcomes on progress in implementation are assessed biannually at the organisational, professional and patient levels using data from the national registry of SUPRANET Care. Generalised linear model repeated measures will be used to analyse if institutions change over time on each quality indicator including registration of a contact person, waiting list duration and safety plan.

To test the effect of the SUPRANET Care implementation approach on the outcome variables, Interrupted Time-Series Analysis Procedure (ITSACORR) will be conducted, designed to analyse short time series that likely have autocorrelated errors.²⁴ ITSACORR is the preferred method above autoregressive integrated moving average in relatively short time series data ²⁵. The result is a ‘repeated time series’ that, unlike preintervention and postintervention means or percentage difference tests, enables investigation of the pattern of change over time and includes its mean level (the average of all time points) and changes in its slope.²⁴ To strengthen this uncontrolled study design, healthcare organisations’ level of implementation is added to the study. If organisations with better, or greater number of, implemented quality indicators show greater change in the outcomes, it strengthens the argument that the SUPRANET Care approach led to the changes.

Patient and public involvement

A member of the clients’ advisory board participates in the board of the SUPRANET Care Foundation. Experts with experiences in suicidal behaviour are involved in the development of SUPRANET Care in the Delphi study to create useful quality indicators for implementation. Furthermore, they actively participate in the workgroups: the Quality of Care Group and the Registration Group, in which professionals recruited from the participant organisations participate. Results of the study will be disseminated to the study participants, through feedback reports, presentations and messages on our website (www.supranetggz.nl).

Discussion

This paper describes the study protocol of a longitudinal study investigating the activities of SUPRANET Care by examining its feasibility and impact in a network formed by 13 specialist mental healthcare institutions. It will be the first study worldwide to report on the results of a confidential learning network approach in suicide prevention. We expect that SUPRANET Care will improve shared knowledge of professionals, increase the registration of suicide attempts and decrease suicide rates in Dutch mental healthcare.

Suicide is the worst outcome of mental illness. Recent evidence shows that suicide prevention in mental healthcare can be enhanced considerably by creating a culture that puts patient and staff safety first, and by systematically improving the quality and organisation of care.^{8 9} This involves the implementation of guideline best practices addressing contextual barriers and facilitators at different levels, continually addressing targeted quality and safety issues using plan-do-check-act cycles. Given the low base rate of suicides and suicide attempts, large and longitudinal databases are needed to assess the impact of quality improvement and guideline best practice implementation. The SUPRANET Care programme contains these elements and may prove to be a successful new approach to enhance suicide prevention in mental healthcare.

The strength of the study is that SUPRANET Care is a bottom-up initiative covering almost half of the large mental healthcare organisations in The Netherlands, with a clear ambition to work together to improve guideline implementation, suicide prevention and quality of care in Dutch mental health settings. Also, experts with experiences in suicidal behaviour are involved in the organisation of SUPRANET Care.

A limitation of our study is the aggregation of the collected patient and treatment data to protect the privacy of patients. Due to the aggregation, SUPRANET Care nor the data analyst is able to decrypt personal patient information to follow patients in time. However, for feedback reports and our implementation and study goals, the aggregated data appear sufficient.

SUPRANET Care is a unique project worldwide. When successful, all Dutch mental health settings will be invited to join SUPRANET and to include the quality indicators into their policy for suicide prevention. As the results will be of high relevance for countries in and outside of Europe, the implementation approach of SUPRANET Care and the gained knowledge of the evaluation study will be shared with an international audience.