Abstract
Background
The community-based mental health care programme GBV is based on the British Community Mental Health Teams and the Dutch Flexible Assertive Community Treatment model. In addition, the programme offers crisis-intervention services. A special feature of this integrated care programme is the initial standardised assessment process regarding empowerment, unmet care needs, and psychosocial functioning, used to verify the need for such a comprehensive form of care. The project evaluates the assessment process and analyses the effectiveness and cost-effectiveness of GBV compared to treatment as usual.
Methods
This randomised, controlled study includes five assessments over 2 years. In twelve regions in Germany, 1000 patients with severely impaired psychosocial functioning and unmet care needs will be recruited. Study eligibility relies on an indication for GBV based on the results of the initial assessment. The primary outcome is improved self-reported empowerment. Further outcomes include improved treatment satisfaction and subjective quality of life, reductions in patients’ unmet needs and illness-related clinical and social impairment, and an improved cost-effectiveness ratio of the resources used (from the perspectives of both statutory health insurance and the national economy). In addition, the GBV’s effects on the burden and quality of life of informal caregivers of patients will be investigated.
Discussion
The study’s results are expected to provide information on whether the community-based mental health care programme GBV contributes to improving mental health care provision in Germany. In addition, the study will show whether the GBV successfully overcomes the weaknesses that former research has identified regarding a German integrated care programme. Such improvement is particularly expected with respect to the semi-structured assessment within GBV.
Trial registration
German Clinical Trial Register, DRKS00019086. Registered on 3 January 2020.
Keywords: Community-based mental health care, Cross-sectoral care, Assessment, Severe mental illness, Depression, Schizophrenia, Empowerment, Cost-effectiveness
Administrative information
Note: the numbers in curly brackets in this protocol refer to SPIRIT checklist item numbers. The order of the items has been modified to group similar items (see http://www.equator-network.org/reporting-guidelines/spirit-2013-statement-defining-standard-protocol-items-for-clinical-trials/).
Title {1} | Effectiveness and cost-effectiveness of a community-based mental health care programme (GBV) for people with severe mental illness in Germany: study protocol for a randomised controlled trial |
Trial registration {2a and 2b}. |
German Clinical Trial Register DRKS00019086 Item 2b is met |
Protocol version {3} | 03/04/2020 Version 1.0 |
Funding {4} | This study is funded by the Innovation Fund of the Federal Joint Committee (G-BA), grant number 01NVF18028 (GBV). |
Author details {5a} |
Dr Annabel Sandra Mueller-Stierlin,1,2 Friedrich Meixner,1 Anne Kohlmann,1 Mara Schumacher,1 Anke Hänsel,1 Melanie Pouwels,1 Nicole Bias,1 Sabrina Hartl,1 Jessica Reichstein,3 Dr Elke Prestin,3 Nils Greve,3 Prof Dr Thomas Becker,1 Prof Dr Reinhold Kilian1 1 Department of Psychiatry II, Ulm University, Bezirkskrankenhaus Günzburg, Günzburg, Germany 2 Institute of Epidemiology and Medical Biometry, Ulm University, Ulm, Germany 3 Dachverband Gemeindepsychiatrie, Cologne, Germany. |
Name and contact information for the trial sponsor {5b} |
Nils Greve, Dachverband Gemeindepsychiatrie, Cologne, Germany. Phone: +49 163 2482112, mail: greve@psychiatrie.de |
Role of sponsor {5c} | The sponsor initiated the trial and was involved in designing the study. Representatives of the sponsor—NG, EP, and JR—belong to the trial steering committee. The sponsor will not be involved in the collection, management, analysis, or interpretation of the data, or in the decision to submit the report for publication. |
Introduction
Background and rationale {6a}
Beyond controlling psychiatric symptoms and ensuring the satisfaction of elementary needs, contemporary psychiatric services are expected to support individual recovery and comprehensive social inclusion among their main targets [1]. For this purpose, the World Health Organization considers the provision of comprehensive, integrated, and responsive mental health and social care services in community-based settings as the universal state of the art of mental health care systems [2].
As a country, Germany has one of the highest expenditures on mental health care in the world, in terms of population and gross domestic product. The German mental health care system is characterised by a well-developed system of psychiatric hospitals and psychiatric departments at general hospitals providing in- and outpatient treatment of patients with severe mental illness [3]. Psychiatric outpatient treatment of people with common mental disorders is mainly provided by psychiatrists and psychological psychotherapists working in private practice. Access to mental health care, including medication, is guaranteed under social law by the statutory health insurance scheme, which covers about 90% of the German population [3]. In comparison to this medical service-based mental health care system, community mental health care (with a psychosocial focus) is underdeveloped in most parts of Germany. Beyond historical developments in the German health and social care system, the current disparities in the provision of community mental health care mainly exist because, in contrast to medical care, the responsibility for providing and financing community care is at the level of the federal states and the communities [4, 5]. This fragmentation of the provision and financing of mental health care is suspected to have caused several shortcomings regarding the effectiveness and efficiency of mental health care in Germany. In particular, many experts criticise the lack of systematic coordination between in- and outpatient care resulting in the discontinuation of treatment, which in turn leads to unnecessary inpatient admissions and increases the risk of social and occupational disintegration due to mental disorders [5].
Several changes to the social code have recently been implemented to overcome the fragmentation of mental health care provision, thus extending the possibilities for the common provision and financing of medical and community mental health services. With regard to developments in other countries, over the last decade, various pilot projects have been conducted in Germany to implement and evaluate different approaches for integrating medical- and community-based mental health care. The results of these studies are inconclusive with regard to the assessed effectiveness and efficiency of integrated mental health care approaches but also with regard to the generalisability of their outcomes [6–11].
This is mainly because, with one exception [8], none of these projects implemented evidence-based approaches to integrated mental health care, such as assertive community treatment or intensive case management. A majority of the projects used certain components, like case management or multiprofessional treatment, which were provided in addition to standard care. In most cases, these additional treatment components were financed on the basis of lump sums, in contrast to the usual fee-for-service payment in the German health care system.
Moreover, to date, no German study has examined the effectiveness and efficiency of integrated mental health care compared to care as usual (CAU) in a randomised controlled trial (RCT) design. Consequently, mental health care decision-makers in Germany still lack an empirical basis by which to assess the advantages and disadvantages of implementing integrated mental health services into routine psychiatric treatment.
In a recent study, we revealed that the additional provision of integrated mental health care components based on case management to people with mental illness, on the whole, was not more effective than CAU with regard to the primary outcome (empowerment) and most of the secondary outcomes (psychosocial impairment, met needs, and satisfaction). We suspected that this result was mainly because the services provided were insufficiently tailored to the individual needs of the service users [10].
In the current study, a new approach to community-based integrated mental health care, called Gemeindepsychiatrische Basisversorgung (GBV), will be evaluated using a RCT design. GBV includes an initial need assessment and an individually tailored mental health care plan as core elements of an integrated mental health care process. All relevant medical and social services are integrated and coordinated in close consultation with the service users and their family, or other persons to whom they feel close [12, 13].
Objectives {7}
The aims of this study are to examine the adequate implementation, effectiveness, and cost-effectiveness of the community-based mental health care programme GBV for people with severe mental illness in Germany.
The following hypotheses will be examined individually:
Community-based mental health institutions can integrate GBV into regular mental health care for people with severe mental illness.
The indications for GBV can be assessed validly and reliably based on objective criteria.
Compared to CAU, the use of GBV leads to a stronger empowerment effect, made apparent by an improved subjective ability to live a self-determined and self-reliant way of life, independently arrange social relationships, actively participate in mental health care, and have increased expectations of self-efficacy and future expectations (primary outcome criterion).
Compared to CAU, the use of GBV leads to a stronger improvement of subjective quality of life as well as reductions to unmet care needs and the clinical and social impairment of patients related to their illness.
Compared to CAU, the use of GBV leads to a greater reduction in the burden on informal caregivers and to improvements in their quality of life.
From a health economic perspective, the use of GBV improves the cost-effectiveness ratio of the resources used, compared to CAU.
From the perspective of the statutory health insurance, the implementation of GBV does not lead to a significant increase in expenditures.
Trial design {8}
The present study is a multisite RCT involving people with severe mental illness and, if available, their informal caregivers, with five measurement points at 6-month intervals. The participants are randomly assigned to either the GBV or CAU group, with a 1:1 allocation. The study is primarily aimed at determining the superiority of GBV over CAU. Exploratory analysis includes superiority analysis of GBV over CAU for secondary outcomes, equivalence testing of GBV and CAU for costs of statutory health insurance agencies, and descriptive analyses.
Methods: participants, interventions, and outcomes
Study setting {9}
The study will take place in five federal states of Germany, in cooperation with twelve local community mental health care providers. The list of local service providers is available online (https://gbv.online/kontakt/regionale-ansprechpartnerinnen/). The intervention is based on contracts for model projects according to § 64b SGB V, which were concluded between the health insurance companies involved in the project and the local service providers.
Eligibility criteria {10}
The inclusion criteria for participants are as follows:
Minimum age: 18 years
Presence of mental illness in ICD-10 diagnosis groups F2, F3, F4, F5, or F6
Membership in a participating health insurance company
Permanent residence in one of the twelve defined project regions
Indications for GBV established by a screening assessment
Within a structured assessment, the local GBV teams will assess the indications for GBV in consultation with a physician or psychotherapist. The following standardised scales will be applied as part of the assessment:
The Assessment of Empowerment in Patients with Affective and Schizophrenic Disorders (EPAS), to determine the participants’ self-assessed ability to lead a self-determined and responsible way of life.
The Camberwell Assessment of Need (CAN), to determine unmet needs, which is an external assessment by members of the GBV team.
The Health of the Nation Outcome Scale (HoNOS), to determine patients’ clinical and social impairment due to illness, which is an external assessment by members of the GBV team.
The cut-off values for the three scales were derived from data from a previous study on integrated mental health care in Germany [10]. These cut-offs are ≤ 3.3 for mean EPAS total score, ≥ 4 for CAN unmet needs (sum score), and ≥ 12 for HoNOS total sum score.
Based on the results of these scales and upon consideration of the clinical evaluation in consultation with a physician or psychotherapist, the following decision regulations will be applied:
Within cut-off values for all scales: no indication for GBV
Outside cut-off values for one or two scales: indication only with a clinical recommendation
Outside cut-off values for all three scales: indication for GBV
The exclusion criteria for participants are as follows:
Primary clinical diagnosis in ICD-10 groups F0, F1, F7, F8, or F9
Participation in other integrated mental health care programmes in the last 6 months before recruitment
Who will take informed consent? {26a}
At the beginning of the assessment, local community mental health care providers will provide the patients with verbal and written information regarding the study and ask the patients to give written informed consent if they agree to participate in this study.
The patients will be asked to nominate an informal caregiver who can be interviewed with patient approval. The nominated informal caregivers will receive the written study information and will be invited to contact the local research associate if they have any remaining questions. The informal caregivers’ participation requires their written consent.
Additional consent provisions for collection and use of participant data and biological specimens {26b}
Informed consent includes consent for using data from the statutory health insurance.
Interventions
Explanation for the choice of comparators {6b}
The range of offerings for participants in the control group (CAU) is restricted to standard care. Participants in the control group have no access to the GBV’s specific services. The current standard mental health care in Germany is mainly provided by psychiatric clinics and day clinics as well as psychiatrists and psychotherapists in private practice. Vocational rehabilitation centres, community mental health care centres, and various residential and nursing facilities offer a wide range of nonmedical professional and psychosocial services. Crisis support teams have so far been implemented in only few regions of Germany, although they are now required by law in Bavaria, based on the Bavarian Mental Health Assistance Act (BayPsychKHG). In sum, mental health service offerings are wide-ranging but vary from place to place.
Intervention description {11a}
Participants in the intervention group will receive GBV in addition to routine care. GBV comprises the coordination and provision of community-based mental health care through multiprofessional GBV teams involving a physician. GBV is modelled on the British Community Mental Health Teams and the Dutch Flexible Assertive Community Treatment model in consideration of the S3 guideline Psychosocial Therapies for Severe Mental Illnesses by the German Society of Psychiatry and Psychotherapy, Psychosomatic Medicine and Neurology (DGPPN) [1].
Case management
A team member will act as a case manager for the participant and maintain continuous contact with him/her at a frequency appropriate to his/her needs. In addition, the case manager will organise network meetings based on systemic concepts between the patient, his/her informal caregivers, and all service providers involved to coordinate and harmonise their services.
Service planning
An individual mental health service plan will be developed with each participant on the basis of the participant’s needs identified at the initial assessment. Network meetings involving patients, service providers, and informal caregivers or other related persons will be used to review and adjust the mental health service planning as required but at least every 6 months.
Crisis services
A locally organised four-level crisis service will be available around the clock to the participants of the intervention group. It comprises a telephone hotline staffed by psychosocial specialists, an equally staffed crisis-intervention team, a crisis home that can be occupied and is staffed at all times, and a medical service that can be contacted by telephone around the clock and can be attended, if necessary.
Criteria for discontinuing or modifying allocated interventions {11b}
GBV is a complex mental health care programme and follows a need-adapted approach. Consequently, the intervention varies between patients and over time depending on the patients’ current mental health state and preferences.
Strategies to improve adherence to interventions {11c}
All GBV teams participate in multicentre training activities in order to ensure the standardisation of key features of the intervention.
The GBV concept includes psychosocial measures, such as relationship building with the case manager and considering patient preferences, which are expected to improve adherence to the intervention.
Relevant concomitant care permitted or prohibited during the trial {11d}
There are no restrictions regarding concomitant care during the trial.
Provisions for post-trial care {30}
Post-trial care is not planned, as it is not expected that anybody will suffer harm from trial participation. However, the concluding phase (months 19 to 24) of the GBV interventions is aimed at transferring the client to CAU.
Outcomes {12}
All outcomes will be assessed over 24 months at 6-month intervals: t0, t0 + 6 months (t1), t0 + 12 months (t2), t0 + 18 months (t3), t0 + 24 months (t4).
The primary outcome is change in empowerment (EPAS, mean total score) [14] over 24 months. This outcome was chosen because contemporary care for people with permanent severe mental illness is aimed not only at controlling disease symptoms but also at empowering patients by increasing their capacities for a largely independent lifestyle and comprehensive social and professional inclusion.
The secondary outcomes include changes over 24 months in (1) satisfaction with psychiatric treatment using the Questionnaire of Treatment Satisfaction (ZuF8, sum score) [15, 16], (2) subjective quality of life using the World Health Organisation Quality of Life—short version (WHO-QoL-BREF, mean score rescaled to the range between 0 and 100) [17], (3) count of unmet needs for psychiatric and psychosocial services using the Camberwell Assessment of Need—European version (CAN) [18, 19], and (4) psychosocial and clinical impairment using the Health of the Nation Outcome Scale (HoNOS, sum score) [20–23]. Furthermore, the cost-utility of the GBV treatment in comparison to CAU will be investigated from the societal perspective and from the perspective of the statutory health insurance (payer perspective). Costs will be estimated from the view of the statutory health insurance and of the national economy using the Client Sociodemographic and Service Receipt Inventory (CSSRI) [24]. The cost-utility ratio is defined as the costs of gaining a “healthy” year of life (QALY), measured by the Euro Quality of Life—5 dimensions questionnaire (EQ-5D) [25].
The secondary outcomes assessed in informal caregivers will be changes over 24 months in (1) the perceived burden related to support of patient suffering from serious mental illness using the Involvement Evaluation Questionnaires (IEQ-EU, sum score) [26], (2) satisfaction with the patient’s psychiatric treatment using the ZuF8 (sum score) [15, 16], and (3) subjective quality of life using the WHO-QoL-BREF (mean score rescaled to the range between 0 and 100) [17].
The participants are asked to immediately report serious adverse events. Inpatient stays will be recorded at the next follow-up assessment, at the latest, by completing the CSSRI.
Participant timeline {13}
The start of patient recruitment is set for June 1, 2020, and is to be completed within 12 months. Recruitment of and data collection from informal caregivers will take place simultaneously. The individual participation period in the study will be 24 months. In both study groups, baseline and four follow-up assessments will take place at 6-month intervals (t0, t0 + 6 months [t1], t0 + 12 months [t2], t0 + 18 months [t3], and t0 + 24 months [t4]). The follow-up windows are defined as ± 4 weeks for the intended time point. The last data collection of the last study participant should be completed by June 1, 2023 (last patient out [LPO] planned).
The following study schemes depict the course of the study (Tables 1 and 2). GBV team members of the local community mental health providers will enrol patients (including providing informed consent and screening), while evaluation team members at Ulm University will enrol informal caregivers and collect the data.
Table 1.
The activities will be conducted either by (1) GBV team members or (2) evaluation team members
Table 2.
All activities will be conducted by evaluation team members
The intervention will start immediately after t0 is completed and participants are assigned to the intervention group. Therefore, a GBV team member will join the research worker and the participant at the end of the baseline assessment. If the participant is assigned to the intervention group, the GBV team member will schedule an initiation meeting with the participant. If the participant is assigned to the CAU group, the GBV team member will inform the client about other regional services, as would be expected for CAU.
Sample size {14}
The calculation of the expected effect size is based on the results of a previous study (IVPOWER) [10]. Since the total score of the scale for measuring empowerment in patients with affective and schizophrenic disorders (EPAS) as the primary outcome proved to be rather difficult and slow to change, a low effect size of 0.20 and a high power of 0.90 were chosen for the sample size calculation of the present study.
The sample size calculation was based on a linear mixed-effects regression model with an interaction effect between study group and measurement time. The expected effect size, alpha value, beta value, number of measurements, number of study sites, and expected number of dropouts over all follow-up measurements were taken into account.
Based on the results of the IVPOWER study, an increase in the effect size of the primary outcome parameter is expected, from 0 standard deviations (SD) at baseline (t0) to 0.2 SD at the last assessment (t4). Furthermore, a decreasing dropout rate of 15% between t0 and t1, 7% between t1 and t2, 4% between t2 and t3, and 2% between t3 and t4 is expected, so that the expected sample size at t4 still comprises 72% of the initial sample.
A sample size calculation using the Web-based programme RMASS (www.rmass.org) yielded a number of at least 978 subjects for twelve study sites. On this basis, the target sample size was set at 1000 participants.
We expect to include about 350 informal caregivers, of whom about 225 will complete the study based on the experience gained in the IVPOWER study (recruitment rate = 36.8%; dropout rate = 35.6%).
Recruitment {15}
Various service providers in the regional mental health care system and the participating health insurance agencies will inform the study participants about the project. The information will be provided in personal interviews, via telephone, and via written information.
All patients included in the study will be asked to name an informal caregiver with whom he/she is in regular contact and who supports him/her in coping with his/her illness.
Assignment of interventions: allocation
Sequence generation {16a}
In order to avoid selection bias, random allocation into the two study groups—GBV and CAU—will be undertaken. Random assignment to the control or intervention group with a 1:1 allocation will occur using the ROM software [27] according to a computer-generated randomisation schedule stratified by site using permuted blocks of random sizes. The block sizes will not be disclosed, to ensure concealment.
Concealment mechanism {16b}
The Institute of Epidemiology and Medical Biometry at Ulm University will carry out the randomisation, guaranteeing the independence of the group allocation from the data collection and analysis processes.
All participants who give consent to participation and fulfil the inclusion criteria will be randomised. For this, GBV teams will contact the local research worker who will schedule the baseline assessment (t0) and request randomisation by e-mail from the Institute of Epidemiology and Medical Biometry, Ulm University, Germany. The randomisation form includes the participant ID (including his/her study ID), date of informed consent, and approval of eligibility. The requesting research worker will obtain a response by mail within one working day. Allocation disclosure will occur after the baseline assessment is completed, with the attendance of the participant, the research worker, and a member of the GBV team.
Implementation {16c}
The Institute of Epidemiology and Medical Biometry at Ulm University will generate the allocation sequence. The GBV teams will enrol participants and inform the research workers at Ulm University about the participants’ enrolment. The research workers will initiate randomisation, collect baseline data, and assign the participants to interventions.
Assignment of interventions: blinding
Who will be blinded {17a}
Due to the type of intervention, blinding of the participants and service providers is not feasible. Furthermore, research workers responsible for data collection cannot be blinded, since they would have been unblinded at the latest when documenting GBV-specific services in the CSSRI and when collecting the data for process evaluation. The same applies for all researchers involved in data monitoring and data management. Thus, only the researcher performing the main analysis will be blinded until analysis has been completed.
Procedure for unblinding if needed {17b}
Group affiliation of study participants will be revealed to the statistician by the data manager on duty after the finalisation of the main data analysis.
Data collection and management
Plans for assessment and collection of outcomes {18a}
Screening will take place at local mental health care facilities, and GBV team members will collect the data in paper-based form. The local research worker will collect further data (from t0 to t4) at a location of the study participant’s choosing, primarily in electronic form. If electronic data collection is not possible, the data will be collected on paper and electronically transferred later. Informal caregivers will be asked to fill out paper-based questionnaires on their own and send them by mail to the coordinating centre. Centralised electronic data capture is intended for these questionnaires. If necessary, specific issues will be clarified by phone.
Study-specific questionnaires will be compiled for sociodemographic data, medical history, living conditions, and social life. At the follow-up assessments (t1, t2, t3, and t4), a study-specific fidelity scale will be used to assess interventional compliance among the service providers and to gain ancillary information about the importance of and the satisfaction with local community mental health services from the patients’ perspective. In addition, all local GBV providers will be asked to answer a catalogue of quality criteria for community mental health care as defined on the basis of the S3 guideline “Psychosocial Therapies for Severe Mental Illnesses” of the DGPPN [1] in order to check for adequate implementation of GBV. Standardised questionnaires will be used to assess further process variables such as interprofessional collaboration and recovery orientation.
Only standardised questionnaires with known and at least satisfactory psychometric characteristics will be used to assess the outcomes:
Assessment of Empowerment in Patients with Affective and Schizophrenic Disorders (EPAS) [14]
World Health Organisation Quality of Life—short version (WHO-QoL-BREF) [17]
Camberwell Assessment of Need—European version (CAN) [18, 19]
Euro Quality of Life—5 dimensions (EQ-5D) [25]
Client Sociodemographic and Service Receipt Inventory (CSSRI) [24]
Involvement Evaluation Questionnaire (IEQ-EU) [26]
The primary outcome will be measured by means of the EPAS, which measures empowerment as the patient’s perceived possibilities to control his or her own life on five dimensions: daily living, social relationships and sexuality, psychiatric treatment, hope and self-efficacy and self-esteem. This self-assessment instrument has 33 items and five additional items each for patients who are employed and for patients with minor children. Cronbach’s α = 0.94 was obtained for the total scale [14]. Originally developed to assess empowerment in patients with affective and schizophrenic disorders, the EPAS has also proved to be reliable in patients with other types of severe mental disorders [10, 28].
The German short version of the Client Satisfaction Questionnaire (CSQ8), published as Fragebogen zur Patientenzufriedenheit (ZUF8) [15, 16], is a suitable tool for measuring patient satisfaction with their mental health care.
The WHO-QoL-BREF captures the patients’ subjective quality of life in the dimensions of physical health, mental well-being, social relationships, and environmental conditions. The WHO-QoL-BREF is a self-assessment tool with 25 items [17].
The CAN records patients’ perceived needs and the extent to which they are met, covering 23 areas. No special knowledge is required to work with this instrument [18, 19].
The HoNOS measures the clinical and psychosocial impairment of patients in twelve dimensions, independent of diagnosis. After an appropriate briefing, members of all mental health professions can carry out the HoNOS assessment [20–23].
The EQ-5D enables the assessment of health conditions as a basis for determining quality-adjusted life years (QALY) [25].
The use of health and psychosocial care services will be recorded using the CSSRI [24]. Health care costs are estimated by multiplying the service units used by the billing costs per unit for a 6-month period. For the health economic analysis from the perspective of statutory health insurance, expenditures will be determined on the basis of billing data.
The German version of the Involvement Evaluation Questionnaire (IEQ-EU) [11] will be used to assess the burden on informal caregivers.
To assess treatment satisfaction and subjective quality of life, the same instruments will be used for informal caregivers as for patients, specifically the ZUF8 [15, 16] and the WHO-QoL-BREF [17].
Dropouts and adverse events will be documented on standardised forms at the next follow-up assessment, at the latest.
Only trained GBV team members will administer the EPAS, CAN, and HoNOS and lead the enrolment procedure. A contact person at the coordinating centre will be available to GBV team members throughout the course of the study to answer questions on participant enrolment and screening. Refresher trainings will be offered as needed.
All local research workers have been trained and will follow uniform standard operating procedures (SOP) as to good clinical practice (GCP), recruitment, assessment, and questionnaires, for example. Advanced training sessions (e.g. training in HoNOS evaluation) of the research associates will be part of biannual project meetings. During the study, local research workers will be regularly tested for interrater reliability on the HoNOS and CAN using case vignettes.
Research workers at the coordinating centre will check the quality of data collection at each site during semiannual monitoring visits. The results will be documented in a monitoring report, which will be sent back to the local research workers and considered for the final analysis and interpretation of the data. The focus of the monitoring will primarily be on site monitoring, i.e. whether the local research workers are conducting the study in accordance with the given SOP. Furthermore, the monitoring will involve checking the completeness and plausibility of the available study documents. During the 1-year recruitment phase, the inclusion of study participants in accordance with the study protocol will be checked (e.g. by checking their eligibility and informed consent). Thereafter, the monitor will focus on the completeness and plausibility of the collected data and the documentation of serious adverse events. In addition, the monitor will offer support in case of questions or ambiguities in terms of participant recruitment, study conduct, and data collection and reporting, thus contributing to increased data quality.
Plans to promote participant retention and complete follow-up {18b}
Participants will receive reimbursement for study participation (€25 per study visit, from t0 to t4) and, if applicable, for their expenses for travelling to the interview location.
Data management {19}
Data will primarily be collected using the survey software SoSci Survey (SoSci Survey GmbH, Munich). The value ranges for the individual questions are defined in the electronic form of the questionnaire. A check for missing values is performed for each questionnaire in the electronic form. If not feasible, a paper version of the questionnaire will be used and transcribed into electronic format by the local research worker within 1 week. In order to check the data entry, staff at the coordinating centre in Ulm will re-enter a randomly chosen 5% sample of the paper-based data and subsequently compare the data sets.
All collected data will be regularly requested and stored on servers at the Department for Psychiatry and Psychotherapy II of Ulm University. Research workers at the coordinating centre at Ulm University will process and analyse the data. The data processing will include electronic data checks for plausibility and completeness as well as subsequent preparation for analysis. The data will be processed and evaluated with the software packages SPSS and SAS, among others.
The coordinating centre will oversee the whole study process by frequently communicating with local research workers regarding outstanding issues, checking the data collection procedure, and sending queries, if necessary.
Confidentiality {27}
The collection, storage, and analysis of study data will be carried out in compliance with the relevant data-protection regulations. All personal data will be pseudonymised during the collection. The study participants’ personal identifying data will be replaced by identifiers. The index tables for the assignment of personal data and identifiers will be access-protected and stored separately from the data.
Communication and data transfer between the research team and both the local community mental health care providers and statutory health insurances will take place using screening identifiers. Local research workers will use study identifiers (different from screening identifiers) for data collection during the baseline and follow-up assessments (from t0 to t4). The local research workers will be the only persons with access to the major index table with personal data, screening identifiers, and study identifiers.
Prior to enrolment in the study, the potential study participants will be informed orally and in writing about the study’s aims, nature, scope, and implications by local GBV team members. In addition, the potential study participants will be informed about the data-protection regulations within the study. The potential study participants will be given sufficient time to think about their participation and to ask questions. By completing the informed consent form, the potential study participant will agree to participate in the study and to the data-protection regulations.
The study results will be published in anonymous form. This means that it will not be possible to allocate study data to individual study participants since no access to the index tables will be given.
The study data will only be transferred in anonymised form to third parties.
The study data will be archived on servers at the Department for Psychiatry and Psychotherapy II of Ulm University.
Plans for collection, laboratory evaluation, and storage of biological specimens for genetic or molecular analysis in this trial/future use {33}
No biological specimens for genetic or molecular analysis will be collected in this trial.
Statistical methods
Statistical methods for primary and secondary outcomes {20a}
The data will be managed using IBM SPSS 25 and analysed using STATA 15 and SAS 9.4.
In order to evaluate the indication for GBV, the interrater reliability will be determined (using Spearman rank correlation) for the EPAS, HoNOS, and CAN ratings between screenings carried out by GBV team members and the baseline assessments carried out by local research workers. Furthermore, the criteria for serious mental illness will be assessed in the baseline data, and descriptive statistics will provide information on whether the target population has been recruited.
The analyses of primary and secondary outcomes will follow the intention-to-treat (ITT) principle using mixed-effects models with random time effects and time × treatment interaction effects. The covariance structure will take the study sites and individuals into account.
The cost-effectiveness ratios will be determined using the net benefit method, from the perspectives of both the statutory health insurance and national economy.
The implementation of GBV at the study sites will mainly be evaluated using descriptive methods. Mediator and moderator analysis will be conducted for the process variables, such as recovery orientation and multiprofessional collaboration.
Interim analyses {21b}
No interim analyses are planned.
Methods for additional analyses (e.g. subgroup analyses) {20b}
Diagnosis-specific effects will be examined using mixed-effect models extended by the interaction effect of time × treatment × diagnosis group. In addition, effects of GBV intensity on primary and secondary outcomes will be examined.. In-depth analysis should reveal effects of specific features of the GBV teams (e.g. peer-led teams, attitudes, recovery orientation, and interprofessional collaboration).
Methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data {20c}
In addition to the main analysis, a per-protocol (PP) approach will be followed to analyse the primary outcome in order to assess the maximal intervention efficacy in ideal conditions based on comparable outcome measurements. Criteria for the PP population will be defined in the statistical analysis plan prior to data analysis and may include
More than 2 months between baseline assessment (t0) and initial meeting with the GBV team for participants in the intervention group
Noncompliance with or infrequent use of GBV by participants in the intervention group
Follow-up assessments (t1, t2, t3, and t4) being far outside the initially planned time periods (more than 2 months before or after the intended time points: 6 months, 12 months, 18 months, and 24 months after baseline assessment t0)
No imputation of missing data will be needed for the main analysis via mixed linear models. For the sensitivity analysis, missing values will be handled using the multiple imputations method [29]. Missing data will be imputed ten times. Then, each of the completed datasets will be analysed using the proposed statistical method. According to Rubin’s rule, the final result will be the average result of all completed datasets.
Plans to give access to the full protocol, participant-level data, and statistical code {31c}
The full protocol (including the data-management and statistical-analysis plans) and the statistical code will be available on request from Prof Reinhold Kilian. Parts of the data sets will be published with open access according to data-protection regulations. More data will be accessible on request from Prof Reinhold Kilian.
Oversight and monitoring
Composition of the coordinating centre and trial steering committee {5d}
The team at the coordinating centre in Ulm comprises four persons:
RK, as the principal investigator, will oversee the entire study process. AMS will coordinate the study across sites and lead the data management and data analysis as a biostatistician. TB, as a psychiatrist, and FM, as a clinical psychologist, will address clinical issues in the study process, such as training the GBV team members and local research workers to use the assessment tools.
The trial-steering committee consists of RK, AMS, and TB from the coordinating centre and NG, EP, and JR from the Dachverband Gemeindepsychiatrie as the sponsoring organisation.
Composition of the data monitoring committee, its role, and reporting structure {21a}
The team from the coordinating centre will monitor the data.
Adverse event reporting and harms {22}
The study participants will be requested to immediately inform the local research worker (if possible) about serious adverse events. The occurrence of serious adverse events will be recorded, at the latest, at the next follow-up assessment. Subsequently, all serious adverse events will be reported to the coordinating centre in Ulm using standardised forms.
Community-based treatment, as intended in GBV, might lead to higher burden on relatives, and therefore, the burden on relatives was chosen as secondary outcome. No other adverse events are expected from trial participation.
Frequency and plans for auditing trial conduct {23}
There are no plans for independent trial auditing.
Plans for communicating important protocol amendments to relevant parties (e.g. trial participants, ethical committees) {25}
Any modifications to the protocol which may impact the study’s conduct or the potential benefit to the participants or which may affect participant safety will require a formal amendment to the protocol. Important protocol amendments must be submitted to local ethical committees and be communicated to all directly involved parties, specifically the umbrella association of community psychiatry (Dachverband Gemeindepsychiatrie), the local mental health care providers, the statutory health insurances, and the Institute for Epidemiology and Medical Biometry at Ulm University, which will randomise the participants. The entry at the German Clinical Trial Register DRKS00019086 is kept updated.
Dissemination plans {31a}
The research results will be disseminated in peer-reviewed journals, together with anonymised data, where possible. Moreover, the results will be shared through oral and poster presentations at international conferences and at national stakeholder events. All project partners are encouraged to contribute to the dissemination of project outcomes. The researchers employed at the University of Ulm will lead the scientific dissemination of the study results. Authorship will provide credit for the researcher's contributions to the study and will carry accountability.
A project report in plain language will be sent to all participants and will be accessible on the project’s website. The only publication restrictions involve the national data-protection regulations.
Discussion
This is the first RCT in Germany to examine the effectiveness and efficiency of integrated mental health care compared to CAU. This multicentre study will evaluate the impact of a community-based mental health care concept (GBV) on empowerment, quality of life, treatment satisfaction, and health economic measures among people suffering serious mental illnesses. The results of this study will help to decide whether GBV should be integrated into routine care for people with mental illness and financed by the statutory health insurance in the future.
A major strength of this study is the RCT design, which is recognised as the gold standard for examining the efficacy of interventions. High levels of academic rigour will be maintained throughout the data collection, data management, and data analysis by the involvement of independent, trained research associates and the use of standardised, validated scales. An additional strength is that the study will be conducted at different locations in Germany covering a broad range of catchment areas with different levels of urbanisation, different local mental health service systems, and different service providers. This will be advantageous when it comes to generalisation of the study results. In contrast, randomisation reduces the willingness to participate and therefore may lead to a selection effect. Another strong point of this study design is the economic evaluation from both national economic and statutory health insurance views. Patient reports on service use will provide the basis for estimating disease-related costs independently of funding agencies. In addition, health insurance claims data will be used to assess real costs for statutory health insurance.
The integrated care concept has been refined based on the results of the previous IVPOWER-study, which examined the efficacy of integrated care according to network mental health contracts. Fundamental changes were made with regard to participant enrolment and to the comparability of services across sites. An initial standardised assessment process regarding empowerment, unmet care needs, and psychosocial functioning was introduced to verify the need for such a comprehensive form of care by all community-based mental health service providers. Moreover, quality standards for GBV were set, and training activities across service providers were organised.
However, it is still to be assumed that service provision will vary across sites. As GBV is a complex intervention relying on good networking with local service providers, GBV depends on the available local service providers and their willingness to cooperate. Furthermore, it is not enough to define GBV’s individual components. Rather, the service providers’ processes and principles should be harmonised. Thus, differences in multiprofessional cooperation and the recovery orientation of GBV teams are assumed to impact GBV’s effectiveness. These aspects will be examined within the process evaluation, though the study is not designed to estimate site-specific effects. But when performing effectiveness analysis, the clusters of study sites will be taken into account in the covariance structure.
Another difficulty in this project is to avoid indirect changes in CAU by implementing GBV. GBV will be provided by teams working for community-based mental health service providers that offer a broad spectrum of services. Staff training and changes in infrastructure, especially communication channels, may also impact standard care. This applies especially to cases in which staff members are active in different services (e.g. GBV and outpatient sheltered living) at the same time. Nevertheless, this RCT should make it possible to demonstrate the added value of GBV, i.e. the additional care services such as case management, network discussions, and crisis services, even if the underlying standard care should improve in both groups.
Trial status
Protocol version 1.0
Recruitment is planned to start on June 1, 2020, and is expected to last for 12 months until June 1, 2021.
Acknowledgements
We acknowledge the support of the local service providers for community-based mental health care (GBV), specifically Netzwerk integrierte Gesundheitsversorgung (NiG) Pinel gGmbH, Berlin; Medizinisch technischen Versorgungszentrum (MtVZ) Dresden GmbH, Dresden; Gesellschaft für psychische Gesundheit in Nordrhein-Westfalen (GpG NRW) gGmbH, Solingen; Ivita Rheinland-Pfalz und Saarland gGmbH, Koblenz; INTEGRE GmbH, Augsburg; and VICENTRO gGmbH, München. Moreover, we thank all of the local health service providers who will contribute to patient recruitment. Furthermore, deep appreciation goes to the statutory health insurances, which will contribute to successful project performance through diverse supportive actions.
Abbreviations
- CAN
Camberwell Assessment of Need—European version
- CAU
Care as usual
- CSSRI
Client Sociodemographic and Service Receipt Inventory
- DGPPN
Deutsche Gesellschaft für Psychiatrie, Psychotherapie, Psychosomatische Medizin und Neurolgie (engl. German Society of Psychiatry, Psychotherapy, Psychosomatic Medicine and Neurology)
- EPAS
Assessment of Empowerment in Patients with Affective and Schizophrenic Disorders
- EQ-5D
Euro Quality of Life—5 dimensions
- GBV
Gemeindepsychiatrische Basisversorgung (engl. community-based mental health care)
- GCP
Good clinical practice
- HoNOS
Health of the Nation Outcome Scale
- IEQ-EU
Involvement Evaluation Questionnaire
- ITT
Intention-to-treat
- LPO
Last patient out
- PP
Per-protocol
- QALY
Quality-adjusted life years
- RCT
Randomised controlled trial
- SD
Standard deviation
- SOP
Standard operating procedures
- ZUF8
Questionnaire of Treatment Satisfaction; German version of the Client Service Questionnaire CSQ8
- WHO-QoL-BREF
World Health Organisation Quality of Life—short version
Authors’ contributions {31b}
AMS, NG, TB, and RK developed the study design. RK will oversee the whole study process. AMS will coordinate the study across sites and lead the data management and data analysis. AMS and RK wrote the first draft of the manuscript. All of the authors commented on and contributed to successive drafts. All of the authors read and approved the final manuscript.
Authors’ information
Not applicable.
Funding {4}
This study is funded by the Innovation Fund from the Federal Joint Committee (G-BA), grant number 01NVF18028 (GBV).
Availability of data and materials {29}
Parts of the datasets in anonymous form will be available from the corresponding author upon reasonable request. All data supporting the results reported in future articles will be included in anonymous form in these published articles and their supplementary information files.
Ethics approval and consent to participate {24}
This study was approved by the Ethics Committee of Ulm University on August 28, 2019 (application number: 259/19) and by local ethics committees (Landesärztekammer Bayern, Landesärztekammer des Saarlandes, Landesärztekammer Berlin, Landesärztekammer Sachsen, Landesärztekammer Nordrhein, Landesärztekammer Westfalen-Lippe). All of the participants will be asked to give written informed consent if they agree to participate in this study. The informed consent form describes the objective of the study and the study procedures for participants and explains what a randomised, controlled trial is. Moreover, details are given on pseudonymisation, voluntariness, risks, insurances, confidentiality, and study allowances in this study. Special attention is paid to data management, data storage, and data protection. Contact data of the research institution, service providers, and health insurances and their data protection officers are given. Informed consent forms in German are available on request from the corresponding author.
Consent for publication {32}
Not applicable.
Competing interests {28}
All of the authors declare that they have no competing interests.
The project is funded by the Innovation Fund from the Federal Joint Committee (G-BA), grant number 01NVF18028 (GBV). The Federal Joint Committee (G-BA) is the highest decision-making body of the joint self-government of physicians, dentists, hospitals, and health insurance funds in Germany.
The funding body had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Annabel Sandra Mueller-Stierlin, Email: Annabel.Mueller-Stierlin@uni-ulm.de.
Friedrich Meixner, Email: Friedrich.Meixner@uni-ulm.de.
Anne Kohlmann, Email: Anne.Kohlmann@uni-ulm.de.
Mara Schumacher, Email: Mara.Schumacher@uni-ulm.de.
Anke Hänsel, Email: Anke.Haensel@uni-ulm.de.
Melanie Pouwels, Email: Melanie.Pouwels@uni-ulm.de.
Nicole Bias, Email: Nicole.Bias@uni-ulm.de.
Sabrina Hartl, Email: Sabrina.Hartl@uni-ulm.de.
Jessica Reichstein, Email: Reichstein@psychiatrie.de.
Elke Prestin, Email: Prestin@psychiatrie.de.
Nils Greve, Email: Greve@psychiatrie.de.
Thomas Becker, Email: T.Becker@uni-ulm.de.
Reinhold Kilian, Email: Reinhold.Kilian@uni-ulm.de.
References
- 1.Gühne U, Weinmann S, Riedel-Heller SG, Becker T. S3-Leitlinie Psychosoziale Therapien bei schweren psychischen Erkrankungen. Berlin: Springer; 2019. [Google Scholar]
- 2.World Health Organization . WHO mental health action plan 2013 - 2020. 2012. [Google Scholar]
- 3.OECD . Health at a Glance 2019. Paris: OECD Publishing; 2019. [Google Scholar]
- 4.Salize H-J, Roessler W, Becker T. Mental health care in Germany: current state and trends. Eur Arch Psychiatry Clin Neurosci. 2007;257:92–103. doi: 10.1007/s00406-006-0696-9. [DOI] [PubMed] [Google Scholar]
- 5.Brieger P. Psychiatrische Versorgung in Deutschland – ein Überblick. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2019;62:121–127. doi: 10.1007/s00103-018-2861-5. [DOI] [PubMed] [Google Scholar]
- 6.Karow A, Bock T, Daubmann A, Meigel-Schleiff C, Lange B, Lange M, et al. The Hamburg-model of integrated care for patients with psychosis: part 2. Results of the clinical course over 2- and 4-years of treatment. Psychiatr Prax. 2014;41:266–273. doi: 10.1055/s-0033-1349496. [DOI] [PubMed] [Google Scholar]
- 7.Karow A, Reimer J, Koenig H-H, Heider D, Bock T, Huber CG, et al. Cost-effectiveness of 12-month therapeutic assertive community treatment as part of integrated care versus standard care in patients with schizophrenia treated with quetiapine immediate release (ACCESS trial). J Clin Psychiatry. 2012;73:e402–8. 10.4088/JCP.11%20m06875. [DOI] [PubMed]
- 8.Lambert M, Bock T, Daubmann A, Meigel-Schleiff C, Lange B, Lange M, et al. The Hamburg-model of integrated care for patients with psychosis: part 1. Rationale, treatment concept and results of the pre-study. Psychiatr Prax. 2014;41:257–265. doi: 10.1055/s-0033-1349497. [DOI] [PubMed] [Google Scholar]
- 9.Fischer F, Hoffmann K, Mönter N, Walle M, Beneke R, Negenborn S, et al. Kostenevaluation eines Modells der Integrierten Versorgung für schwer psychisch Kranke. Gesundheitswesen. 2014;76:86–95. doi: 10.1055/s-0033-1343438. [DOI] [PubMed] [Google Scholar]
- 10.Mueller-Stierlin AS, Helmbrecht MJ, Herder K, Prinz S, Rosenfeld N, Walendzik J, et al. Does one size really fit all? The effectiveness of a non-diagnosis-specific integrated mental health care program in Germany in a prospective, parallel-group controlled multi-centre trial. BMC Psychiatry. 2017;17:283. doi: 10.1186/s12888-017-1441-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kaestner D, Buechtemann D, Warnke I, Radisch J, Baumgardt J, Giersberg S, et al. Clinical and functional outcome of assertive outreach for patients with schizophrenic disorder: results of a quasi-experimental controlled trial. Eur Psychiatry. 2015;30:736–742. doi: 10.1016/j.eurpsy.2015.04.003. [DOI] [PubMed] [Google Scholar]
- 12.Greve N. Gemeindepsychiatrische Basisversorgung - ein dickes Brett. Soziale Psychiatrie. 2019;1:4–6.
- 13.Greve N, Heuchemer P. Die Gemeindepsychiatrische Basisversorgung. Psychosoziale Umschau. 2019;2:47–8.
- 14.Kilian R, Becker T, Schleuning G, Welschehold M, Hertle C, Matschinger H, Hörand S. Die Entwicklung eines standardisierten Verfahrens zur Messung von Empowerment im Prozess der psychiatrischen Behandlung von Patienten mit schweren psychischen Erkrankungen. Ulm; 2012. http://forschung-patientenorientierung.de/files/abschlussbericht_01gx0743.pdf.
- 15.Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5:233–237. doi: 10.1016/0149-7189(82)90074-X. [DOI] [PubMed] [Google Scholar]
- 16.Schmidt J, Lamprecht F, Wittmann WW. Zufriedenheit mit der stationären Versorgung. Entwicklung eines Fragebogens und erste Validitätsuntersuchungen. Psychother Psychosom Med Psychol. 1989;39:248–255. [PubMed] [Google Scholar]
- 17.Angermeyer MC, Kilian R, Matschinger H. WHOQOL-100 und WHOQOL-BREF: Handbuch für die deutschsprachigen Versionen der WHO-Instrumente zur internationalen Erfassung von Lebensqualität. Göttingen: Hogrefe; 2000. [Google Scholar]
- 18.Kilian R, Bernert S, Matschinger H, Mory C, Roick C, Angermeyer MC. Die standardisierte Erfassung des Behandlungs- und Unterstützungsbedarfs bei schweren psychischen Erkrankungen - Entwicklung und Erprobung der deutschsprachigen Version des Camberwell Assessment of Need-EU. Psychiatr Prax. 2001;28(Suppl 2):S79–S83. doi: 10.1055/s-2001-17789. [DOI] [PubMed] [Google Scholar]
- 19.Phelan M, Slade M, Thornicroft G, Dunn G, Holloway F, Wykes T, et al. The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. Br J Psychiatry. 1995;167:589–595. doi: 10.1192/bjp.167.5.589. [DOI] [PubMed] [Google Scholar]
- 20.Andreas S, Harfst T, Dirmaier J, Kawski S, Koch U, Schulz H. A psychometric evaluation of the German version of the ‘Health of the Nation Outcome Scales, HoNOS-D’: on the feasibility and reliability of clinician-performed measurements of severity in patients with mental disorders. Psychopathology. 2007;40:116–125. doi: 10.1159/000098492. [DOI] [PubMed] [Google Scholar]
- 21.Kawski S, Andreas S, Harfst T, Dirmaier J, Koch U, Schulz H. Die “Health of the Nation Outcome Scales”: Entwicklung und erste psychometrische Analysen einer deutschen Version (HoNOS-D) Psychother Psychosom Med Psychol. 2004;54:AB49 (abstract). doi: 10.1055/s-2004-822511. [DOI] [Google Scholar]
- 22.Wing JK, Lelliott P, Beevor AS, Trauer T, Slade M, Thornicroft G. HoNOS update. Br J Psychiatry. 2000;176:392–395. doi: 10.1192/bjp.176.4.392. [DOI] [PubMed] [Google Scholar]
- 23.McClelland R, Trimble P, Fox ML, Stevenson MR, Bell B. Validation of an outcome scale for use in adult psychiatric practice. Qual Health Care. 2000;9:98–105. doi: 10.1136/qhc.9.2.98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Roick C, Kilian R, Matschinger H, Bernert S, Mory C, Angermeyer MC. Die deutsche Version des Client Sociodemographic and Service Receipt Inventory – Ein Instrument zur Erfassung psychiatrischer Versorgungskosten. Psychiatr Prax. 2001;28(Suppl 2):S84–S90. doi: 10.1055/s-2001-17790. [DOI] [PubMed] [Google Scholar]
- 25.Brooks RG, Rabin R, de Charro F. The measurement and valuation of health status using EQ-5D: a European perspective: Evidence from the EuroQol BIOMED Research Programme. Dordrcht: Kluwer; 2003. [Google Scholar]
- 26.Bernert S, Kilian R, Matschinger H, Mory C, Roick C, Angermeyer MC. Die Erfassung der Belastung der Angehörigen psychisch erkrankter Menschen: Die deutsche Version des Involvement Evaluation Questionnaires (IEQ-EU) Psychiatr Prax. 2001;28(Suppl 2):97–101. doi: 10.1055/s-2001-17792. [DOI] [PubMed] [Google Scholar]
- 27.Rohlmann F, Muche R, Goldschmidt L. Randomisierung in klinischen Studien.Praktische Umsetzung mit dem Randomisationsprogramm ROM. In: Schweizer B, Großmann C, Meule M, Gaus W, editors. Tagungsband zur 8. DVMD-Tagung in Ulm. Ulm: Universitätsverlag Ulm; 2004. p. 168–71.
- 28.Panzirsch M, Frasch K, Kilian R. Empowerment und Stigma als Mediatorvariablen zwischen Krankheitsschwere und Lebensqualität bei Patienten mit einer affektiven Erkrankung. Psychiatr Prax. 2019;46:439–444. doi: 10.1055/a-1002-0508. [DOI] [PubMed] [Google Scholar]
- 29.Rubin DB. Multiple Imputation for Nonresponse in Surveys. Hoboken: Wiley-Interscience; 2004. ISBN: 978-0-471-65574-9.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Parts of the datasets in anonymous form will be available from the corresponding author upon reasonable request. All data supporting the results reported in future articles will be included in anonymous form in these published articles and their supplementary information files.