Abstract
Background
Improvement of quality of care for psychiatric patients is a key objective of health care systems worldwide. Consequently, there is an increasing interest in documenting quality of care; however, little is known about the validity of the available data on psychiatric care.
Objective
To assess the validity of process of care data recorded in the Danish National Indicator Project (DNIP), a national population-based registry containing quality of care data of patients diagnosed with schizophrenia in Denmark.
Methods
A random sample of 1% of patients with schizophrenia registered in the DNIP between 2004 and 2009 (111 inpatient and 85 outpatient) was identified for validation. Medical records for these patients, which were used as the gold standard, were retrieved and reviewed for information on the processes of care received. Agreement between the data in the DNIP and the medical records were assessed by computing sensitivity, specificity, and positive and negative predictive values.
Results
The agreement between the recorded processes of care in the DNIP and in the medical records varied substantially across the individual process of care variables. However, a collection of the processes of care demonstrated a high agreement (80% or more) between data in the DNIP and the medical records, according to all examined aspects of data validity (sensitivity, specificity, and positive and negative predictive values). The medical records contained varying levels of missing information regarding the processes of care, from 1% for antipsychotic medication prescription to 54% for psychoeducation.
Conclusion
Current documentation practices in Danish psychiatric hospitals appear to be inconsistent and may preclude the use of psychiatric medical records as the gold standard when validating registry data.
Keywords: validation, quality of care, registry, schizophrenia, The Danish National Indicator Project
Introduction
Psychiatric registries are frequently used for administration, quality of care improvement, and research. The advantages of using well-established registries include substantial time and cost savings, availability of large sample sizes, high levels of representation, and lower recall bias and nonresponse bias.1,2 The usefulness of registries is entirely dependent on the validity of the data, including the completeness and quality of the recorded data; however, little is known about the validity of psychiatric registries. Byrne et al3 identified and reviewed 14 studies that examined the validity of administrative registries used in psychiatric research. The methodological quality of the constituent studies varied widely, making it difficult to draw any conclusions. Consequently, further information on the validity of psychiatric registries is needed to ensure that available data are used in a meaningful way, to avoid misleading results.
Working systematically with improvements in psychiatric health care relies on access to empirical evidence of the care provided to patients in routine clinical settings. Consequently, there is increasing interest in data on the basic processes of psychiatric health care, including access, detection, appropriateness of treatment, safety, and continuity of care.4 Nonetheless, very little is known about the validity of the available data regarding the processes of psychiatric care.
The aim of this study was to examine the validity of electronically maintained records of the processes of care for individuals with schizophrenia in the Danish National Indicator Project (DNIP) – a national schizophrenia registry.
Methods
The Danish health care system provides tax-paid health care to the country’s 5.5 million residents, including free access to hospital care and general practitioners. Patients with schizophrenia who require psychiatric treatment are exclusively admitted to public psychiatric hospitals. Data from this entire population is recorded through a large number of administrative and medical registries, which are used for monitoring and regulating all central aspects of the public sector, including the health care system. All Danish citizens are assigned a unique, ten-digit civil registration number that is used in all registries,5 enabling unambiguous linkage among the sources of data.
The DNIP – schizophrenia
Participation in the DNIP is mandatory for all Danish psychiatric hospitals, relevant clinical departments, and units treating patients with schizophrenia. The objective of the DNIP is to document, monitor, and improve diagnosis and care provided by the Danish psychiatric health care system among patients with schizophrenia. The registry has been established through collaboration among the Ministry of Health, the National Board of Health, the Danish Regions, health care professional organizations, and scientific societies.6 Data collection was initiated in 2004 and the data collected include information on key processes of care among psychiatric inpatients and outpatients. These processes of care were identified by an expert panel consisting of physicians, psychologists, nurses, occupational therapists, and social workers,6 and are described in next column. Data for the DNIP are collected using a registration form with detailed written instructions. For inpatients, the registration form is completed based on documentation in the medical record at the time of discharge from the psychiatric ward. Data from outpatients are recorded once a year. In 2009, the DNIP registry contained records for 98% of all inpatients and 93% of all outpatients with schizophrenia in the Danish psychiatric health care system, when compared with the National Registry of Patients, which captures all admissions and discharges from Danish hospitals. 7 The DNIP defines incident patients with schizophrenia as individuals who have been diagnosed with schizophrenia 12 months before the date of hospital discharge.
Study population
We aimed to retrieve and review medical records from a random sample of adult (≥18 years) patients (including inpatients and outpatients) recorded in the DNIP between January 1, 2004, and March 31, 2009, to determine agreement between information in the medical records and electronically entered information. Schizophrenia was defined according to the International Classification of Diseases version 10 (ICD-10)7 F20.0 to F20.9. A computer-generated 1% random sample was drawn among inpatients (n = 10,757) and outpatients (n = 7800). A total of 125 inpatients and 100 outpatients were included, 32 of whom were incident patients. The sample size was not determined based on a formal analysis but, rather, as a pragmatic tradeoff between achieving a reasonable statistical precision of the estimated sensitivity, specificity, and positive and negative predictive values and having a feasible number of medical records to retrieve and review.
Processes of care
Table 1 lists the definitions of recorded processes of care contained within the DNIP. The main processes are: antipsychotic medical treatment, contact with relatives, and psychoeducation. In addition, inpatients’ data included scheduled psychiatric aftercare, professional support, and suicide risk assessment. Data on individuals receiving outpatient treatment include continued contact with outpatient clinics and professional contact person(s). Finally, data on the processes of diagnosing schizophrenia in the incident patients were collected, including assessment of psychopathology, assessment of psychopathology by a psychiatric specialist, assessment of psychopathology using interview forms, assessment of cognitive function by a psychologist, and assessment by a social worker.
Table 1.
Definitions of the processes of care provided for patients with schizophrenia
| Processes of care | Definition |
|---|---|
| Antipsychotic medical treatment | Prescription for antipsychotic medical treatment at psychiatric hospital discharge or status period as psychiatric outpatient14–17 |
| Contact with relatives | Staff contact with the patient’s relatives during hospitalization or status period18–20 |
| Psychoeducation | Overall indication of whether the patient received psychoeducation during hospitalization or status period21–23 |
| Psychiatric aftercare | Planned professional support for inpatients after discharge in patient’s own homes, residential facilities, or care homes, as well as for housing24 |
| Professional support | Scheduled visits for inpatients with psychiatric aftercare, including outpatient psychiatric treatment team, psychiatric clinic, or general practice after hospital discharge |
| Suicide risk assessment | In the week leading up to the patient’s discharge, a required clinician’s assessment of the patient’s risk of suicide, including an evaluation of depressive symptoms |
| Ongoing contact with outpatient | Only outpatients: indication of whether the staff has ongoing contact with the patient during outpatient treatment |
| Professional contact person | Only outpatients: indication of whether the patient has an assigned contact person during outpatient treatment |
| Only incident patients | |
| Assessment of psychopathology | An overall record of whether the patient has been assessed for psychopathological characteristics |
| Assessment of psychopathology by specialist in psychiatry | Indication of whether the patient’s psychopathological assessment was performed by a specialist in psychiatry |
| Assessment of psychopathology by interview form | Indication of whether the patient received a diagnostic interview with an established interview instrument, such as the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) or Present State Examination (PSE)25,26 |
| Assessment of cognitive function | Indication of whether cognitive testing of the patient was performed by a psychologist |
| Assessment by social worker | Assessment of need for acute or longer-term support, such as help with changing housing, financial help to purchase medicine, educational guidance, rehabilitation, and application for disability benefits |
Medical record review
For patients with multiple registrations in the DNIP, we randomly selected a hospitalization record among inpatients and a status record date among outpatients and located data from the equivalent dates in the medical records. We were unable to retrieve the medical paper records for 10% (n = 13) of the inpatients and 15% (n = 15) of the outpatients; therefore, analyses were conducted on 111 inpatients and 85 outpatients. The 13 inpatients had been hospitalized in ten different hospitals, and the 15 outpatients had contact with eleven different outpatient clinics. No systematic differences were found in relation to the distribution of processes of care recorded in the DNIP, sex, or age, when comparing patients with and without available medical records.
All medical records were reviewed by a single reviewer (CGP, who is a certified and clinically experienced nurse) using the DNIP data definitions. The reviews were performed systematically, in accordance with the DNIP definitions for individual processes of care. Problematic records were discussed with specialists in psychiatry and clinical epidemiology, and consensus was reached.
Data from the medical records were entered into a database, and their accuracy was confirmed by re-entering the data a second time.
Ethics
The study was approved by the Danish Data Protection Agency (J no 2008-41-2894) and the Danish National Board of Health (J no 7-604-04-2/80/AHA).
Statistical analysis
The sensitivity, specificity, and positive and negative predictive values of data recorded in the DNIP were computed using the reviewed medical records as the gold standard. Sensitivity of DNIP registry data was expressed as the proportion of patients who were registered with a verified and received process of care in the DNIP (numerator), divided by the total number of patients in our random sample who received a process of care, based on the medical records (denominator). Specificity was computed as the number of patients without a specified process of care according to the DNIP (numerator), divided by the total number of patients without a process of care, according to the medical records. The positive predictive value (PPV) of data recorded in the DNIP was computed as the number of patients with a verified received process of care as the numerator and the total number of patients registered with a received process of care in the DNIP as the denominator. Similarly, negative predictive values were computed as the number of patients without a process of care as the numerator and the total number of patients registered without a process of care in the DNIP as the denominator. All estimates are presented with a 95% confidence interval (CI). The data were analyzed using STATA software (v 11.1; StataCorp LP, College Station, TX).
Results
A total of 18,557 patients with schizophrenia were registered in the DNIP from January 1, 2004, until March 31, 2009. Of these patients, 58% (n = 10,757) were inpatients and 42% (n = 7800) were outpatients (Table 2).
Table 2.
Patient characteristics and processes of care among patients with schizophrenia
| Total n = 18,557 |
Inpatients n = 10,757 |
Outpatients n = 7800 |
|
|---|---|---|---|
|
|
|||
| n (%) | n (%) | n (%) | |
| Sex | |||
| Female | 7445 (40) | 4352 (40) | 3093 (40) |
| Male | 11,112 (60) | 6405 (60) | 4707 (60) |
| Age (years) | |||
| ≥18–29 | 4032 (22) | 2632 (24) | 1400 (18) |
| 30–39 | 4608 (25) | 2761 (26) | 1847 (24) |
| 40–49 | 4562 (25) | 2628 (24) | 1934 (25) |
| 50–60 | 3198 (17) | 1700 (16) | 1498 (19) |
| 60+ | 2157 (11) | 1036 (10) | 1121 (14) |
| Antipsychotic medical treatment | |||
| Yes | 15,836 (85) | 9264 (86) | 6572 (84) |
| No | 1117 (6) | 417 (4) | 700 (9) |
| Unknown | 1604 (9) | 1076 (10) | 528 (7) |
| Contact with relatives | |||
| Yes | 7325 (39) | 4581 (43) | 2744 (36) |
| No | 10,147 (55) | 5438 (50) | 4709 (60) |
| Unknown | 1085 (6) | 738 (7) | 347 (4) |
| Psychoeducation | |||
| Yes | 10,851 (58) | 6338 (59) | 4513 (58) |
| No | 6753 (37) | 3792 (35) | 2961 (38) |
| Unknown | 953 (5) | 627 (6) | 326 (4) |
| Psychiatric aftercarea | |||
| Yes | – | 5325 (49) | – |
| No | – | 1079 (10) | – |
| Unknown | – | 4353 (41) | – |
| Professional supporta | |||
| Yes | – | 1218 (11) | – |
| No | – | 210 (2) | – |
| Unknown | – | 9,329 (87) | – |
| Suicide risk assessmenta | |||
| Yes | – | 4528 (42) | – |
| No | – | 1058 (10) | – |
| Unknown | – | 5171 (48) | – |
| Ongoing contact with outpatientb | |||
| Yes | – | – | 3433 (44) |
| No | – | – | 1367 (18) |
| Unknown | – | – | 3000 (38) |
| Professional contact personb | |||
| Yes | – | – | 3704 (47) |
| No | – | – | 375 (6) |
| Unknown | – | – | 3721 (47) |
Notes:
Only inpatients;
Only outpatients.
Table 3 illustrates the proportion of individuals lacking registration of processes of care in medical records among the random sample selected for the current study. The completeness of medical information on antipsychotic medical treatment was high, as only 1% (n = 2) of the patients lacked this registration information. In contrast, the majority of medical records (54%) did not contain information on the use of psychoeducation (n = 106), and 50% of the cases lacked information regarding staff contact with patients’ relatives (n = 98). Among the inpatients, 43% of the records had no information on professional support after discharge (n = 48), and 40% lacked information on suicide risk assessment (n = 45). Among the diagnostic processes of care for incident patients, the highest proportions of medical records with missing data were 37.5% without information on the assessment of psychopathology by psychiatric specialists (n = 12) and 37.5% lacking information on the assessment of psychopathology by interview form (n = 12). The records with missing data were not restricted to a few departments; they represented a large number of institutions.
Table 3.
Proportions of medical records that lacked information on individual processes of care among patients with schizophrenia
| Processes of care | Medical records n (%) |
|---|---|
| All patients | n = 196 |
| Antipsychotic medication | 2 (1) |
| Contact with relatives | 98 (50) |
| Psychoeducation | 106 (54) |
| Only inpatients | n = 111 |
| Psychiatric aftercare | 11 (10) |
| Professional support | 48 (43) |
| Suicide risk assessed | 45 (41) |
| Only outpatients | n = 85 |
| Ongoing contact with patients | 0 (−) |
| Professional contact person | 0 (−) |
| Only incident patients | n = 32 |
| Assessment of psychopathology | 2 (6) |
| Psychopathology by specialist | 3 (9) |
| Psychopathology by interview form | 12 (38) |
| Cognitive test | 12 (38) |
| Social worker | 6 (19) |
Table 4 shows the overall sensitivity, specificity, and predictive values for each of the 13 processes of care in the DNIP compared with the information available in the medical records of the current sample. In general, agreement between the recorded processes of care in the DNIP and the medical records varied substantially across the individual processes of care. A few of the processes demonstrated high agreement (80% or more) between data in the DNIP and the medical records for all examined parameters (sensitivity, specificity, positive and negative predictive values). Sensitivity ranged from 11% to 96%, with four processes having a sensitivity of 80% or more: medical antipsychotic treatment had 96% sensitivity (95% CI: 92%, 98%); psychoeducation had 80% sensitivity (95% CI: 69%, 87%); evaluated psychopathology had 81% sensitivity (95% CI: 62%, 94%); and assessment of psychopathology by psychiatric specialist had 85% sensitivity (95% CI: 62%, 97%). Specificity ranged from 11% to 100%, with three processes having a specificity of 80% or more: contact with relatives had 82% specificity (95% CI: 48%, 98%); cognitive testing had 100% sensitivity (95% CI: -); and social worker involvement had 100% specificity (95% CI: -). The ranges of the positive and negative predictive values were 44%–100% and 5%–100%, respectively, with six and one processes, having a predictive value of 80% or more. This included medical antipsychotic medication with a PPV of 97% and 95% CI of 94%, 99%; psychiatric aftercare with a PPV of 89% and 95% CI of 78%, 95%; ongoing contact with patients with a PPV of 100% and 95% CI -; professional contact person with a PPV of 98% and 95% CI of 91%, 99%, evaluated psychopathology with a PPV of 85% and 95% of 65%, 96%; and assessment by a social worker with a PPV of 86% and 95% CI of 65%, 97%. The negative predictive value included psychopathology by specialist 100%.
Table 4.
Sensitivity, specificity, and predictive values of data on processes of care registered in the DNIP, using medical records as gold standard
| Verified positive/total N | Sensitivity % (95% CI) | Verified negative/total N | Specificity % (95% CI) | Verified N/total N | PPV % (95% CI) | Total negative N/total N | NPV % (95% CI) | |
|---|---|---|---|---|---|---|---|---|
| Processes of care | ||||||||
| Antipsychotic medication | 179/186 | 96 (92, 98) | 5/8 | 17 (24, 91) | 179/184 | 97 (94, 99) | 5/11 | 45 (17, 77) |
| Contact with relatives | 62/87 | 71 (60, 80) | 9/11 | 82 (48, 98) | 62/78 | 79 (69, 88) | 9/109 | 8 (1, 15) |
| Psychoeducation | 56/70 | 80 (69, 87) | 6/62 | 10 (4, 20) | 56/127 | 44 (35, 53) | 6/62 | 9 (1, 20) |
| Psychiatric aftercarea | 55/97 | 57 (46, 67) | 1/3 | 33 (1, 91) | 55/62 | 89 (78, 95) | 1/10 | 9 (1, 45) |
| Professional supporta | 10/58 | 17 (9–29) | 0/5 | – | 10/14 | 71 (42, 92) | 0/1 | – |
| Suicide risk assesseda | 39/66 | 56 (46, 71) | –d | –d | 39/55 | 71 (57, 82) | –d | –d |
| Contact with patientsb | 52/83 | 63 (51, 73) | 1/2 | 50 (1, 99) | 52/52 | 100 (−) | 1/18 | 5 (1, 27) |
| Professional contact personb | 61/84 | 73 (62, 82) | –d | –d | 61/62 | 98 (91, 99) | –d | –d |
| Evaluated psychopathologyc | 22/27 | 81 (62, 94) | 0/3 | – | 22/26 | 85 (65, 96) | 0/1 | – |
| Psychopathology by specialistc | 17/20 | 85 (62, 97) | 1/9 | 11 (2, 48) | 17/25 | 68 (46, 85) | 1/1 | 100 (−) |
| Psychopathology by interview formc | 2/18 | 11 (1–35) | 0/2 | – | 2/2 | 100 (−) | 0/1 | – |
| Cognitive testc | 6/19 | 32 (13, 57) | 1/1 | 100 (−) | 6/8 | 75 (35, 97) | 1/17 | 5 (1,18) |
| Social worker assessmentc | 19/25 | 76 (55, 90) | 1/1 | 100 (−) | 19/22 | 86 (65, 97) | 1/4 | 25 (1,80) |
Notes:
Only inpatients;
Only outpatients;
Only incident patients;
Patients did not have “no” recorded in the medical records.
Abbreviations: PPV, positive predictive value; NPV, negative predictive value; CI, confidence intervals.
Discussion
In this validation study, we found varying levels of agreement between data on delivered processes of schizophrenia care recorded in a national schizophrenia registry and the original medical records. The variation was found in relation to sensitivity, specificity, and positive and negative predictive values. Documentation of the provided care was incomplete in a high proportion of the examined medical records, and the records were consequently a problematic gold standard in the validation process.
Our findings indicate a concerning pattern of documentation practices in the Danish psychiatric hospitals regarding patients with schizophrenia. A variety of factors may potentially have contributed to this problem, including a limited availability of electronic medical records systems in the psychiatric clinical wards during the study period, difficulty locating a substantial proportion of the paper versions of medical records, and a tradition of inadequate documentation of psychiatric care, compared with current legislative requirements. Several studies have found that electronic records are beneficial in clinical documentation practices, with the rationale that they contain more complete and readily available information than paper records.9 This convenience ultimately promotes a better quality of care and patient safety. In addition, integrated electronic reminder systems found in electronic medical records may improve the quality of care,10 although this is not a consistent finding,11 and the use of electronic checklists has been reported to improve the quality of patient care.12 The disadvantages of electronic medical records include the high financial costs, difficulty in technical training and support for the staff, and complexity of the software and its interface (which may be combined with a lack of customizability, reliability, and standardization). Furthermore, due to difficulty in accessing computers and concerns that the use of electronic patient records will reduce the time spent bedside with the patients, logistical problems also may play a role in the underuse of electronic medical records.13 However, as demonstrated in this study, the use of paper records, which is still the current practice in many clinical settings, may be associated with serious logistical problems regarding locating and retrieving the records from institutional or hospital archives. This problem may constitute a particular challenge with patients with schizophrenia, due to the frequent readmissions in this patient group. The insufficient documentation of the provided care in the medical records that we were able to locate and retrieve may reflect the combination of a health care system under pressure due to financial restrictions and an inappropriate tradition of not giving high priority to organized documentation.
The insufficient access to a valid gold standard of provided care represents a major challenge in attempts to validate the DNIP, as well as other clinical registries. In addition to striving toward the implementation of electronic patient records and improved documentation practices, alternative information sources could be considered for the purpose of validation. Sources other than medical records could potentially contain valid corroborating data, such as nursing records and laboratory information systems. Therefore, combining different information sources could potentially result in a more accurate picture of the provided care for validation purposes and beyond, rather than relying solely on patient medical records.
The strengths of our study include the use of a random sample of both inpatients and outpatients with schizophrenia, prospectively registered in a national, population-based registry. Therefore, the findings are likely to be representative of the data recorded in the registry. The use of the unique civil registration number system in Denmark enabled us to locate the majority of the relevant medical records and accurately link information between medical records and the DNIP. All records were reviewed by the same experienced individual. While the criteria used to make the judgments were developed in collaboration with experienced colleagues and according to the detailed data definitions of the DNIP, the validation decisions within the current study are consistent, as they rely on the judgment of a single person. The most important limitation of this study, and perhaps the most important finding, was the high proportion of medical records with missing information on several of the examined processes of care, which made it difficult to draw firm conclusions about the validity of the data recorded in the DNIP.
The tasks of conducting quality assessments, facilitating the coordination of care, and managing an individual patient’s treatment all rely on complete and up-to-date medical records. Inaccurate documentation in medical records complicates each of these necessary activities. Current medical record documentation practices in Danish psychiatric hospitals appear to be inconsistent and may preclude the use of psychiatric medical records as the gold standard when validating registry data.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Sørensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–442. doi: 10.1093/ije/25.2.435. [DOI] [PubMed] [Google Scholar]
- 2.Olsen J. Using secondary data. In: Rothman KJ, Greenland S, Lash TL, editors. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott Williams and Wilkins; 2008. pp. 481–491. [Google Scholar]
- 3.Byrne N, Regan C, Howard L. Administrative registers in psychiatric research: a systematic review of validity studies. Acta Psychiatr Scand. 2005;112(6):409–414. doi: 10.1111/j.1600-0447.2005.00663.x. [DOI] [PubMed] [Google Scholar]
- 4.Hermann RC. Improving Mental Healthcare: A Guide to Measurement-Based Quality Improvement. Washington DC: American Psychiatric Press, Inc; 2005. [Google Scholar]
- 5.Pedersen CB. The Danish Civil Registration. Scand J Public Health. 2011;39(7 Suppl):22–25. doi: 10.1177/1403494810387965. [DOI] [PubMed] [Google Scholar]
- 6.Mainz J, Krog BR, Bjørnshave B, et al. Nationwide continuous quality improvement using clinical indicators: the Danish National Indicator Project. Int J Qual Health Care. 2004;16:i45–i50. doi: 10.1093/intqhc/mzh031. [DOI] [PubMed] [Google Scholar]
- 7.Danish National Board of Health. The Danish National Indicator Project Audit Schizophrenia [webpage on the Internet] Aarhus: Danish National Board of Health; 2009. [Accessed December 15, 2011]. Available from: http://www.nip.dk/ Danish. [Google Scholar]
- 8.World Health Organization. International Statistical Classification of Diseases and Related Health Problems (ICD) Geneva: World Health Organization; 2012. [Accessed March 28, 2012]. 10th Revision. Available from: http://www.who.int/classifications/icd/en/ [Google Scholar]
- 9.Tsai J, Bond G. A comparison of electronic records to paper records in mental health centers. Int J Qual Health Care. 2008;20(2):136–143. doi: 10.1093/intqhc/mzm064. [DOI] [PubMed] [Google Scholar]
- 10.Sequist TD, Gandhi TK, Karson AS, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12(4):431–437. doi: 10.1197/jamia.M1788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J. Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010;182(5):E216–E225. doi: 10.1503/cmaj.090578. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wolff AM, Taylor SA, McCabe JF. Using checklists and reminders in clinical pathways to improve hospital inpatient care. Med J Aust. 2004;181(8):428–431. doi: 10.5694/j.1326-5377.2004.tb06366.x. [DOI] [PubMed] [Google Scholar]
- 13.Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. doi: 10.1186/1472-6963-10-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Davis JM, Chen N, Glick ID. A meta-analysis of the efficacy of second-generation antipsychotics. Arch Gen Psychiatry. 2003;60(6):553–564. doi: 10.1001/archpsyc.60.6.553. [DOI] [PubMed] [Google Scholar]
- 15.Geddes J, Freemantle N, Harrison P, Bebbington P. Atypical anti-psychotics in the treatment of schizophrenia: systematic overview and meta-regression analysis. BMJ. 2000;321(7273):1371–1376. doi: 10.1136/bmj.321.7273.1371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Leucht S, Wahlbeck K, Hamann J, Kissling W. New generation antipsychotics versus low-potency conventional antipsychotics: a systematic review and meta-analysis. Lancet. 2003;361(9369):1581–1589. doi: 10.1016/S0140-6736(03)13306-5. [DOI] [PubMed] [Google Scholar]
- 17.Tuunainen A, Wahlbeck K, Gilbody SM. Newer atypical antipsychotic medication versus clozapine for schizophrenia. Cochrane Database Syst Rev. 2000:CD000966. doi: 10.1002/14651858.CD000966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Pharoah FM, Mari JJ, Streiner D. Family intervention for schizophrenia. Cochrane Database Syst Rev. 2000:CD000088. doi: 10.1002/14651858.CD000088. [DOI] [PubMed] [Google Scholar]
- 19.Pilling S, Bebbington P, Kuipers E, et al. Psychological treatments in schizophrenia: II. Meta-analyses of randomized controlled trials of social skills training and cognitive remediation. Psychol Med. 2002;32(5):783–791. doi: 10.1017/s0033291702005640. [DOI] [PubMed] [Google Scholar]
- 20.Pitschel-Walz G, Leucht S, Bauml J, Kissling W, Engel RR. The effect of family interventions on relapse and rehospitalization in schizophrenia – a meta-analysis. Schizophr Bull. 2001;27(1):73–92. doi: 10.1093/oxfordjournals.schbul.a006861. [DOI] [PubMed] [Google Scholar]
- 21.Pekkala E, Merinder L. Psychoeducation for schizophrenia. Cochrane Database Syst Rev. 2002:CD002831. doi: 10.1002/14651858.CD002831. [DOI] [PubMed] [Google Scholar]
- 22.Merinder LB. Patient education in schizophrenia: a review. Acta Psychiatr Scand. 2000;102(2):98–106. doi: 10.1034/j.1600-0447.2000.102002098.x. [DOI] [PubMed] [Google Scholar]
- 23.Zygmunt A, Olfson M, Boyer CA, Mechanic D. Interventions to improve medication adherence in schizophrenia. Am J Psychiatry. 2002;159(10):1653–1664. doi: 10.1176/appi.ajp.159.10.1653. [DOI] [PubMed] [Google Scholar]
- 24.Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. Cochrane Database Syst Rev. 2000:CD001089. doi: 10.1002/14651858.CD001089. [DOI] [PubMed] [Google Scholar]
- 25.Wing JK, Babor T, Brugha T, et al. SCAN. Schedules for clinical assessment in neuropsychiatry. Arch Gen Psychiatry. 1990;47(6):589–593. doi: 10.1001/archpsyc.1990.01810180089012. [DOI] [PubMed] [Google Scholar]
- 26.Wing JK, Birley JL, Cooper JE, Graham P, Isaacs AD. Reliability of a procedure for measuring and classifying “present psychiatric state”. Br J Psychiatry. 1967;113(498):499–515. doi: 10.1192/bjp.113.498.499. [DOI] [PubMed] [Google Scholar]