Abstract
Background:
Stroke patients frequently experience medical complications; yet, data on incidence, causes, and consequences are sparse.
Objective:
To examine the data validity of medical complications among patients with stroke in a population-based clinical registry and a hospital discharge registry.
Methods:
We examined the predictive values, sensitivity and specificity of medical complications among patients admitted to specialized stroke units and registered in the Danish National Indicator Project (DNIP) and the Danish National Registry of Patients (NRP) between January 2003 and December 2006 (n = 8,024). We retrieved and reviewed medical records from a random sample of patients (n = 589, 7.3%).
Results:
We found substantial variation in the data quality of stroke-related medical complication diagnoses both within the specific complications and between the registries. The positive predictive values ranged from 39.0%–87.1% in the DNIP, and from 0.0%–92.9% in the NRP. The negative predictive values ranged from 71.6%–98.9% in the DNIP and from 63.3% to 97.4% in the NRP. In both registries the specificity of the diagnoses was high. The sensitivity ranged from 23.5% (95% confidence interval [CI]: 14.9–35.4) for falls to 62.9% (95% CI: 54.9–70.4) for urinary infection in the DNIP, and from 0.0 (95% CI: 0.0–4.99) for falls to 18.1% (95% CI: 2.3–51.8) for pressure ulcer in the NRP.
Conclusion:
The DNIP may be useful for studying medical complications among patients with stroke.
Keywords: stroke, medical complications, data validity, clinical database
Introduction
Stroke patients frequently experience medical complications such as pneumonia, urinary infection, pressure ulcer, falls, venous thromboembolism, and severe constipation. These complications may hinder optimum rehabilitation and have been associated with increased mortality.1–6 However, the available data on incidence, causes, and consequences of serious medical complications are sparse and inconclusive.1–5
Large-scale population-based studies, reflecting real-life conditions in modern stroke care, are much needed. Primary data collection is often time-consuming and costly; consequently, it is often only done on a smaller scale. Furthermore, there is a likelihood of bias due to recall, nonresponse, and effects on the diagnostic process as a result of the research question.6–8 Clinical and administrative registries are possible alternatives to primary data collection, they have the advantage of readily available data and often contain complete registration of information on people in the target population. Registries are generally valuable tools for answering clinical, administrative, and research questions and may also be useful for studying medical complications in patients with stroke; however, documented reasonable data validity is a prerequisite for using such data sources.
To our knowledge, information on the validity of registry data on medical complications among stroke patients has not been reported previously. Therefore, we examined the data validity of medical complications among stroke patients in two population-based Danish registries: the Danish National Indicator Project (DNIP) and the Danish National Registry of Patients (NRP).
Methods
Data sources
The Danish healthcare system provides tax-supported health care to the country’s 5.5 million residents, all of whom have free access to hospital care and general practitioners. Patients with acute medical conditions, including stroke, are exclusively admitted to public hospitals. The entire population is covered by a large number of administrative and medical registries, which are used for monitoring and regulating all central aspects of the public sector including the health care system. Through the use of a civil registry number, which is unique to every Danish citizen and encodes sex and date of birth, unambiguous linkages between the population-based registers can be made.9
The Danish National Indicator Project (DNIP)
In 2000, the DNIP was established as a nationwide initiative to monitor and improve the quality of treatment and care provided by the Danish health care system for specific diseases, including stroke. The project develops evidence-based quality criteria related to the structure, process, and outcome of health care and monitors the fulfillment of these criteria. Project participation is mandatory for all Danish hospitals, relevant clinical departments, and units treating patients with stroke.10 Upon hospital admission, data on care and patient characteristics (eg, age, sex, marital status, housing, Scandinavian Stroke Scale score, history of stroke or myocardial infarction, previous and/or current atrial fibrillation, hypertension, diabetes mellitus or intermittent claudication, smoking habits, and alcohol intake) are collected for all patients admitted with stroke. Detailed written instructions are available to the staff to ensure the validity of the data collected and completeness of patient registration in the DNIP. After hospital discharge the data are entered into a central database. A structured audit process is carried out regularly (every year) on a national, regional, and local basis to assess critically the quality of the dataset and results. After the audit process is completed the data are released publicly, including comments on the results from the audit groups. To ensure completeness of patient registration in DNIP, its enrollees are compared with local hospital discharge registries.10
The Danish National Registry of Patients (NRP)
The NRP is an administrative nationwide public registry that covers all discharges from somatic hospitals in Denmark since January 1, 1977. The data include the civil registry number, which is unique to every Danish citizen, the dates of admission and discharge, the surgical procedure(s) performed, and up to 20 diagnoses for every discharge classified, since 1994, according to the Danish version of the International Classification of Diseases, 10th edition (ICD-10). All discharge diagnoses are assigned by the physician who discharges the patient. Reporting of all hospital contacts to the NRP is mandatory by law. Data reported to the NRP are also used for classifying the patients in relation to Diagnosis-related groups (DRG), which determines the financial compensation given to the hospital for treating the individual patients.11
Study population
We identified all admissions for acute stroke, from 10 specialized stroke units in the former Copenhagen Hospital Corporation and Aarhus County, registered in the DNIP from January 13, 2003 to December 31, 2006. All patients (≥18 years) admitted to Danish hospitals with stroke, as defined by WHO criteria, ie, an acute disturbance of focal or global cerebral function with symptoms lasting more than 24 hours or leading to death of presumed vascular origin, are eligible for inclusion in the DNIP database. This includes patients with intracerebral hemorrhage and ischemic stroke (ICD-10: I63, infarction; I61, hemorrhage; and I64, unspecified). Patients with subarachnoidal or epidural hemorrhage, subdural hematoma, retinal infarct, and infarct caused by trauma, infection, or an intracranial malignant process are excluded. Patients with diffuse symptoms, such as isolated vertigo or headache, and asymptomatic patients with infarct detected only by computed tomography (CT) or magnetic resonance imaging (MRI) scans are also excluded.
Although some patients had multiple events, we included only the first stroke event registered in the DNIP during the study period. Furthermore, only patients with a valid civil registry number (a unique personal identification number issued to all Danish citizens that allows unambiguous linkage between public registries) were included in the study. In total, we identified 8,024 (99.5%) admissions in the registry which were eligible for inclusion in our study population.
Medical complications in the DNIP
During hospital admission, detailed data on each patient are prospectively registered using a standardized form. The data includes the presence (and if so, date of diagnosis) or absence of the following seven medical complications: pneumonia, urinary infection, pressure ulcer, falls, deep venous thrombosis (DVT), pulmonary embolism (PE), and constipation. Only complications that occurred after hospital admission are registered.
Medical complications in the NRP
Information on medical complications during the hospital admission was obtained through linkage to the NRP. We retrieved data on the following ICD-10 discharge diagnoses: pneumonia (DJ12–DJ18), urinary infection (DN30.0, DN30.8, DN30.9, and DN10), pressure ulcer (DL899), falls (DR297 and EUHE), DVT (DI82.9, DI82.9A-E), PE (DI26), and constipation (DK590). Both primary diagnoses, the condition, which is the principal reason for the hospital admission, and secondary diagnoses, all other clinically relevant diagnoses, including any significant complications and symptoms that appear during the admission, were identified from the registry.
Medical record review
We aimed to retrieve and review medical records from a random sample of approximately 5% of patients in the defined study population (n = 417). A computer-generated 5% random sample was obtained from the entire study population, stratified according to the 10 stroke units serving the study population. Additionally, we retrieved and reviewed 25 medical records for each of the seven types of complications registered in the DNIP. The 25 extra medical records were selected randomly among all patients registered with medical complications in the DNIP. The later records were added to ensure a reasonable statistical precision of the estimated positive predictive value for relatively rare complications. In total, 589 records were retrieved and reviewed.
All medical records were retrieved and reviewed by a single reviewer (AI) using a standardized form. The form was developed and tested in close collaboration with 2 consultants in neurology and clinical epidemiology (GA, SPJ). The criteria used to define the presence of a complication were in accordance with existing international defined criteria or, if such criteria were not available, defined a priori by consensus among the authors.
The diagnostic criteria are presented in Table 1. All cases with uncertain diagnosis, based on the available information, were discussed with the consultants. The review was based on all available information in the medical records including written radiology reports, results from laboratory tests, and the nurse records. The actual brain imaging films were not re-interpreted. The study was approved by The Danish Data Protection Agency (J.no. 2007-41-0563) and The National Board of Health (J.no. 7-604-04-2/26/EHE).
Table 1.
The diagnostic criteria for the seven medical complications
| Medical complication | Diagnostic test/objective examinations | Clinical indications in the medical record | Diagnosis is defined as correct if: |
|---|---|---|---|
| Pneumonia | X-ray examination showed an infiltration, Body temperature ≥37.5°C Microorganism isolated from blood culture, increased infection parameters |
Cough Dyspnea Chest pain Rales coincident with the area of infiltration Increased sputum, purulent sputum |
|
| Urinary infection | Culture and resistance examination (D + R) positive with significant bacteriuria (>105/ml) | Pain over the symphysis Frequent urge to urinate strangury Malodorous urine Fever |
|
| Decubitus | Graduation 1: Skin: Red, intact. The red color does not disappear by finger pressure, the condition is painful. | Decubitus was documented in the medical record as symptoms describing one of the graduations 1–4. | |
| Graduation 2: Blisters or superficial wounds are seen, which rarely penetrate the dermis, the condition is painful. | |||
| Graduation 3: The wound has penetrated the dermis and entered subcutaneous tissue; there may be necrosis, undermining, and possibly fistula in the wound. Muscle fascia will stop progression in depth. | |||
| Graduation 4: The wound has broken through fascia down into muscle tissue and possibly bone and joint, is often infected. Generally there is no pain. | |||
| Falls after stroke | The accident was inadvertent, the patient inadvertently fell down, the patient has been confined to bed for at least one day because of the trauma. | It was documented that the accident was inadvertent, and that the patient fell down; ie, with or without bed rest. Both must be documented. | |
| Deep vein thrombosis | Phlebography Doppler ultrasound |
Leg pain Reddening Swelling in the area |
DVT has been suggested by objective examination and there was at least one clinical indication of DVT. |
| Pulmonary embolism | Perfusion ventilation, pulmonary scintigraphy (P/V), CT-angio Echo diagram, Increased D-Dimer, A-puncture resulting in the form of hyperventilation | Dyspnea Tachypnea Chest pain Cough |
|
| Constipation | Oil phosphate clysma | Less than three feces a week Feces is described as inspissated A feeling of not having finished Defecation was painful/strenuous |
|
Statistical analysis
The positive and negative predictive values of the registered complications in the DNIP and the NRP were assessed using the review of the medical records as the gold standard. Positive predictive values of the complications in the registries were computed as proportions with the number of patients with a verified complication, after review of medical records using the diagnostic criteria in Table 1, as the numerator and the total number of patients registered in the DNIP/NRP with this specific complication as the denominator. Negative predictive values of the complications in the registries were computed as proportions with the number of patients verified to not have the specific complication, after review of medical records using the diagnostic criteria in Table 1, as the numerator and the total number of patients registered in the DNIP/NRP without this specific complication as the denominator. We computed the sensitivity of the complications as the proportion of patients with a verified complication registered in the DNIP/NRP divided by the total number of patients in our random sample with a complication in the medical record. Specificity of the complications was similarly defined as the number of patients without any verified complication registered in the DNIP/NRP divided by the total number of patients in our random sample without a complication in the medical record.
Positive and negative predictive values, the sensitivity, and the specificity were calculated for all complications combined and for the specific complications (pneumonia, urinary infection, pressure ulcer, falls, DVT, PE, and constipation). Finally, we estimated the proportion of patients who had at least one symptom of a complication registered in their medical record among patients registered in the DNIP with the complication.
For all estimates, a 95% confidence interval (CI) was calculated. Data were analyzed using STATA® version 10.0 (StataCorp, College Station, TX, USA).
Results
Table 2 shows demographic and clinical characteristics of the 8,024 patients with acute stroke registered in the DNIP.
Table 2.
Descriptive characteristics of the 8,024 patients with acute strokea registered in the Danish National Indicator Project, 2003–2006
| Characteristic | N (%) |
|---|---|
| Age (years): | |
| 18–65 | 2308 (28.76) |
| 65–80 | 3204 (39.93) |
| >80 | 2512 (31.31) |
| Sex: | |
| – Men | 4009 (49.96) |
| – Women | 4015 (50.04) |
| Marital status: | |
| – Living with partner, family, or friend | 3511 (43.7) |
| – Living alone | 4008 (49.95) |
| – Other | 202 (2.5) |
| – Unknown | 303 (3.8) |
| Housing: | |
| – Own home | 6994 (87.2) |
| – Nursing home or other institution | 576 (7.2) |
| – Other | 138 (1.7) |
| – Unknown | 316 (3.9) |
| Type of stroke: | |
| – Intracerebral hemorrhage | 847 (10.6) |
| – Ischemic | 6076 (75.7) |
| – Unspecified | 1101 (13.5) |
| Scandinavian stroke scale: | |
| – Very severe, 0–14 points | 654 (8.2) |
| – Severe, 15–29 points | 696 (8.7) |
| – Moderate, 30–44 points | 1271 (15.8) |
| – Mild, 45–58 points | 3098 (38.6) |
| – Unknown | 2305 (28.7) |
| Previous stroke: | |
| – Yes | 1908 (23.8) |
| – No | 5585 (69.6) |
| – Unknown | 531 (6.6) |
| Previous myocardial infarction: | |
| – Yes | 745 (9.3) |
| – No | 6638 (82.7) |
| – Unknown | 641 (8.0) |
| Atrial fibrillationb: | |
| – Yes | 1454 (18.1) |
| – No | 6050 (75.4) |
| – Unknown | 520 (6.5) |
| Hypertensionb: | |
| – Yes | 3856 (48.1) |
| – No | 3529 (44.0) |
| – Unknown | 639 (7.9) |
| Diabetes mellitusb: | |
| – Yes | 1237 (15.4) |
| – No | 6276 (78.2) |
| – Unknown | 511 (6.4) |
| Claudicatio intermittensb: | |
| – Yes | 363 (4.5) |
| – No | 6214 (77.4) |
| – Unknown | 1447 (18.0) |
| Smoking habits (%): | |
| – Current | 2875 (35.3) |
| – Former (Quit more than ½ yr previously) | 1402 (17.5) |
| – Never | 2183 (27.2) |
| – Unknown | 1564 (19.5) |
| Alcohol intake (%)c: | |
| – ≤21/14 drinks/week | 5853 (72.9) |
| – >21/14 drinks/week | 671 (8.4) |
| – Unknown | 1500 (18.7) |
| Rankin score (%): | |
| 0–1 | 4739 (59.0) |
| 2–3 | 1313 (16.4) |
| 4–5 | 502 (6.3) |
| – Unknown (+ missing) | 1470 (18.3) |
Notes:
Patients are from the former Copenhagen Hospital Corporation and Aarhus County;
Known history or diagnosed during current admission;
Drinks per week for men and women, respectively.
Table 3 shows the number of medical complications registered in the DNIP and NRP among the patients. During hospitalization, a total of 25.4 % (n = 2,039) of the patients were registered with at least one medical complication in the DNIP. The most common complications in the DNIP were urinary infection in 15.5% (n = 1,240), pneumonia in 8.4% (n = 670), and constipation in 6.8%, (n = 544) of the patients; whereas DVT and PE were only registered in 0.4% (n = 32 and n = 28, respectively) of the patients.
Table 3.
Proportions of medical complications registered in the Danish National Indicator Project (DNIP) and the National Registry of Patients for the 8,024 patients with acute stroke registered in the DNIP, 2003–2006
| Medical complication |
The Danish National Indicator Project |
The National Registry of Patients |
|---|---|---|
| N (%) | N (%) | |
| Pneumonia | ||
| – Yes | 670 (8.4) | 122 (1.5) |
| – No | 6,637 (82.7) | 7,902 (98.5) |
| – Unknown | 717 (8.9) | – |
| Urinary infection | ||
| – Yes | 1,240 (15.5) | 391 (4.9) |
| – No | 6,070 (75.7) | 7,633 (95.1) |
| – Unknown | 714 (8.9) | – |
| Decubitus | ||
| – Yes | 110 (1.4) | 20 (0.25) |
| – No | 7,225 (90.0) | 8,004 (99.75) |
| – Unknown | 689 (8.6) | – |
| Falls after stroke | ||
| – Yes | 204 (2.5) | 10 (0.12) |
| – No | 7,082 (88.3) | 8,014 (99.88) |
| – Unknown | 738 (9.2) | – |
| Deep vein thrombosis | ||
| – Yes | 32 (0.40) | 15 (0.2) |
| – No | 7,323 (91.3) | 8,009 (99.8) |
| – Unknown | 669 (8.3) | – |
| Pulmonary embolism | ||
| – Yes | 28 (0.4) | 24 (0.3) |
| – No | 7,314 (91.2) | 8,000 (99.7) |
| – Unknown | 682 (8.5) | – |
| Constipation | ||
| – Yes | 544 (6.8) | 57 (0.7) |
| – No | 6,667 (83.1) | 7,967 (99.3) |
| – Unknown | 813 (10.1) | – |
Table 4 presents the positive and negative predictive values of all the reviewed medical complications in the DNIP and NRP. We found the overall positive predictive value for the medical complications registered in the DNIP was 71.7% (95% CI: 67.4–75.8) compared to 76.1% (95% CI: 75.9–84.6) in the NRP. The positive predictive values in the DNIP ranged from 39.0% (95% CI: 24.2–55.5) for pressure ulcer to 87.1% (95% CI: 78.0–93.4) for pneumonia. In the NRP, there was even more variation in the positive predictive values, which ranged from 0.0 % (95% CI: 0–97.5) for falls to 92.9% (95% CI: 66.1–99.8) for pneumonia. The overall negative predictive value registered in the DNIP was 91.3% (95% CI: 90.3–92.2). The negative predictive values in the DNIP ranged from 71.6% (95% CI: 67.2–75.7) for pneumonia to 98.9% (95% CI: 97.5–99.6) for pulmonary embolism. The overall negative predictive value for the medical complications registered in the NRP was 85.1% (95% CI: 83.9–86.1), the values ranged from 63.3% (95% CI: 59.2–67.3) for pneumonia to 97.4% (95% CI: 95.8–98.5) for pulmonary embolism.
Table 4.
Positive and negative predictive values for all the reviewed medical complications in the Danish National Indicator Project and the National Registry of Patients
| Medical complication |
The Danish National Indicator Project |
The National Registry of Patients |
||||||
|---|---|---|---|---|---|---|---|---|
| Verified N/total N | Positive predictive value % (95% CI) | Total negative N/total N | Negative predictive value % (95% CI) | Verified N/total N | Positive predictive value % (95% CI) | Total negative N/total N | Negative predictive value % (95% CI) | |
| Pneumonia | 74/85 | 87.1 (78.0–93.4) | 330/461 | 71.6 (67.2–75.7) | 13/14 | 92.9 (66.1–99.8) | 364/575 | 63.3 (59.2–67.3) |
| Urinary infection | 122/149 | 81.9 (74.7–87.7) | 340/399 | 85.2 (81.3–88.5) | 30/39 | 76.9 (60.7–88.9) | 390/550 | 70.9 (66.9–74.7) |
| Decubitus | 16/41 | 39.0 (24.2–55.5) | 498/506 | 98.4 (96.9–99.3) | 4/8 | 50 (15.7– 84.3) | 561/581 | 96.6 (94.7–97.9) |
| Falls after stroke | 35/47 | 74.5 (59.7–86.1) | 444/496 | 89.5 (86.5–92.1) | 0/1 | 0.0 (0.0–97.5) | 497/588 | 84.5 (81.3–87.4) |
| Deep vein thrombosis | 20/28 | 71.4 (51.3–86.8) | 521/524 | 99.4 (98.3–99.9) | 7/8 | 87.5 (47.3–99.7) | 564/581 | 97.1 (95.4–98.3) |
| Pulmonary embolism | 17/26 | 65.4 (44.3–82.8) | 520/526 | 98.9 (97.5–99.6) | 10/11 | 90.9 (58.7–99.8) | 563/578 | 97.4 (95.8–98.5) |
| Constipation | 46/84 | 54.8 (43.5–65.7) | 416/450 | 92.4 (89.6–94.7) | 3/7 | 42.9 (9.9–81.6) | 493/582 | 84.7 (81.5–87.5) |
| Overall | 330/460 | 71.7 (67.4–75.8) | 3069/3362 | 91.3 (90.3–92.2) | 67/88 | 76.1 (75.9–84.6) | 3432/4035 | 85.1 (83.9–86.1) |
Abbreviation: CI, confidence interval.
Table 5 shows the sensitivity and specificity of the medical complications in the DNIP and NRP. The estimates are based on a 5% random sample of all patients with stroke registered in DNIP in the study period.
Table 5.
Sensitivity and specificity of the medical complications in the Danish National Indicator Project and the National Registry of Patientsa
| Medical complication |
The Danish National Indicator Project |
The National Registry of Patients |
||||||
|---|---|---|---|---|---|---|---|---|
| Verified positive/total N | Sensitivity % (95% CI) | Verified negative/total N | Specificity % (95% CI) | Verified positive/total N | Sensitivity % (95% CI) | Verified negative/total N | Specificity % (95% CI) | |
| Pneumonia | 51/182 | 28.0 (21.6–35.1) | 330/339 | 97.3 (95.0–98.8) | 12/201 | 6.0 (3.1–10.2) | 362/363 | 99.7 (98.5–100) |
| Urinary infection | 100/159 | 62.9 (54.9–70.4) | 340/364 | 93.4 (90.3–95.7) | 24/168 | 14.3 (9.4–20.5) | 388/396 | 98.0 (96.1–99.1) |
| Decubitus | 3/11 | 24.4 (6.02–61.0) | 498/511 | 97.5 (95.7–98.6) | 2/11 | 18.11 (2.3–51.8) | 550/553 | 99.5 (98.4–99.9) |
| Falls after stroke | 16/68 | 23.5 (14.9–35.4) | 444/450 | 98.7 (97.1–99.5) | 0/72 | 0 (0–4.99) | 491/492 | 99.8 (98.9–100) |
| Deep vein thrombosis | 2/5 | 40.0 (5.27–85.3) | 521/525 | 99.2 (98.1–99.8) | 1/6 | 16.7 (0.4–64.1) | 561/561 | 100 (99.3–100) |
| Pulmonary embolism | 0/6 | – | 520/521 | 99.8 (98.9–100) | 0/8 | – | 556/556 | 100 (99.9–100) |
| Constipation | 31/65 | 47.7 (35.1–60.5) | 416/444 | 93.7 (91.0–95.8) | 3/77 | 3.9 (0.8–11.0) | 484/487 | 99.4 (98.2–99.9) |
| Overall | 203/496 | 40.9 (36.6–45.4) | 3069/3154 | 97.3 (96.7–97.8) | 42/543 | 7.7 (5.6–10.3) | 3392/3408 | 99.5 (99.2–99.7) |
Notes:
Based on a 5% random sample of all patients registered in the study period.
Abbreviation: CI, confidence interval.
Sensitivity
The overall sensitivity of the seven medical complications in the DNIP and NRP was 40.9% (95% CI: 36.6–45.4) and 7.7% (95% CI: 5.6–10.3), respectively. The lowest sensitivity in both registries was found for falls (ie, 23.5% [95% CI: 14.9–35.4] in the DNIP and 0.0% [95% CI: 0.0–4.99] in the NRP). The highest sensitivity in the DNIP was for urinary infection, 62.9% (95% CI: 54.9–70.4); whereas, the highest in the NRP was for pressure ulcer, 18.1% (95% CI: 2.3–51.8).
Specificity
The overall specificity of the seven medical complications was 97.3% (95% CI: 96.7–97.8) in the DNIP and 99.5% (95% CI: 99.2–99.7) in the NPR. The specificity ranged from 93.4% (95% CI: 91.0–95.8) for urinary infection to 99.8% (95% CI: 98.9–100) for PE in the DNIP. In the NRP, the specificity ranged from 98.0% (95% CI: 96.1–99.1) for urinary infection to 100% (95% CI: 99.9–100) for PE. In a sub analysis (data not shown), we found that of the 130 complications registered in DNIP that were not verified during the review of the medical records, 57% (95% CI: 48.7–66.3) (n = 75) of these patients had a least one or more symptoms registered in their medical record which indicated the presence of a medical complication, but was not sufficient to fulfill the diagnostic criteria listed in Table 1.
Discussion
We found substantial diagnosis- and register-specific variation when validating the data quality of seven stroke-related medical complications registered in public Danish registries. The positive and negative predictive values were generally moderate to high for most complications in the two registries. In addition, the specificity of the medical complication diagnoses was high for all complications in the two registries. In contrast, major differences were found in the two registries for the sensitivity of the medical complication diagnoses. The sensitivity for all types of complications was moderate to low in the DNIP, whereas it was extremely low in the NRP.
The strengths of our study included the population-based design, the access to medical records with detailed clinical data, and the standardized evaluation of the records including the use of well-defined diagnostic criteria developed in accordance with available existing literature and/or international consensus. The use of a single reviewer, who was not blinded for the diagnosis in DNIP, of the medical records was a potential methodological limitation as there exist a possibility of error in extracting information from the medical records. The use of retrospective review of medical records may also be a limitation as the collection and the quality of the data are not under control of the researcher (eg, the required information to make a valid conclusion about the presence of specific medical complications were always available in the medical records).
However, detailed diagnostic criteria and a standardized form were used to reduce any risk of bias in the data collection. Furthermore, all cases with uncertain diagnosis, based on the available information, were discussed with senior consultants. Although the number of reviewed medical records was relatively large compared to other validation studies based on detailed review of medical records, some of the complications (eg, deep venous thrombosis and pulmonary embolism) were rare with relatively few cases available for assessment. Consequently, the precision of the estimated predictive values, sensitivity, and specificity for these complications were only moderate. Finally, it is not known to what extent our findings can be extrapolated to other settings, where the prevalence of complications and the diagnostic strategy may be different.
We are unaware of other studies validating the quality of diagnoses of medical complications among patients with stroke in administrative and clinical registries. There may be different possible explanations for the low sensitivity we found in the NRP. However, the fact that there is no financial incentive for the stroke units to register medical complications in NRP due to the DRG system used in Denmark (ie, medical complications do not result in additional compensation to the hospital), is likely to be a major factor. Furthermore, registration of data to the NRP is done by physicians, whereas nurses are much more involved in the reporting of data to the DNIP. It is our impression that the nurses in general are much more aware of registration of medical complications, which may explain the higher sensitivity found in DNIP.
However, our finding of moderate to high positive predictive values in both registries are generally in accordance with the results from a number of validation studies concerning medical complications in different patient populations (eg, unselected internal medicine patients, pregnant women, patients from geriatric wards, general practice, or with a former cancer diagnosis).12–22 The positive predictive values reported from these studies have ranged from 20.7% for pressure ulcer to 96.2% for pneumonia.12–15,17–22 Only a few studies have previously reported the sensitivity and specificity of diagnoses of medical complications in registries. Quan et al examined diagnoses of complications in Canadian administrative hospital discharge data and found that the sensitivity ranged from 0% to 57.1% (higher than 50% for only two conditions). In contrast, specificity was generally high (range: 99.0%–100%).16 These results are very consistent with our findings and also in line with findings from validation studies on other diagnoses not related to medical complications.23,24 The implications of the often low sensitivity and high specificity of diagnoses in administrative and clinical registries depend on the research question and the design of the study.8 A low sensitivity would result in misleading rates in a study determining incidence (eg, of medical complications among patients with stroke), whereas analytical studies examining associations would likely produce unbiased relative risk estimates due to the high specificity.25 Thus, in analytical studies, a low specificity would be far more troublesome in most situations than a low sensitivity.23 The fact that half of the complications not verified had a least one or several symptoms of the specific complications registered in the medical record supports this and indicates data on complications in the DNIP are valid, although the international criteria of the specific complication were not completely fulfilled.
Conclusion
In conclusion, we found moderate to high predictive values for medical complication diagnoses among patients with stroke in two population-based registries. However, the sensitivity varied substantially between the two registries with the stroke registry having a much higher sensitivity compared to the hospital discharge registry. The specificity of the diagnoses was high in both registries. These findings indicate that data from the DNIP may be useful for studying medical complications in patients with stroke, particularly for analytical studies.
Acknowledgments
This study was supported by The Danish Heart Foundation and the Central Denmark Research Foundation. The authors report no conflicts of interest in this work.
References
- 1.Davenport RJ, Dennis MS, Wellwood I, Warlow CP. Complications after acute stroke. Stroke. 1996;27(3):415–420. doi: 10.1161/01.str.27.3.415. [DOI] [PubMed] [Google Scholar]
- 2.Dromerick A, Reding M. Medical and neurological complications during inpatient stroke rehabilitation. Stroke. 1994;25(2):358–361. doi: 10.1161/01.str.25.2.358. [DOI] [PubMed] [Google Scholar]
- 3.Heuschmann PU, Kolominsky-Rabas PL, Misselwitz B, Hermanek P, Leffmann C, Janzen RW, et al. Predictors of in-hospital mortality and attributable risks of death after ischemic stroke: the German Stroke Registers Study Group. Arch Intern Med. 2004;164(16):1761–1768. doi: 10.1001/archinte.164.16.1761. 13. [DOI] [PubMed] [Google Scholar]
- 4.Johnston KC, Li JY, Lyden PD, Hanson SK, Feasby TE, Adams RJ, et al. Medical and neurological complications of ischemic stroke: experience from the RANTTAS trial. RANTTAS Investigators. Stroke. 1998;29(2):447–453. doi: 10.1161/01.str.29.2.447. [DOI] [PubMed] [Google Scholar]
- 5.Langhorne P, Stott DJ, Robertson L, MacDonald J, Jones L, McAlpine C, et al. Medical complications after stroke: a multicenter study. Stroke. 2000;31(6):1223–1229. doi: 10.1161/01.str.31.6.1223. [DOI] [PubMed] [Google Scholar]
- 6.Vernino S, Brown RD, Jr, Sejvar JJ, Sicks JD, Petty GW, O’Fallon WM. Cause-specific mortality after first cerebral infarction: a population-based study. Stroke. 2003;34(8):1828–1832. doi: 10.1161/01.STR.0000080534.98416.A0. [DOI] [PubMed] [Google Scholar]
- 7.Baron JA, Weiderpass E. An introduction to epidemiological research with medical databases. Ann Epidemiol. 2000;10(4):200–204. doi: 10.1016/s1047-2797(00)00039-9. [DOI] [PubMed] [Google Scholar]
- 8.Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–442. doi: 10.1093/ije/25.2.435. r; [DOI] [PubMed] [Google Scholar]
- 9.The Ministry of Health and Prevention . Health Care in Denmark. Copenhagen, Denmark: The Ministry of Health and Prevention; 2008. Available from: http://www.sum.dk/Aktuelt/publikationer/UK_health-care_in_DK.aspx. Accessed on February 10, 2010. [Google Scholar]
- 10.Mainz J, Krog BR, Bjornshave B, Bartels P. Nationwide continuous quality improvement using clinical indicators: the Danish National Indicator Project. Int J Qual Health Care. 2004;16(Suppl 1):i45–i50. doi: 10.1093/intqhc/mzh031. [DOI] [PubMed] [Google Scholar]
- 11.Andersen TF, Madsen M, Jorgensen J, Mellemkjoer L, Olsen JH. The Danish National Hospital Register. A valuable source of data for modern health sciences. Dan Med Bull. 1999;46:263–268. [PubMed] [Google Scholar]
- 12.Arnason T, Wells PS, van WC, Forster AJ. Accuracy of coding for possible warfarin complications in hospital discharge abstracts. Thromb Res. 2006;118(2):253–262. doi: 10.1016/j.thromres.2005.06.015. [DOI] [PubMed] [Google Scholar]
- 13.Gunningberg L, Dahm MF, Ehrenberg A. Accuracy in the recording of pressure ulcers and prevention after implementing an electronic health record in hospital care. Qual Saf Health Care. 2008;17(4):281–285. doi: 10.1136/qshc.2007.023341. [DOI] [PubMed] [Google Scholar]
- 14.Larsen TB, Johnsen SP, Moller CI, Larsen H, Sorensen HT. A review of medical records and discharge summary data found moderate to high predictive values of discharge diagnoses of venous thromboembolism during pregnancy and postpartum. J Clin Epidemiol. 2005;58(3):316–319. doi: 10.1016/j.jclinepi.2004.07.004. [DOI] [PubMed] [Google Scholar]
- 15.Lawrenson R, Todd JC, Leydon GM, Williams TJ, Farmer RD. Validation of the diagnosis of venous thromboembolism in general practice database studies. Br J Clin Pharmacol. 2000;49(6):591–596. doi: 10.1046/j.1365-2125.2000.00199.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J Clin Epidemiol. 2004;57(4):366–372. doi: 10.1016/j.jclinepi.2003.01.002. [DOI] [PubMed] [Google Scholar]
- 17.Severinsen MT, Kristensen SR, Overvad K, Dethlefsen C, Tjonneland A, Johnsen SP. Venous thromboembolism discharge diagnoses in the Danish National Patient Registry should be used with caution. J Clin Epidemiol. 2010;63(2):223–228. doi: 10.1016/j.jclinepi.2009.03.018. [DOI] [PubMed] [Google Scholar]
- 18.Skull SA, Andrews RM, Byrnes GB, Campbell DA, Nolan TM, Brown GV, et al. ICD-10 codes are a valid tool for identification of pneumonia in hospitalized patients aged > or = 65 years. Epidemiol Infect. 2008;136(2):232–240. doi: 10.1017/S0950268807008564. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Thomsen RW, Riis A, Norgaard M, Jacobsen J, Christensen S, McDonald CJ, et al. Rising incidence and persistently high mortality of hospitalized pneumonia: a 10-year population-based study in Denmark. J Intern Med. 2006;259(4):410–417. doi: 10.1111/j.1365-2796.2006.01629.x. [DOI] [PubMed] [Google Scholar]
- 20.White RH, Brickner LA, Scannell KA. ICD-9-CM codes poorly indentified venous thromboembolism during pregnancy. J Clin Epidemiol. 2004;57(9):985–988. doi: 10.1016/j.jclinepi.2004.02.003. [DOI] [PubMed] [Google Scholar]
- 21.Zhan C, Battles J, Chiang YP, Hunt D. The validity of ICD-9-CM codes in identifying postoperative deep vein thrombosis and pulmonary embolism. Jt Comm J Qual Patient Saf. 2007;33(6):326–331. doi: 10.1016/s1553-7250(07)33037-7. [DOI] [PubMed] [Google Scholar]
- 22.Zhan C, Elixhauser A, Richards CL, Jr, Wang Y, Baine WB, Pineau M, et al. Identification of hospital-acquired catheter-associated urinary tract infections from Medicare claims: sensitivity and positive predictive value. Med Care. 2009;47(3):364–369. doi: 10.1097/MLR.0b013e31818af83d. [DOI] [PubMed] [Google Scholar]
- 23.Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58(4):323–337. doi: 10.1016/j.jclinepi.2004.10.012. [DOI] [PubMed] [Google Scholar]
- 24.Wilchesky M, Tamblyn RM, Huang A. Validation of diagnostic codes within medical services claims. J Clin Epidemiol. 2004;57(2):131–141. doi: 10.1016/S0895-4356(03)00246-4. [DOI] [PubMed] [Google Scholar]
- 25.Kelsey JL, Whittemore AS, Evans AS, Thompson WD. Methods in observational epidemiology. 2nd ed. New York: Oxford University Press; 1996. [Google Scholar]