Abstract
Context
Suicidal behavior has gained attention as an adverse outcome of prescription drug use. Hospitalizations for intentional self-harm, including suicide, can be identified in administrative claims databases using external cause of injury codes (E-codes). However, rates of E-code completeness in US government and commercial claims databases are low due to issues with hospital billing software.
Objective
To develop an algorithm to identify intentional self-harm hospitalizations using recorded injury and psychiatric diagnosis codes in the absence of E-code reporting.
Methods
We sampled hospitalizations with an injury diagnosis (ICD-9 800–995) from 2 databases with high rates of E-coding completeness: 1999–2001 British Columbia, Canada data and the 2004 U.S. Nationwide Inpatient Sample. Our gold standard for intentional self-harm was a diagnosis of E950-E958. We constructed algorithms to identify these hospitalizations using information on type of injury and presence of specific psychiatric diagnoses.
Results
The algorithm that identified intentional self-harm hospitalizations with high sensitivity and specificity was a diagnosis of poisoning; toxic effects; open wound to elbow, wrist, or forearm; or asphyxiation; plus a diagnosis of depression, mania, personality disorder, psychotic disorder, or adjustment reaction. This had a sensitivity of 63%, specificity of 99% and positive predictive value (PPV) of 86% in the Canadian database. Values in the US data were 74%, 98%, and 73%. PPV was highest (80%) in patients under 25 and lowest those over 65 (44%).
Conclusions
The proposed algorithm may be useful for researchers attempting to study intentional self-harm in claims databases with incomplete E-code reporting, especially among younger populations.
Introduction
Suicidal behavior has gained increasing attention as a potential adverse outcome of prescription drug use. In October 2004, the U.S. Food and Drug Administration (FDA) issued an advisory regarding a possible increased risk of suicidal thoughts and attempts among children and adolescents taking antidepressants.1 This warning was prompted by a meta-analysis of data from randomized controlled trials of antidepressants in this age group, in which patients randomized to antidepressants had nearly twice the rate of suicidal ideation or behavior relative to those given placebo.2 More recently, FDA has issued warnings regarding increased suicidality among patients receiving anticonvulsant agents3 and the smoking cessation drug Chantix (varenicline),4 and is currently investigating a possible association between Singulair (montelukast sodium) use and suicidality.5 FDA is now requiring some drug manufacturers to provide data on suicidality before and after approval.6
While spontaneous adverse event reports and analyses of RCT data are useful in identifying signals of increased suicidality rates, these data are limited. Information regarding suicidal ideation and behavior was not collected systematically in older trials, and while these safety outcomes can be incorporated into new trials, an increase in risk may be missed due to the relatively low incidence of suicidality and the frequent exclusion of high-risk patients from trials of psychiatric medications. Observational studies in administrative claims data have the potential to provide valuable information on the association between drug use and suicide risk among large patient populations, provided that deliberate self-harm events, the majority of which are suicide attempts,7 can be identified.
Suicides can be identified using data from the National Death Index,8 and in instances where the subject dies without reaching an emergency room, must be identified in this manner. However, the majority of suicide attempts are non-fatal and must be identified through alternative means.9 Intentional self-harm emergency room visits and hospitalizations can be identified in administrative claims databases using external cause of injury codes (E-codes).10, 11 These codes are part of the International Statistical Classification of Diseases and Related Health Problems (ICD) coding scheme and are used to provide supplemental information about the cause and intent of an injury. E-coding is mandatory in approximately half of the US states, and the completeness of E-coding in state hospital discharge databases typically exceeds 90%12. Even higher completeness was reported for Canadian administrative databases. However, insurance claims databases such as Medicare have low rates of E-code completeness, presumably because the billing software used by many hospitals removes E-codes since they have no relevance for hospital payments 13 A recent study reported that only 28% of injury hospitalizations in the 1999 Medicare Provider Analysis and Review (MedPAR) data had an E-code reported,13 and our own analyses have found similarly low E-coding rates in more recent data from Medicare, Medicaid, and commercial insurers.
The objective of this study was to create an algorithm to identify intentional self-harm hospitalizations in the absence of complete E-codes and evaluate the validity of the results that would be obtained using this algorithm in several hypothetical scenarios.
Methods
Data sources
Data were derived from two large population-based hospital discharge abstract databases. We chose to use two different databases in the interest of gaining some insight regarding the generalizeability of our findings. These data were drawn from two countries with different suicide rates, different practice patterns, and different hospital payment schemes, which may translate into differences in coding practice. In addition, there is variation in the number of diagnosis and procedure codes recorded and the availability of patient-level linkable data on prior inpatient hospitalizations, physician visits, and prescription drug use.
Data from British Columbia (BC), Canada were obtained from the BC Ministry of Health. The database includes records of hospitalizations for all patients in BC’s publicly funded healthcare system. Data elements include patient age and sex, up to 25 diagnosis codes including E-codes, up to 5 procedure codes, length of stay, and discharge disposition. An evaluation of this database found good specificity and completeness of diagnosis codes.14 We used data from 1999 through 2001, a period immediately prior to the transition from ICD-9-CM to ICD-10-CA diagnosis codes in BC.
Data from the United States came from the Nationwide Inpatient Sample (NIS), a publicly available dataset designed to approximate a 20% representative sample of all community hospitals in the United States.15 The NIS is produced by the Agency for Healthcare Research and Quality (AHRQ) from hospital inpatient discharge records submitted by state health data organizations. The 2003 NIS included data from 37 states. Data elements include hospital location (state), patient age, sex, and race, up to 15 diagnosis codes, up to 15 procedure codes, up to 4 E-codes, length of stay, primary payer, and discharge disposition. Data are subjected to internal consistency checks and formatted uniformly by AHRQ. An analysis of 2001 data by AHRQ found that E-codes were 86% complete for hospitalizations with an injury as a primary diagnosis.16 Because E-code collection and reporting requirements vary by state, we restricted our analysis to states with E-coding rates above 85% (see Appendix A). We quantified E-coding rates (the proportion of hospital discharges with an injury as a diagnosis that had a valid E-code) according to the method recommended by the State and Territorial Injury Prevention Directors Association, modified to include discharges that had an injury as a secondary rather than primary diagnosis. 17
Study sample
We identified hospitalizations with a primary or secondary diagnosis of injury or poisoning (800.x – 995.x), excluding cases where the only injury was an adverse effect of surgical or medical care (909.3, 909.5, 995.0 – 995.4, 995.88, 995.89, 995.9x), an adverse food reaction (995.6 – 995.7), or a late effect of a previous injury (905.x – 909.x). We restricted our sample to hospitalizations where an E-code other than E849.x, which denotes place of injury occurrence, was present. We restricted our population to subjects age 10 and over because the rate of intentional self-harm among subjects younger than 10 is less than 2 per 100,000.9 We included young children in a secondary analysis. Individual patients were allowed to contribute multiple hospitalizations. Cases with an injury as a secondary diagnosis were included in the interest of capturing injury hospitalizations where a condition precipitating or discovered as a result of the injury or a post-injury complication was coded as the primary diagnosis.18
Our gold standard for intentional self-harm was the presence of a diagnosis of E950 E958. We excluded E959 (late effects) from this definition as our intent was to identify acute outcomes rather than history of self-harm.
To construct algorithms to identify intentional self-harm cases, we collected information on types of injuries and psychiatric diagnoses recorded and psychiatric evaluations and procedures performed during the hospitalization. We created indicators for the presence of a diagnosis of each of the following psychiatric conditions (see appendix B for ICD-9 diagnosis codes): depression, anxiety disorder, sleep disorder, attention-deficit/hyperactivity disorder, substance abuse, psychotic disorder, dementia, delirium, personality disorders, unspecified non-psychotic mental disorders, adjustment reaction with disturbance other than depressed mood, and other psychiatric disorder. In addition, we created indicators for the presence of a diagnosis of each of the following types of injuries: open wound to elbow, forearm, or wrist; other open wound to upper limb; other open wounds and injuries to blood vessels; intracranial injury, internal injury, or injury to nerves and spinal cord; poisoning by psychotropic agents; poisoning by other drugs; toxic effects; asphyxiation; and other injury. We also collected information on patient age, sex, and race, as these factors may modify the risk of intentional self-harm and the performance of algorithms to identify it. In the BC dataset, we collected information on inpatient or outpatient depression diagnoses in the 180 days prior to the injury hospitalization and antidepressant use during this period, as patient history of depression is likely to affect intentional self-harm risk and may affect the performance of algorithms to identify it. Antidepressant prescriptions were identified in the PharmaNet database that includes drug name, dose, and quantity for all prescription drugs dispensed in British Columbia pharmacies. This information is entered by pharmacists via a province-wide network that assures minimal underreporting and misclassification and is recorded for all dispensings independent of the payor (provincial government, self-pay, commercially insured). PharmaNet data and data on physician services, maintained by the BC Ministry of Health, are linkable by unique patient identifiers to data on hospitalizations. Data on prescription drug use and history of depression diagnoses was not available in the NIS.
Appendix B.
ICD-9 Diagnosis Codes Used to Define Psychiatric Conditions and Injuries
|
Psychiatric Conditions | ||
| Depression | 293.83 | organic affective syndrome, depressive type |
| 296.2x | major depressive episode, single episode | |
| 296.3x | major depressive disorder, recurrent episode | |
| 296.90 | unspecified affective psychosis (e.g., melancholia NOS) | |
| 298.0x | depressive type psychoses | |
| 300.4x | neurotic depression | |
| 309.0x | brief depressive reaction | |
| 309.1x | prolonged depressive reaction | |
| 309.28 | adjustment reaction with anxiety and depression | |
| 311 | depressive disorder not elsewhere classified | |
| Anxiety disorder | 300.0x | anxiety states |
| 300.2x | phobic disorders | |
| 300.3x | obsessive-compulsive disorders | |
| Sleep disorder | 307.4x | specific disorders of sleep of non-organic origin |
| 347.xx | cataplexy and narcolepsy | |
| 780.5x | sleep disturbances | |
| Mania | 296.0x | manic disorder, single episode |
| 296.1x | manic disorder, recurrent episode | |
| 296.4x | bipolar affective disorder, manic | |
| 296.5x | bipolar affective disorder, depressed | |
| 296.6x | bipolar affective disorder, mixed | |
| 296.7x | bipolar affective disorder, unspecified | |
| 296.8x | bipolar affective disorder, other and unspecified | |
| 296.99 | other specified affective psychoses (e.g., mood swings) | |
| Attention-deficit/hyperactivity disorder | 312.xx | |
| 314.xx | ||
| Substance abuse | 291.0x | alcohol withdrawal delirium |
| 291.1x | alcohol amnestic syndrome | |
| 291.2x | other alcoholic dementia | |
| 291.3x | alcohol withdrawal hallucinosis | |
| 291.4x | idiosyncratic alcohol intoxication | |
| 291.5x | alcoholic jealousy | |
| 291.8x | other specified alcoholic psychosis | |
| 291.9x | unspecified alcoholic psychosis | |
| 303.xx | alcohol dependence syndrome | |
| 305.0x | alcohol abuse | |
| 292.0x | drug withdrawal syndrome | |
| 292.1x | paranoid and/or hallucinatory state induced by drugs | |
| 292.2x | pathological drug intoxication | |
| 292.8x | other specified drug-induced mental disorders | |
| 292.9x | unspecified drug-induced mental disorder | |
| 304.xx | drug dependence | |
| 305.2x-305.9x | non-dependent abuse of drugs | |
| Psychotic disorder | 290.8x | other specified senile psychotic conditions |
| 290.9x | unspecified senile psychotic condition | |
| 295.xx | schizophrenia | |
| 297.xx | paranoid states | |
| 298.1x-298.9x | other non-organic psychoses | |
| 299.xx | psychoses with origin specific to childhood | |
| 780.1x | hallucinations | |
| Dementia | 290.0x | senile dementia |
| 290.1x | presenile dementia | |
| 290.2x | senile dementia with delusional or depressive features | |
| 290.3x | senile dementia with delirium | |
| 290.4x | arteriosclerotic dementia | |
| 291.1x | alcoholic amnestic syndrome | |
| 291.2x | other alcoholic dementia | |
| 292.82 | drug induced dementia | |
| 294.0x | amnestic syndrome | |
| 294.1x | dementia in conditions classified elsewhere | |
| 294.8x | other specified organic brain syndromes (chronic) | |
| 294.9x | unspecified organic brain syndrome (chronic) | |
| 330.xx | cerebral degenerations usually manifest in childhood | |
| 331.0x | Alzheimer’s disease | |
| 331.1x | Pick’s disease | |
| 331.2x | senile degeneration of brain | |
| 331.7x | cerebral degeneration in diseases classified elsewhere | |
| 331.8x | other cerebral degeneration | |
| 331.9x | cerebral degeneration, unspecified | |
| Delirium | 290.11 | presenile dementia with delirium |
| 290.3 x | senile dementia with delirum | |
| 290.41 | arteriosclerotic dementia with delirium | |
| 291.0x | alcoholic delirium | |
| 292.81 | drug-induced delirium | |
| 293.0x | acute delirium | |
| 293.1x | subacute delirium | |
| 293.8x | other specified transient organic mental disorder | |
| 293.9x | unspecified transient organic mental disorder | |
| 348.3x | encephalopathy, unspecified | |
| 349.82 | toxic encephalopathy | |
| Personality disorders | 301.xx | |
| Unspecified non-psychotic mental disorder | 300.9 | |
| Adjustment reaction with disturbance other than depressed mood | 309.2x-309.9x | |
| Other psychiatric disorder | 300.1x | hysteria |
| 300.5 | Neurasthenia | |
| 300.6 | Depersonalization disorder | |
| 300.7 | Hypochondriasis | |
| 300.8x | somatoform disorder | |
| 302.xx | sexual deviations and disorders | |
| 306.xx | physiologic malfunction arising from mental factors | |
| 307.0x-307.3x | special symptoms or syndromes, not elsewhere classified | |
| 307.5x-307.9x | special symptoms or syndromes, not elsewhere classified | |
| 308.xx | acute reaction to stress | |
| 310.xx | nonpsychotic mental disorders due to organic brain damage | |
| 313.xx | disturbance of emotions specific to childhood or adolescence | |
| 315.xx | specific delays in development | |
| 316.xx | psychic factors associated with diseases classified elsewhere | |
| Injuries | ||
| Fracture | 800 – 829 | |
| Open wound to elbow, forearm, or wrist | 881 | |
| Other open wound to upper limb | 880 – 887, excluding 881 | |
| Other open wounds, injuries to blood vessels | 870 – 879 | Open wound to head, neck, trunk |
| 890 – 897 | open wound to lower limb | |
| 900 – 904 | injury to blood vessels | |
| Intracranial injury, internal injury, injury to nerves and spinal cord | 850 – 854 | Intracranial injury excluding skull fracture |
| 860 – 869 | Internal injury | |
| 950 – 957 | Injury to nerves and spinal cord | |
| Poisoning by psychotropic agents | 969 | |
| Poisoning by other drugs | 960 – 979 excl. 969 | |
| Toxic effects | 980 – 989 | |
| Asphyxiation | 994.7 | |
| Other injury | 830 – 839 | Dislocation |
| 840 – 848 | Sprains and strains | |
| 905 – 909 | Late effects | |
| 910 – 919 | Superficial injuries | |
| 920 – 924 | Contusion | |
| 925 – 929 | Crushing injury | |
| 930 – 939 | Foreign body | |
| 940 – 949 | Burns | |
| 958 – 959 | Traumatic complications | |
| 996 – 999 | Complications of medical/surgical care | |
| 990 – 995 | Other/unspecified | |
Statistical analysis
Each dataset was divided into a 50% development sample and a 50% validation sample. Using the development sample, we explored a series of simple algorithms based on the presence of a psychiatric diagnosis recorded during the hospitalization or on the type of injury reported to identify intentional-self harm hospitalizations. For example, a simple algorithm could assume that the presence of a depression diagnosis indicates that an injury hospitalization was the result of intentional-self harm. Based on these results, we identified more complex definitions (e.g. the presence of a depression or mania or psychotic disorder diagnosis) that might have desirable test characteristics. We created definitions based on the presence of one of a number of individual psychiatric conditions, selecting the individual conditions for inclusion based on their positive predictive values for intentional self-harm, using stricter and more lenient cut-points. The same process was used to create definitions based on the presence of one of a number of specific types of injuries. From these definitions, we created several stricter and more lenient definitions requiring both the presence of specific psychiatric conditions and specific injuries. Our final algorithms were assessed in the validation sample.
We assessed the performance characteristics of each algorithm. Sensitivity was defined as the probability that “true” intentional self-harm hospitalizations (“true” according to the recorded E-code) tested positive according to the algorithm, and were thus identified by it. Specificity was defined as the probability that hospitalization not due to self-harm (i.e., according to the recorded E-code) tested negative (i.e., were correctly ruled out using the algorithm). Positive predictive value (PPV) was calculated as the probability that hospitalizations identified by the algorithm were in fact due to self-harm based on E-codes.
We chose a preferred definition based on very high specificity and relatively high sensitivity. Very high specificity is essential to obtain unbiased ratio estimates for epidemiologic studies19. Relatively high sensitivity is desirable – as long as specificity remains high – because the larger the proportion of true cases that can be identified, the greater the power a study will have to identify relationships, especially when studying rare outcomes.20
To test the magnitude of the bias that might result from using the preferred algorithm to identify intentional self-harm hospitalizations, we created a sample dataset for a hypothetical study of antidepressant safety. We calculated intentional self-harm hospitalization rate ratios for antidepressant users versus non-users under several realistic scenarios using 1) the “true” event rates and 2) those that would be observed using our algorithm.
We evaluated three different scenarios. The first was based on the BC data; we assumed that intentional self-harm hospitalization rates in the absence of antidepressant treatment are equal to those observed in the general BC population21 and, in the presence of antidepressant treatment, are elevated at a ratio equal to the increased risk of self-harm events noted in randomized controlled trials (RCT) (a 1.90-fold increase for adolescents and a 1.57-fold increase for adults).2, 22 Based on the sensitivity and specificity of the algorithm in the BC sample, we calculated the rate of intentional self-harm hospitalizations that would be identified in each treatment arm. The equation is true positives [true intentional self-harm hospitalizations *sensitivity] + false positives [non intentional self-harm hospitalizations * (1− specificity)]. An observed rate ratio was computed and bias was calculated as the rate ratio minus the expected rate ratio from the RCT data.
Because intentional self-harm rates are likely to be higher in a depressed population, we conducted a second analysis using intentional self-harm hospitalization rates observed in the BC sample during the first year of antidepressant treatment as the rate among antidepressant users.10, 11 The rate among depressed non-users was calculated by dividing this rate by the RCT rate ratio. Finally, we calculated observed rates using intentional self-harm and other injury hospitalization rates observed in the general US population9 and the sensitivity and specificity values calculated from the NIS.
The study is covered by data use agreements with AHRQ (NIS) and the British Columbia Ministry of Health and was approved by the Brigham and Women’s Institutional Review Board.
Results
For the period 1999 through 2001, we identified 177,618 hospital discharges of patients ages 10 and over in the BC dataset that had a valid E-code and a primary or secondary ICD-9-CM diagnosis code for an injury other than an adverse effect of medical care, late effect, or adverse food reaction. In the 2003 NIS, we identified 527,798 injury hospitalizations meeting these criteria. After deleting 4,110 with missing age or sex, 254,910 from states with E-code reporting rates less than 85%, and 13,431 hospitalizations without E-codes from remaining states, we were left with a sample size of 245,164.
Characteristics of the hospital discharges randomly allocated to the development sample -- 88,808 of 177,618 BC hospitalizations and 122,574 of 254,910 NIS hospitalizations -- are presented in Table 1. The mean patient age was 55 in BC and 57 in the NIS, with the population roughly split between males and females. Race was not reported in the BC data; in the NIS sample 75.9% of patients were white.. In-hospital death resulted from 1.1% of BC admissions and 2.7% of NIS admissions. Fractures were the most common injury type, accounting for 34.0% of BC admissions and 51.4% of NIS admissions. Substance abuse was the most commonly recorded psychiatric diagnosis (8.0% in BC, 13.7% in NIS), followed by depression (4.1% in BC, 12.2% in NIS) and dementia (3.6% in BC, 8.5% in NIS). A greater proportion of injury hospitalizations in younger subjects were due to intentional self-harm (in BC, 13.8% in age 10 – 25, 10.2% in age 25 – 64; in the NIS, 15.8% in age <25, 12.2% in age 25 – 64) than were in the elderly (0.8% in BC, 0.7% in the NIS).
Table 1.
Patient characteristics in the 50% development sample of hospital discharges with a primary or secondary injury diagnosis
| British Columbia | US Nationwide Inpatient Sample | |||
|---|---|---|---|---|
| N (%) | Patients with intentional self- harm E-code – N (row %) | N (%) | Patients with intentional self- harm E-code – N (row %) | |
| Total N | 88,808 | 6,127 (6.9) | 122,574 | 9,345 (7.6) |
| Age | ||||
| 10–25 | 11,887 (13.4) | 1,643 (13.8) | 16,375 (13.4) | 2,599 (15.8) |
| 25–64 | 41,160 (46.4) | 4,187 (10.2) | 52,112 (42.5) | 6,369 (12.2) |
| 65+ | 35,761 (40.3) | 297 (0.8) | 54,094 (44.1) | 377 (0.7) |
| Sex | ||||
| Male | 45,458 (50.9) | 2,307 (5.1) | 58,626 (47.8) | 3,781 (6.5) |
| Female | 43,650 (49.2) | 3,820 (8.8) | 63,955 (52.2) | 5,564 (8.7) |
| Geographic region | ||||
| Northeast US | 0 (0%) | N/A | 46,330 (37.8) | 3,313 (7.2) |
| South US | 0 (0%) | N/A | 30,211 (24.7) | 2,492 (8.3) |
| Midwest US | 0 (0%) | N/A | 11,067 (9.0) | 1,030 (9.3) |
| Western US | 0 (0%) | N/A | 34,973 (28.5) | 2,510 (7.2) |
| BC | (100%) | 0 (0) | N/A | |
| Length of stay – median | 4.0 | N/A | 3.0 | N/A |
| In-hospital death | 965 (1.1) | 30 (3.1) | 3,243 (2.7) | 135 (4.2) |
| Race | ||||
| White | Unknown | N/A | 80,540 (75.9) | 6,055 (7.5) |
| Black | Unknown | N/A | 11,353 (10.7) | 869 (7.7) |
| Other | Unknown | N/A | 14,278 (13.4) | 1,140 (8.0) |
| Type of injury | ||||
| Fracture | 30,179 (34) | 195 (0.6) | 63,030 (51.4) | 181 (0.3) |
| Open wound to wrist, elbow, or forearm | 1,342 (1.5) | 93 (6.9) | 2,607 (2.1) | 784 (30.1) |
| Open wound to upper extremity | 5,309 (6.0) | 298 (5.6) | 3,568 (2.9) | 131 (3.7) |
| Open wound to lower extremity or injury to blood vessels | 7,488 (8.4) | 170 (2.3) | 15,797 (12.9) | 397 (2.5) |
| Intracranial injury, internal injury, injury to nerves and spinal cord | 756 (0.9) | 28 (3.7) | 19,964 (16.3) | 257 (1.3) |
| Burns | 2,946 (3.3) | 2,348 (79.7) | 2,027 (1.7) | 77 (3.8) |
| Poisoning by psychotropic agent | 5,230 (5.9) | 3,273 (62.6) | 5,540 (4.5) | 3,721 (67.2) |
| Poisoning by other drugs | 1,021 (1.1) | 566 (55.4) | 10,102 (8.2) | 5,053 (50.0) |
| Toxic effects of non-medicinal substances | 1,065 (1.2) | 464 (43.6) | 2,367 (1.9) | 861 (36.4) |
| Asphyxiation | 84 (0.1) | 71 (84.5) | 70 (0.1) | 55 (78.6) |
| Other injuries | 40,842 (46) | 374 (0.9) | 110,754 (90.4) | 9,125 (8.2) |
| Presence of mental health diagnoses | ||||
| Psychiatric procedure | 210 (0.2) | 95 (45.2) | 1,497 (1.2) | 449 (30.0) |
| Depression | 3,685 (4.1) | 1,861 (50.5) | 14,985 (12.2) | 5,710 (38.1) |
| Anxiety | 564 (0.6) | 169 (30) | 2,977 (2.4) | 506 (17.0) |
| Mania | 2,067 (2.3) | 1,192 (57.7) | 3,069 (2.5) | 1,418 (46.2) |
| Sleep disorder | 116 (0.1) | * | 1,448 (1.2) | 115 (7.9) |
| Attention Deficit Hyperactivity Disorder | 197 (0.2) | 88 (44.7) | 723 (0.6) | 293 (40.5) |
| Substance Abuse | 7,103 (8.0) | 1,883 (26.5) | 16,841 (13.7) | 3,822 (22.7) |
| Psychotic Disorder | 2,307 (2.6) | 361 (15.6) | 3,011 (2.5) | 677 (22.5) |
| Dementia | 3,213 (3.6) | 38 (1.2) | 10,474 (8.5) | 57 (0.5) |
| Delirium | 2,033 (2.3) | 77 (3.8) | 2,936 (2.4) | 301 (10.3) |
| Personality Disorder | 1,688 (1.9) | 1,131 (67) | 1,298 (1.1) | 853 (65.7) |
| Adjustment reaction, non-depressed | 607 (0.7) | 404 (66.6) | 1,115 (0.9) | 630 (56.5) |
| Unspecified nonpsychotic mental disorder | 598 (0.7) | 467 (78.1) | 825 (0.7) | 603 (73.1) |
| Other mental disorder | 1,112 (1.3) | 379 (34.1) | 1,374 (1.1) | 339 (24.7) |
| Presence of chronic conditions | ||||
| Cancer | 6,968 (7.8) | 66 (0.9) | 4,448 (3.6) | 96 (2.2) |
| HIV | 118 (0.1) | 16 (13.6) | 595 (0.5) | 110 (18.5) |
cell sizes less than 12 are not reported
The performance characteristics of possible algorithms to identify intentional self-harm hospitalizations are presented in Table 2. In the BC data, the presence of a substance abuse diagnosis had the greatest sensitivity (30.7%), indicating that 30.7% of intentional self-harm hospitalizations had a substance abuse diagnosis recorded. Other diagnoses that were commonly assigned to intentional self-harm hospitalizations were depression (sensitivity = 30.4%), mania (19.5%), personality disorder (18.5%), unspecified psych (6.2%), adjustment reaction (6.6%), other psych disorder (6.2), and psychotic disorder (5.9%). The specificity – i.e. the probability that hospitalizations for injuries other than intentional self-harm didn’t have the diagnosis coded – was above 96% for all psychiatric diagnoses excluding substance abuse (specificity = 94%). The positive predictive value – i.e. the probability that a hospitalization assigned a psychiatric diagnosis of interest is an intentional self-harm hospitalization – was highest for unspecified non-psychotic mental disorders (78.1%), personality disorder (67.0%), adjustment reaction (66.6%), mania (57.7%) and depression (50.5%). Similar patterns were observed in the NIS data, although depression was recorded with a greater frequency among NIS intentional self-harm cases (sensitivity = 61.1%) and in the overall population (12.2% in the NIS versus 4.2% in BC). Algorithms based on a combination of psychiatric diagnoses had sensitivities ranging from 62.9% to 69.3% in the BC data and 77.6% to 82.1% in the NIS; PPVs ranged from 43.9% to 55.1% in the BC data and 35.5% to 39.4% in the NIS.
Table 2.
Operating characteristics of alternative algorithms to identify intentional self-harm cases among injury hospital discharges – development sample
| British Columbia development sample | NIS development sample | |||||||
|---|---|---|---|---|---|---|---|---|
| Algorithm operating characteristics | Algorithm operating characteristics | |||||||
| % hospital discharges with diagnosis coded | Sensitivity | Specificity | Positive Predictive Value | % hospital discharges with diagnosis coded | Sensitivity | Specificity | Positive Predictive Value | |
| Algorithms Based on Presence of Inpatient Diagnoses | ||||||||
| Individual Psychiatric Diagnoses | ||||||||
| Attention-deficit/hyperactivity disorder (ADHD) | 0.22 | 1.44 | 99.87 | 44.67 | 0.59 | 3.14 | 99.62 | 40.53 |
| Depression | 4.15 | 30.37 | 97.79 | 50.5 | 12.22 | 61.1 | 91.81 | 38.1 |
| Mania | 2.33 | 19.45 | 98.94 | 57.67 | 2.5 | 15.17 | 98.54 | 46.2 |
| Other psychiatric disorder | 1.25 | 6.19 | 99.11 | 34.08 | 1.12 | 3.63 | 99.09 | 24.67 |
| Personality disorder | 1.90 | 18.46 | 99.33 | 67 | 1.06 | 9.13 | 99.61 | 65.72 |
| Psychotic disorder | 2.60 | 5.89 | 97.65 | 15.65 | 2.46 | 7.24 | 97.94 | 22.48 |
| Adjustment reaction other than depressed mood | 0.68 | 6.59 | 99.75 | 66.56 | 0.91 | 6.74 | 99.57 | 56.5 |
| Unspecified non-psychotic mental disorder | 0.67 | 7.62 | 99.84 | 78.09 | 0.67 | 6.45 | 99.8 | 73.09 |
| Substance abuse | 8.00 | 30.73 | 93.69 | 26.51 | 13.74 | 40.9 | 88.5 | 22.69 |
| Combinations of Psychiatric Diagnoses | ||||||||
| Depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder | 7.88 | 62.92 | 96.2 | 55.1 | 15.02 | 77.58 | 90.14 | 39.37 |
| or ADHD or psychotic disorder or other psychiatric disorder | 10.9 | 69.32 | 93.42 | 43.86 | 17.64 | 82.07 | 87.67 | 35.46 |
| Injury Type | ||||||||
| Open wound to wrist, elbow, or forearm | 1.2 | 7.57 | 99.27 | 43.57 | 2.13 | 8.39 | 98.39 | 30.07 |
| Other open wound to upper extremity | 1.51 | 1.52 | 98.49 | 6.93 | 2.91 | 1.4 | 96.96 | 3.67 |
| Open wound to lower extremity, injury to blood vessels | 5.98 | 4.86 | 93.94 | 5.61 | 12.89 | 4.25 | 86.4 | 2.51 |
| Poisoning by psychotropic agent | 3.32 | 38.32 | 99.28 | 79.7 | 4.52 | 39.82 | 98.39 | 67.17 |
| Poisoning by other drugs | 5.89 | 53.42 | 97.63 | 62.58 | 8.24 | 54.07 | 95.54 | 50.02 |
| Toxic effects of non-medicinal substances | 1.15 | 9.24 | 99.45 | 55.44 | 1.93 | 9.21 | 98.67 | 36.38 |
| Aphyxiation | 0.09 | 1.16 | 99.98 | 84.52 | 0.06 | 0.59 | 99.99 | 78.57 |
| Combinations of Injury types | ||||||||
| Poisoning, toxicity of substance chiefly non-medical in nature, or asphyxiation | 8.96 | 84.33 | 96.62 | 64.9 | 12.36 | 82.32 | 93.41 | 50.76 |
| or open wound to wrist, elbow, or forearm | 10.04 | 90.29 | 95.9 | 62.02 | 14.38 | 89.4 | 91.81 | 47.4 |
| Combinations of injury type and presence of psychiatric diagnoses | ||||||||
| Depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder – PLUS – poisoning, toxicity of substance chiefly non-medical in nature, asphyxiation, or open wound to elbow, wrist or forearm | 4.5 | 57.25 | 99.41 | 87.83 | 7.3 | 70.75 | 97.93 | 73.86 |
| Depression, personality disorder, mania, adjustment reaction, unspecified non-psychotic mental disorder, ADHD, psychotic disorder or other mental disorder – PLUS – poisoning, toxicity of substance chiefly non-medical in nature, asphyxiation, or open wound to elbow, wrist or forearm | 5.09 | 62.61 | 99.17 | 84.85 | 7.85 | 74.08 | 97.62 | 71.94 |
Definitions of algorithm operating characteristics:
Sensitivity: probability that intentional self-harm hospitalization has diagnosis/diagnoses coded and is detected by algorithm
Specificity: probability that non intentional self-harm hospitalization does not have diagnosis/diagnoses coded and is ruled out by algorithm
Positive predictive value (PPV): probability that hospitalization detected by the algorithm is a true intentional self-harm hospitalization, based on gold standard of external cause of injury codes (E-codes)
Certain types of injuries were frequently recorded among intentional self-harm hospitalizations. In the BC data, 38.3% of intentional self-harm hospitalizations had a diagnosis of poisoning by a psychotropic agent, 53.4% had a diagnosis of poisoning by another drug, 9.2% had a diagnosis of toxic effects of non-medicinal substances, and 7.6% had a diagnosis of an open wound to the wrist, forearm, or elbow. While asphyxiation was an uncommon diagnosis (sensitivity was 1.2%), the PPV for this diagnosis was high – 84.5% -- indicating that the majority of asphyxiation cases resulted from intentional self-harm. Similar patterns were observed in the NIS, although PPVs were lower. An algorithm defining intentional self-harm as a hospitalization for poisoning, toxicity of substances chiefly non-medical in nature, or asphyxiation had a sensitivity of 84.3% in the BC development sample (82.3% in the NIS) and a PPV of 64.9% in the BC development sample (50.8% in the NIS). Including open wound to wrist, elbow or forearm increased the sensitivity to 90.3% (89.4% NIS) but reduced the PPV to 62.0% (47.4% NIS). Algorithms based on type of injury and the presence of psychiatric diagnoses provided an improvement in PPV at the loss of some sensitivity. Defining intentional self-harm hospitalizations as those with a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder plus a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm yielded a sensitivity of 59.8%, a specificity of 99.4%, and PPV of 88.3% in the BC data; values in the NIS were 71.1%, 98.0%, and 74.1%. Adding ADHD, psychotic disorder, and other mental disorders to the list of allowed psychiatric diagnoses increased the sensitivity to 65% but reduced the PPV to 85.8% in the BC data. In the NIS, the values were 74.2 and 72.3%. In the interest of maintaining a high specificity at the expense of some sensitivity, we elected to omit the ADHD, psychotic disorder, and other mental disorder diagnoses from our final algorithm.
Thus, our final algorithm classifies a hospitalization due to injury as resulting from intentional self-harm if a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder is recorded as well as a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm.
The performance of the derived algorithms in the validation sample varied by patient characteristics, as shown for the more restrictive algorithm above in Table 3. In both BC and the NIS, the specificity was highest for hospitalizations among patients age <25, resulting in the highest PPV in this group (92.2% in BC, 83.4% NIS). The low prevalence of intentional self-harm hospitalizations among all injury hospitalization in subjects aged > 65 resulted in a low PPV in this group (65.8% in BC, 48.6% in NIS). The PPV was slightly lower in males. In the BC sample, which included history of depression diagnosis or antidepressant prescription in the past 180 days, PPV was slightly lower in subjects with no antidepressant use (87.5% versus 88.9% for those with past antidepressant use in BC) and differed little by prior history of depression diagnosis. Results from a secondary analysis including children younger than 10 were essentially identical.
Table 3.
Operating characteristics of the preferred algorithm by patient characteristics in the 50% validation sample. The preferred algorithm defines intentional self-harm hospitalizations as those with a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder plus a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm.
| British Columbia (N = 88,810) | US – Nationwide Inpatient Sample (N= 122,590) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Hospital discharges | Algorithm operation characteristics | Hospital discharges | Algorithm operation characteristics | |||||||
| % due to intentional self-harm | % meeting algorithm criteria | Sensitivity | Specificity | Positive Predictive Value | % due to intentional self-harm | % meeting algorithm criteria | Sensitivity | Specificity | Positive Predictive Value | |
| All subjects | 6.90 | 4.74 | 59.81 | 99.4 | 88.32 | 7.55 | 7.24 | 71.06 | 97.97 | 74.06 |
| By age | ||||||||||
| < 25 | 13.82 | 8.63 | 57.12 | 99.22 | 92.24 | 16.08 | 13.7 | 71.04 | 97.29 | 83.4 |
| 25 – 64 | 10.17 | 7.2 | 61.05 | 99.09 | 88.69 | 12.0 | 11.79 | 71.47 | 96.35 | 72.77 |
| 65+ | 0.83 | 0.64 | 56.6 | 99.78 | 65.79 | 0.72 | 0.96 | 64.63 | 99.5 | 48.57 |
| By sex | ||||||||||
| Female | 8.75 | 6.45 | 63.62 | 99.18 | 88.42 | 8.68 | 8.93 | 76.93 | 97.53 | 74.71 |
| Male | 5.11 | 3.11 | 53.45 | 99.61 | 88.13 | 6.33 | 5.42 | 62.34 | 98.43 | 72.88 |
| By antidepressant use in the past 180 days | ||||||||||
| Use | 19.40 | 14.89 | 68.25 | 97.95 | 88.92 | N/A | ||||
| No use | 4.14 | 2.4 | 50.65 | 99.69 | 87.45 | N/A | ||||
| By depression diagnosis in the past 180 days | ||||||||||
| Diagnosis | 16.68 | 12.94 | 68.19 | 98.12 | 87.92 | N/A | ||||
| No diagnosis | 5.72 | 3.66 | 56.58 | 99.55 | 88.51 | N/A | ||||
Definitions of algorithm operating characteristics:
Sensitivity: probability that intentional self-harm hospitalization is detected by algorithm
Specificity: probability that non intentional self-harm hospitalization is ruled out by algorithm
Positive predictive value (PPV): probability that hospitalization detected by the algorithm is an intentional self-harm hospitalization
Figure 1 summarizes the bias in estimated relative rates of intentional self-harm that might result from using the algorithm to identify intentional self-harm hospitalizations in a hypothetical study of antidepressant safety conducted under three scenarios. In the first example (Figure 1a) based on data from the general BC population where intentional self-harm hospitalizations accounted for 10% – 23% of injury hospitalizations depending on age and antidepressant use, intentional self-harm rate ratios comparing antidepressant use to non-use were underestimated by about 0.07 (1.83 versus 1.90 for adolescents, 1.50 versus 1.57 for non-senior adults). In the second example, based on patients initiating antidepressants in BC where rates of self-harm are substantially higher and intentional self-harm hospitalizations are highly prevalent among injury hospitalizations (62 – 76% of injury hospitalizations in those under 25, 24 – 33% in non-senior adults), bias was reduced to −0.007 and −0.03. Figure 1c depicts the results from an analysis based on rates of intentional self-harm and other injury hospitalizations observed in the general US population9 and the sensitivity and specificity values calculated from the NIS. Although the prevalence of intentional self-harm hospitalizations among all injury hospitalizations was comparable to that in the BC general population, the reduced specificity of the algorithm in the US data led to greater bias. Rate ratios were underestimated by 0.12 – 0.13.
Figure 1.
Bias in Rate Ratios Calculated Using the Algorithm to Identify Outcomes
Discussion
In an analysis of 1999 – 2001 hospital data from the province of British Columbia and 2003 data from the US Nationwide Inpatient Sample, we found that intentional self-harm hospitalizations could be identified with reasonable sensitivity and high specificity without the use of external cause of injury codes (E-codes). Our preferred algorithm, which identified intentional self-harm hospitalizations based on the presence of a diagnosis of depression, personality disorder, mania, adjustment reaction, or unspecified non-psychotic mental disorder in combination with a diagnosis of poisoning, toxicity of a substance chiefly non-medical in nature, asphyxiation, or open wound to the elbow, wrist, or forearm had a sensitivity of 59.8%, a specificity of 99.4%, and a positive predictive value of 88.3% in the BC data to predict E-codes for intentional self-harm among a population aged 10 and over. In the NIS, the values were 71.1%, 97.9%, and 74.1%. While the algorithm will underestimate rates of intentional self-harm due to its moderate sensitivity, based on current E-code completeness rates in administrative data, the algorithm will detect twice as many cases as would be found using E-codes alone. Because intentional-self harm is a relatively rare outcome, this increase in statistical power is likely to be important.
The proposed algorithm had the highest positive predictive value (PPV) in patients age 10 to 25 and a reasonable PPV in patient age 25 – 64. The PPV was low in patients aged 65 and over, in part due to the low prevalence of intentional self-harm hospitalizations in this age group; among subjects age 65 and over, fewer than 1% of injury hospitalization were due to intentional self-harm according to recorded E-codes. The low prevalence of intentional self-harm hospitalizations is likely due to several factors. While the suicide rate in this population is high and the case fatality rate (CFR) is higher than in younger populations (44% CFR in age 65+ versus 3% in adolescents and 8% in non-senior adults),9 the rate of hospital visits for nonfatal self-harm among seniors is very low relative to that among adolescents and non-senior adults.23 In addition, the rate of hospitalization for other types of injuries is higher in subjects 65 and over than in younger age groups. 9. The lower PPV in seniors suggests the algorithm may be of greatest use in adolescent and non-senior adult populations.
Several limitations should be considered. We used recorded E-codes from hospital discharge records as the gold standard for defining intentional self-harm hospitalizations. However, these E-codes are likely not 100% accurate. In a 1996 study from the state of Washington, agreement between hospital discharge data and chart review for intent of injury was 95% and for mechanism of injury was 87%, as measured by kappa statistics.24 A study conducted in a Canadian teaching hospital found that rates of deliberate self-poisoning were underestimated by E-codes relative to medical chart review, particularly among subjects over the age of 55.25 The under-coding of intentional self-harm E-codes among older adults may in part explain the low specificity of our algorithm in seniors. A direct validation of the algorithm against medical chart review would be preferable to fully assess its validity.
Physician service records are not available for the hospitalizations included in the NIS. Because most hospital physicians are salaried rather than paid on a fee-for-service basis in British Columbia, complete records are not available for BC hospitalizations either. It may be possible to improve the performance of these algorithms by including diagnoses and procedures recorded in physician service claims. Although our samples included patients admitted to the hospital through the emergency department, we did not have data to test the algorithms in treated-and-released emergency department (ED) visits. Because 42% of cases are seen in the ED only,9 this will be an important area of research. Also, we tested the algorithm only on data using ICD-9-CM coding. Canadian hospitals began migrating to ICD-10-CA, a Canadian clinical modification of the tenth revision of the ICD, in the early and mid-2000s, and it will be useful to test the algorithm in this context. Finally, because our algorithm is based on a limited number of injury types, it will systematically miss self-harm by other methods such as drowning, firearm, fire, blunt trauma, jumps, motor vehicle, and open wound to an area other than the wrist, forearm, or elbow. Together, methods other than those included in the algorithm accounted for 10% of E-coded self-harm injuries in the BC sample and 11% in the NIS sample.
In summary, we have developed and tested an algorithm that identifies intentional self-harm hospitalizations with very high specificity and reasonable sensitivity, particularly in younger patients, relative to the gold standard of the presence of an intentional self-harm E-code. Use of this algorithm in a hypothetical study of the effects of antidepressant use on suicidality resulted in little bias in relative risk estimates. While steps should be taken to ensure high E-code rates in insurance databases, in the meantime this algorithm may be of value in studies of deliberate self-harm that rely on claims databases from Medicaid, or commercial insurers where E-codes are substantially under-recorded.
Acknowledgments
This project was funded by a grant from the National Institute of Mental Health (RO1-MH078708). Amanda Patrick had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Funding:
This project was funded by a grant from the National Institute of Mental Health (RO1-MH078708).
Appendix A: States by region
Northeast: Connecticut*,#, Maine, Massachusetts*,#, New Jersey*,#, New Hampshire*, New York*,#, Pennsylvania*,#, Rhode Island*,#, and Vermont*,#.
South: Florida*, Georgia*,#, Maryland*,#, North Carolina*,#, South Carolina*, Virginia*, West Virginia*, Alabama, Kentucky*, Mississippi, Tennessee*,#, Arkansas, Louisiana, Oklahoma and Texas*
Midwest: Ohio*, Indiana*, Michigan*, Illinois*, Wisconsin*,#, Iowa*, Kansas*, Missouri*,#, Minnesota*, Nebraska*,#, South Dakota*, and North Dakota.
West: Alaska, Arizona*, California*,#, Colorado*,#, Hawaii*, Idaho, Montana, New Mexico, Nevada*, Oregon*, Utah*,#, Washington*,#, and Wyoming.
Footnotes
States included in 2003 NIS
States with E-coding rate > 85%
References
- 1.Food and Drug Administration. Summary minutes of the September 13–14, 2004. Center for Drug Evaluation and Research Pharmachopharmacologic Drugs Advisory Committee and the FDA Pediatric Advisory Committee; [Accessed February 1, 2005.]. http://www.fda.gov/ohrms/dockets/ac/04/minutes/2004-4065M1_Final.htm. [Google Scholar]
- 2.Hammad TA, Laughren T, Racoosin J. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry. 2006 Mar;63(3):332–339. doi: 10.1001/archpsyc.63.3.332. [DOI] [PubMed] [Google Scholar]
- 3.Food and Drug Administration. [Accessed December 22, 2008.];Information for Healthcare Professionals Suicidality and Antiepileptic Drugs. 2008 January 31; http://www.fda.gov/cder/drug/InfoSheets/HCP/antiepilepticsHCP.htm.
- 4.Food and Drug Administration. Public Health Advisory. [Accessed January 15, 2009.];Important Information on Chantix (varenicline) http://www.fda.gov/cder/drug/advisory/varenicline.htm.
- 5.Food and Drug Administration. [Accessed January 15, 2009.];FDA Safety Update: Asthma Medications. http://www.fda.gov/consumer/asthmameds051308.html.
- 6.Mundy A. Drugs’ Links to Suicide Risk Draw Concern. The Wall Street Journal (Eastern edition) 2008 July 9; [Google Scholar]
- 7.Iribarren C, Sidney S, Jacobs DR, Jr, Weisner C. Hospitalization for suicide attempt and completed suicide: epidemiological features in a managed care population. Soc Psychiatry Psychiatr Epidemiol. 2000 Jul;35(7):288–296. doi: 10.1007/s001270050241. [DOI] [PubMed] [Google Scholar]
- 8.Patorno E, Bohn RL, Wahl PM, et al. Anticonvulsant medications and the risk of suicide, attempted suicide, or violent death. Jama. 2010 Apr 14;303(14):1401–1409. doi: 10.1001/jama.2010.410. [DOI] [PubMed] [Google Scholar]
- 9.Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Web-based Injury Statistics Query and Reporting System (WISQARS) www.cdc.gov/ncipc/wisqars.
- 10.Schneeweiss S, Patrick AR, Solomon DH, et al. Comparative safety of antidepressant agents for children and adolescents regarding suicidal acts. Pediatrics. 2010 May;125(5):876–888. doi: 10.1542/peds.2009-2317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Schneeweiss S, Patrick AR, Solomon DH, et al. Variation in the risk of suicide attempts and completed suicides by antidepressant agent in adults: a propensity score-adjusted analysis of 9 years’ data. Arch Gen Psychiatry. 2010 May;67(5):497–506. doi: 10.1001/archgenpsychiatry.2010.39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Council of State and Territorial Epidemiologists; Data Committee Injury Control and Emergency Health Services Section, American Public Health Association; and State and Territorial Injury Prevention Directors Association. How states are collecting and using cause of injury data: 2004 update to the 1997 report 2005.
- 13.Clark DE, DeLorenzo MA, Lucas FL, Wennberg DE. Epidemiology and short-term outcomes of injured medicare patients. J Am Geriatr Soc. 2004 Dec;52(12):2023–2030. doi: 10.1111/j.1532-5415.2004.52560.x. [DOI] [PubMed] [Google Scholar]
- 14.Williams JI, Young W. Technical report. Institute for Clinical Evaluative Sciences (ICES); Dec, 1996. Inventory of studies on the accuracy of Canadian health administrative databases. [Google Scholar]
- 15.HCUP Nationwide Inpatient Sample (NIS). Healthcare Cost and Utilization Project (HCUP) Agency for Healthcare Research and Quality. Rockville, MD: 2000–2001. www.hcup-us.ahrq.gov/nisoverview.jsp. [Google Scholar]
- 16.Barrett M, Steiner C, Coben J. HCUP Methods Series Report # 2004–06 ONLINE. April 14,2005. U.S. Agency for Healthcare Research and Quality; Healthcare Cost and Utilization Project (HCUP) E Code Evaluation Report. 2004. [Google Scholar]
- 17.Injury Surveillance Workgroup. Consensus Recommendations for Using Hospital Discharge Data for Injury Surveillance. Marietta (GA): State and Territorial Injury Prevention Directors Association; 2003. [Google Scholar]
- 18.Lawrence BA, Miller TR, Weiss HB, Spicer RS. Issues in using state hospital discharge data in injury control research and surveillance. Accid Anal Prev. 2007 Mar;39(2):319–325. doi: 10.1016/j.aap.2006.08.001. [DOI] [PubMed] [Google Scholar]
- 19.Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3. Philadelphia: Lippincott Williams & Wilkins; 2008. [Google Scholar]
- 20.Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005 Apr;58(4):323–337. doi: 10.1016/j.jclinepi.2004.10.012. [DOI] [PubMed] [Google Scholar]
- 21.British Columbia Injury Research and Prevention Unit. [Accessed March 18, 2009.];BCIRPU Online Data Tool. Available at: http://www.injuryresearch.bc.ca/categorypages.aspx?catid=10&catname=Online%20Data%20Tool.
- 22.Gunnell D, Saperia J, Ashby D. Selective serotonin reuptake inhibitors (SSRIs) and suicide in adults: meta-analysis of drug company data from placebo controlled, randomised controlled trials submitted to the MHRA’s safety review. Bmj. 2005 Feb 19;330(7488):385. doi: 10.1136/bmj.330.7488.385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Centers for Disease Control and Prevention (CDC) Nonfatal self-inflicted injuries among adults aged > or = 65 years--United States, 2005. MMWR Morb Mortal Wkly Rep. 2007 Sep 28;56(38):989–93. [PubMed] [Google Scholar]
- 24.LeMier M, Cummings P, West TA. Accuracy of external cause of injury codes reported in Washington State hospital discharge records. Inj Prev. 2001 Dec;7(4):334–338. doi: 10.1136/ip.7.4.334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Rhodes AE, Links PS, Streiner DL, Dawe I, Cass D, Janes S. Do hospital E-codes consistently capture suicidal behaviour? Chronic Dis Can. 2002 Fall;23(4):139–145. [PubMed] [Google Scholar]