Emergency department and inpatient coding for self‐harm and suicide attempts: Validation using clinician assessment data

Abstract

Administrative data have been used to determine the occurrence of suicide attempts and deliberate self‐harm, but research about the accuracy of these sources is limited. This study used a clinical sample (n = 5719) containing psychiatry consultations from the emergency departments and inpatient units of the two major tertiary hospitals in Winnipeg, Canada to validate the accuracy of inpatient hospital diagnosis codes at identifying presentations for self‐harm and suicide attempts. The Columbia Classification Algorithm of Suicide Assessment (C‐CASA) was used as the gold standard. International Classification of Diseases version 10 Canadian Enhancement codes for intentional self‐harm, undetermined intent self‐harm, and accidental poisoning were assessed. Measures of validity included Kappa (κ), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity of hospitalized attempts was low using intentional intent codes (36.9%, 95% confidence interval [CI]: 32.4–41.4%) but improved using unknown intent and accidental poisoning codes (44.8%, 95% CI: 40.2–49.4%). Agreement for suicide attempts did not increase with the addition of unknown intent and accidental poisoning codes (κ = 0.465–0.481), but were better for any self‐harm (κ = 0.395–0.478). Hospital diagnosis codes undercount attempts and self‐harm admissions. Including more data sources might improve the detection of events.

1. INTRODUCTION

Suicidal behavior has been the subject of a substantial amount of research. Many studies of suicidal behavior have focused on assessing potential risk factors (Johnson, Krug, & Potter, 2000; Johnson et al., 2002; Hawton & van Heeringen, 2009; Hawton, Saunders, & O'Connor, 2012; Turecki & Brent, 2016). Understanding the factors leading to suicidal behavior is expected to improve treatment for patients and to prevent the occurrence of the significant morbidity and mortality associated with suicidal behavior (Christiansen et al., 2014). In the last few decades research has increasingly used administrative databases and registries to provide information on the occurrence of risk factors and suicide attempts and deaths (Bolton et al., 2015; Christiansen & Stenager, 2012; Katz et al., 2011; Morrison & Laing, 2011; Nordentoft, Mortensen, & Pedersen, 2011; Randall et al., 2014; Spirito & Esposito‐Smythers, 2006; Spittal et al., 2014; Tran et al., 2014). These registries contain records of the use of health services by individuals and allow population‐based work with significant power to be conducted. However, administrative data are not primarily intended to be used for research and require validation of their contents and appropriateness for use as research variables (Lix et al., 2012; Roos et al., 2005).

The validity of the occurrence of suicide attempts and deaths derived from these data is important to the interpretation of work relying on this information. Research assessing methods of identifying suicide attempts in administrative data is insufficient and a recent systematic review determined that more research is needed (Walkup et al., 2012). This review identified only six small sample studies and none of which used the newest version of the International Classification of Disease coding system, version number 10 (ICD‐10). These studies also chiefly relied on chart reviews to establish a gold standard; such work relies on proper documentation in the charts. Larger samples with better gold standards are required to understand the validity of diagnostic coding schemes for these outcomes.

Some prior research has suggested that many suicide attempts are not being detected in administrative data (Walkup et al., 2012). Attempts that are never treated medically will not be detected using these data. However, even when treated in the emergency department or an inpatient unit, a substantial portion of suicide attempts might be missed. This can be due to several factors including physicians listing underlying mental illnesses as the diagnosis and not properly charting the occurrence of suicidal behavior, and gaps in administrative data (such as limited information on emergency department triage presenting complaint). Using different combinations of ICD codes can improve the sensitivity of detecting suicide attempts but may cause a high rate of false positives. Properly understanding the accuracy of identifying suicide attempts with these codes is crucial to future work. Currently there is no evidence‐based consensus on the best way to code for suicide attempts and deliberate self‐harm using ICD‐10 diagnostic codes. Research on the relationship among ICD‐10 codes for external cause of injury, suicide attempts and deliberate self‐harm is needed to guide researchers and understand the strengths and limitations of studies using these outcomes.

This paper examined the validity of using ICD‐10 diagnostic codes in hospital discharge records to identify suicide attempts and deliberate self‐harm presentations admitted to inpatient units. It used clinician assessment of the occurrence of self‐injury as a gold standard to determine whether an admission was a suicide attempt and/or deliberate self‐harm (including both suicidal and non‐suicidal self‐injury). Diagnostic codes of interest included those for intentional self‐harm, self‐harm of undetermined intent, and accidental poisonings derived from hospital discharge data. These analyses were focused on admitted patients due to ICD‐10 codes not being recorded for patients discharged from the emergency department. This study also examined the triage complaints from the emergency department records and their occurrence among those presenting with suicide attempts, self‐harm, or with no self‐harm. We hypothesized that the intentional self‐harm ICD codes would have high specificity but modest sensitivity, and that including ICD codes for self‐harm of undetermined intent and accidental poisoning codes in the definition would improve the sensitivity slightly while reducing the specificity. We also hypothesized that triage complaints would not be able to accurately identify those with suicide attempts and self‐harm.

2. METHODS

2.1. Sample

The study sample was obtained from emergency departments of the two tertiary care hospitals in the city of Winnipeg, Manitoba, Canada (population 650,000). Due to the single‐payer health care system in the province, these hospitals are accessible to all residents of the province and are not limited to individuals based on insurance coverage or other factors. Those hospitals have fulltime psychiatrists and psychiatry residents. As part of their assessment, the psychiatric staff and residents servicing these emergency departments are required to fill out the Suicide Assessment Form in Emergency (SAFE). Data from all psychiatric consultations was collected between January 1, 2009 and December 31, 2012. These consultations consisted of all consecutive adult patients referred for psychiatry consultation after presentation to these emergency departments. The first visit for each individual in the specified time period was selected to form the analysis sample.

2.2. Clinical assessment for suicide attempt and self‐harm

SAFE contains several risk factor assessments as well as information on date of presentation and the Columbia Classification Algorithm of Suicide Assessment (C‐CASA). The C‐CASA is a standardized tool that classifies suicidal and self‐harm behaviors, ideation and planning, sorting individuals into mutually exclusive categories (Posner et al., 2007). The C‐CASA served as the gold standard in this study. The following C‐CASA categories were used: suicide attempts, self‐injurious behavior (no suicidal intent), self‐injurious behavior (intent unknown), suicidal ideation, and preparatory acts toward imminent suicidal behavior. Self‐harm presentations included any presentation assigned to the following C‐CASA categories: suicide attempts, self‐injurious behavior (no suicidal intent), self‐injurious behaviour (intent unknown). Classification is based on the current presentation, and past self‐harming thoughts and behaviors are not included in this assessment.

2.3. Administrative data linkage

Records from the SAFE data set were linked to administrative records housed at the Manitoba Center for Health Policy (the Repository) using patients' Provincial Health Insurance Numbers collected during the assessment. The Repository contains information on medical services provided by the province of Manitoba's single‐payer health care system. The Repository contains individual‐level data on over 99% of the province's population. Two databases from the Repository were used: the Emergency Department Information System (EDIS), and the Hospital Discharge Abstract Database. EDIS is used in both of the hospitals to track and record visits by individuals to the emergency department. EDIS captures the triage presenting complaint; relevant categories of complaints include “Mental health”, “Substance use”, and “Trauma”. More specific categorizations within these categories include “Depression, suicidal, self‐harm”, “Overdose”, and “Laceration”. Moreover, supplementary information can be used to identify the following: “Attempted suicide, clear plan”, “Active thoughts”, “High risk/unknown substance”, and “Altered level of consciousness”. Only the chief complaint is recorded in EDIS, therefore patients presenting with issues other than suicide attempts might have those issues listed as the chief complaint.

The Hospital Discharge Abstract Database identified individuals admitted to inpatient units following emergency department presentation and was the source of diagnosis codes for validation. This database contains records for all admissions in the province and captures up to 25 diagnosis codes for each inpatient admission. The diagnoses are coded using International Statistical Classification of Diseases and Related Health Problems version 10 Canadian Enhancement (ICD‐10‐CA; Canadian Institute for Health Information [CIHI], 2001). Coding was performed by trained medical coders, coding of disorders is thorough and comprehensive based on the documentation of clinicians. The following codes were used to determine suicide attempts: Intentional self‐harm (ICD‐10‐CA codes X60–X84), self‐harm of undetermined intent (Y10–Y34), and accidental poisoning (ICD‐10‐CA codes X40–X49). These codes were selected because of their use in previous validation and epidemiology research (Fransoo et al., 2009; Walkup et al., 2012). All 25 diagnoses were scanned to detect the occurrence of these codes. These codes are not available for patients discharged from the emergency department.

No variable recorded in the SAFE dataset can link individual SAFE records to specific emergency department and inpatient visit records held in the Repository; only date and personal health insurance numbers are available. Therefore, the administrative records were grouped together based on dates. Emergency department visits were identified and records on the same date or subsequent dates were combined into one record, or event. A space of two days without any administrative record was used as an indication that an event had ended. Hospital records provide both admission and discharge dates for this purpose. All analysis is based on these grouped records, which will be referred to as “events” to differentiate them from “emergency department presentations”. Since multiple emergency department visits are possible within the same event, a specific event may have multiple triage codes. Between‐facility transfers may lead to one event being linked to multiple hospital records. All hospital discharge records were scanned for the diagnostic codes of interest.

2.4. Statistical analysis

Frequencies were calculated for individuals based on their EDIS complaint categorization and whether they were coded as a suicide attempt or undetermined intent self‐harm in hospital records. The distribution of these classifications was assessed for those with a suicide attempt and self‐harm. The agreement of the ICD‐10‐CA codes from administrative records and suicide attempts/self‐harm determined by C‐CASA was assessed using the kappa (κ) statistic (Viera & Garrett, 2005). Kappa was interpreted using the following groups: κ < 0, less than chance; 0.01–0.20, slight agreement; 0.21–0.4, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; > 0.81, almost perfect agreement (Landis & Koch, 1977; Viera & Garrett, 2005). These heuristics should be used with caution since one number statistics provide a limited assessment of validity and usefulness and the kappa statistic weighs sensitivity and specificity equally. Three definitions of ICD codes were used: X60–X84; X60–X84 and Y10–Y34; X60–X84, Y10–Y34, and X40–49. Frequencies comparing C‐CASA with these codes were derived twice – once for all of the emergency department events and again with the sample restricted to events with inpatient admissions. Further validity statistics are determined for admitted patients since ICD‐10 codes are only available for admitted patients. Predictive statistics and their 95% confidence intervals (CIs) were derived, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Frequencies and standardized differences for the five most relevant EDIS complaints among admitted patients were compared between the C‐CASA determined self‐harm/attempt patients and those detected by the ICD coding algorithms to assess for potential bias from using ICD codes.

Ethics approval was obtained from the University of Manitoba's Health Research Ethics Board (HREB#: H2015:009) and data access was granted by the Government of Manitoba's Health Information Privacy Commission (HIPC# 2014/2015–41).

3. RESULTS

The sample consisted of 10150 assessment records in the SAFE dataset, of which 9319 were linkable to the Repository. A total of 6025 index visits were identified, 5719 of these had C‐CASA assessment information in the SAFE data and comprised the study sample (Table 1). Among the linked presentations, 3013 were male (52.7%) with a mean age of 41.4 years (median = 39; standard deviation [SD] = 17.4). According to the C‐CASA classification 780 presentations were classified as suicide attempts; 1147 presentations were due to self‐harm (20.1%; including those previously assessed as attempts). Visits categorized as having imminent plans consisted of an additional 190 (3.3%), while suicidal ideation was the classification for 1495 (26.1%).

Table 1.

Descriptive statistics of index emergency department events captured in the SAFE datasets

	N	Percent
Sample	5719	100
Sex(male)	3013	52.7
Admitted to hospital	3328	58.2
C‐CASA classification
Suicide attempt	780	13.6
Self‐harma	1147	20.1
Imminent plans	190	3.3
Suicidal ideation	1495	26.1

	Mean	SD
Age	41.4	17.4

^aIncludes suicide attempts.

Table 2 contains the frequencies of individuals classified into the four C‐CASA categories according to the inpatient diagnosis records (X60–X84). The estimate of kappa for suicide attempt categorization for all emergency department visits (both admitted and non‐admitted visits) was κ = 0.295 (95% CI = 0.252–0.339), which is considered fair agreement (Viera & Garrett, 2005). Agreement between self‐harm and the hospital records was κ = 0.229 (95% CI = 0.187–0.270, fair agreement). Restricting the sample to the admitted visits improved κ to 0.465 (95% CI = 0.414–0.516) and 0.393 (95% CI = 0.347–0.443) respectively; these figures translate into moderate and fair agreement. Using the broader suicide attempt definition including unknown intent codes led to better agreement in all cases, but particularly improved the accuracy of identifying self‐harm. The value of κ after including unknown intent codes for predicting self‐harm admissions was 0.434 (95% CI = 0.388–0.479). Suicide attempt admissions had κ = 0.482 (95% CI = 0.433–0.531). Further adding the accidental poisoning codes had little impact on the kappa for suicide attempts (κ = 0.481, 95% CI = 0.434–0.528) but improved κ to 0.478 (95% CI = 0.435–0.521) for self‐harm. Frequencies for the two expanded definitions are displayed in Table 3.

Table 2.

Administrative data diagnostic codes compared to C‐CASA classification codes

		ICD codes for Intentional self‐harm (X60–X84)
		All emergency visits		Admitted to hospital
C‐CASA classification		Yes	No	Yes	No
Suicide attempt	Yes	164	616	164	280
	No	36	4900	36	2848
Self‐harm	Yes	185	962	185	437
	No	15	4554	15	2691

Table 3.

Administrative data diagnostic codes with expanded definitions compared with C‐CASA classification

	Emergency visits		Admitted to hospital
C‐CASA classification	X60–X84, Y10–Y34
	Yes	No	Yes	No
Suicide attempt	Yes	182	598	182	262
	No	60	4876	60	2824
Self‐harm	Yes	213	934	213	409
	No	29	4540	29	2677
X60–X84, Y10–Y34, X40–X49
Suicide attempt	Yes	199	581	199	245
	No	99	4837	99	2785
Self‐harm	Yes	249	898	249	373
	No	49	4520	49	2657

Note: X60–X84 are the ICD codes for intentional self‐harm; Y10–Y34 are the ICD codes for undetermined intent self‐harm; X40–X49 are the ICD‐10 codes for accidental poisoning.

Using the C‐CASA assessment as the gold standard, the predictive ability of the administrative data coding for the admitted patients is shown in Table 4. The sensitivity of the diagnostic codes was poor. The X60–X84 codes identified only 36.9% (95% CI = 32.4–41.4%) of those admitted with an attempt, that is 63.1% of admitted suicide attempts were not detected using the intentional self‐harm codes. Adding the Y10–Y34 codes marginally improved the sensitivity to 41.0% (95% CI = 36.4–45.6%). The sensitivity for capturing self‐harm was lower at 29.7% (95% CI = 26.2–33.3%), with sensitivity increasing to 34.2% (95% CI = 30.5–38.0%) with the Y10–Y34 codes. The PPV and specificity were fairly good, however. These statistics were slightly higher for the more general self‐harm group than for the suicide attempt group. Adding accidental poisoning increased the sensitivity for self‐harm admissions to 40.0% (95% CI = 36.2–43.9%). The poisoning codes only slightly increased the sensitivity for suicide attempt admissions (43.6%, 95% CI = 40.2–49.4%) while decreasing the PPV to 66.8% (95% CI = 61.4–72.1%). In all instances the majority of people that were determined to have made a suicide attempt or engaged in self‐harm by the clinician were not detected using all of these codes. Since non‐admitted patients do not receive ICD‐10‐CA codes they will be missed entirely. Sensitivity for all events was 21% (95% CI = 18.2–23.9%) for the X60–X84 codes, increasing to 23.3% and 25.5% when the Y10–Y34 codes and X40–X49 were added. Without further data on emergency department treatment, a solid majority of emergency department events were not detected.

Table 4.

Validation statistics for admitted patients using different coding methods

	X60–X84		X60–X84, Y10–Y34		X60–X84, Y10–Y34, X40–X49
Statistic	Estimate (95% CI)		Estimate (95% CI)		Estimate (95% CI)
Suicide attempt
Sensitivity	36.9%	(32.4–41.4%)	41.0%	(36.4–45.6%)	44.8%	(40.2–49.4%)
Specificity	98.8%	(98.3–99.2%)	97.9%	(97.4–98.4%)	96.6%	(95.9–97.2%)
PPV	82.0%	(76.7–87.3%)	75.2%	(69.8–80.6%)	66.8%	(61.4–72.1%)
NPV	91.0%	(90.0–92.0%)	91.5%	(90.5–92.5%)	91.9%	(90.9–92.9%)
Self‐harm
Sensitivity	29.7%	(26.2–33.3%)	34.2%	(30.5–38.0%)	40.0%	(36.2–43.9%)
Specificity	99.4%	(99.2–99.7%)	98.9%	(98.5–99.3%)	98.2%	(97.7–98.7%)
PPV	92.5%	(88.8–96.2%)	88.0%	(83.9–92.1%)	83.6%	(79.3–87.8%)
NPV	86.0%	(84.8–87.2%)	86.7%	(85.6–87.9%)	87.7%	(86.5–88.9%)

Note: X60–X84 are the ICD codes for intentional self‐harm; Y10–Y34 are the ICD codes for undetermined intent self‐harm; X40–X49 are the ICD‐10 codes for accidental poisoning.

Table 5 presents the occurrence of the various EDIS triage complaint categorization by suicide attempt and self‐harm classification. Mental health was the most common primary classification with 48.7% of the self‐harm presentations in this group compared to 73.7% of the people with no self‐harm. Of these presentations, 83.5% were classified under “Depression, suicidal, self‐harm”. Approximately a third of the self‐harmers were categorized under the “substance use” and “overdose” presentation categories. There were 195 (25.0%) suicide attempts and 233 (20.3%) self‐harmers with the “Attempted suicide, clear plan” classification, but 162 (3.5%) non‐self‐harmers were also given this classification. Table 6 shows the occurrence of the five most relevant EDIS complaints among admitted patients and how common they were between individuals classified as self‐harm or attempt patients by C‐CASA, and the three ICD coding methods. A downward bias was apparent for three of the complaints; “Mental health”, “Depression/suicidal/self‐harm”, and “Attempt suicide, clear plan”. “Mental health” and “Depression/suicidal/self‐harm” was approximately half as common among the group detected through ICD codes than expected based on the frequency in the C‐CASA attempt group (ratios between 0.6 and 0.49 relative to C‐CASA attempt group frequency). These are equal to standardized differences of between 0.37 and 0.49. The bias for these three complaints was least when using only X60–X84 codes. Overdose triage complaints were overrepresented in those detected by ICD codes by 1.09 to 1.23 times the expected frequency compared to the CASA attempt group. In this instance the “X60–X84, Y10–Y34” coding was the least biased. The standardized differences for overdose are between 0.06 and 0.15. The “Attempted suicide, clear plan” was 0.89 to 0.7 times less common among those identified with the ICD codes compared to the C‐CASA attempt group. The standardized differences between these groups were between 0.06 and 0.18.

Table 5.

Emergency Department Information System (EDIS) triage category by C‐CASA classification

	Suicide attempt		Self‐harmb		No self‐harmb
	N = 780		N = 1147		N = 4569
	N	column %	N	column %	N	column %
Primary triage category
Mental health	344	44.1	559	48.7	3368	73.7
Substance misuse	276	35.4	348	30.3	104	2.3
Trauma	44	5.6	61	5.3	68	1.5
Multiplea	13	1.7	15	1.3	19	0.4
Secondary triage category
Depression/suicidal/self‐harm	326	41.8	467	40.7	1596	34.9
Overdose	269	34.5	335	29.2	46	1.0
Laceration	15	1.9	32	2.8	9	0.2
Multiplea	9	1.2	10	0.9	8	0.2
Tertiary triage category
Attempted suicide, clear plan	195	25.0	233	20.3	162	3.5
Active thoughts	56	7.2	79	6.9	271	5.9
High risk/unknown substance	78	10.0	104	9.1	17	0.4
Altered level‐of‐consciousness	29	3.7	42	3.7	62	1.4
Multiplea	c	c	c	C	c	c

^aVisit contains more than one of the above codes.

^bSelf‐harm includes all C‐CASA classifications involving self‐injury.

Note: c, censored due to small number of events.

Table 6.

Assessment of bias in triage chief complaint among admitted patients using ICD‐derived attempt and self‐harm groups

	Suicide attempta		Self‐harma		X60–X84		X60–X84, Y10–Y34		X60–X84, Y10–Y34, X40–X49
	N = 444		N = 622		N = 200		N = 242		N = 298
	N	column %	N	column %	N	column %	N	column %	N	column %
Primary triage category
Mental health	205	46.2	311	50.0	55	27.5	63	26.0	71	23.8
Secondary triage category
Depression/suicidal/self‐harm	193	43.5	248	39.9	52	26.0	58	24.0	63	21.1
Overdose	141	31.8	159	25.6	76	38.0	84	34.7	116	38.9
Tertiary triage category
Attempted suicide, clear plan	105	23.6	123	19.8	42	21.0	45	18.6	49	16.4

^aC‐CASA classification of suicide attempt or any self‐harm.

4. DISCUSSION

Our findings suggest that the use of ICD‐10 codes in administrative data for detecting hospital‐treated suicide attempts and self‐harm is an imperfect method of detecting these outcomes. The main issue is the poor sensitivity of the codes. Even among those admitted with a suicide attempt, less than half of the sample was identified from the intentional self‐harm codes. This finding has substantial impact for population‐based administrative data research relying on suicide attempts as an outcome measure. The PPV of the codes is better, but still not perfect. Adding the undetermined intent codes improved the sensitivity, although at the expense of a reduced PPV. When the codes are used to detect suicide attempts, approximately half of the false positives are due to other instances of self‐harm. This study suggests that at least 75% of suicide attempts presenting to the emergency department will be missed if ICD‐10 codes from hospital discharge records are used. Even the majority of individuals admitted to the hospital are missed if ICD‐10 codes are the only data used to detect attempts.

A recent systematic review of validation studies on using ICD codes to detect suicide attempts treated at hospitals determined that research in this area is lacking (Walkup et al., 2012). No studies examining ICD‐10 coding were located, as all previous validation studies were based on ICD‐9. A current literature search did not locate any additional relevant studies. Using presenting complaints in systems like EDIS may prove helpful in improving the accuracy of detecting self‐harm using ICD codes. However, the EDIS third level category for “Attempt suicide, clear plan” appears to capture very few of those with self‐harm and a large number of non‐self‐harming individuals. This field is also optional and is often left blank. Conversely, two of the secondary categories (Overdose and Depression/suicidal/self‐harm) both managed to identify a large number of those with self‐harm. The overdose category also had a good PPV in the sample, though this is likely to decrease in a sample containing all emergency department visits. Using overdose presentation in conjunction with mental health ICD codes or with psychiatric treatment/assessment tariffs may prove effective in detecting a sizeable portion of the self‐harm presentations. A limitation of triage systems similar to EDIS is the reliance of a single presenting complaint. This means that other comorbid conditions may be triaged as the main complaint instead of self‐harm, and this will occur more often among those with less medically serious self‐harm. This may explain why some individuals do not appear to have relevant triage complaints.

The PPV estimates found in this study are similar to the previous results for intentional self‐harm derived from ICD‐9 codes E950–E959 (Iribarren et al., 2000; Simon & Savarino, 2007). The PPV of the codes were acceptable, especially for self‐harm without specifying intent. About half of the false positives that occur when deriving suicide attempt outcomes appear to be due to other forms of self‐harm. This will hinder research aiming to use suicide attempts, specifically, as the outcome. However, this should not be a serious impediment, since individuals that attempt suicide and individuals that self‐injure share similarities across many risk factors and non‐suicidal self‐injurers are the minority of those detected by the codes (Nock et al., 2006). These data also suggest that these codes are specific enough to determine that people missed will be largely true negatives. It is likely that the NPV from a sample of all hospital admissions would be higher than the estimates here.

Few studies have examined the sensitivity of ICD codes. The work validating suicide attempt coding in the systematic review did not determine the sensitivity of the codes at detecting hospital‐treated suicide attempts, only the sensitivity of the codes at detecting suicide deaths (Walkup et al., 2012). Depending on the importance of PPV or sensitivity in detection of suicide attempts, researchers can opt to restrict to X60–X84 codes or use the expanded definition including Y10–Y34 and X40–X49. For deriving self‐harm, the use of the expanded definition appears to be superior.

The ability to separate non‐suicidal self‐harm from suicidal self‐harm is a potential challenge when using administrative data. Some researchers have opted to label events derived from these data as being self‐harm events whereas others have persisted in describing these events as suicidal. This study suggests that using expanded criteria (including Y10–Y34) in order to detect self‐harm outcomes is likely the most accurate method of coding. The accuracy of detecting suicide attempts specifically is similar when using this expanded coding as well. Therefore, using these codes to detect suicide attempts is feasible. However, restricting the definition to intentional codes (X60–X84) would reduce the occurrence of false positives and is advisable when PPV is more valued than sensitivity. This is potentially the case when conducting risk factor or causal studies and this is supported by the smaller bias in EDIS triage complaints found when using only the X60–X84 codes. Including the unintentional codes would be superior when sensitivity is the primary concern. Specifically, this would be advisable when estimating incidences of suicide attempts. Expanding the codes used in detection would produce numbers more closely approximating the actual incidence of self‐harm and suicide attempts. Including accidental poisoning improved the sensitivity and kappa agreement, but in practice the opposite might occur as non‐mental health related accidental poisonings will be identified. Adding an additional requirement of at least one mental health diagnostic code or treatment code before including accidental poisonings would potentially address this issue.

These results have implications for the use of administrative data to track the prevalence of suicide attempts and as the data source for suicide outcomes for causal studies. The low sensitivity indicates that there is likely a significant undercounting of the number of admitted patients estimated using these codes. The implication is less clear for causal studies. Missing cases are likely to bias estimates towards the null if the missing cases are similar to the identified individuals. However, if the identified individuals are not similar to the missed cases then other biases could also occur. It is possible that more severe cases would be detected disproportionately and this could cause risk factors to be differentially biased away from or towards the null, depending on whether they are associated more often with serious or less serious attempts. Overdoses presentations appear to be overrepresented according to the bias assessment potentially due to these reasons.

These results should be generalizable to other similar acute care settings. Institutions with comprehensive record keeping and trained medical coders deriving ICD codes should produce similar results. Areas with standardized psycho‐social assessment of suicidality included in the chart should have increased validity using these codes. Hospitals without psychiatric assessment available 24 hours a day (e.g. rural hospitals) might perform poorer due to less thorough psycho‐social assessment and documentation. Areas without comprehensive coding of charts will likely also perform poorly and poor assessment and coding will likely result in more false negatives particularly.

The main reason for errors in coding is probably poor documentation in the charts. Individuals miscoded with Y10–Y34 codes are a good example as these are likely individuals with injuries and overdoses listed in the chart, but no clear documentation of intention. Medically minor self‐harm might be treated in the emergency department, or not require any treatment at all, while the inpatient records might fail to mention self‐harm due to a focus on what the treating physician determined to be the main issue (e.g. depression). The coding of charts is performed by trained medical coders who are taught to code diagnoses comprehensively; they should correctly code self‐harm and attempts that are clearly documented in the charts.

4.1. Strengths and limitations

This study used consecutive sampling of patients receiving psychiatric consultations in the emergency department. The C‐CASA assessment of treating psychiatrists/psychiatric residents was used to determine whether specific visits were attempts, any form of self‐harm, or not. Previous validation efforts have normally used chart reviews to determine whether visits were suicidal in nature. However, such reviews allow for a considerable degree of interpretation and assume that charting contains all of the relevant information. Classification of the presentations using the C‐CASA completed by the clinician who performed the psychiatric assessment is likely to be more accurate than chart reviews. This work also used a sample considerably larger than previous studies.

A study limitation was the inability to assess the accuracy of determining suicide attempts using emergency department ICD codes; external cause of injury codes specifying intent are not recorded for visits to emergency departments in Manitoba. Another limitation is that approximately 13% of the sample either was not linkable to specific emergency department visits in the administrative records or was missing the C‐CASA. This was mostly due to missing dates or missing health insurance numbers in the SAFE data. While this information appears to be missing randomly, the possibility of bias cannot be entirely dismissed. However, due to the relatively small percent missing, the bias is unlikely to be large enough to substantially affect the statistics obtained. Another limitation is this study's restriction to individuals assessed by the psychiatry consult service. Without assessment by psychiatry there is no C‐CASA to use as a gold standard. Individuals not consulted to psychiatry may have engaged in less serious self‐harming behavior or to have less severe comorbid conditions at presentation. The psychiatry consult service also only assesses adult patients. Therefore, this study was limited to assessing the accuracy of these codes for those 18 years of age and older. Another limitation is that these results reflect the accuracy of coding done an inpatient setting and may not be generalizable to other psychiatric or outpatient settings. Other settings are likely to have less thorough coding and documentation and the performance of these codes could potentially be worse. This is likely to be reflected with even lower sensitivity for suicide attempts or self‐harm, and the possibility of false positives exists but is likely less common. Generalization of these results outside of inpatient settings is not recommended.

5. CONCLUSION

This study assessed the accuracy of ICD‐10 diagnosis codes at detecting patients admitted due to a suicide attempt or self‐harm from among all patients consulted to psychiatry from the emergency department and non‐psychiatric units. Relevant individuals were likely to be accurately identified by the codes, but the sensitivity of the codes is low. These findings suggest that research using these codes to identify suicidal behavior outcomes is likely missing in the vicinity of one half to two thirds of their outcomes. This is likely to cause significant undercounting of prevalence and incidence and may bias the results of causal studies. Future research should continue in this area. Examining the potential usefulness of non‐ICD‐based data sources, such as EDIS, to derive detection algorithms is a potential area that should be examined.

DECLARATION OF INTEREST STATEMENT

The authors have no conflicts of interest to declare.

Randall JR, Roos LL, Lix LM, Katz LY, Bolton JM. Emergency department and inpatient coding for self‐harm and suicide attempts: validation using clinician assessment data. Int J Methods Psychiatr Res. 2017;26:e1559 10.1002/mpr.1559