Current Assessment and Classification of Suicidal Phenomena using the FDA 2012 Draft Guidance Document on Suicide Assessment: A Critical Review

Abstract

Objective: Standard international classification criteria require that classification categories be comprehensive to avoid type II error. Categories should be mutually exclusive and definitions should be clear and unambiguous (to avoid type I and type II errors). In addition, the classification system should be robust enough to last over time and provide comparability between data collections. This article was designed to evaluate the extent to which the classification system contained in the United States Food and Drug Administration 2012 Draft Guidance for the prospective assessment and classification of suicidal ideation and behavior in clinical trials meets these criteria.

Method: A critical review is used to assess the extent to which the proposed categories contained in the Food and Drug Administration 2012 Draft Guidance are comprehensive, unambiguous, and robust. Assumptions that underlie the classification system are also explored.

Results: The Food and Drug Administration classification system contained in the 2012 Draft Guidance does not capture the full range of suicidal ideation and behavior (type II error). Definitions, moreover, are frequently ambiguous (susceptible to multiple interpretations), and the potential for misclassification (type I and type II errors) is compounded by frequent mismatches in category titles and definitions. These issues have the potential to compromise data comparability within clinical trial sites, across sites, and over time.

Conclusion: These problems need to be remedied because of the potential for flawed data output and consequent threats to public health, to research on the safety of medications, and to the search for effective medication treatments for suicidality.

INTRODUCTION

Guided by findings from meta-analyses that some drugs might be implicated in suicidal behavior, the United States Food and Drug Administration (FDA) began mandating the collection of retrospective narrative data on suicidal ideation and behavior in clinical trials falling under its authority as early as 2003. Subsequently, it began classifying suicide events using the Columbia Classification Algorithm for Suicide Assessment (C-CASA), a nine-category classification system developed at Columbia University.¹

Based on these data and later data from prospective trials using the Columbia–Suicide Severity Rating Scale (C–SSRS),² the FDA issued draft guidance on assessing suicidal ideation and behavior in drug trials in 2010.³ This draft guidance was revised and updated in August of 2012 when the FDA released new official draft guidance⁴ recommending the prospective capture of suicide information using 11 categories that appear to be based on the C–SSRS. The FDA Draft Guidance was released during a time of growing concern about suicide and suicide behavior globally and nationally.^5,6 With broad interest in detecting and preventing suicidality, the FDA’s 2012 official endorsement of the C–SSRS helped cement the C–SSRS’s use in a widening range of settings stretching from the pharmaceutical industry (for clinical trials) to the United States Centers for Disease Control and Prevention (CDC) to schools and correction systems in some states and even the United States Military.^7,8 As the C–SSRS diffuses into new settings and the FDA’s 2012 classification system becomes increasingly set, as it were, in stone, it may be useful to step back and look at the extent to which the 11 categories the FDA has apparently adopted from the C-SSRS accurately capture the spectrum of suicidal ideation and behavior.

Ideally, national data collection and classification efforts targeting suicide, whether for drug trials or other epidemiological uses, need to be able to consistently and reliably identify individuals who are suicidal and those who are not. In addition, to be useful beyond the borders of any one nation (an important concern for pharmaceutical companies who operate globally), data collection and classification efforts should meet international standards. Among other “essential components,” according to recent United Nations guidance to national statistical agencies, data collection and classification efforts should contain “categories that are mutually exclusive and exhaustive,” and “definitions that are clear and unambiguous, and which define the content of each category.” Data collection and classification efforts should also be “robust enough to last for a period of time,” “meet user needs,” and “provide comparability over time and between collections.”⁹ Finally, as with any technical or scientific document, the assumptions that guide the text should be as explicit as possible.¹⁰

This paper evaluates the FDA’s 2012 classification system for suicidal ideation and behavior (we will call it the FDA-CASA 2012) in the context of these standards.

METHODS

A critical review was used to evaluate the suicidality classification system (FDA-CASA 2012) adopted by the FDA in its 2012 Draft Guidance. In line with internationally accepted criteria for classification, we focused specifically on whether the FDA-CASA 2012 is comprehensive (covering all potential categories of suicidal ideation and behavior that could be seen in practice) and on whether the definitions that guide specific categories are unambiguous, mutually exclusive, and robust. We also examined the underlying assumptions that appear to guide the FDA 2012 Draft Guidance. These assumptions are important to uncover and highlight since, as Professor Agger of the University of Texas points out in relation to technical and scientific writing in general, the assumptions that guide a scientific text are all too often buried under heavy technical prose. As a result they can be “hidden from the community of science and thus protected from external challenges.” Uncovering these assumptions and putting them out for debate can help identify blind spots, omissions, tensions, and contradictions, or what Agger calls the “internal fissures and fault lines” in a text.¹⁰

We divided this article into three parts. In the first part, we examine general issues and problems relating to the FDA-CASA 2012 classification system. In the second part, we focus on specific categories highlighting individual issues within these categories. We conclude with a discussion of the implications of the review.

GENERAL ISSUES AND PROBLEMS

Missing combinations of suicidal ideation (type II error). An important component of any classification system is that the categories be complete (i.e., no categories be omitted). The FDA-CASA 2012, since it is now more closely aligned to the C-SSRS, divides active suicidal ideation into four categories: 1) “Active suicidal ideation: nonspecific (no method, intent, or plan),” 2) “Active suicidal ideation: method, but no intent or plan,” 3) “Active suicidal ideation: method and intent, but no plan,” and 4) “Active suicidal ideation: method, intent, and plan.”⁴ The four acknowledged categories are anchored in the presence or absence of a method, intent, or a plan. These four categories, however, do not represent all of the possible combinations of method, intent, or plan. As shown in Table 1, there are a total of 16 possible combinations of the active ideation domains of method, intent, and plan (2 response options to the power of 4 domains=16) and a total of 32 possible combinations when passive ideation is also used as a factor (2 response options to the power of 4 domains=16 multiplied by 2 types of ideation: active and passive). The FDA framework, however, is limited to five of these 16 combinations for active ideation (i.e., the 4 identified combinations plus the null of the 4 [combination number 1]) and six of the 32 combinations for active and passive ideation combined. Table 1 illustrates the combinations captured and not captured based on category titles, while Table 2 illustrates the combinations captured and not captured based on category definitions.

TABLE 1.

Possible combinations of suicidal ideation and missing combinations in FDA-CASA 2012 titles

Possible Combination Number	Passive Ideation	Active Ideation	Method	Intent	Plan	Which FDA-CASA 2012 Category Title (1–5) Captures This Combination?		Combinations Not Captured by FDA-CASA 2012 Titles
1	0	0	0	0	0	0	Covered by default
2	0	1	0	0	0	2	“Active suicidal ideation: nonspecific (no method, intent, or plan)”
3	0	0	0	1	0			✓
4	0	1	0	1	0			✓
5	0	0	1	0	0			✓
6	0	1	1	0	0	3	“Active suicidal ideation: method, but no intent or plan”
7	0	0	1	1	0			✓
8	0	1	1	1	0	4	“Active suicidal ideation: method and intent, but no plan”
9	0	0	0	0	1			✓
10	0	1	0	0	1			✓
11	0	0	0	1	1			✓
12	0	1	0	1	1			✓
13	0	0	1	0	1			✓
14	0	1	1	0	1			✓
15	0	0	1	1	1			✓
16	0	1	1	1	1	5	“Active suicidal ideation: method, intent, and plan”
17	1	0	0	0	0	1	“Passive suicidal ideation: wish to be dead”
18	1	1	0	0	0			✓
19	1	0	0	1	0			✓
20	1	1	0	1	0			✓
21	1	0	1	0	0			✓
22	1	1	1	0	0			✓
23	1	0	1	1	0			✓
24	1	1	1	1	0			✓
25	1	0	0	0	1			✓
26	1	1	0	0	1			✓
27	1	0	0	1	1			✓
28	1	1	0	1	1			✓
29	1	0	1	0	1			✓
30	1	1	1	0	1			✓
31	1	0	1	1	1			✓
32	1	1	1	1	1			✓
Totals	32					6		26

FDA-CASA 2012: United States Food and Drug Administration-Classification Algorithm for Suicide Assessment

TABLE 2.

Possible combinations of suicidal ideation and missing combinations in FDA-CASA 2012 definitions

Possible Combination Number	Passive Ideation	Active Ideation	Method	Intent	Plan	Which FDA-CASA 2012 Category Definition (1–5) Captures This Combination?	Combinations Not Captured by FDA-CASA 2012 Definitions
1	0	0	0	0	0	0
2	0	1	0	0	0	2
3	0	0	0	1	0		✓
4	0	1	0	1	0	4
5	0	0	1	0	0		✓
6	0	1	1	0	0	3
7	0	0	1	1	0		✓
8	0	1	1	1	0		✓
9	0	0	0	0	1		✓
10	0	1	0	0	1		✓
11	0	0	0	1	1		✓
12	0	1	0	1	1	5
13	0	0	1	0	1		✓
14	0	1	1	0	1		✓
15	0	0	1	1	1		✓
16	0	1	1	1	1		✓
17	1	0	0	0	0	1
18	1	1	0	0	0		✓
19	1	0	0	1	0		✓
20	1	1	0	1	0		✓
21	1	0	1	0	0		✓
22	1	1	1	0	0		✓
23	1	0	1	1	0		✓
24	1	1	1	1	0		✓
25	1	0	0	0	1		✓
26	1	1	0	0	1		✓
27	1	0	0	1	1		✓
28	1	1	0	1	1		✓
29	1	0	1	0	1		✓
30	1	1	1	0	1		3
31	1	0	1	1	1		✓
32	1	1	1	1	1		✓
Total	32					6	26

FDA-CASA 2012: United States Food and Drug Administration-Classification Algorithm for Suicide Assessment

Regrettably, reducing active ideation to four categories leaves no room for patients who have events that fall into other categories that are not specified. More importantly, limiting the number to four gives a false sense of safety, communicating that there is not a problem where there could well be one (type II error) and making it seem as if other categories do not exist or do not matter. Consider the patient who has active suicidal ideation with no method or intent but does have a plan or the patient with active suicidal ideation, intent, and a plan but no method. There are as many as 11 other potential combinations of active suicidal ideation. Unfortunately these assumptions can lead to a situation where a significant proportion of suicidal ideation is simply not captured. The following two examples illustrate these issues:

Example 1: Single case report. One suicidal person presented detailed records of suicidal ideation she kept over a nine-month period. Instead of aggregating suicidality phenomena into specific combinations over a specific timeframe (e.g., a week or since the last visit), as is now usually done in suicide assessment scales, the subject classified each event individually into its exact combination category.¹¹ The results (Figures 1 and 2) indicate that although the FDA-CASA 2012 categories captured almost 80 percent of the suicidal ideation events the subject experienced, the 20 percent of events not captured (Figure 1) constituted almost 60 percent of the total time she experienced suicidality (Figure 2). As shown in Figure 2, the FDA-CASA 2012 categories captured suicidal phenomena that accounted for 40.3 percent of the time spent (28.5% ideation combinations + 11.8% behaviors). This left a residual 59.7 percent of the time spent during which she experienced suicidal phenomena not captured by the FDA categories. To put this in a numeric perspective, the FDA-CASA 2012, applied as written, did not capture 4,256 of the 21,210 suicidality events this person experienced because these events fell outside the combinations allowed. More importantly, the FDA-CASA 2012 did not capture almost 17,785 minutes (296 hours, 25 minutes) of the 29,804 minutes of the suicidal ideation she suffered over the nine-month time frame.¹¹ This example, while it needs to be replicated, is a valuable one since it is the first time, to our knowledge, that every single suicidality event was individually classified into its exact combination category over such an extended period of time.

FIGURE 1.

Distribution of suicidality events from suicidal subject’s diary over 9 months

Note: Total number of days of data collection: 274; total number of suicidality events: 21,210. Overall, the FDA-CASA 2012 misses the combinations experienced in 4,256 events of suicidality (20.06% of all such events)

FDA: United States Food and Drug Administration; C-SSRS: Columbia-Suicide Severity Rating Scale

FIGURE 2.

Distribution of time spent in suicidality from suicidal subject’s diary over 9 months

Note: Total number of days of data collection: 274; total number of minutes with suicidality: 29,804. Overall, the FDA-CASA 2012 misses the combinations experienced over 17,785.4 minutes (296 hours, 25 minutes and 24 seconds) of suicidality (59.67% of minutes)

FDA: United States Food and Drug Administration; C-SSRS: Columbia-Suicide Severity Rating Scale

Example 2: Validation study dataset. The Sheehan-Suicidality Tracking Scale (S-STS)/InterSePT Scale for Suicidal Thinking-Plus (ISST-Plus)/C–SSRS validation study¹² provided a unique opportunity to further examine the extent of missing combinations in the FDA-CASA 2012. This study tested the agreement of two test instruments (S-STS and ISST-Plus) with the C–SSRS (treated as the gold standard). The three scales were administered in a random order sequence by independent, blinded raters. The results showed that the C–SSRS and the FDA-CASA 2012 categories did not capture combinations of suicidal ideation, method, intent, and plan that were detected in 67 percent of the subjects using the clinician-rated S-STS, 80 percent of the subjects using the patient-rated S-STS, and 76 percent of the subjects using the ISST-Plus. These results are undoubtedly a function of the fact that the S-STS and the ISST-Plus were both designed to capture all 32 combinations, as shown in Table 1, while C–SSRS and the FDA-CASA 2012 are confined to a much more limited number of combinations.

Problematic assumptions. Part of the problem seems to be that the FDA 2012 Draft Guidance appears to make several problematic assumptions. First, an assumption is made that passive ideation precludes method, intent, and plan. This is problematic since a suicidal subject can experience hallucinations, nightmares, and even daydreams in which method and plan are powerful features. Patients may even respond to command hallucinations with suicide attempts.

Second, an assumption is made that suicidal ideation generally reflects a staircase of escalating severity in which patients move up through the steps of suicidal ideation, abandoning passive ideation, for example, for active ideation and adopting more features (method, intent, plan) as the severity of their ideation increases. This is problematic for two reasons: 1) We know that passive and active types of ideation often coexist. For example, a daydream about shooting oneself can convert to passive ideation (where the subject wishes to be dead) and back again to active ideation (shooting oneself) in seconds; and 2) There is no evidence that patients always follow these escalating steps or that more features necessarily means more severity. In some cases, the patient with a violent method but no intent or plan may be at greater risk than one with method, intent, and plan.¹³

Third, an assumption is made that method, intent, and plan do not exist independent of ideation when in fact patients do report having a method, plan, or even intent quite independent of ideation, active or passive. Not considering this group is a potential safety issue.

Fourth, an assumption is made that the definition of plan includes intent and, as clarified in a footnote on page 4 of the FDA 2012 Draft Guidance, “thus, there is no need for the category method and plan, but no intent.”⁴ This assumption is problematic because patients sometimes make detailed plans years in advance. To dismiss the existence of a plan—since the subject has no current intent—and place the subject at the same level as someone who endorses active ideation without intent or plan poses a relative safety issue. The existence of a plan could well make this person much more vulnerable than someone without a plan.

Fifth, the assumption is made (presumably based on the C–SSRS) that reporting of ideation should be anchored only in events to the exclusion of time spent enduring such events. This is problematic since a patient could present in two different time frames with the same number of events in the same categories but show a dramatic increase in the amount of time spent in suicidal ideation or behavior. For example, one case study found evidence of as much as a 26-fold increase in time spent in suicidal ideation or behavior on two different days, each with the same number of suicidal events.¹⁴ Since the FDA 2012 Draft Guidance does not require that the amount of time be tracked, the clinician may miss the increase in time spent experiencing suicidality and, more importantly, miss the impact this change has in the life of the patient.¹⁴

Misalignment of category titles and category definitions. More often than not, the category titles in the FDA-CASA 2012 do not align with the definitions below them, and all too often definitions are ambiguous, which can create confusion. Misalignment of title and definition (Table 3) is a critical problem since some clinicians will only use the title to classify individual patients, others will only use the definition, a third group may try to use the title plus the definition, and a fourth group may cherry pick using unique combinations of the above.

TABLE 3.

Possible combinations of suicidal ideation and missing combinations in FDA-CASA 2012 titles and definitions

Possible Combination Number	Passive Ideation	Active Ideation	Method	Intent	Plan	Which FDA-CASA 2012 Category Title (1–5) Captures This Combination?	Which FDA-CASA 2012 Category Definition (1–5) Captures This Combination?	Combinations Not Captured by FDA-CASA 2012 Titles	Combinations Not Captured by FDA-CASA 2012 Definitions	Do the FDA-CASA 2012 Title and Definition Both Capture This Combination?
1	0	0	0	0	0	0	0			Yes
2	0	1	0	0	0	2	2			Yes
3	0	0	0	1	0			✓	✓	Overlooked
4	0	1	0	1	0		4	✓		No
5	0	0	1	0	0			✓	✓	Overlooked
6	0	1	1	0	0	3	3			Yes
7	0	0	1	1	0			✓	✓	Overlooked
8	0	1	1	1	0	4			✓	No
9	0	0	0	0	1			✓	✓	Overlooked
10	0	1	0	0	1			✓	✓	Overlooked
11	0	0	0	1	1			✓	✓	Overlooked
12	0	1	0	1	1		5	✓		No
13	0	0	1	0	1			✓	✓	Overlooked
14	0	1	1	0	1			✓	✓	Overlooked
15	0	0	1	1	1			✓	✓	Overlooked
16	0	1	1	1	1	5			✓	No
17	1	0	0	0	0	1	1			Yes
18	1	1	0	0	0			✓	✓	Overlooked
19	1	0	0	1	0			✓	✓	Overlooked
20	1	1	0	1	0			✓	✓	Overlooked
21	1	0	1	0	0			✓	✓	Overlooked
22	1	1	1	0	0			✓	✓	Overlooked
23	1	0	1	1	0			✓	✓	Overlooked
24	1	1	1	1	0			✓	✓	Overlooked
25	1	0	0	0	1			✓	✓	Overlooked
26	1	1	0	0	1			✓	✓	Overlooked
27	1	0	0	1	1			✓	✓	Overlooked
28	1	1	0	1	1			✓	✓	Overlooked
29	1	0	1	0	1			✓	✓	Overlooked
30	1	1	1	0	1			✓	✓	Overlooked
31	1	0	1	1	1			✓	✓	Overlooked
32	1	1	1	1	1			✓	✓	Overlooked
Total	32					6	6	26	26	Yes: 4/32; No: 4/32; Overlooked: 24/32

FDA-CASA 2012: United States Food and Drug Administration-Classification Algorithm for Suicide Assessment

SPECIFIC PROBLEMS RELATED TO FDA-CASA 2012 CATEGORIES

“Suicide attempt.” This category definition requires the suicide attempt to be “a potentially self-injurious behavior” (line 513).⁴ This is a limited definition of a suicide attempt and does not allow for a full range of suicide attempt behavior phenomena. For example, after watching the movie Snow White with his family, a young child is teased by his older brother. The older brother offers the child an apple that the older brother claims is poisoned like the one in the movie. Assuming it will make him sleep forever, the child eats the apple. Because the child thinks that eating the apple will kill him, just as it put Snow White into the “sleeping death,” this event should count as a suicide attempt. However the category definition for “Suicide attempt” in FDA-CASA 2012 does not allow this event to be coded as a suicide attempt and thereby misses what could be argued to be a clear suicide attempt. This is because the definition assumes that everyone making a suicide attempt has a clear understanding of what behaviors truly are potentially self-injurious. This same issue is present in the “Interrupted suicide attempt” and “Aborted suicide attempt” categories.

“Interrupted suicide attempt” and “Aborted suicide attempt.” In both cases, the definitions require the patient to have refrained from engaging in the potentially self-injurious or self destructive act. These definitions are problematic for two reasons. First, it is misleading to classify the defined behavior as a suicide attempt since the attempt by definition never started. Second, there is a safety issue. Starting a self-injurious action that is then interrupted or aborted is more worrisome than engaging in preparatory behavior. To the extent that such started actions are excluded, per FDA-CASA 2012 definition, true incidents of interrupted or aborted attempts will be missed. It is more appropriate to use language that accurately describes the phenomenon than to mislabel something as a suicide attempt, when it is not a suicide attempt but is a preparatory behavior.

A further problem with the FDA-CASA 2012 category “Interrupted suicide attempt” is contained in its definition stating that “if not for that [the interruption], [an] actual attempt would have occurred” (line 521).⁴ The problem here is that the clinician is required to know what the future would have been, if not for the interruption. It is quite possible that the patient would have aborted the attempt if the interruption had not occurred, but there is no way to know if the attempt would have been aborted or not because the interruption did occur. This wording presents serious challenges in reasonably coding any event as an “Interrupted suicide attempt” based upon the category definition.

“Preparatory acts toward imminent suicidal behaviors.” Again, this category has a title that does not match the definition below it. Specifically, the title contains the word “imminent,” but the definition does not require the preparatory actions to be connected to any imminent suicidal behavior.⁴ Moreover, the word “imminent” is not defined. Does it refer to a time span of a week, four weeks, four months, one year, or some other time frame? Without specification, clinicians may apply wildly different timeframes to determine if a preparatory behavior fits into the category or not. This can result in inconsistency. There is also the potential to miss important behaviors if the time frame applied is short. In fact, some patients make preparatory behaviors months or even years in advance of an attempt. These patients may not require additional preparations just prior to an attempt. In sum, using the qualifier “imminent” is ambiguous and may result in a clinician completely missing or dismissing preparatory behavior because the behavior did not occur within a subjective period of time (e.g., consider the patient who purchased a gun and bullets over a year ago and wrote suicide letters to loved ones months ago). The clinician may not ask about all of the steps the patient has taken in the past and by not doing so will have a much more difficult time helping to keep the patient safe.

The definition under this category is problematic in another way. In identifying relevant behavior, the definition (lines 533–534)⁴ refers to “assembling a specific method (e.g., buying pills, purchasing a gun).” Most of us would consider these examples as gathering “means” or tools, not assembling methods. The clinician reading this definition, however, may interpret the wording to include “planning the way” for a suicide, causing further complications in the interpretation of FDA-CASA 2012 by raters or clinicians.

“Self-Injurious Behavior Without Suicidal Intent.” According to the definition, this category is specifically intended “purely for other reasons, either to relieve distress […] or to effect change in others or the environment” (lines 539–541).⁴ Unfortunately, that excludes a wide range of self-injurious behaviors. Take the example of a car accident while the patient is driving or other types of accidents patients might have. In the blind-rater, case-history, narrative reviews for the FDA in its investigation on the effect of antidepressants on suicidality in adults, there were many cases of accidents in depressed patients that required adjudication in drug trials. The first author of this article (D.S.), who served as a blind rater in these adjudications recalls a typical narrative: This depressed study patient crashed into a tree and suffered severe injuries while skiing. Could this event have been a suicide attempt? Many examples of such accidents need careful investigation and coding and are too frequent to ignore. And what about the patient who engages in self-injurious behavior for reasons other than those specified? For example, the category definition here precludes such behavior for religious purposes, such as Tatbir, which is the ritual act of mourning performed by some Shi’a Muslims where they strike themselves on the head with a sword in order to cause blood flow in an act of remembrance. It also precludes classifying anyone engaging in self-injury strictly for the purposes of sadomasochism (S & M), such as a submissive being instructed by a dominant to self-flagellate or cut.¹⁵ We have seen such behaviors labeled as suicide attempts in emergency rooms, especially when the patient admitted to having some recent suicidal ideation, although the suicidal ideation was not related to the sadomasochistic acts that resulted in the self-injury needing medical treatment in the emergency room. If this definition for this category was broadened to cover self-injurious behaviors such as those described above, it may avoid their being incorrectly labeled as suicide attempts.

“Active suicidal ideation: nonspecific (no method, intent, or plan).” This FDA-CASA 2012 category requires, in part, active suicidal ideation “without general thoughts of ways to kill oneself/associated methods, intent, or plan during the assessment period” (lines 478–479).⁴ The clinician might interpret this to mean without general thoughts of ways to kill oneself, as stated by the definition, or might interpret it to mean without general thoughts but with specific thoughts of associated methods, intent, or plan. To avoid ambiguity, the definition might be better worded to read “without general thoughts of ways to kill oneself and without any associated methods, intent, or plan during the assessment period.” Otherwise the existing wording could be interpreted to allow inclusion of those with specific methods, intent, or plan while allowing exclusion of those with general ideation, general thoughts about method, general intent, and general plan, even though this was probably not the intent of the FDA wording. The wording and the unclear use of the “/” is a potential source of confusion because the title and the first part of the definition, “general nonspecific thoughts”⁴ clearly indicate that this category is for nonspecific active suicidal ideation, but the second part of the definition can be interpreted to also include specific active suicidal ideation.

While the reader may regard some of these interpretations of definitions as overly literal, we raise them because the experience of the first author of this article (D.S.) in doing clinical trials is that the exact FDA-CASA 2012 wording is often used to adjudicate disputes between study investigators, scale raters, clinical monitors, contract research organizations (CROs), and rater training agencies when item responses on rating scales could be interpreted in different ways.

“Active suicidal ideation: method, but no intent or plan.” For this FDA-CASA 2012 category, the definition is that the “patient […] has thought of at least one method during the assessment period” (lines 483–484)⁴ [italics ours]. However, we then read, “This situation is different than a specific plan with time, place, or method details worked out” (lines 484–485)⁴ [italics ours]. The presence of a method is necessary based on the category title and on the first sentence of the definition. However, the qualifying phrase “or method details worked out” in the following sentence indicates that if the patient HAS worked out the details of the method, the patient no longer qualifies for this category. What is the demarcation line for the amount of detail about the method necessary to meet or not meet criteria for this category? For example, consider the patient who says, “I have thought about hanging myself with a rope, but I have not worked out the rest of the details of the plan on carrying it out.” In this case, the patient has a specific method in mind. It is not clear whether the patient does or does not meet criteria for this category. This ambiguity can be interpreted in different ways by different readers and by different raters, with little guidance on how to disentangle this ambiguity or how to precisely operationalize classifying to this category in practice. Such ambiguity contributes to inter-rater unreliability.

“Active suicidal ideation: method and intent, but no plan.” The current definition wording is as follows: “Active suicidal thoughts of killing oneself, and patient reports having some intent to act on such thoughts, as opposed to ‘I have the thoughts but I definitely will not do anything about them’” (lines 492–494).⁴

For this category, the title suggests that the patient has suicidal intent, but the definition wording could be interpreted to erroneously include a patient without intent. The problem is the instruction in the definition wording not to include someone who says “I have the thoughts but I definitely will not do anything about them” (lines 493–494).⁴ This exclusion is problematic because there are at least two ways to interpret it. One interpretation is that the patient will not act on the suicidal thoughts in a way that is suicidal (i.e., no suicidal intent). This patient should properly be excluded from the category. But consider a patient who has been thinking for quite a while about “doing something” about these thoughts in terms of seeking treatment. This patient might respond that indeed it is at the top of his mind to do something (i.e., get treatment). In this case, the rater would be forced to include the patient in this category when in fact the patient has no suicidal intent. The authors of this paper have encountered this issue clinically.

“Active suicidal ideation: method, intent, and plan.” Here, there is a mismatch between the FDA-CASA 2012 title and the definition since the definition only requires intent and plan. This definition is inconsistent with the title since the title requires method as well as intent and plan.

“Passive suicidal ideation: wish to be dead.” The definition for this category limits passive ideation to “a wish to be dead or not alive anymore, or wish to fall asleep and not wake up” (lines 472–473).⁴ But patients may experience passive suicidal ideation in other ways. All three of the authors of this article (D.S., J.G., and K.S.) have frequently heard from patients that they experienced the thought or feeling that they would be “better off dead.” Unfortunately, the FDA-CASA 2012 does not allow experiences like this to be captured. In fact, one of the authors of the C–SSRS, on which the FDA-CASA 2012 appears to be based, has specifically stated “[...] we don’t consider better off dead to be anything [...]” and should not be counted as suicidal ideation.¹⁶ In addition to feeling they would be better off dead, some patients reported to the first and second authors of this article (D.S. and J.G.) that they felt a need to be dead.¹⁷ This also does not fit the current definition of passive suicidal ideation. This restrictive definition limits reporting of passive suicidal phenomena. In our opinion, both of these omitted phenomena should be viewed as danger or warning signals, because they have been shown to be associated with impulsive suicidality.¹⁷

OTHER ISSUES

Confusing conceptions of passive and active ideation. In general, the phrases “passive suicidal ideation” and “active suicidal ideation” suggest meanings that do not align with the FDA-CASA 2012 category definitions.⁴ The word passive suggests that an event or experience is not willful while active suggests a level of willfulness. In the case of passive suicidal ideation, the definition indicates that the patient must have “thoughts about a wish to be dead or not alive anymore, or wish to fall asleep and not wake up.”⁴ This clarification creates confusion for patients and clinicians (personal communication to the authors by suicidal study subjects). How should one classify an event that includes a hallucination about killing oneself that is not at all willful? In our view, different phrases may need to be used to avoid this confusion (e.g., inactive suicidal ideation versus proactive suicidal ideation). Alternately, the observation that willfulness is not central to the distinction between active and passive suicidal ideation needs to be widely communicated in training.

Can there be no intent in the presence of a plan within a time frame? Another source of confusion is found in footnote 6 of the FDA 2012 Draft Guidance with the statement “the definition of plan includes intent.”⁴ Here the document fails to precisely define “plan” and appears to make the assumption that if there is a plan there must be intent. The FDA 2012 Draft Guidance may assert these phenomena are linked; however, in reality a patient can have ideation about a plan, such as the location or date of an attempt, without intending at all to act upon the plan (examples of this may include a suicidal delusion, suicidal hallucination, or suicidal dream). This also raises the issue of the term intent not being defined precisely. The FDA-CASA 2012 definition of the “Active suicidal ideation: method and intent, but no plan” category suggests that intent refers to the intent to act.⁴ It is also possible for patients to experience the intent to plan or the intent to die. There is the potential that a patient intends to act years into the future and does not intend to act anytime soon. One is left questioning what combination of these ways of interpreting the word intent the creators of FDA-CASA 2012 intended. The lack of definition of these terms even further complicates the many ways a rater or clinician may interpret the FDA-CASA 2012 categories.

The patient’s suicidal experience. The FDA-CASA 2012 does not allow a clinician to properly capture the suicidal experience of a patient. It does not enable a clinician to track if the suicidal ideation presented, for example, with impulsivity or as a delusion or a hallucination. There is the potential of one medication effective in treating the suicidality in major depressive disorder to also, at the same time, trigger impulsive suicidality. Some suicidal patients have reported to the first and second authors of this article (D.S. and J.G.) that their experience of suicidality is different while they are taking antidepressants and can “shift” from active suicidal ideation to ideation with more acute urgency and impulsivity that seems more automatic in nature. We believe it is possible that this “shift” in the experience of suicidal ideation may be one reason why younger patients taking antidepressants report an increase in suicidality. If such a change in the experience of the patient occurs, the current structure of the FDA-CASA 2012 will not give clinicians any awareness of the change. FDA-CASA 2012 does not give clinicians enough information to review data in order to determine if a particular medication causes such a shift.

Number of events versus time spent. The FDA 2012 Draft Guidance, through its use of the C–SSRS as a standard reference assessment instrument, focuses on the total number of times the behaviors occur. There is an implicit assumption that the number of events correlates with severity or seriousness or risk of suicidal behaviors. While this is an interesting idea, it has not been adequately tested, and we found in tracking over 31,000 events of suicidality in a single case over a year¹⁴ that the evidence in support of this association is very weak. In contrast, we found that the evidence supporting the value of tracking time spent in suicidality was much more valuable for this purpose.¹⁴

Suicidal ideation and behavior categories and definitions citation. At the top of Appendix A (line 466)⁴ containing the FDA categories and definitions used in the FDA 2012 Draft Guidance is a reference to a 2007 document by Posner et al. This reference is inadequately sourced. The source is almost certainly the 2007 paper delineating the nine-category classification developed by Posner et al,¹ but the corresponding footnote (#10) only provides a website address for a Columbia University website for the 11-category C–SSRS, an instrument that is described in a 2011 paper by Posner et al.² The website in turn does not provide a link to the 2007 paper or to the 2011 paper by Posner et al² that describes the C–SSRS. We find this confusing since the FDA definitions contained in this appendix are central to the FDA 2012 Draft Guidance. The absence of a proper source citation makes it difficult to easily find the source and then to compare the FDA definitions in the FDA 2012 Draft Guidance with the source. And as it turns out, the categories and definitions in Appendix A do not match those in the 2007 classification and, as we show elsewhere,¹⁸ they are not always in precise alignment with the C–SSRS.

CONCLUSION

The results of our review suggest that the 2012 FDA Draft Guidance still needs work. In particular, the categories do not capture the full range of suicidal ideation and behavior (they are not exhaustive) in that they select and categorize only some combinations of relevant ideation and behavior while not capturing many others (26 not captured out of 32 total ideation combinations). Moreover, definitions are not clear enough to be unambiguous and all too often misalign with category names (titles above definitions). Users, whether they be raters, investigators, or sponsors, who consult the document as an aid in decision making on an item response on a scale are hampered by this kind of ambiguity. At best, end users will ask questions and make consistent decisions in a given interview. At worst, they will cherry pick, using the title to classify one patient and the definition or some part or phrase of the definition to classify another patient. This impacts comparability over time and between collections, especially when data is collected at multiple sites with hundreds of end users in a drug trial and can render data output flawed. For pharmaceutical company sponsors of studies, the problem is further compounded since the categories for the 2012 Draft Guidance do not match categories in previous 2010 Draft Guidance, making it difficult to pool data over time to properly detect trends for a particular drug or drug class.

These problems need to be remedied because they pose potential threats to public health, to research on the safety of medications, and to the search for effective medication treatments for suicidality.