Selective TNF Inhibition for Chronic Stroke and Traumatic Brain Injury: An Observational Study Involving 629 Consecutive Patients Treated with Perispinal Etanercept

I read with interest the recent article by Tobinick and colleagues entitled “Selective TNF inhibition for chronic stroke and traumatic brain injury: An observational study involving 629 consecutive patients treated with perispinal etanercept.”¹ My enthusiasm for the findings was greatly diminished by the highly-subjective data collection methods and measures used. These notable shortfalls warrant some attention given that an invited commentary regarding the findings was solicited by the journal,² and since the article has generated some publicity in the media.

Among the methodologic limitations, one of the most notable is the highly-subjective and arbitrary nature of outcome measures that were used. Specifically, in evaluating subject outcomes, the treating physician or nurse practitioner from the authors’ clinics judged subjects to be “improved” or “not improved” in several domains of impairment. The concerns with this approach are numerous. First and foremost, this measurement approach is highly subjective, relying on the raters to make a determination on whether the patient improved and what factors to consider in coming to this decision. No rules for scoring or manual of procedures to guide decisions of “improved” versus “not improved” decisions appear to have been created, making the evaluative process highly subjective from rater to rater. For example, with regard to the category of “motor impairment,” the relative weight of impairments in each region of the body (e.g., upper versus lower extremity), the movements or positions during which individuals were assessed, and other highly influential factors appear to be unstandardized. Additionally, no attempts were made to confirm or assess intra or inter-rater reliability among the data-collectors, either at each site or across the three sites. It is also unclear what measurement experiences specific to the above mentioned constructs that the raters possessed, if any. The above concerns are especially egregious given that the raters worked for the very organization that was administering and charging patients for the intervention, which constitutes a conflict of interest (which was noted in the paper).

In addition to concerns with administration of the rating system, there are significant concerns with the precision of such a classification system. Indeed, a wide and varied continuum of impairment levels are exhibited after acquired brain injury (i.e., stroke; traumatic brain injury). Blithely labeling a patient as “improved” or “not improved” is imprecise in that it provides little detail on the actual levels of impairment exhibited by subjects before intervention, or the ways in which the intervention impacted impairment following its provision. In particular, this approach provides no detail regarding the particular movements, capabilities, or specific areas of deficit that derived benefit from perispinal etanercept. More detailed information is also important given that varied impairment levels are exhibited in different body regions even within the same patient (e.g., patients with stroke typically retain more upper extremity motor impairment than lower extremity motor impairment in the months and years post ictus³). As a result, the clinician or patient reading this article is left with little information regarding the magnitude of change that might be anticipated as a result of integrating this intervention strategy, or the particular regions in which one might expect change. From a scientific standpoint, the use of such an arbitrary, dichotomous measurement strategy compromises ability to truly replicate the efforts of the authors in a meaningful way, since “improved” and “did not improve” are subjective assessments (as addressed more fully later in this letter) and can be made according to the judgment of the particular rater, and/or relative to the pre-intervention level of each patient (which can vary greatly).

The authors’ choice to use such a subjective and arbitrary measurement strategy is even more perplexing given the multitude of stroke specific, validated, outcome measures available for the very constructs measured in the study (e.g., motor impairment;⁴ spasticity;⁵ sensory impairment⁶). Such measures provide valid and specific ascertainment of constructs that may be responsive to a clinical intervention; information that would presumably be desirable for the authors to know since they are using etanercept clinically. In lieu of using such measures, the authors instead applied their rating strategy to fabricated domains such as “clinical impairment,” “walking impairments,” and “psychological impairment” that are inconsistent with published recommendations in the literature.^e.g.,7 Again this choice speaks to a lack of precision in the study, which, in this case, does not tell the reader in what domains patients actually exhibited change. For example, following acquired brain injury, “cognitive impairment” as assessed by the authors could be due to deficits in attention, working memory, long term memory, agitation, information processing, and/or many other areas. The authors’ choice to use fabricated, “umbrella” impairment domains rather than established impairment categories and measures deviates from norms in clinical rehabilitation, diminishes the reader’s ability to determine in which domains changes occurred, and undermines substantive comparison between the magnitude of changes associated with perispinal etanercept versus those of other interventions. Since clinically meaningful changes (sometimes known as “minimal clinically important differences”) are assessed using established outcome measures, the authors’ use of a subjective, dichotomous rating scale also undermines discernment of whether changes were clinically-significant.

To the authors’ credit, they did deploy some established tools to gauge change, such as the Montreal Cognitive Assessment (MoCA) and the Visual Analog Scale (VAS). The concern here is that these instruments are not typically used as outcome measures, per se; they are usually used as clinical screens to provide a rough estimate of a patient’s level of cognition or pain. Again, though, a bigger issue is that cognition and pain are multifactorial and can vary greatly depending on the domain measured. Consequently, global measures of such constructs provide little insight into what was actually impacted by the intervention as discussed previously. For example, the MoCA assesses recall, registration, subtraction, trailmaking, and several other abilities. If the authors were truly interested in such constructs, there are more robust measures that break down these various aspects of cognition, providing a more succinct depiction of what is actually impacted by the intervention. The authors would be better advised to focus in on the areas in which they have seen the most robust changes in their clinical experiences (or where they hypothesize that the largest changes will occur, and systematically measure these changes using more specific measures.

In addition to the above limitations, the scientific basis for the application of perispinal etanercept for chronic acquired brain injury remains unclear based on the authors’ discussion. Specifically, the authors argue that three lines of inquiry “present a scientific rationale for the use for PSE as a treatment for chronic neurological dysfunction after stroke and TBI.”¹ The presumed, supporting bodies of work that the authors cite are: (a) the clinical development of perispinal administration of etanercept; (b) evidence from clinical and preclinical studies suggesting etanercept’s safety; and (c) studies suggesting that neuroinflammatory mechanisms may constitute a potential therapeutic target after stroke and TBI. The problem is that these areas do not actually constitute scientific rationale for using this drug in this population. Development “a” constitutes an innovation in drug delivery but not an actual scientific reason to administer etanercept to people with chronic stroke and TBI. Likewise, development “b” points to safety of etanercept, but, again, is not a scientific rationale to aid chronic recovery from acquired brain injuries. And, while “c” is certainly a promising development, it is well known that the stroke penumbra has resolved and that these target processes have diminished considerably (if not subsided altogether) in the months or years post ictus. Concurrently, the presumed mechanism of most chronic stroke recovery approaches is neuroplasticity either by unmasking latent pathways and/or by recruiting pathways previously allocated to other brain functions.⁸ In short, the very scientific rationale that the authors cite are not, in fact, scientific rationale for the use of this approach.

In summary, there are significant methodological flaws in the validity and administration of the outcome measures used in this study. Furthermore, the scientific bases for the approach as cited by the authors are not actually bases, and the only literature attesting to the impact of this intervention is that which has been produced by the authors (including at least one study⁹ that has some of the same “fatal flaws” as described herein). As a result, there remain significant questions associated with the efficacy of perispinal etanercept. At best, provision of this approach for acquired brain injury recovery should be considered experimental at this time.