“Black box” pharmacogenetic decision-support tools in psychiatry

The last decade has witnessed a surge in the development and deployment of psychiatric pharmacogenetic testing, fueled by an emerging evidence base and demand for more personalized approaches to medication selection and dosing. However, the successful diffusion of pharmacogenetic testing results into clinical practice requires accompanying decision-support tools capable of translating pharmacogenetic data into actionable prescribing recommendations. To facilitate this translation process, several expert groups such as the Clinical Pharmacogenetics Implementation Consortium (CPIC)1 have developed prescribing guidelines based on the scientific literature and drug label information curated by resources such as the Pharmacogenomics Knowledgebase.2 Despite the availability of these resources, many clinicians are unaware of them or do not have the time to consult them during their day-to-day practice. As a result, pharmacogenetic testing companies have developed and provide decision-support tools that translate test results into prescribing recommendations. However, there is growing concern about how some companies develop and generate these prescribing recommendations, particularly those employing “black box” decision-support tools.

Black box decision-support tools deliberately conceal – for proprietary reasons – the process by which pharmacogenetic testing results are translated into clinical recommendations (Figure 1). This strategy is in conflict with open and peer-reviewed approaches adopted by CPIC and other clinical guideline development groups as well as the European Union’s General Data Protection Regulation requirement for deconvolution of black box approaches before they are used in patient care.3 Proponents however have argued that the black box approach is aligned with the idiosyncratic manner by which most clinicians treat patients4 and the unknown mechanisms of many medications.5 Importantly, many arguments in defense of black box algorithms stem from artificial intelligence systems, where the relationships captured cannot be explicitly understood, and as a result are non-transparent by nature rather than by choice. To our knowledge, artificial intelligence is not yet being used by pharmacogenetic testing companies and hence most arguments defending the black box approach are not applicable to black box decision-support tools used in pharmacogenetics. Nevertheless, these black box tools are utilized by a considerable share of pharmacogenetic testing companies and several have been supported by randomized controlled trials in the treatment of major depressive disorder.6 So, should we shun or should we come to terms with black box pharmacogenetic decision support tools in psychiatry?

Figure 1. Overview of the black box approach to pharmacogenetic-guided decision support tools. Pharmacogenetic decision-support tools take discrete pharmacogenetic inputs (i.e., genotype data), apply a decision algorithm (i.e., set of rules), and produce clinically actionable outputs (e.g., medication selection and dosing recommendations). Although all components of the black box might be known (but not always), the set of rules by which these components are combined to generate actionable outputs is concealed from the end user (e.g., clinicians, patients). Note: the figure does not depict a comprehensive view of resources that could be utilized in black box approaches, nor does it suggest that all black box tools used in pharmacogenetics include all of these resources as part of their decision algorithms.

Before answering this question, it is useful to consider what value a black box approach offers over an open approach. Three queries laid out in a recent paper can guide us in our evaluation.5 First, does the black box approach produce the best results? The current evidence does not provide a clear verdict on whether this is the case. Head-to-head trials comparing performance of black box versus open pharmacogenetic decision support tools have yet to be published, although comparison of recommendations produced by companies using black box and open approaches showed high levels of discordance.7 Thus, it is likely that one approach is superior to the other but it is not clear which approach will prevail. The second query relates to the cost of a wrong answer. The probability of an inaccurate medication or dosing recommendation is a commonly expressed concern among critics of pharmacogenetic testing that is further amplified in the context of a black box approach. Regardless of approach however, this critique assumes that decision-support tools are used in isolation rather than in combination with good clinical judgment that could effectively tailor or refute inaccurate recommendations. In fact, the term “decision-support tool” implies that recommendations should be viewed as a companion source of information used to make clinical decisions. Nonetheless, future research comparing the frequency and potential costs associated with inaccurate recommendations using black box and open approaches are warranted. The third and final query asks what approach inspires new ideas? Here, black box decision-support tools would appear to have an edge, in that their proprietary nature is more attractive to private investors and in turn could expedite the development of more comprehensive tools with greater predictive value. However, there are numerous recent examples of large public investments in open pharmacogenetics (e.g., Ubiquitous Pharmacogenomics Consortium8), signaling that both approaches can inspire new ideas and health innovations.

This brings us back to our original question of whether we should reject or accept black box pharmacogenetic decision support tools in psychiatry. Unfortunately, the current evidence does not lead us to a definitive answer; and we suspect consensus, if sought, would not be reached. What is certain is that black-box decision support tools do not appear to be superior to their open counterparts. Furthermore, the black box will remain a feature in pharmacogenetic testing for the foreseeable future. With this in mind, those keen to implement pharmacogenetics but weary of the black box approach should become familiar with the basic principles of pharmacogenetics, its limitations, and the free guidelines (e.g, CPIC) and resources (e.g., PharmGKB) available to assist with clinical implementation. With this knowledge in hand, it is possible to bypass the black box via direct interpretation of the raw genotype or phenotype (e.g., metabolizer status) results provided by companies. Alternatively, one could simultaneously explore the use of both the black box interpretation and the raw genotype or phenotype data. In fact, the PharmGKB website includes free tools9 that allow users to enter genotype information and produce prescribing recommendations based on published CPIC guidelines, effectively circumventing black box interpretations. This strategy does not guarantee “better” medication selection or dosing recommendations relative to the black box approach, but it does offer full transparency, an attribute deemed necessary by some and inconsequential by others.

Disclosure

CAB has received material support from Assurex Health (Myriad Neuroscience), CNSDose, Genomind, and AB-Biotics for research purposes and has ongoing research collaborations with MyDNA, but does not have equity, stocks, or options in these companies or any other pharmacogenetic companies. HAE owns shares in CNSDose Pty Ltd.