Strategically Guiding Research through Careful Consideration of the Path to Impact

Introduction

Research in medical imaging has the potential to improve human health. This includes basic imaging research that aims to advance medical knowledge, methodologic research to enhance technical capabilities, and clinical research with immediate clinical applications. The investigators pursuing these studies interact regularly with other investigators, conference attendees, grant review panels, clinicians, and other colleagues with similar perspectives. In these interactions, the primary focus is on the scientific details (eg, on the next experiment, grant, or paper). The translation of medical research to improve health outcomes is commonly not evident or even considered when conceiving of and executing a research project. A case in point is, for example, MRI research (1). What follows is a case study of compositional MRI of cartilage research and one approach for guiding researchers to establish a mindset that embeds a consideration of impact into their research perspective.

Case Study: Compositional MRI of Cartilage

Osteoarthritis is a widespread, debilitating disease characterized by degeneration of articular cartilage. For the past 25 years, MRI research of cartilage has been motivated by the promise of earlier detection of osteoarthritis and improved therapeutics. Beginning in the 1990s, a large number of MRI metrics have been proposed and developed to target imaging of the molecular composition of cartilage. Metrics include sodium concentration (2), T1, T2 (3), T1ρ (4), diffusion (5), magnetization transfer (6), delayed gadolinium-enhanced MRI of cartilage (7), glycosaminoglycan-specific chemical exchange saturation transfer (8), diffusion tensor imaging (9), T1ρ dispersion (10), T2* with ultrashort echo times (11) and, most recently, quantitative susceptibility mapping (12).

For the purposes of this case study, we focus on just one metric, diffusion MRI, to illustrate the challenge. The first evidence that diffusion MRI was sensitive to cartilage degeneration was reported in 1993 (5). Ten years later, a report (13) concluded that diffusion might be sensitive to structural rather than compositional degradation. Five years after that, diffusion tensor imaging was shown to monitor the changes in collagen fibers with compression (14). In later years, double-echo steady-state diffusion imaging was established (15), and then stimulated echo diffusion imaging was demonstrated as a means of detecting early cartilage damage (16). Most recently, a report concluded that diffusion of fluid and biopolymers in articular cartilage are sensitive to osteoarthritis (17). When taken in isolation, each of these studies presents a reasonable approach with robust data and provides intellectual stimulation. Each study is also an advance in some (generally technical) way.

However, although these studies (and those of other metrics) are important in showing potential clinical utility (18–22), this body of work has not overall led to improved understanding of cartilage pathophysiology or validated approaches to incorporate into clinical practice or patient management. Although the imaging options and quality have greatly improved, and hundreds of articles have been published, there has been little specific delineation in most of the studies in terms of how they might be used in the short or long term to answer specific biologic, physiologic, or clinical questions and ultimately improve health outcomes.

One Method of Inserting Impact Considerations into Research: The IMPACT Program

We propose that researchers be explicit about, and provide evidence for, how the results of their research would be used, by whom, and for what purpose. This evidence will provide strategic guidance for the project. It will also make a case for how the work, if successful, would bring the field closer to realizing improved health outcomes.

To explore this approach, we implemented a supplemental training program (the “IMPACT program”) for fellows (graduate students and postdoctoral researchers). We recruit from institutions in the greater Boston area and attract fellows from a wide variety of biomedical research areas. Program faculty are drawn from the academic, business, and clinical sectors. In contrast to many mentoring programs, we do not attempt to match the fellows with faculty according to expertise. The only commonality is a shared interest in the biomedical arena.

A series of 10 3-hour sessions are held over the semester, during which the fellows are challenged to articulate their research and its potential impact.

During the first meeting, the fellows are asked the following question: “What do you do, and why is it important?” The first part of the question generates easy flowing answers, although these answers tend to contain much jargon and are difficult to understand by others in the group. The second part of the question evokes an explanation of why the field is important, but the fellows have much more difficulty explaining specifically how and why their research is important.

In subsequent weeks, we iterate between three formats for describing the projects: one to three sentences, one-page bulleted overview with a logical flow, and 5-minute slide presentations. In all formats, they are asked to explicitly and specifically include how the results of the research would be used, by whom, for what purpose, and how the benefit would be realized.

We also require that the fellows interview stakeholders, as stakeholders often shed light on the reality of what happens and how decisions are made. Stakeholders may include researchers who might shift direction due to the findings, clinicians who might incorporate the outcomes into clinical practice, companies that might use the results as a basis for new product development, politicians who might develop new public policies, or patients who may directly benefit from the research.

Reactions toward the IMPACT Approach

The fellows are initially surprised at how difficult it is to provide a convincing exposition of what he or she is doing to a diverse audience, how it will matter, and why it will have an impact. Many assert that it would be impossible to explain their work without being technical or within a few minutes. We tell them that it is possible. Some contend that their work is to develop a new platform technology and believe, therefore, that it is unnecessary and limiting to state a specific application. We explain that it is important to be convincing of at least one application where the new technology will be of clear value and then explain its broader implications. Fellows often assume that making something better (more precise, more quantitative, faster, etc) is intrinsically valuable. We encourage them to articulate specifically how that improvement will enable something that is not possible with current approaches.

To help the fellows prepare to meet stakeholders, the faculty work with the fellows to formulate questions, identify contacts, and use the program’s credibility to approach individuals that they might not otherwise interact with.

Through this process, fellows discern how their initial conception of their work missed several crucial points and discover tacit assumptions in need of validation. As a result, they are able to be more strategic, sometimes shifting the focus of their work as needed to have an effective impact.

Survey Outcome Metrics

Independent surveys for this prospective study were approved by the local institutional review board (Massachusetts Institute of Technology Committee on the Use of Humans as Experimental Subjects approval number 511298089; this study is considered exempt, so it was approved without full institutional review board review). Subjects were sent an e-mail that explained the process and included a link to the survey; participants were considered to provide consent when and if they clicked on the survey. Survey data were under control of one author (R.M.).

Experts in the Massachusetts Institute of Technology Teaching and Learning Laboratory conducted independent surveys each semester at the conclusion of the IMPACT experience. Thus far, 77 of the 78 admitted fellows have completed the program and 71 (92%) returned the survey.

Ninety-four percent of respondents (67 of 71) said they would recommend the program to others. The survey revealed that the program helped the fellows begin to think more strategically about their research projects: Greater than 80% (57 of 71) of respondents stated that they were more able to identify steps needed to guide their research for future impact, that they identified gaps in their thinking about their projects, and that talking with stakeholders was one of the factors leading them to rethink their research directions. In a follow-up survey administered between 6 months and 2 years after completion of the program, 79% of the respondents (27 of 34) stated that their research itself was affected by their participation in the IMPACT program.

The ways in which the research was affected varied. In some cases, there was a dramatic shift; for example, one fellow was designing a diagnostic test based on a molecular pathway that she discovered to be involved in Lyme disease. By talking to stakeholders, she understood that using the same pathway as the basis for a new therapeutic drug would address a more pressing clinical need, and she proceeded to receive new funding on that basis. Another fellow’s work involved using machine learning approaches to augment molecular dynamics to screen candidate pharmaceuticals for their ability to bind a particular molecular target. However, he learned from discussions with faculty and stakeholders that this was not a priority for the pharmaceutical industry. He shifted the focus to rapid screening of molecules to coat electrodes used in large-scale water filtration systems to extend the types of contaminants that can be removed. These examples are analogous to the design of new imaging approaches; deciding on a specific pathophysiologic question, clinical research need, or clinical practice need early on might better inform the design of the technical development and put it into practical use much earlier.

Other projects shifted in more subtle ways (eg, adjusting study design, eliminating anticipated experiments, or finding new ways to contextualize the work). The important point is that the fellow asked the question and as a result she or he was able to define and articulate the path to impact or modify the approach to define a better path.

Take-Home Messages

Although it seems intuitive to anyone doing research that their work should lead to impact, being asked to define that potential impact explicitly turns out to be much harder than expected. The survey results showed that many of the trainees rethought their research direction and, in some cases, actually altered their research path despite the fact that most of these research projects were already peer-reviewed, funded, and ongoing.

After completing the program, the fellows were much more comfortable speaking about their research in terms that professionals in different sectors can understand, improving the potential for those interested in moving between career paths and allowing engagement in cross-disciplinary research and advocacy. The program also provided broad mentoring and networking support to both trainees and faculty. In addition, fellows gained confidence in approaching stakeholders. Finally, participants reported anecdotally that the methodology of articulating a clear and focused research story enabled more compelling research manuscripts and grant proposals.

Integration into Imaging Research

Considering the potential impact of a research project as part of training paradigm will improve our current research mentoring of imaging scientists as well as the overall focus of scientific development toward realistic impactful ends. Moreover, trainees (and scientists) who develop the habit of regularly considering the path ahead with regard to impact are likely to identify and overcome the inevitable roadblocks earlier rather than later. Investigators and clinicians could benefit from periodically evaluating their work from the perspective of impact. This could be done within and between hospital radiology and nonradiology departments, universities, and companies (eg, in the setting of weekly conferences). Shorter workshop-style implementations could also be implemented. Importantly, the gaps in the logic of how research might lead to impact are more easily discerned by those not involved in the field, and, hence, diverse participants are very helpful.

Finally, this approach induces a substantial advance and broadening of one’s mindset that persists well beyond the supplemental experience. To paraphrase one of our alumni, “after this experience, I now listen to talks, read papers, and evaluate my own research with a very different eye—asking not just what was done, but why does it matter and for whom.” That mindset bodes well for the future.