Evaluating Translational Research: A Process Marker Model

Abstract

Objective: We examine the concept of translational research from the perspective of evaluators charged with assessing translational efforts. One of the major tasks for evaluators involved in translational research is to help assess efforts that aim to reduce the time it takes to move research to practice and health impacts. Another is to assess efforts that are intended to increase the rate and volume of translation. Methods: We offer an alternative to the dominant contemporary tendency to define translational research in terms of a series of discrete “phases.”Results: We contend that this phased approach has been confusing and that it is insufficient as a basis for evaluation. Instead, we argue for the identification of key operational and measurable markers along a generalized process pathway from research to practice. Conclusions: This model provides a foundation for the evaluation of interventions designed to improve translational research and the integration of these findings into a field of translational studies. Clin Trans Sci 2011; Volume 4: 153–162

Introduction

The concept of translational research has become critically important in contemporary biomedical research and practice. It is the subject of a rapidly growing literature, catching the attention of most leading biomedical journals, and becoming the central focus of several new publications. Translational research is showing up in everything from research grant proposals to the curricula of leading medical schools and schools of public health. It is the focus of considerable effort in the biomedical industry ¹ and is increasingly central to discussions of public health. The National Institutes of Health (NIH) have made it a central priority, part of their “Roadmap” initiative. One of their primary programs, the Clinical and Translational Science Awards (CTSAs), currently expends over 350 million per year to fund 55 research centers and by 2012 is expected to fund 60 centers at a cost of approximately a half billion dollars per year, making it the largest program at NIH.

What is behind this considerable investment in translational research? One of the most significant motivations comes from a relatively small number of studies that show that it takes a long time to move basic scientific ideas to practice and health impacts. For instance, Westfall, Mold, and Fagnon ² asserted that “It takes an estimated average of 17 years for only 14% of new scientific discoveries to enter day‐to‐day clinical practice” (p. 403). They based their claim on earlier work ³ that similarly stated “Studies suggest that it takes an average of 17 years for research evidence to reach clinical practice” (p. 66) at a rate of 50% use in the relevant population. Other work ⁴ suggests that the median translation lag was 24 years between first description and earliest highly cited article. Because these studies typically only measure part of the process of moving from research to practice and eventually to health outcomes and impacts, these are likely to be significant underestimates. Translational research in many ways can trace its primary impetus to the notion that this time lag is seen as too long, certainly longer than necessary, and that there must be a better way to move research to practice more quickly without sacrificing quality or increasing costs. Proposed solutions include everything from better management of scientific research and increased process efficiency to wholesale rethinking of the biomedical research‐practice endeavor for the 21st century.

This paper focuses on the length of time that translational research takes. We do so because the long duration and time estimates to move research to practice were critical to making the policy case for significant investments in translational research. Nevertheless, temporality and duration concerns need to be considered in the context of many other factors including quality of research, cost, ethics, management, potential impacts, and so on. ⁵ The success of the translational research endeavor will ultimately be judged by whether it reduces the time and duration issue while at least preserving the current status of other factors such as quality or cost.

We argue here that one of the major tasks for evaluators involved in translational research is to help assess whether efforts such as the CTSAs can reduce the time it takes to move research to practice and health impacts and increase the rate and volume of translation—all while ensuring the quality and cost‐efficiency of the conduct of research. This paper examines the concept of translational research from the perspective of evaluators charged with assessing translational efforts. In doing so we hope to: (1) consider the most prominent models of translational research that have been offered in the literature, (2) synthesize the major features that are shared across these models in order to show underlying commonalities, and (3) suggest a new synthetic framework for evaluating progress in enhancing research translation that is consistent with existing models but avoids some of the major current problems. Specifically, we offer here an alternative to the dominant contemporary tendency to define translational research in terms of a series of discrete “phases.” We contend that this phased approach is insufficiently precise for most evaluation purposes and instead argue for the identification of key operational and measurable markers along a generalized process pathway from research to practice.

Models of Translational Research

Models are essential for understanding and representing complex multifaceted constructs such as translational research. They provide a concise description of the concept, often representing it visually in a diagram or graph that depicts its major features and characteristics. They help evaluators clarify and operationalize useful measures. In this section, we review the primary published models of translational research and present a synthesis designed to illuminate important issues for evaluation.

One of the earliest and most straightforward models of translational research was suggested by Sung et al. ⁶ ( Figure 1 ) who describe it with a two‐phase framework that essentially consists of “blocks” that exist in the process of moving from basic research to improved health. “The first translational block involves the transfer of new understandings of disease mechanisms gained in the laboratory into the development of new methods for diagnosis, therapy, and prevention and their first testing in humans. The second translational block affects the translation of results from clinical studies into everyday clinical practice and health decision making”6 (p. 1279). They refer to the first phase as “T1” translational research and the second as “T2” research. They also describe a number of barriers to translational research (also called “blocks,” see top of Figure 1 ) that range from a lack of willing participants to the lack of funding.

Figure 1.

Two‐phase translational research model in Sung et al. ⁶

Westfall et al. ² offer a similar multiphase model of translational research ( Figure 2 ) but divide it differently into three phases. The first (T1) goes from basic to human clinical research (“bench to bedside”) with this latter consisting of early phase clinical trials in humans. The second and third phases of clinical research (T2 and T3) collectively span practice‐based research and are derived by dividing Sung et al.’s ⁶ T2 into two distinct phases. In their T2 phase, knowledge is moved from early clinical trials to use with patients in phase III and IV clinical trials through guideline development, meta‐analyses, and systematic reviews. The third phase, T3, involves translation to practice and encompasses dissemination and implementation research. There are several noteworthy features in the Westfall model. It explicitly argues that both the T2 and T3 phases involve a bidirectional dynamic translational process with practice influencing research and vice versa. The endpoint of their model is clinical practice, not improved health (as in Sung et al. ⁶ and most other models). Westfall et al. ² also argue that “practice‐based research’ should enter the mainstream medical research vocabulary” (p. 406).

Figure 2.

Three‐phase translational research model in Westfall et al. ²

Dougherty and Conway ⁷ also offer a three‐phase model of translational research ( Figure 3 ) that moves from basic biomedical science to clinical efficacy knowledge (T1), then to clinical effectiveness knowledge (T2), and on to improved health quality and value and to population health (T3). They point out that each translational step moves to progressively broader settings over time. They also explicitly include feedback loops throughout their model to indicate the bidirectional nature of the process. Their T2 phase includes clinical effectiveness trials and the development of “practice guidelines and tools for patients, clinicians and policy makers” (p. 2319).

Figure 3.

Three‐phase translational research model in Dougherty and Conway. ⁷

A four‐phase model ( Figure 4 ) has been offered by Khoury et al. ⁸ Their first two phases are similar to Dougherty and Conway’s ⁷ distinction between efficacy and effectiveness studies in clinical research. Like Westfall et al., ² they make a finer distinction of postguideline translational research than is done in the Sung et al. ⁶ model. Their T3 phase encompasses dissemination, implementation, and diffusion research. Perhaps, the most salient feature of their model is their identification of T4 that they describe as “outcomes research” and define as “research that describes, interprets and predicts the impact of various influences, especially (but not exclusively) interventions on ‘final’ endpoints that matter to decision makers. Decision makers include patients, families, individuals at risk, provider, private and public payers, and so forth” (p. 668).

Figure 4.

Four‐phase translational research model based on Khoury et al. ⁸

Translational Research Model Synthesis

The danger in the proliferation of multiple competing models with different and conflicting numbers and definitions of translational research phases is that they complicate communication about translational research generally and run the risk of confusing interpretations of evaluations that rely on them. The particular dilemma for translational research evaluators, and especially for those evaluating CTSA research, is the potential for conceptual cacophony, a translational “Tower of Babel,” where the same phase label is used for very different operational stages in the research‐practice continuum, thus making cross‐evaluation comparisons and syntheses especially problematic.

A close reading of the primary existing models shows that they have several key features in common, even though they may differ in specifics. First, and perhaps most important, all of them characterize translational research as a temporal process moving from basic to clinical to postclinical research and ultimately to use and public health impact. That said, all of them also incorporate the idea of bidirectionality, the notion that sometimes information flows from the right to the left on the models. For example, when basic research is informed or shaped by clinical insights, the translational process, at least temporarily, moves toward the left.

Second, the three alternative models to Sung et al.’s ⁶ T1/T2 model differ in how finely they divide their T1 and T2 phases. For instance, Dougherty and Conway ⁷ seem to divide Sung et al.’s ⁶ T1 into two phases that essentially distinguish efficacy from effectiveness research. Westfall et al. ² divide Sung et al.’s ⁶ T2 into two phases that provide a clearer distinction for practice‐based research. And, the Khoury et al. ⁸ model essentially makes both of the above distinctions, or something very close to it, in arriving at their fourfold classification. Each of these three models subdivides the Sung et al. ⁶ phases in order to make an important point and to help assure that something they deem critical in translational research does not get overlooked or lost in the complexity. Dougherty and Conway ⁷ want to be sure to note the distinction between efficacy and effectiveness studies in clinical contexts. Westfall et al. ² believe that practice‐based research tends to be overlooked and needs to be highlighted. And Khoury et al. ⁸ want to preserve the efficacy–effectiveness distinction and make one between outcomes research and other types of postguidelines work. As this evolution suggests, the subdivision of translational research phases based upon the interests or perspectives of different stakeholders could conceivably continue indefinitely, leading to ever more complex models, and contradictory classification schemes.

Third, all of the models make a basic distinction between research that takes place before and after the development of synthesized clinical trial knowledge in the Sung et al. ⁶ model at the point of demarcation between T1 and T2. This distinction between pre‐ and postclinical synthesis may represent something like a major shift in scale in the research‐practice continuum, a critical jump from individual clinical studies (before) to more synthesized general knowledge that cuts across studies (after). This shift is essentially at the heart of the evidence‐based practice movement in clinical medicine ⁹ , ¹⁰ and is often operationalized through research syntheses such as meta‐analyses, systematic reviews, and the development of practice guidelines. ¹¹ Importantly, perhaps in a less‐formalized way, this same synthetic process occurs in studies at levels of basic science and studies of clinical mechanisms. In these instances, the unit being studied is the basic scientific discovery (e.g., a molecular pathway, drug mechanism of action, etc.) that may at a later point in the pathway be translated into an intervention that would be subject to clinical trials. After each of the shifts, the unit being translated is cross‐study synthetic knowledge. While not every translational process proceeds through this change in scale from study‐specific knowledge to synthetic knowledge, the junctures between basic and clinical research and between clinical research and practice will increasingly be a primary pathway. This scale transition is essentially the focus of a recent Institute of Medicine report. ¹¹ They argue that on one side of the transition are clinical research studies and systematic reviews. On the other is the development of clinical guidelines and recommendations—where “decision makers and developers of clinical recommendations interpret the findings of systematic reviews to decide which patients, health care settings, or other circumstances they relate to” (p. 23). This change in scale from study specific to synthetic knowledge (see center of Figure 5 below) presents an important challenge to CTSA evaluators since: (1) the object of the evaluation shifts dramatically at this point; (2) this juncture is a nearly unanimous point in the predominant models of the translational pathway; and (3) because the area of the translational pathway after the shift from study specific to synthetic knowledge is an important component of the “17 years” lag time from bench to bedside.

Figure 5.

Comparison of the four major translational research models.

In the comparison and synthesis above, we have attempted to place the four major models into an integrated temporal continuum ( Figure 5 ) to highlight some of the commonalities and differences in the models. This synthesis shows five of the major components along the research‐practice continuum: (1) basic research, (2) clinical research, (3) research synthesis (meta‐analyses, systematic reviews, and guidelines) ¹¹ , (4) practice‐based research, and (5) health impacts. For visual simplicity, these components are grossly approximate and leave out considerable detail. For instance, the model does not explicitly show the system of multiphased clinical trials that is a critical part of the clinical research endeavor, nor does it depict the dissemination and implementation research endeavors that are central to practice research.

The differing phases in the four prior models ² , ⁶ , ⁷ , ⁸ can be arrayed along the research‐practice continuum showing how translational research interacts throughout. In Figure 5 , we depict translational research in this synthesis as a continuous endeavor. That is, we do not show distinct translational phases that are interposed between other components such as clinical and practice research. We assume that translational research has potential relevance at many points along this continuum and we depict translational research as a meta‐arching endeavor and not as a disjointed series of potentially distinct sequential phases.

We borrow from the four prior models the notion that the research‐practice continuum can be portrayed as moving across time from basic research to health impacts. But this does not mean that translational efforts are linear in nature or that movement along the continuum is only from left to right over time. In fact, the process of the development of any specific discovery is likely to move backwards and forwards through this continuum. Ideas for potentially useful treatments may originate in clinical practice or epidemiologic research as well as in basic research. Ideas that look promising in basic research may show unanticipated problems in practice, leading to a return to the “bench” and revision before another round of clinical trials begin. Clinical trials may uncover populations unresponsive to particular interventions that suggest fruitful areas for understanding disease mechanisms. Ideas that seemed successful in controlled clinical contexts may be hard to work with in implementation contexts, leading to new ideas that could inform a new round of basic or clinical research. Translational activities and interventions have a role at all of these points, encouraging and facilitating more rapid and effective communication, supporting collaboration, and keeping the process moving along as efficiently as possible. We show bidirectional arrows to suggest these directional shifts, and we assume that in the normal course of things they exist virtually everywhere along the continuum. Moreover, we do not portray replication as an essential component of the scientific process since that also occurs at each step (in both directions) of the process as a prerequisite for synthesis. Note, however, that while the process of translational research may be bidirectional and dynamic, the intention or goal is ultimately to move research from left to right. Whether the bidirectional and dynamic nature of the process increases or decreases the overall time, it takes to move from left to right is, of course, one of the central empirical questions that evaluation needs to answer.

Of course, reality is seldom as simple or straightforward as any of the models or the synthesis might suggest. In the multidisciplinary, multiparadigmatic, multiorigin, multilab, and sometimes nationally and internationally collaborative world of translational research, it may be difficult or impossible to identify a direct lineage that tracks from basic research through to health impacts. Theoretically at least, contributing research may occur concurrently or in parallel with a targeted translational process and the interactions and influences between multiple processes may at times be more complex than any simplifying models can portray.

In summary, the different translational research models synthesized in Figure 5 contribute to our understanding of the process of moving research along a continuum to health impacts. While the prior models suggest both commonalities and differences in phasing, perhaps their most striking feature is the common underlying research‐practice continuum. This suggests that measures may usefully be employed at any point along the continuum from basic research to health impacts, whether it corresponds to one or more models’ phase transitions or not. A major task for evaluation of translational research is to identify better process models for translation (regardless of local definitions of “T” phases) and explore what measures might be most feasible, useful, and highest in quality for evaluating progress throughout these processes.

A General Framework for Translational Research and Its Evaluation

We offer here a framework that provides distinct advantages for evaluation of translational research and can either be used by itself or can be applied to current or prospective multiphase models to help translate between and among them, regardless of the number or type of “T” phases in use. The process marker approach advocated here is both an operationally precise way to structure a systematic evaluation of translational interventions and a complementary methodology that can enhance research and scholarship on the nature of various models of translational research processes that exist, perhaps in phases, at different times and places.

This framework might be termed a process marker model, characterized by the two components that constitute its name. First, it views translational research as a continuous process that moves from basic research through clinical, postclinical, and practice‐based research and ultimately to health policies, outcomes, and impacts. It assumes that this process may be bidirectional, variable, and complex and that any particular discovery may follow a unique pathway through the process. Second, it assumes that there are many different potential markers along this process. The focus in this model is on identifying a set of observable points in the process that can be operationally defined and measured, in order to enable evaluation of the duration of segments of the research‐practice continuum.

For instance, in the three‐phase Westfall et al. ² model described earlier, they describe T1 research as “translation to humans” and use as examples both case series and phase I and II clinical trials. In the model offered here, we might operationally define a process marker such as “date of first accrual of a human subject into a research protocol investigating the target treatment.” Note that the operational process marker is a specific time point in the presumed process of translation, in this case expressed as a date. It is clear what the marker refers to—the accrual of the first human subject in the first research study that employs the treatment in humans. The central point is that the operational marker is one that can be readily understood and measured, since it is based on an action or behavior.

The process marker model assumes that one would define a number of such operational markers along a presumed process continuum. Assuming that all of the markers use a common measurement scale (e.g., dates), it is then relatively easy to operationally define the difference between any two markers as the duration of time between their dates. For instance, later in their model, Westfall et al. ² define T2 as the “translation to patients” and give as examples “guideline development, meta‐analyses or systematic reviews.” In our framework, we might set up a marker such as “date of inclusion of the results of a research study of the target treatment in a meta‐analysis.” Or, even more specifically, we could define it as “date of publication of the results of a research study of the target treatment in a Cochrane Collaboration meta‐analysis.” Then, to determine the amount of time, it took to translate the research in question from the first marker to the second one would need only to subtract the two dates.

One of the obvious questions raised by this approach is what is the “correct” operational marker to use? A simple answer is that there is not a single correct marker—different markers simply represent different reference points in an assumed continuous process. That said, some markers may be better than others for different purposes and in different contexts. For example, some potential markers may be more feasible to measure. We could define an alternative marker for the introduction of a discovery in humans as “the date the first research protocol for use of the target treatment in humans was submitted for human subjects review.” It may or may not be easier to measure this than our earlier suggested marker of date of first accrual. Records of Institutional Review Board submissions may or may not be easier to obtain than dates of subject accrual, and the marker that is more feasibly measured will generally be more useful for evaluation.

Another consideration is that some markers are likely to be encountered by more protocols than others. Since protocols can take different pathways in the process of translation, one would generally want to select markers that are more likely to be commonly passed. For example, it may be that for some discoveries they typically will have research publications (date of first publication of results of a human trial) but may not generate a patentable intervention (date of submission of first patent application). In general, using markers that are more likely to be encountered by more protocols will enhance our ability to explore empirically what factors affect durations for that part of the process.

A third consideration is that there are likely to be subprocesses that get repeated throughout the overall translational process. For instance, the subprocess of conducting, replicating, and using a research study (shown in Figure 6 ) follows the same basic steps regardless of whether it is a basic, clinical, or postclinical study. In some process analyses, it might be useful to look at the durations between two steps in this subprocess, say from application to funding, across all instances, regardless of where in the translational process the study is done. In other analyses, it is may be necessary to separate the results based on whether the studies are basic, clinical, or practice based. In this critical repeating subprocess, it is important to recognize that each step has the potential to be influenced by strategies embodied in the CTSAs and other organizations involved in biomedical research.

Figure 6.

Generic subprocess of a research study.

How does the operational process marker model deal with the issue of directionality? It would be possible to define operational markers that can be encountered moving in either direction (“date a basic research study that is based on a clinical finding is initiated”). Furthermore, if between two markers the durations have great variability this might suggest that there are subprocesses and perhaps even bidirectional loops that occur and might warrant further study.

There are a number of characteristics that commend the process marker framework over multiphase models:

• 1It is pragmatic in that it avoids theoretical presumptions and undefined abstractions. It should help the field focus greater attention on how we can practically measure and improve translation.
• 2This model is an objective one in that it emphasizes observable measurable phenomena, allowing anyone to readily see how any marker is defined.
• 3The model is conceptually clear. It avoids the debates about how many phases there are in translational research, while enabling evaluators to use phased‐based approaches (and even translate between multiple models) as long as they operationally define what they mean. For instance, if one wants to classify a portfolio of research for where it is on a general multiphase model you could do so better using an operationally defined process marker approach than with any of the existing multiphase models.
• 4Because key markers are expected to be defined operationally, this model encourages replicability. It encourages the community of evaluators to look at and adopt others’ markers once they have been demonstrated to be feasible.
• 5The model is robust and forgiving. If a particular discovery does not pass a particular marker one can simply find a subsequent marker that it does hit and again pick up the trail. In other words, missing data and variable protocol pathways can be accommodated.
• 6The process marker model will encourage development of new hypotheses that involve more precise operational definitions. For example, in current CTSA pilot work on IRB processes, the first marker defined in the IRB process was the date of submission of a complete human subjects application and the end marker was the date of final approval without any contingencies. The results indicated that different centers had markedly different median durations. This led to a more refined subsequent hypothesis that some centers may do considerable work with researchers prior to submitting their IRB application. This in turn suggested that to measure this segment of the process well in subsequent evaluations we need to set up markers prior to the submission of the IRB application.
• 7The model avoids debates about the scope of translational research. For instance, there is disagreement about when translation starts. Does it begin with the genesis of the idea in basic research, with the first basic studies that anticipate human applicability, with the first study involving humans, and so on? This model is silent on such questions. The scope of translation being examined in any given process marker evaluation is simply the process that is encompassed between the first and last marker measured.
• 8The process marker model can be applied prospectively or retrospectively. For instance, we could use it to conduct historical analyses of the durations involved to translate research to practice in the same way that others have done. ² , ³ , ⁴ And it can be used prospectively when setting up evaluation monitoring systems of translation in progress.

The process marker model is firmly rooted in a process modeling research tradition in biomedical research ³ , ¹² as well as in other fields such as quality control and assurance. These traditions have begun to be applied in the context of translational research generally and in the CTSAs in particular. For instance, the authors are aware of process studies for segments of the translational process that include the time it takes to: apply for a pilot grant, accomplish an IRB review, complete contract negotiations for a research protocol, accrue subjects into existing protocols, and even the time it takes for a research publication to be included in a research synthesis. As more such studies are completed across the research‐practice continuum, we should be able to get a clearer sense of how the overall translational process occurs, where the major barriers are, and how effective different interventions are in addressing these barriers.

To help concretize the idea of a process marker model, we present an example of how it might look in Figure 7 . Before discussing it, some caveats are in order. We do not pretend to offer this as the process marker model as if there can only be one correct one. Any such model is simply an imperfect representation of some presumed underlying process. And because such a process is assumed to be continuous, it would always be possible to detail more precise subprocesses. The articulation of good process models will be an ongoing endeavor that will need to extend beyond the scope of this paper and involve a broad range of stakeholders and subject matter experts across disciplines and “translational phases.” The common value of such models is that they ultimately depend on operationally definable marker variables. If the model is wrong about the underlying process, other researchers will be able to pose alternative models while still utilizing the findings of the model they are criticizing.

Figure 7.

Examples of process marker models at three levels of scale.

Figure 7 shows an example of a multilevel process marker model. In the top, box of the figure is a very high‐level process model of the research‐practice translational continuum. On the left is the general region of basic research. The center shows the region typically associated with clinical trials research. And the right side depicts applied clinical research, translation to practice and policy, and ultimately use in populations and the health of the public. There are a multitude of operational markers that might be constructed across this continuum, and the ones included in Figure 7 are depicted primarily for illustrative purposes. Attempts to study the entire duration of translation could utilize early and late‐stage markers and follow studies throughout this course, but because of the length of time involved, this is only likely to be feasible for retrospective historical studies of the type mentioned earlier. Prospectively, the more feasible course would be to follow protocols over a comparatively short segment of the model such as illustrated in the middle of the figure in the breakout of a phase II clinical trial process. The durations for the segment described there—typical of the process sequence for almost any basic or clinical research study—could be readily tracked and estimated. By assembling multiple such pieces across the continuum, we can begin to gather contemporary data on translation that would be critical for evaluating interventions designed to reduce translational time. The phase II clinical trial breakout in turn is further specified in the bottom box of the figure in a breakout detailing the segment of the process that involves IRB approval. Again, this sequence is a general one that would apply here and in all clinical trials.

Each vertical “pin” in the figure represents an operationally definable marker. Each of these markers can be operationalized as a specific date. In the translational research continuum at the top, we identify a specific marker at the date when a phase II clinical trial began. In the more detailed breakout of this segment of the model in the middle box, we see that this is in turn operationalized as the date of first accrual of a subject into the trial. We also see that in the process segment for the phase II trial, we have a marker for IRB approval. This in turn is operationalized in the IRB approval process segment at the bottom of the figure as the date the IRB proposal was approved.

The hypothetical process marker model offered here illustrates several important features of this approach. It shows how we can evaluate translational research at any level of scale from the overarching basic research to health impacts scope to the assessment of the length of any segment or subsegment. It suggests that such studies should be a major feature of the evaluation of translational research and that it would be possible to integrate the results of many such studies at a macro or meta level, essentially stitching together duration estimates of segments to arrive at a more integrated understanding of how long the process takes.

Perhaps, the most important feature of this model is that it provides a foundation for the evaluation of interventions designed to improve translational research and the integration of these findings into a field of translational studies. Our expectation is that the CTSAs and many others will be experimenting with a wide range of interventions to enhance translation: improved efficiency of IRB and contract review processes, better clinical research management, new methods for interdisciplinary team research, and so on. The process marker model provides a common framework that can link these many and varied studies together, a common basis for assessing whether such interventions contribute to reducing time to translation.

There are some important challenges that need to be addressed with respect to the operational process marker framework. Perhaps most obvious is that it will lead to more complex and difficult to communicate models of translational research. It is much easier to describe translational research in broader terms that are not operationally defined. The more precisely stated operational definitions are cumbersome. Nevertheless, they provide distinct advantages from an evaluation and research perspective and they help to address the conceptual confusions present in the current literature.

Another major challenge is analytic in nature. Much of the process modeling literature relies upon descriptive statistics—such as median durations—as the heart of the results. But an inferential statistical analysis framework would both enable one to ask whether process changes lead to statistically significant improvements in translation and whether there are statistically significant variables that predict translational duration. Several possible analytic frameworks seem worth investigating. For instance, many of the analyses are likely to be hierarchical in nature. Research protocols may be nested within CTSA centers. Or research publications may be nested within fields or disciplines. Consequently, a general framework, such as hierarchical linear modeling, ¹³ may be applicable for testing hierarchical hypotheses using process data. Alternatively, we can view process data on durations using a survival analysis framework, ¹⁴ sometimes referred to as a Cox proportional hazard model that would enable us to provide statistical inferences regarding how long protocols “survive” or stay in various process intervals.

Increased application of an operational process marker approach to the study of translational research is likely to lead to considerable evolution and adaption over time. The field will be able to determine empirically the degree to which various markers are feasible to measure and yield results that have value for our understanding of translation. Over time, it is likely that a set of markers will emerge across the research‐practice continuum that has survived the test of repeated application. Such a set of measures would be critical to establishing a basis for the emerging field of translational research.

Conclusion

Translational research is critical to the evolution of biomedical research and practice in the 21st century. The key problems that led to its emergence—the relatively long time from discovery to use and impact—and the relatively low proportion of discoveries that survive that journey—remain a challenge. Significant investments in translational research are already being made. Certainly, the CTSAs are a major one, but it appears that many other institutions and individuals in the public and private sectors are joining in this endeavor.

There is little doubt that evaluation will be essential for managing translational research effectively, learning what works and what does not, and being accountable for these investments. However, good evaluation will depend on a deep understanding of the object that is being evaluated. The current state of conceptual models and definitions of translational research poses significant challenges to our ability to evaluate. There is considerable disagreement about many of the key characteristics associated with translational research including where it’s start and endpoints are; what is being translated; whether translational research is a bridging process or a continuous one; whether it is a multiphase process or a series of interventions and activities to encourage progress through such a process; and the number and demarcation points of any phases of the translational research.

There are, however, several definitional issues on which there does appear to be an emerging consensus. Translational research involves movement along the research‐practice continuum, ultimately to health impacts. There is a broad consensus that translational research is bidirectional with respect to this continuum with information and feedback flowing throughout. There is also agreement that translational research will often, although not invariably, involve multidisciplinarity, collaboration, and new models and modes of communication. It is clear from even this brief review that the “unit” of translation—the “thing” that is being translated—can change dramatically across the span of the research‐practice continuum. This is critical for evaluations, especially for those that seek to estimate the effects of translational interventions over time or across different parts of the continuum. The unit that you begin evaluating may shift into a different unit as you track it over time. What begins as a study in genetics may transform into a pathway of work on molecular mechanisms, a study of a new drug, a study of a variation of that drug that emerged from refinement based on interactions with clinical practitioners, guidelines based on many studies of that drug, refinements based on implementation challenges, new policies for insurance provision, and so on. This makes it extremely challenging to trace this evolution in evaluations and determine how long it took and how that process may be made more efficient.

While there is also wide acceptance that the end point of translational research is ultimately in health outcomes and impacts, many of the translational research activities that are engaged in by CTSAs and others will not directly touch on health impacts. This poses one of the most significant challenges to evaluators of translational research efforts—how do we link the many and varied translational interventions to these ultimate outcomes? This issue of causality is familiar if uncomfortable territory for researchers and evaluators alike. We know well the challenges that are involved in building causal inferences across complex dynamic chains of interventions within the context of systemic factors that also drive the outcomes we are trying to influence, even if these challenges remain daunting.

While we await a consensus on translational phase models that may or may not emerge, evaluators still need to get about their work. The operational process marker model offered here provides a way to do that. Many of the most likely markers are already familiar to evaluators: the beginning of the first clinical trial, the successful submission of a research proposal, the filing of a patent, the publication of research results, the inclusion of a study in a meta‐analysis, assessment of rates of use in populations, and so on. This is essentially the approach that was taken by researchers who empirically established the relatively long time that it takes to translate research to practice and health impacts. ³ They did not use a “phase” model but instead relied on operationally definable markers (publication of a “landmark” study; time to recognition as evidenced in >1,000 citations) to measure the time spans involved. In effect, evaluators can still accomplish much of their work while remaining either agnostic or multilingual with respect to translational “phaseology.”

We conclude with some simple suggestions for our evaluation colleagues in the CTSAs and in the field of translational research generally. Stay with the data in the form of operationally definable markers (measures) on the pathways from basic research to health impacts. If multiphase classification systems are used in evaluations, indicate clearly whose definition of phases you are employing and how the different phases are being operationalized in your context. In the meantime, we should work with the broader translational research community to collect feedback regarding our various evaluation measurement challenges. Many of our translational research colleagues are unaware of the definitional confusion and multiple conflicting models, or do not recognize the implications these have for their efforts and evaluations of them. Evaluators can play a useful role in educating the field about these issues and, in doing so, contribute to the next chapter in the evolution of translational research.