Articulating the “So, What?” in Clinical Research: Insight from the M-CHOIR Group

Summary:

With the academic culture of “publish or perish,” authors must ensure that they are delivering high-quality data with a meaningful impact on clinical practice. Even for physician–scientists at the top of their fields, establishing the relevance of a study to clinical practice is a challenge. Thus, it is essential that research proposals ask questions that are clinically important, use appropriate methodologies, and examine outcomes that are relevant to both the physicians and the patients. The question of “so, what?” or in other words, “who cares?” is one that can make or break a study’s impact on clinical practice. Researchers should use models such as PICOS (Population, Intervention, Comparison, Outcomes, and Study design) and FINER (Feasible, Interesting, Novel, Ethical, Relevant) and ask why readers will care about their study’s findings before the study is conducted. By doing so, researchers can ensure the successful execution of their study and a meaningful impact of their findings, in both academia and clinical practice. This Special Topic article aims to guide researchers in producing relevant, impactful conclusions of their studies by providing input and resources from the Michigan Center for Hand Outcomes and Innovation (M-CHOIR) group.

INTRODUCTION

A cost-analysis of the various types of implants for proximal interphalangeal (PIP) joint arthroplasty, particularly the costs associated with complications and recurrent surgeries.

—An idea proposed by the M-CHOIR team members.

At first glance, this project is seemingly a good research question with an obvious cohort of patients, a clear objective in mind, and a vision of how it will be executed. However, a thorough breakdown of this research question demonstrates a clear lack in the understanding and consideration of the basic elements of research that separate the realistic, intriguing research questions from the fantasy world (Fig. 1). First, a cost-analysis is not a recognized term in economic analysis. The researchers may have wanted to conduct a cost-effectiveness or cost-benefit analysis, but the identification of this study as a cost-analysis discredits the researcher and demonstrates to the journal editors, reviewers, and other colleagues that the authors are not familiar with economic modeling. Second, there is no database that will show the complications associated with proximal interphalangeal (PIP) joint arthroplasty, which hinders the feasibility of this study. Most large databases are administrative in nature for billing purposes and are devoid of accurate clinical information. Finally (and considerably, the most important critique of this research question), it is well known that complications and recurrent surgeries will cost more, regardless of the condition, treatment, or patient.^1,2 Thus, researchers are challenged with the most critical questions: “Do I need to conduct this study when the answer is obvious? How will this excite anyone? So, what?”

Fig. 1.

The breakdown of a failed research proposal.

“SO, WHAT?”

Eighty-five percent of medical research is wasted.³ As research waste remains an issue in the health sciences, there are several initiatives to promote quality over quantity.⁴ In the past few decades, the number of publications in the plastic surgery literature has more than doubled.⁵ Although the quantity of plastic surgery publications has doubled, the quality has not changed.^6,7 For instance, a citation analysis of the top 50 cited articles in plastic surgery showed no correlation between level of evidence and number of citations.⁶ The authors found that >80% of the articles are considered as level IV or V evidence and that no studies of level I or II evidence are present in the top 50.⁶ With an increase in the number of publications, researchers must understand that the relevancy, influence, and overall quality of each article are important factors to consider. Researchers in the health sciences can ensure the delivery of high-quality research by optimizing methodology, generalizability, and relevance, among other factors. Developing a research project with an effective methodology, meaningful results, and an impactful conclusion is not an easy task. In particular, executing a project with meaningful results that can be translated into clinical practice is a challenge among many physicians. Bridging the gap between clinical research and clinical practice is a challenge seen in many specialties, not just in plastic surgery.⁸ In fact, the American Psychological Association launched a special section series in Psychotherapy to provide strategies to integrate research into clinical practice.⁸ As physicians, the responsibility lies in searching for evidence and applying it to their practice. As researchers, one must provide evidence that is not only applicable to practice, but feasible and relevant to patients. Thus, researchers must ask themselves constantly, “So, what? Who cares?” and be able to provide the answers to these 2 questions to ensure successful study execution.

A challenge that researchers may face is the ability to identify the “relevance, significance, and wider value of their writing.”⁹ Certainly, this is a challenge for the members of the Michigan Center for Hand Outcomes and Innovation Research (M-CHOIR) group. As the research arm of the University of Michigan Comprehensive Hand Center, M-CHOIR aims to conduct clinical and health services research that will advance the field of hand surgery with a comprehensive application of the full spectrum of research designs, including decision analyses, large database studies, and clinical trials. In the past year alone, M-CHOIR has published over 45 articles in 14 journals, including JAMA Surgery and JAMA Network Open, and received a National Institutes of Health (NIH)-funded U01 grant for a multicenter randomized clinical trial. With an aspirational value to deliver high-quality research efficiently and promptly, the members of M-CHOIR implement different strategies to ensure execution of impactful studies. The M-CHOIR team comprised researchers from varying levels and phases in career, including undergraduate students to senior surgeons. Thus, the team members may experience challenges in research design and implementation, especially as junior members learn how to formulate and execute clinical research studies that are impactful and novel. We have strategized and exercised different methods to overcome these challenges. In this Special Topic article, we offer those strategies and provide several examples in (1) drafting a “first page”; (2) critically asking evaluative questions; and (3) devising relevance from the published literature to answer a vital question in clinical research: “So, what?”

CONCEPTUAL MODELS

Conceptual models provide a visual representation of theoretical constructs or variables, which can be translated into action. In clinical research, conceptual frameworks can be used to generate a proposal that ensures the successful execution of the project. For example, Sterbenz et al¹⁰ developed a conceptual model depicting an approach to develop strategies for an effective team. In this model, the authors guide team members in how to strategize and execute a plan to reach a shared goal, leading to desired outcomes.¹⁰ To develop an effective research proposal, researchers must think along the theories of this conceptual framework and ask themselves, “Why?” In the book Start with Why, the New York Times bestselling author Simon Sinek introduces the idea of the “Golden Circle” to represent the “naturally occurring pattern, grounded in the biology of human decision making.”^11,12 This concept emphasizes the importance of the “why” before considering the “how” and “what.” Similarly, in clinical research, researchers must know the purpose of their research question before considering how the study will be implemented or what it will do. Following this idea, M-CHOIR uses 2 models to detail a study plan: FINER (Feasible, Interesting, Novel, Ethical, Relevant) and PICOS (Population, Intervention, Comparison, Outcomes, and Study design).^13–15 Implementing these 2 conceptual models ensures that the 5 Ws (Who? What? When? Where? Why?) and How are answered.

FINER

Following Sinek’s idea of the “Golden Circle,” the framework FINER encourages researchers to consider the “why” of their research proposal.¹² Hulley et al¹³ introduced the concept of FINER¸ which ensures that a study idea is Feasible, Interesting, Novel, Ethical, and Relevant (Fig. 2). First, feasibility considers the adequacy of participants, technical expertise, affordability in time and money, and manageability.¹⁶ This is an important aspect to consider in larger clinical trials, such as randomized controlled trials. Pilot and feasibility studies can be used to “test-drive” the methodology of a study, identifying any issues or areas of weakness before launching the clinical trial.¹⁷ Second, a study that is interesting and novel ensures that the study’s results will add new insight to the scientific literature and be intriguing to the researchers.¹⁶ Third, ethical studies guarantee that they will follow institutional review board guidelines and national policies to ensure the optimal safety of study participants.¹⁶ Finally, developing an interesting, novel research question is not a challenge because most researchers pursue topics in which they have a natural interest. However, the challenge for most researchers comes in establishing the last element of FINER: relevancy.¹⁶

Fig. 2.

The FINER model.

PICOS

Whereas the FINER framework depicts the development and purpose of a proposal, PICOS provides the answers to the “how” and “what.”^12,14 Population, Intervention, Comparison, Outcomes, and Study design are the 5 main categories within PICOS.^14,15 Population refers to the patients who are being studied or the condition that is being observed, clearly indicating the eligibility criteria for the study.^14,15 Whereas the intervention is the new treatment, therapy, or surgery that is being tested, the comparison is the treatment, therapy, or surgery that serves as the “control” group.^14,15 Furthermore, the outcomes are the endpoints that are used to measure the effectiveness of the different treatment groups.^14,15 Finally, the study design includes the methodology and the statistical design, which will determine the level of evidence of the project.^14,15 Figure 3 provides additional details about the PICOS model. A viable research question should address all components of PICOS. For instance, Figure 4 displays the impact statement from an article published by M-CHOIR, identifying the 5 elements of PICOS.¹⁸

Fig. 3.

The PICOS model.

Fig. 4.

Example of research question emphasizing PICOS and clinical relevance. *Saito et al¹⁸.

MINIMIZING “RESEARCH WASTE” AND ESTABLISHING CLINICAL RELEVANCE

The goal of clinical research is to improve the delivery of health care to patients.¹⁹ Thus, researchers must address questions that are prioritized by the health care community and examine outcomes that are important to both physicians and patients.³ However, clinical trials may be subject to “research waste” by inadequate planning and incomplete reporting of outcomes.²⁰ “Research waste” can be seen in various ways throughout the proposal and execution of a project. For instance, Chalmers and Glasziou³ identify 4 successive stages of research that are subject to waste: the research question, design and methodology, accessibility of publication, and the quality of research reporting. As professionals who prioritize patient-centered care, researchers should aim to include the input or consideration of patients when developing their research protocol. Figure 5 displays the steps that should be taken to ensure a successful study execution.

Fig. 5.

Steps to execute a successful research proposal.

Relevancy is established by considering the impact of the study’s findings to scientific knowledge, clinical and health policy, and future research.¹⁶ The NIH prioritize 3 arms of research: health, society, and scientific knowledge. Their website states “…[The] NIH improve health by promoting treatment and prevention, contribute to society by driving economic growth and productivity, and expand the biomedical knowledge base by funding cutting-edge research and cultivating the biomedical workforce of today and tomorrow.”²¹ As the largest funder of biomedical research, this vision should be embodied by researchers in the health sciences to ensure that their study is relevant to health care professionals, patients, and scientists and establishes impactful conclusions.²² Cruz Rivera et al²³ define research impact as “any identifiable benefit to, or positive influence on, the economy, society, public policy or services, health, the environment, quality of life, or academia.” Authors should be able to apply their study’s empirical findings to a more general idea, providing insight into how their results could make a broader, societal influence. By doing so, authors can minimize research waste to ensure real improvements in health care practice.²⁴ Saito et al¹⁸ establish relevancy in their research question by emphasizing the impact that their study will have on clinical practice: “…to guide clinicians to better understand which types of drugs are most useful for preventing subsequent fractures” (Fig. 4). Stating this impact directly provides readers with a clear understanding of the societal effects that the study will have.

THE “FIRST PAGE”

Before the start of any research project, the members of M-CHOIR create a “first page” of a NIH grant, whether additional funding is needed or not. In a previous publication from the M-CHOIR group, Sterbenz et al¹⁰ provide insight into the purpose of a “first page,” which is to establish relevancy, define project aims, provide hypotheses, and state any projected conclusions or impact of the study.¹⁰ Sterbenz et al¹⁰ provide an example to show readers how the “first page” should be formatted: (1) an introductory paragraph discussing what is known, any gaps in knowledge, and the proposed project; (2) a section identifying the aims and hypotheses; and (3) a final section stating any conclusion and impact statements.¹⁰ As seen in the requirements of a “first page,” the members of M-CHOIR are challenged to state the projected conclusion and impact statements of their proposed study before execution. Over the past few years, the expectations of the “first page” have changed to represent several pages of justification, establishing relevance, and considering the impact and implications of the projects. As seen in Figure 6, researchers are entrusted to consider the current state and gap in knowledge, the relevance and impact, the methodology, the potential limitations and biases, and the main take-home points and implications. This exercise informs researchers to practice writing effective and successful grant proposals. However, it also motivates members of the team to think about the clinical relevance of the study before it is executed. By doing so, researchers are keeping in mind the “So, what?” before the study even begins. Some of the questions that should be considered when establishing the “So, what” are outlined in Figure 7.

Fig. 6.

Example of a “First Page.”

Fig. 7.

Questions to consider while writing the “First Page.”

What Is the Current State of Knowledge? What Is the Gap in Knowledge?

The answer to these questions requires a reflective evaluation of what is known in clinical practice and an extensive search of the literature.²⁵ Without a thorough search, researchers may fail to recognize studies that have already been conducted and published with the same research question. Furthermore, a search of the current evidence may guide researchers in formulating their study question and choosing the appropriate methodology. As a growing field, plastic surgery researchers should find the areas in which there lacks evidence and data. For instance, Waljee et al²⁶ conducted a national, population-based study to examine the utilization of opioid analgesics following common upper extremity surgery. The authors report that there is an abundance of research regarding the prevalence of opioid use among patients undergoing orthopedic surgery.²⁶ However, there is little evidence regarding the use of opioids after ambulatory services, such as hand surgery.²⁶ In the introduction alone, Waljee et al²⁶ cite 23 different references to allude to the evidence that exists in the current literature.²⁶ In addition, Fracol et al²⁷ evaluated women after subpectoral implant-based breast reconstruction. The authors present evidence that the use of this particular surgery is growing, but the evidence regarding patient-reported outcomes is scarce. Furthermore, the authors provide 2 studies that have examined patient-reported outcomes, but state that there are no outcomes studies assessing patient satisfaction with validated questionnaires. By explicitly stating the gaps in knowledge, the authors assured readers of the relevance and need for their study.

How Will We Contribute to the Gap in Knowledge? Why Will Readers Care about Our Findings?

The methodology and outcomes of a study are important elements of the “So, what?” to establish relevancy to the health care community. For instance, the treatment options for PIP joint osteoarthritis have changed in the past few years.²⁸ With multiple treatment options emerging, there are a variety of factors that can be examined, such as patient preference, cost, and postoperative complications, among others. However, researchers should be cautious in designing research questions with outcomes that may not be of major importance to the readers and ask themselves, “What factors do patients and physicians care about?”

A reflection of what is seen in everyday clinical practice is necessary to inspire the questions that are of high clinical importance.³ For instance, procedures such as breast reconstruction may be costly and require a longer inpatient stay than other plastic surgery procedures. Typically, cost and outcomes can be considered the top 2 priorities of patients. Thus, a cost–utility analysis of inpatient flap monitoring is helpful to understand the point at which the costs start to outweigh the outcomes.²⁹ However, other procedures such as PIP joint surgery may not need this type of analysis because it is not costly enough to be a concern for patients. Thus, researchers should consider other factors and outcomes of the surgery that may be of interest to patients. For example, Harris et al³⁰ conducted a conjoint analysis of patient preferences in PIP joint surgery. Whereas cost may not be a factor in a patient’s decision, other factors such as joint mobilization, reoperations, and strength may strongly influence a patient’s desire for one treatment over another. Although the study’s empirical findings are important, the clinical implications of the study are what contribute to the gap in knowledge. Surgeons can use the findings of this conjoint analysis to gain insight into how osteoarthritis patients make decisions regarding their treatment options. Although physicians may rely on their own opinions and experience with patients, the authors are able to provide real evidence to augment the clinical expertise of the physicians.

USING THE LITERATURE

Establishing the “So, what?” of a research project is certainly a challenge, but there are various sources that may assist researchers in this rigorous process. A source of evidence to not only guide clinical practice but inspire future research is systematic reviews and meta-analyses.³¹ Particularly in clinical research, systematic reviews and meta-analyses provide the highest quality evidence for decision-making.³² Systematic reviews answer a clear, clinically relevant research question and conduct a systematic, thorough search of the literature to answer that specific question.³² Furthermore, they provide a synthesis of the current evidence in the literature, interpret those findings, and address the strengths or weaknesses of those findings.³² By providing this analysis, researchers could use the results of systematic reviews and meta-analyses to identify knowledge gaps in the literature clearly and precisely. For example, Chan et al³³ conducted a systematic review to compare the outcomes of silicone and pyrocarbon arthroplasties for patients with PIP joint arthritis. The authors were able to synthesize the literature to conclude that pyrocarbon arthroplasty does not demonstrate any clear benefits over silicone implants. However, the authors are also able to demonstrate the lack of evidence in the literature that examined the quality-of-life outcomes of patients who undergo PIP joint arthroplasty, particularly using validated measurement tools.³³ Thus, the authors are able to not only answer their primary research question with a systematic search of the literature but they are able to identify clear gaps in the literature regarding these treatments.³³

In addition to the use of systematic reviews, the M-CHOIR group has conducted root cause analyses to understand the cause of a failed proposal on Dupuytren contracture.³⁴ The authors used the 5-whys method, which forced researchers to ask themselves the question “why” at least 5 times to assess the cause and effect relationship of a failed study proposal.³⁴ The authors identified a few take-away points from this exercise: (1) the possible conclusions derived from the analysis will not differ; (2) previous studies have conclusions that provide more insight; (3) there is insufficient depth of analysis and clinical details; and (4) there is a lack of knowledge about the limitations of the dataset.³⁴ By using a root cause analysis, researchers may prevent the failure of projects before time, money, and resources are wasted to conduct the study.³⁴

CONCLUSIONS

With the push for evidence-based medicine, research proposals should ask important clinical questions using the appropriate methodology and relevant outcomes. However, researchers should take caution in not only considering the clinical questions that may be of interest to them but those that are relevant to their colleagues, patients, and other health care professionals. Using models, such as PICOS and FINER, may seem elementary in practice but they are essential in developing a research proposal. Applying the correct methodology, choosing the relevant outcomes, and establishing the study’s relevance to clinical practice, health policy, and academia are not easy challenges to overcome. Considering the broader sense of why, they are asking the question that is being asked and why it is relevant will be essential in translating a study’s findings to clinical practice. “So, what? Who cares?”…the constant presence of these questions will drive a proposal through to successful execution.