Back to the basics: guidance for formulating good research questions

Abstract

Good science is driven by rigorous questions. Much like the foundation of a house, a research question must be carefully constructed to prevent downstream problems in project execution. And yet, pharmacy researchers and scholars across all career stages may find themselves struggling. The purpose of this commentary is to provide useful guidance on composing and evaluating rigorous research questions. A variety of frameworks, such as PICO (Patient/population; Intervention; Comparison; Outcome), are available to researchers and can assist them in ensuring that their research question has covered all relevant components. Additionally, the FINER (Feasible; Interesting; Novel; Ethical; and Relevant) criteria can help researchers with evaluating their research questions for practical considerations. Finally, building awareness of common pitfalls when composing research questions can aid researchers to avoid issues that they may not otherwise discover until their manuscript undergoes peer review.

Background

The beginning of a research project often brings excitement in the opportunity to explore something new. For many, this can be accompanied by nervousness in getting started and the project unknowns lying in the road ahead. Early career researchers are building confidence in their skills and learning how to navigate the process, but even experienced researchers recognize that every project will challenge them with unique and new problems to solve. Everyone is impacted by the external pressures of science, including limits in time, funding, and resources, as well as institutional and professional pressures to “publish or perish.” In some cases, this can lead researchers not giving the early stages of the research process the time and effort they require. One place that researchers may neglect to think carefully or deliberately about is the creation of a research question.

Just like a house is composed of walls, floors, and ceilings, scientific inquiry is composed of different essential elements. A research question serves as the foundation a research project is built upon. It is therefore important to ensure that the question is well-designed and rigorously crafted, otherwise errors made at this stage will impact the design, interpretation, value, and ultimately the ability of the study to be published; this is not unlike failing to build a house to code, leading to challenges later on for living in or selling the investment. To prevent wasted time and effort, investing fully in the creation of rigorous research questions is key. As such, the purpose of this commentary is to provide useful guidance on composing and evaluating rigorous research questions.

A framework for formulating research questions

Although every research project is unique, they share common domains that a researcher should consider and define a priori. Frameworks for developing research questions aim to help researchers ensure that they are contemplating all the relevant domains in the design of their project. The most common framework used is PICO (or one of several similar variations), which refers to Patient/population, Intervention, Comparison, and Outcome.^1,2 Consideration of each of these components is important for the composition of a research question, but also for prompting discussions about other study design decisions that should occur in the early planning phase of the research process.

The first component (P) focuses on the subject of the research itself. In clinical research, this is a specific patient group, but in epidemiological research, it may be a population of interest. Within this component, researchers need to define who they plan to study, considering relevant baseline and clinical characteristics such as age, sex/gender, race/ethnicity, medical condition, or disease severity. Defining these characteristics not only helps to compose the research question, but also aids discussions about the inclusion/exclusion criteria, as well as potential selection bias and ways to lessen its impact. It also prompts consideration of the sampling method, the setting in which the study will be conducted (e.g., inpatient; ambulatory care; community; academia), and how the participants will be recruited.

The second component (I) refers to the intervention, process or action being tested. Although this part of the framework is defined with an experimental design in mind, one should also consider the relevant exposures of interest in this component when designing observational research. For qualitative studies, consider what specific phenomenon you want to study, which can include domains such as behavior, attitude, practice, experience, culture, decision, perspective, or beliefs. For this component, researchers must clearly define what they are studying and how they plan to study it, including both intrinsic (e.g., medication type, dose, and route; service provided; etc.) and contextual factors (e.g., patient education; training of involved parties; workflow integration; etc.). Defining this component of the research question will enable an overall description of the study’s specific methodology (e.g., interview; survey; cohort; case-control; pre-post; etc.), whether descriptive or evaluating correlation or causation.

The third component (C) refers to what the intervention is being compared against, primarily in the setting of an experimental design. In this case, researchers should consider if a control (e.g., active; sham; historical; usual care; etc.) will or should be used, and what further design choices will accompany this, such as masking/blinding. It is important that the details of comparators are specified as closely as are the interventions of interest. Alongside this component, a discussion of potential confounders that may influence the study (and how they should be measured and handled) can occur.

The fourth component (O) is ultimately what the researchers hope to learn in the study. In quantitative studies, this is some measure of an effect, whereas in qualitative studies this is usually a description of the phenomenon of interest (e.g., attitude toward something, experience of something, how a strategy is implemented). In quantitative studies, a researcher should aim to define primary and secondary outcomes, considering relevant economic, clinical, or humanistic domains. Additionally, studies of interventions should consider both potential beneficial outcomes as well as potential harms. It is also important to consider the temporal aspects of the study, including the schedule of intervention itself, as well as the duration of the evaluation, which each have implications upon outcome development, recall bias, attrition bias, and drop-outs. Delineation of this component feeds in other design decisions, including ascertainment of sample size, and pre-planning of subgroup and statistical analyses.

Table 1 provides an example of how the PICO framework can be used to map development of a research question. In addition to making sure that each component is contained with the question itself, researchers should recognize that fulfillment of each component occurs on a scale of detail. Iteration of a research question through self-evaluation and peer feedback can incrementally improve upon specification of a research question’s components and important aspects within each domain.

Table 1.

Definitions and examples using the PICO framework

Component	Definition	Examples
		Good	Better	Best
Patient/population	The subject(s) of interest	Patients with DM	Adult patients (18–64 years old) with type II DM	Adult patients (18–64 years old) with uncontrolled type II DM (A1c <7%) seen in the primary care setting
Intervention	The action/exposure being studied	Patient disease education and management	Patients receiving pharmacist-led education on lifestyle and medication optimization	Patients receiving three visits over a 12-month period with an ambulatory care pharmacist providing nutritional and activity guidance and medication adjustments via a collaborative care agreement
Comparison	The alternative action/exposure to measure against	Patients receiving usual care	Patient receiving visits with PCP	Patient receiving three visits with PCP over a 12-month period
Outcome	The effect being evaluated	Change in blood glucose control	Change in A1c and FBG	% change in A1c and FBG from baseline at 3-, 6, and 12-months

% = percentage; A1c = glycated hemoglobin; DM = diabetes mellitus; FBG = fasting blood glucose; PCP = primary care physician

It should be noted that several frameworks exist beyond PICO, albeit with similar components.³ Researchers engaging in qualitative and mixed-methods research may gravitate toward the SPIDER framework (Sample; Phenomenon of interest; Design; Evaluation; Research type).⁴ This framework was originally developed to help identify studies for systematic reviews, but can also be used to develop primary research questions. For public health researchers focused on evaluations of projects, services, or policies, the SPICE (Setting; Perspective; Intervention/interest/exposure; Comparison; Evaluation)⁵ or ECLIPSE (Expectation; Client group; Location; Impact; Professionals; Service)⁶ frameworks may be valuable approaches. Regardless of the framework a researcher chooses, each function to encourage researchers to fully delineate the “who,” “what,” “why,” and “how” of their project before they get started.

Criteria for evaluating research questions

Beyond being well-constructed with the proper components, a good research question should be capable of producing results that are valuable, useful, and achievable. The FINER criteria, defined as Feasible; Interesting; Novel; Ethical; and Relevant, is a tool that can be used to critically appraise research questions.^7–9

The first component (F) addresses the practicality of the research question. When planning, it is important to consider the availability of resources for a given study. Such aspects may include funding, time, institutional support, data or participant availability, and required personnel and expertise. This recognizes that a research question can be rigorously constructed, but sometimes not possible within the constraints of a particular researcher’s environment. Ultimately, planning a feasible research study increases the likelihood of the project completion.

The second component (I) focuses on the appeal the research question has on a larger scale. Given the aforementioned resources needed to successfully bring a project into fruition, it is imperative that the researcher is dedicated to the work. In addition to a researcher having personal engagement in their topic, it is also essential to evaluate the interest of the project to the wider scientific community. This can be achieved through discussions with peers/mentors, assessing emerging literature, and reviewing funding agency priorities. A research question with high levels of interest is ultimately more competitive for funding and publication.

The third component (N) ensures that the research question furthers knowledge in the current field, and fills a clear knowledge gap. The most important way to evaluate novelty is by conducting a rigorous literature review to evaluate available data for what is known and not known. This allows for researchers to decide on the general approach of their work, which might include designing to: (1) improve upon limitations in previous studies, (2) investigate unknown areas that other studies have not yet addressed, or (3) purposefully replicate existing work to validate their conclusions. Research questions that are novel are more likely to move the field forward, and researchers will likely encounter less problems during peer review.

The fourth component (E) acknowledges the ethical implications of many types of research and the need to consider the risk of harm to potential participants. Engaging with appropriate oversight in the early stages of research can assist with the conceptualization of the project, especially in projects which may navigate grey areas or those with unique considerations (e.g., embryonic stem cells, artificial intelligence, etc.). Researchers should be acquainted with whether their research requires review by an institutional review board (for human subjects), or is subject to other procedures and protocols. Beyond official approvals, researchers should also engage in conversations regarding specific vulnerabilities in their subject area and how their research can uphold the safety and privacy of the subjects of study.

The final component (R) focuses on the impact the research will have on present and future society. It is essential for a researcher to consider the greater purpose of the research question and how it advances our communities and world. While research may be pursued sometimes purely for academic interest, a larger duty to society to create positive impact, through knowledge generation or practice advances, should be an ever-present consideration. One way to achieve this level of applicability is engagement with stakeholders. Not all research is important to everyone, so reflecting on the needs of end-users of your work can have a positive influence research design and enable integration of the work into future decision-making. For instance, tools provided by field of implementation science can provide a guide for thinking through how research may be translated into practice and what can be done to facilitate that transition. Ultimately, critically evaluating the relevance of your work ensures that investments in science truly make a difference.

Table 2 provides a description of the FINER criteria as well as guiding questions that a researcher may use to self-assess their research question. Diligent use of this criteria often causes a researcher to go back to the PICO ‘drawing board’ to re-formulate their question. When FINER is used together with PICO, there is increased likelihood that the resulting research question will be structurally sound and practically achievable to move forward.

Table 2.

Definitions and self-assessment using the FINER criteria

Component	Definition	Self-assessment questions
Feasible	Can the research question be answered given the available resources? Consider required materials for success, including time, expertise, funding, or data availability.	Do you have the items needed for the research? Do you have the skills to execute the research? Is the planned timeframe appropriate for the work?
Interesting	Is the research question exciting to you and the scientific community? Objectively evaluate the external value of the research through peer interest, discussion in scientific publications, and funding agency calls for proposals.	Does your project align with the funding agency? Are you personally interested/engaged in your project? Is the project of interest to the broader scientific field?
Novel	Does the research question seek to fill an existing gap in knowledge? Conduct a thorough literature review to understand existing research, identifying what is known and not yet known in an area of science.	What is known and what is unknown? What are limitations in existing research in the field? Has it been done before?
Ethical	Does the research question protect/respect the subjects of interest? Evaluate need for guidance from an institutional review board or other standard (e.g., animal care, radioactive substances, etc.).	Does your research minimize potential harm? Are you working with a vulnerable population or researching an area with special ethical concerns? Have you engaged in ethical oversight processes?
Relevant	Will the research question lead to societal improvements? Consider the larger purpose of the work and how it is meant to improve our communities and world beyond academic interest.	What is the potential impact of your research? What are the needs of the stakeholders? Are you producing value for money?

Common pitfalls when developing research questions

There are several pitfalls for researchers to be mindful of when creating a research question, many of which can be tied back to different areas of the PICO framework and FINER criteria.

Many researchers start with a question that is either too narrow or too broad. If a question is too narrow, it may limit the external validity of the study and broader interest in your results, generating difficulty when seeking publication and dissemination. Alternatively, if a question is too broad, it may impact the internal validity and the subsequent relevance of the results by rendering unclear conclusions. Being that crafting a question with good balance is challenging, it should be done through iteration, driven by feedback from peers/mentors, existing literature, and reflection. For example, consider a study regarding cancer. When reflecting on the patients/population, are there important subgroups that would introduce significant heterogeneity (specific cancer type)? Are different patient subgroups of interest to you, or has one been thoroughly studied while others have not (early-stage vs late-stage)? Would subgroup choice create feasibility challenges (finding early-stage cases to sample when most patients are diagnosed late-stage), ethical concerns (requiring standard of care as an active control vs using a placebo), or otherwise impact the relevance of the study? Alternatively, the sample may be so circumscribed that it would be difficult to recruit for and not be widely generalizable. Critically considering questions like this can help researchers scope their questions appropriately.

A second pitfall can occur when researchers poorly specify the individual components of their research question, which can lead to problems during the conduct of the study. Even when the research question is appropriately scoped at a high-level, each individual component needs to be specified to remove ambiguity. Failure to do so can impact the reliability of the study if research team members, peers, and patients interpret the components of the question differently. Think about a study focused on hypertension. It is unlikely that the researcher is interested in all patients with hypertension, so how will the researchers ensure patients are evaluated according to inclusion/exclusion criteria (an intake interview with the research team, patients filling out an electronic questionnaire, medical record verification)? When considering the intervention, what deviations from protocol are acceptable versus not (lack of adherence, dose escalations, pauses in treatment due to tolerability)? If the outcome of the study is a change in blood pressure from baseline, how will this be determined (measurement by a healthcare professional or based on patient self-report)? Accordingly, detail in individual research question components is just as important as overall scope.

Finally, researchers may fail to consider the importance of coherence across different aspects of the research question. Although the PICO framework asks the researcher to individually consider each component, it is imperative that alignment exists among them. Some researchers may design and fully execute a study without full and clear specification of the research question, and only finally doing so at the time of writing the manuscript. In such cases, the research question can read like an afterthought when it fails to effectively connect the different elements of the work. A final example may be a study about attention deficit hyperactivity disorder (ADHD). What the researcher chooses for the methodological design ultimately will affect the choice of outcome and type of conclusions that be reached (an experimental study to assess stimulant efficacy vs a qualitative study to explore patient experiences with stimulant shortages). Similarly, which patient population the researcher chooses will influence the optimal sampling strategy (adults vs children with ADHD, sourced from primary care vs psychiatry). Additionally, the chosen intervention will heavily relate to the appropriate timeframe for assessment (time for medication titration and efficacy, scheduling a patient interview during optimal focus time). As such, after ‘zooming in’ to specific each component, one should ‘zoom out’ to ensure the path they forge is coherent.

Conclusion

For early career and experienced researchers alike, foundational investment in a well-constructed and rigorous research question pays dividends later on the scientific process. The PICO framework and the FINER criteria are powerful tools available to researchers for them to invest in during the early phases of research. The effort and time spent in this planning will allow researchers to avoid a number of downstream problems in research and publication, avoiding common pitfalls that many researchers find themselves dealing with.

ABBREVIATIONS:

CENTER

Creating an Educational Nexus for Training in Experimental Rigor

ECLIPSE

Expectation; Client group; Location; Impact; Professionals; Service

FINER

Feasible; Interesting; Novel; Ethical; and Relevant

METER

Materials to Enhance Training in Experimental Rigor

NIH

National Institutes of Health

NINDS

National Institute of Neurological Disorders and Stroke

PICO

Patient/population; Intervention; Comparison; Outcome

SPICE

Setting; Perspective; Intervention/interest/exposure; Comparison; Evaluation

SPIDER

Sample; Phenomenon of interest; Design; Evaluation; Research type