How to Build and Assess the Quality of Healthcare-Related Research Questions

Abstract

The objective of this article is to describe a simplified process for building and assessing the quality of healthcare-related research questions. This process consisted of three stages. The first stage aimed to select and explore a field of science. This field would be the area for which to identify outputs, such as units of analysis, variables, and objectives. The second stage aimed to write structured research questions, taking into account the outputs of the first stage. In general, the structure of research questions starts with interrogative adverbs (e.g., what and when), auxiliary verbs (e.g., is there and are there), or other auxiliaries (e.g., do, does, and did); followed by nouns nominalized from verbs of research objectives, such as association, correlation, influence, causation, prediction, application; research variables (e.g., risk factors, efficiency, effectiveness, and safety); and units of analysis (e.g., patients with hypertension and general hospitals). The third stage aimed to assess the quality and feasibility of the research questions against a set of criteria such as relevance, originality, generalizability, measurability, communicability, availability of resources, and ethical issues. By following the proposed simplified process, novice researchers may learn how to write structured research questions of sound scientific value.

INTRODUCTION

A critical factor for the success of research projects is selecting research questions of good quality.[1,2] Research questions are meant to address knowledge gaps and generate hypotheses (e.g., unknown and controversial knowledge). Research projects answer such questions through the application of the scientific method.

Novice researchers who are inexperienced and untrained in the steps and procedures of the scientific method may not know how to develop a research question of high value in an efficient manner.[1,3]. Even if novice researchers try to self-educate themselves on the processes of identifying research questions, they might fail to learn well because of the large body of literature published on the subject. The task of building research questions is time-consuming and challenging.

The objective of this article is to provide a simplified process aimed to build and assess the quality of healthcare-related research questions. The process is explained in three stages, step by step, for novice researchers who will design and conduct research projects on their own and publish their results in peer-reviewed journals. An overview of the process is illustrated in Figure 1.

Figure 1.

Overview of process to build and assess the quality of healthcare-related research questions.

STAGE 1: SELECTING AND EXPLORING A SCIENCE FIELD

Step 1: Selecting a science field

The task of step 1 is to select one field of science to study. A science field, simply defined, is a subject area targeted for research purposes by scientific communities. Classifications of science fields are potential sources for identifying such fields. For example, the Qatar National Research Foundation[4] classifies science into broad fields such as natural sciences, engineering and technology, medical and health sciences, agricultural sciences, social sciences, and humanities. They further classify such broad fields into fields of intermediate size. For example, they classify the medical and health sciences field into health sciences, basic medicine, clinical medicine, medical biotechnology, and other medical sciences. Finally, they classify intermediate size fields into smaller fields. For example, they classify the health science field into hospital administration, health policy, nursing, nutrition, dietetics, public health, environmental health, tropical medicine, epidemiology, occupational health, social biomedical health, and so forth.

Whether you select the science field provided by someone else or from your own knowledge, look for evidence (e.g., articles published in peer-reviewed journals) supporting its scientific legitimacy. Often, this evidence is easy to find because scientists and editors from research centers and scientific journals publish definitions and descriptions of their fields of interest on web pages, in journal articles,[5] and on research agendas.[2] If such evidence is difficult to find, then try an internet keyword query, such as the following. First, identify the keywords or phrases that you believe are names of science fields. Then, if necessary, combine your keywords or phrases with one or more of the following terms: science, research, research agenda, research center, research at World Health Organization, international forum, and journal. Second, add quotation marks around keywords or phrases and enter them into an internet search engine. Finally, review a few (e.g., 10) of the first hits of the search engine results pages, especially those that match the query phrase, looking for evidence indicating the legitimacy of the science field. For example, I identified the keywords healthcare quality and combined it with the word journal into healthcare quality journal. Then I added quotation marks in the Google search engine. The search produced 13,800 pages. I found a peer-reviewed journal on healthcare quality among the first 10 pages. With this evidence, I felt confident about the legitimacy of the science field named healthcare quality. If there are various science fields of interest, prioritize one that you believe has social relevance, feasibility (e.g., technical, material, financial), and a low risk of ethical issues for research purposes.

Step 2: Identifying units of analysis (UAs)

The two tasks of step 2 are, first, to identify, from the perspective of the science field selected at step 1, potential UAs of interest for scientific inquiry, and second, to select one or a few of them. The UA is a definable and measurable entity, either natural or social, for which projects seek to answer research questions. There are two main purposes for developing a list of UAs. The first purpose is to promote a novice researcher's awareness that within a science field there are several types of UAs. In medicine, for example, this awareness would help the researcher to consider not only patients but also other types of UAs for research projects. The second purpose is to raise awareness of the possibility of building research questions for studies that target not only one but two or more UAs, as in multilevel analysis studies. Four tips on how to develop the list of UAs are given below.

The first tip is to identify UAs based on published lists or classification systems of UAs. For example, one published list of UAs includes control projects; single patients; single clinicians; clinics; populations of patients in nursing homes, hospital wards, microsystems, or entire hospitals.[6] Babbie[7] classifies UAs into five categories: individuals, groups, organizations, social interactions, and social artifacts. Lofland[8] classifies UAs for social science fields in cultural practices, episodes, encounters, roles, social and personal relationships, groups and cliques, organizations, settlements and habitats, and subcultures and lifestyles. Table 1 lists UAs related to the field healthcare quality.

Table 1.

Example units of analysis for the field of healthcare quality

Category	Examples
Individuals	Patient with type I diabetes, patients with substance abuse disorder, and individual nurses
Groups	Nurses, doctors, quality specialists, families, and surgical teams
Organizations	Hospital networks, health centers, nursing homes, and medical schools
Social interactions	Communication, relationships, and conflicts
Social artifacts	Strategic health plans, quality policies, hospital buildings, health technologies, vaccines, drugs, and medical charts

Content based on Babbie.[7]

The second tip is to use specific phrases to distinguish between UAs. For example, the phrase individual humans includes all humans, but the phrase human with diabetes type I above 40 years old is specific enough to include only humans who meet those criteria. Other examples of criteria to distinguish between groups of humans would include a specific condition or disease such as hypertension, diabetes, dengue; behaviors such as physical activity, diet, sexual practices; functions and capabilities such as attention, memory, and judgment; and sociodemographic characteristics such as age, ethnicity, religion, and marital status.

The third tip is to list several UAs that are homogeneous, especially when there is interest in analyzing consolidated data, as in quantitative studies. A hint to help understand the homogeneous population UA would be to examine the standard “production” process of each unit. On one hand, UAs such as goods (e.g., vaccines) mass produced with standardized industrial processes are quite homogenous. On the other hand, UAs such as social constructs (e.g., neighborhoods) are usually not homogenous because they grow spontaneously by social processes, often with few limitations and flexible constraints. Neighborhoods, for example, might vary in size, social composition, economic status, and so on, even neighborhoods that have grown in the same city.

The fourth tip is to include UAs that represent single, small, and large natural or social entities to build research questions for qualitative and quantitative studies. For example, the World Health Organization is a single entity, the number of continents in the world would be a small number, whereas the number of cities in the world would be a large number. In quantitative studies, the inferential power of statistical analysis of a small sample of UAs may be compromised.

The second part of step 2 is selecting one or a few UAs to focus on developing in step 3, which is to list variables related to each UA. Prioritize UAs of social relevance, feasibility (e.g., technical and financial), and low risk of ethical issues for research purposes.

Step 3: Identifying research variables and phenomena

The task of step 3 is to create a list of variables and phenomena related to each UA selected in step 2. Variables are measurable features of UAs whose values would vary depending on changing situations. Phenomena describe the meaning, interpretation, or explanation related to variables of the UA.[9,10]

At this step, researchers should be aware of the broad array of variables and phenomena related to each UA. A research project might be conducted with one, two, or more variables, depending on what research questions the project seeks to answer. With a longer list of variables, there are more possibilities to build univariate, bivariate, and multivariate research-related questions. Table 2 illustrates an example of UAs and variables related to the field of healthcare quality. Below are three tips on how to develop a list of variables and phenomena.

Table 2.

Example units of analysis and variables for the field of healthcare quality

Units of analysis	Variables
	Characteristics	Frequencies	Factors or causes	Processes	Effects	Solutions
Patient with diabetes type I	Age and gender	Percentage of patients dissatisfied	Look-alike drugs	Self-care	Adverse events	Drugs and diets adherences
				Aging
Professional groups of nurses, doctors	Group size and average years of professional experience	Incidence of conflicts between nurses and doctors	Professional subculture	Time spent on verbal communication	Adverse events	Multiprofessional teamwork practice in health-related careers
Hospital	Score of PSC	The proportion of hospitals measuring PSC annually	Determinants of PSC	Quality improvement systems	Adverse events	Cognitive debiasing training strategy

PSC: patient safety culture.

The first tip is to search variables and phenomena, one by one, through an extensive literature search for relevant scientific publications. During your search, keep in mind that publications of quantitative approaches often use the term variable, but publications of qualitative approaches might use other terms (e.g., phenomena).[9,10] This difference might be because “qualitative research aims to understand the how and why of certain behaviors, decisions, and individual experiences”[9] and such understanding is not measured numerically.

The second tip is to identify a useful framework to classify different variables. For example, Lofland's[8] typology of questions might help to classify variables and phenomena by type, frequency, magnitude, structure, process, cause, consequence, and agency. Another classification, promoted from the field of quality management systems,[11] includes six categories of variables: physical (e.g., physical activity, safety of drugs, effectiveness of surgical procedures); sensory (e.g., vision, smell, hearing); behavioral (e.g., courtesy, honesty, veracity); temporal (e.g., punctuality, reliability, availability); ergonomic (e.g., physiological characteristics or related to human safety); and functional (e.g., speed of emergency care).

The third tip is to consider each variable, either simple or complex, as one unit. Simple variables can be measured with just one indicator (i.e., age, sex). Complex variables are usually divided into several dimensions (e.g., patient safety culture may be divided into teamwork and communication), each of which might be measured with multiple indicators. If a researcher adds complex variables and their dimensions as separate units in the same list, the total number of variables becomes inflated, which may overcomplicate the analysis.

Step 4: Identifying possible research objectives

The task of step 4 is to develop a list of possible objectives of interest for the research project. A research objective is the expected outcome that a research project seeks to achieve. Research objectives clarify whether a project seeks to describe one or more variables and phenomena or to study how two or more variables relate to each other (e.g., exposure, intervention, mediating, and outcome variables).

Objectives and research questions are closely related. Thus, developing a list of objectives would help develop a list of research questions. Objectives are often classified into generic categories, such as exploratory, descriptive, correlational, explicative, predictive, and applicative. Exploratory objectives aim to discover, understand, and characterize phenomena and their interactions with other phenomena. Descriptive objectives aim to count frequencies of variables without making comparisons. Correlational objectives aim to assess how one or more variables behave while interacting with each other. Explicative objectives aim to study cause and effect relationships among variables. Predictive objectives aim to forecast the behavior of one variable through understanding the behavior of other variables. Applicative objectives aim to study the effectiveness of interventions on changing other variables' status and behaviors. Table 3 illustrates an example of a UAs with variables, objectives, and research questions related to healthcare quality.

Table 3.

Examples of research questions related to hospitals (UA) and healthcare quality (science field)

UA	Variables	Objectives	Examples of research questions
Hospital	Dimensions of PSC	Exploratory: To identify dimensions of PSC in hospital	What are the dimensions of PSC in hospitals?
Hospital	PSC	Descriptive: To assess the hospital PSC	What are the strengths and weaknesses of PSC in hospitals?
Hospital	PSC and adverse events	Correlational: To correlate the score of PSC and incidence of adverse events in hospitals	What is the relationship between PSC and adverse events in hospitals?
Hospital	System-level factors and adverse drug events	Explicative: To analyze the system-level factors contributing to adverse drug events in hospitals	What are the system-level factors contributing to noncompliance with timely administration of drugs in hospitals?
Hospital	Team trust and willingness to discuss patient safety issues	Predictive: To determine if team trust is a predictor of the willingness to speak on patient safety issues in hospitals	Does team trust predict the willingness to speak on patient safety issues in hospitals?
Hospital	Electronic prescription and medication error rate	Applicative: To assess the effectiveness of using electronic prescriptions to reduce the medication error rate in hospitals	How can the medication error rate be reduced by using electronic prescriptions in hospitals?

UA: units of analysis; PSC: patient safety culture.

STAGE 2: WRITING STRUCTURED RESEARCH QUESTIONS

The task of the second stage is to build a list of possible research questions for the project. A research question is “a logical statement that progresses from what is known or believed to be true to that which is unknown and requires validation.”[12] A research question clarifies the specific knowledge that a research project expects to discover through the study of one or a few variables or phenomena related to one or a few UAs.

In general, the structure of research questions would include interrogative adverbs (e.g., what, why, who, when, and where), auxiliary verbs (e.g., is there and are there), or other auxiliaries (e.g., do, does, and did); nouns nominalized from verbs of research objectives, such as association, correlation, causation, prediction, and application; research variables (e.g., incidence, prevalence, risk factors, causes, effects, and interventions); and UAs (e.g., patients with hypertension, mothers, pregnant women with diabetes, and general hospitals). Research questions related to prognosis and relationships between interventions and outcomes would be structured in formats such as those outlined in Table 4.[1,3,9,10]

Table 4.

Examples of framework used to write research questions

Acronym	Elements of the framework
PICO	Population, Intervention, Comparison, Outcome
PICOS	Patient population or problem, Intervention (treatment or test), Comparison (group or treatment), Outcomes, and Setting or study type
PICOT	Patient population of interest, Intervention or issue of interest, Comparison with another intervention or issue, Outcome of interest and Time frame
PESICO	Person, Environment, Stakeholders, Intervention, Comparison, Outcomes,
SPICE	Setting, Perspective, Intervention or Exposure or Interest, Comparison, Evaluation
SPIDER	Sample, Phenomenon of Interest, Design, Evaluation, Research Type

Content based on Fandino,[1] Thabane et al,[3] Cooke et al,[9] and Cañón and Buitrago-Gómez.[10]

To write structured research questions, use the outputs of the first stage (e.g., UA list, variables, and objectives) and general or specific formats for structuring research questions. Table 3 includes examples of research questions.

STAGE 3: ASSESSING AND SELECTING RESEARCH QUESTIONS

Step 1: Assessing the quality and feasibility of a research question

The task of this step is to assess the quality of the research question against a set of criteria, such as those shown in Figure 2. A more detailed discussion on such criteria is provided elsewhere.[2,3,13,14] Other criteria to assess the quality of research questions include answerability, effectiveness, innovativeness, implementation, burden reduction, and equity.,[2] Researchers may also use criteria based on the acronyms, FINER (feasible, interesting, novel, ethical, and relevant)[1,3] and I-SMART (important, specific, measurable, achievable, relevant, and timely).[14] Rank the quality and feasibility of the criteria using a scale of 1–10 points (i.e., 10 being highest quality and 1 being lowest). Remember to ensure that each criterion is scored using the same scale. Finally, calculate the total score for the research question.

Figure 2.

Examples of criteria used to assess the quality of research questions.

Step 2: Selecting the research question

Select the research questions that score highest in the quality assessment from the previous step. Avoid selecting high-risk ethical questions, regardless of other criteria that had high scores. Finally, if answering a research question is not feasible (e.g., too expensive or complex) with the available resources, then that question might be kept on standby until the circumstances for feasibility improve.

SUMMARY

Building a research question of high quality just by guessing is unlikely, especially for novice researchers who do not have any training in the scientific method.

The proposed simplified process aimed to help novice researchers write structured, high quality research questions. The key concepts and framework of the scientific method presented illustrate the myriad of questions that researchers can develop in any field of science. The importance of using a well-known set of criteria to assess the quality of research questions was highlighted. This simplified process teaches novice researchers how to build research questions of sound scientific value in a systematic way.

Funding Statement

Source of Support: None.