Table 2.

Transdisciplinary research quality assessment framework (adapted from [10]).

Relevance: The importance, significance, and usefulness of the research problem(s), objectives, processes, and findings to the problem context.
Criteria	Definition	Observations
Clearly defined problem context1	The context is well defined, described, and analyzed sufficiently to identify a research problem and corresponding entry points.	•The research documentation demonstrates holistic understanding of the problem context in which the research is situated (description of the system, including actors situated in the context) •Connection is made between the problem context and the research problem •Research entry points are determined by the problem context
Socially relevant research problem2	The research problem is well defined and described, and considers application to the problem context and current academic discourse.	•The research problem is a timely issue in society or aligns with current actions (e.g., international commitments, governmental mandate, policy development, social movements, etc.) •There is a demand from system actors3 for the research problem to be addressed
Engagement with problem context	Researchers demonstrate appropriate4 breadth and depth of understanding of and sufficient interaction with the problem context.	•Understanding drawn from the literature •System actor perspectives are understood •Where possible, researchers incorporate insights from prior research or professional experiences relevant to the problem context
Explicit theory of change	The research explicitly identifies its main intended outcomes5, how they are expected to be realized, and how they are expected to contribute to longer term outcomes and impacts.	•The logic of the research contributions to a process of change is well described and sound •Key actors, processes, and assumptions are identified •End-of-project outcomes are reasonable to expect with the resources available
Relevant research objectives and design	The research objectives are appropriate to the research problem, and the research design is aligned with the objectives.	•Objectives identify what the research project aims to do or produce •Objectives can be justified in how they address the research problem (e.g., fill a knowledge gap) •The research design logically plans how the project will meet the objectives (i.e., identify what methods, activities, and engagement are needed)
Relevant communication6	Communication during and after the research process7 is appropriate to the context and accessible to stakeholders, users, and other intended audiences.	•Communications with system actors help focus the research, source information, and co-generate and share learning •Communications are timely and responsive to other system processes •Communications are tailored to the target audience
Credibility: The research findings are robust and the sources of knowledge are dependable. This includes clear demonstration of the adequacy of the data and the methods used to procure the results, including clearly presented and logical interpretation of findings.
Criteria	Definition	Observations
Broad preparation	The research is based on a strong integrated theoretical and empirical foundation.	•Breadth and depth of literature and theory from relevant disciplines are reviewed and integrated •Empirical demonstration of gaps is based on previous research or interventions, or identified by system actors (e.g., joint problem formulation)
Clear research problem definition	The research problem is clearly stated and defined, researchable, and grounded in the academic literature and problem context.	•A research/knowledge gap is identified •The importance of and need for the research are demonstrated •The research problem can be answered empirically
Clear research question	The research question(s) is clearly stated and defined, researchable, and justified as an appropriate way to address the research problem.	•The research question(s) is logically derived from the research problem •The research question(s) can be answered empirically (i.e., is researchable) •Justification is given on how answering the research question will address the research problem
Comprehensive objectives	Research objectives8 are clearly stated and sufficient to answer the research question(s).	•Objectives are clear, coherent, and feasible •Objectives indicate what knowledge is needed, and how that knowledge will be acquired •Collectively, satisfying all objectives will answer the research question(s)
Feasible research project	The research design and resources are appropriate and sufficient to meet the objectives as stated, and adequately resilient to adapt to unexpected opportunities and challenges throughout the research process.	•Research design is logically derived from the objectives •The project can be completed with the resources available (i.e., budget, time, hardware, software, human capital, and social capital) •Research design is flexible to accommodate unexpected changes
Adequate competencies	The skills and competencies of the researcher(s), team, or collaboration (including academic and societal actors) are sufficient and in appropriate balance (without unnecessary complexity) to succeed.	•The knowledge, skills, and expertise needed to carry out the research are identified •The necessary knowledge, skills, and expertise are represented in the research team
Appropriate research framework	Disciplines, perspectives, epistemologies, approaches, and theories are combined and/or integrated to meet stated objectives and answer the research question(s).	•Explanation of the theoretical/analytical framework is given •Explanation is provided for why and how disciplines, epistemologies, and theories are used •The process of integration of disciplines, epistemologies, and theories is explained, including how paradoxes and conflicts between integrated components are addressed •Justification is given for the framework selected in relation to the problem context
Appropriate methods	Methods are fit to purpose and well suited to achieve the objectives and answer the research question(s).	•Clear descriptions of methods and how they were applied are given •Methods are justified and logically connected to the objectives •Novel (unproven) methods or adaptations are explained and justified, including why they were used and how they maintain rigour
Sound argument	The logic from analysis through interpretation to conclusions is clearly described. Sufficient evidence is provided to clearly demonstrate the relationship between evidence and conclusions.	•The argument is logical and defensible •Analyses and interpretations are adequately explained and supported by evidence •If applicable, alternative explanations of results are explored
Transferability and/or generalizability of research findings	The degree to which the research findings are applicable in other contexts is assessed and discussed. In cases that are too context-specific to be generalizable, aspects of the research process or findings that may be transferable to other contexts and/or used as learning cases are discussed.	•The potential to transfer results and/or methods to other contexts is discussed •Justification of transferability/generalizability of results is logical
Limitations stated	An explanation is given regarding how the characteristics of the research design or method may have influence on the results or conclusions.	•The influence of internal (e.g., sampling) and/or external factors (e.g., responsiveness of interviewees) on the results is acknowledged and discussed •Researcher(s) assess the extent to which the limitations influence the results
Ongoing monitoring and reflexivity9	Researchers engage in ongoing reflection and adaptation of the research process, making changes as new obstacles, opportunities, circumstances, and/or knowledge surface.	•There is an indication that the researcher(s) considers the need to reflect on and adapt during the research process •Efforts to monitor progress and identify, consider, and respond to changes in context or understanding are discussed •Processes of reflection (whether formal or informal), and the resulting action(s) taken, are explained
Legitimacy: The research process is perceived as fair and ethical. This encompasses the ethical and fair representation of all involved and the appropriate and genuine inclusion and consideration of diverse participants, values, interests, and perspectives.
Criteria	Definition	Observations
Disclosure of perspective	Actual, perceived, and potential bias is clearly stated and accounted for.	•Potential for actual or perceived bias (e.g., positionality, sources of funding, partnerships, mandate, etc.) is identified and acknowledged •Implications of potential bias on the conclusions are discussed
Effective collaboration10	Individuals11 involved in the research process pool their knowledge, experience, and skills together in a constructive atmosphere and in appropriate measure to produce new knowledge and/or social processes that contribute to a common goal.	•A shared understanding of goals and expectations is established •Roles and responsibilities are clear and explicitly agreed upon •Decision-making structures are transparent and fair •A synergistic process capitalizes on the strengths of collaborators (across disciplinary, professional, organizational, and cultural boundaries)
Genuine and explicit inclusion12	The research offers authentic opportunities to involve relevant actors to share their perspectives, knowledge, and values, and/or participate in the research process.	•Participants’ roles and contributions, perspectives, and cultural backgrounds are described •Steps taken to ensure the respectful inclusion of diverse actors and views are explained
Research is ethical	The research adheres to standards of ethical conduct.	•Ethical practice is followed: research does no harm; participants have informed consent; anonymity and confidentiality are maintained •Procedural ethics (e.g., ethical review process) are pursued and documented
Positioning for Use: The research process is designed and managed to enhance sharing, uptake, and use of research outputs and stimulates actions that address the problem and contribute to solutions.
Criteria	Definition	Observations
Strategic engagement	The research process stimulates and/or engages with change opportunities.	•Engagements are timely and responsive to other system processes •Researcher(s) is well positioned to have influence within the problem context •Opportunities to influence change processes are identified and/or generated, and acted upon •Resources are mobilized to influence/act on change processes
New knowledge contribution	The research generates new knowledge and understanding in academic and social realms in a timely, relevant, and significant way.	•An academic knowledge gap is filled •System actors’ knowledge gaps are filled •System actors gain a better understanding of the problem context
Influencing attitudes	The research process and/or findings stimulates and supports system actors to reflect on and/or change their attitudes or perspectives on the problem and solutions to address it.	•Awareness-building of the research problem, the research findings, or a solution/innovation is a first step in changing attitudes •System actors gain a different perspective on the targeted problem as a result of the research process and/or findings
Capabilities	System actors develop skills relevant to the problem context and/or skills to solve the societal problem through the research process and/or findings.	•Research capacities of the researcher(s) and/or partners are developed (e.g., gain research experience, training, testing of new methods/approaches) •Participants and partners gain new or build on existing skills as a result of the research process and/or findings •Skills developed are transferable to other aspects of system actors’ professional or personal lives
Relationship-building	The research process supports new or fortifies existing relationships, networks, and ways of working for solution-building in the problem context.	•Trust between system actors is fostered by the research process •Mutual interests between system actors are recognized •A forum, platform, or network is created or strengthened as a result of the research process •System actors work together in new ways as a result of the research process •The research contributes to shifting the power dynamics toward solution-building •Open communication, equality and equity, co-identification/co-development across the research process, feedback processes, and conflict management and resolution are important components of effective relationships
Practical application	The findings, process, and/or products of research have high potential for use by system actors.	•The potential utility of the research outputs for system actors are discussed •System actors convey intentions to use or apply the research •System actors pilot, adopt, or adapt a method, tool, approach, or innovation from the research •System actors use or refer to the research findings to inform their work
Significant results	The research contributes to the solution of the targeted problem or provides unexpected solutions to other problems.	•The research process and/or findings contribute to behaviour change in the problem context •Expected changes are realized or have potential to be realized in the future

¹Problem context refers to the social and environmental setting(s) that gives rise to the research problem, including aspects of: location; culture; scale in time and space; social, political, economic, and ecological/environmental conditions; resources and societal capacity available; uncertainty, complexity, and novelty associated with the societal problem; and the system actors and processes [19].

²A research problem is a specific gap in understanding or knowledge that needs to be filled in order to help solve a societal problem.

³System actors include other researchers, practitioners, policy actors, and intended beneficiaries.

⁴ Words such as ‘appropriate’, ‘suitable’, and ‘adequate’ are used deliberately to allow for quality criteria to be flexible and specific enough to the needs of individual research projects [20].

⁵ Outcomes are defined as “changes in knowledge, attitudes, skills, and relationships manifested as changes in behaviour” ([13], p.9).

⁶Communication refers to both written communication (e.g., proposal, documents, presentation of findings, etc.) as well as engagement communications (e.g., scoping, data collection activities, meetings, workshops, etc.).

⁷Research process refers to the series of decisions made and actions taken throughout the entire duration of the research project and encompasses all aspects of the research project.

⁸Objectives explain what the research will do (i.e., generate specific knowledge, create or facilitate specific processes) and what steps will be undertaken in order to answer the research question(s).

⁹Reflexivity refers to an iterative process of formative, critical reflection on the important interactions and relationships between a research project's process, context, and product(s).

¹⁰ Collaboration encompasses both internal dynamics within the core research team and external processes with participants, collaborators, partners, and allies. Collaboration comes in many forms in research, ranging from general advice-giving to co-generated knowledge production.

¹¹ Within and external to the core research team.

¹² Some system actors may not want to participate in the research process, but still want their views to be represented in the findings. It is the task of the researcher(s) to ensure that their perspectives are accurately represented.