Skip to main content
NCBI home page
F1000Research logoLink to F1000Research
. 2024 Nov 18;13:1382. [Version 1] doi: 10.12688/f1000research.156907.1

Methodological literature on the reporting of systematic reviews of health economic evaluations: a scoping review protocol

Phuong Bich Tran 1,a, Joseph Kwon 1, Anastasios Bastounis 2, Stavros Petrou 1, Andrew Booth 2; PRISMA-EconEval Management Group
PMCID: PMC11865710  PMID: 40018300

Abstract

Systematic reviews of health economic evaluations play a crucial role in informing evidence-based healthcare decisions, yet they lack standardized reporting guidelines. A project has been initiated that aims to extend the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline for systematic reviews of health economic evaluations (PRISMA-EconEval). This scoping review forms a foundation for the PRISMA-EconEval project, aiming to identify, map, and extract candidate reporting items from the methodological literature.

The scoping review will follow the PRISMA Extension for Scoping Reviews (PRISMA-ScR) Checklist and involve comprehensive searches in databases such as PubMed MEDLINE, Embase, and Web of Science, covering the period from 2015 to 2024. Supplementary searching, reference checking and citation searching will target grey literature, overlooked studies and evidence prior to 2015. Inclusion criteria will focus on methodological papers that provide frameworks or recommendations for reporting systematic reviews of health economic evaluations and enhanced case studies that critically discuss methods and reporting structures. The extracted data will be coded and analyzed to produce an initial list of candidate reporting items, structured according to conventional sections of a systematic review (e.g., title, abstract, methods, results). This initial list will be used in the subsequent stages of the project and disseminated through a peer-reviewed publication and presentations at international conferences.

The outcome of this scoping review will significantly contribute to the development of a comprehensive PRISMA-EconEval reporting guideline, aimed at enhancing the transparency, consistency, and quality of systematic reviews of health economic evaluations, and provide an essential tool for authors, editors, peer-reviewers, and stakeholders.

Keywords: reporting, systematic review, health economic evaluation, review protocol

Introduction

A systematic review summarizes the available evidence and provides an unbiased, reliable assessment of the current state of knowledge, essential for making evidence-based decisions. 1 The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement was designed to enhance the reporting of systematic reviews and meta-analyses – primarily of the effects of interventions, though also applicable to systematic reviews and meta-analyses of studies with a different objective – by improving their clarity, consistency, transparency, quality, and overall value. 2

Systematic reviews of health economic evaluations differ from other systematic reviews in their approach to reporting key methodological steps, including methods for study selection, data extraction and presentation, and the assessment of quality and relevance. Such reviews consider health economic evaluations, which compare the effects of different healthcare interventions on economic outcomes (e.g. incremental cost-effectiveness ratios, net monetary benefits, etc.). Additionally, they differ in their potential for combining costs, preference-based health-related quality of life outcomes, and cost-effectiveness outcomes across studies.

A literature search in PubMed Central from January 1, 2015 to March 25, 2017 found 202 systematic reviews of health economic evaluations listed within this 27-month period. 3 Extending the search to other databases and grey literature and from 2017 to date is likely to increase this number substantially. These reviews lack common standards for reporting quality, highlighting the need for an additional reporting guideline specifically for systematic reviews of health economic evaluations.

The ‘Development of a PRISMA extension for systematic reviews of health economic evaluations (PRISMA-EconEval)’ project was initiated to address this gap. A core international multidisciplinary working group, including PRISMA members, authors, journal editors, and stakeholders, will identify and extract candidate reporting items from various sources. Such sources include this scoping review of the methodological literature on the reporting of systematic reviews of health economic evaluations together with a purposive sample of previous systematic reviews of health economic evaluations. They will refine these items to improve reporting quality. A multi-round online Delphi survey will then evaluate these items with input from academic researchers, specifically health economists (including authors of systematic reviews of health economic evaluations) and systematic review methodologists, journal editors, reporting guideline developers, information specialists, and stakeholders, including health care decision-makers, health research funders, and patient and public representatives. The development of common reference standards for reporting should enhance completeness, transparency and structure in the reporting of systematic reviews of health economic evaluations and generate user-friendly tools for authors, editors, peer-reviewers and stakeholders that facilitate reporting.

As a first step, it is necessary to review the methodological literature on the reporting of systematic reviews of health economic evaluations. This process will help map the current methods, guidance, and recommendations for reporting such reviews and is part of the process of identifying common reference standards for reporting them.

Research objectives

  • 1.

    To identify and review the methodological literature on the reporting of systematic reviews of health economic evaluations.

  • 2.

    To summarize and map methods, guidance, and recommendations for reporting such reviews.

  • 3.

    To identify and extract an initial dataset of candidate reporting items.

Method

Protocol

The scoping review forms part of the larger PRISMA-EconEval project. The protocol for the scoping review follows the Preferred Reporting Items for Systematic Review and Meta -Analysis Protocols (PRISMA-P) 2015 Checklist, 4 with the protocol and PRISMA-P checklist deposited on the Open Science Framework. 5 We will report the scoping review in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist. 6

The scoping review will be conducted in various stages as shown in Figure 1 and will aim to identify an initial list of candidate reporting items. This initial list of reporting items will have generic relevance to systematic reviews of health economic evaluations regardless of the types of health economic evaluation (e.g., cost-minimization analysis, cost-benefit analysis, cost-effectiveness analysis, cost-utility analysis, cost-consequences analysis) and the vehicles for health economic evaluation (e.g., economic evaluations based on randomized controlled trials, economic evaluations based on observational studies with patient-level data, decision-analytic models) adopted by the individual studies they cover. Specific reporting items relevant to different categories of health economic evaluations may also be identified. The initial list will then be enhanced by findings from an additional sample of published systematic reviews of health economic evaluations. After refinement by the PRISMA-EconEval Management Group, this list will inform the multi-round online Delphi survey. Detailed plans for subsequent stages (post-scoping review) will be outlined in the comprehensive PRISMA-EconEval project protocol.

Figure 1. Stages of the PRISMA-EconEval project.

Figure 1.

Open in a new tab

The scoping review will focus on two types of articles (either published in journals or elsewhere):

  • •

    Methodological papers: Articles that describe methods, frameworks, guidance for conduct, or recommendations on the reporting of systematic reviews of health economic evaluations.

  • •

    Enhanced case studies: Systematic reviews of health economic evaluations that include a minimum of two paragraphs that discuss methods, conceptual approaches, limitations (e.g., items not analyzed/reported in the review), and/or thorough categorization and reporting of health economic evaluation features that can inform the initial list of candidate reporting items.

Conditions or domains being studied

We used the ECLIPSE Framework to help in framing the scope and objectives of the scoping review ( Table 1). 7

Table 1. The ECLIPSE framework.

Element Definition Description
Expectation What are you looking to improve or change? What is the information going to be used for? We aim to collate and summarize methods, guidance, and recommendations for the reporting of systematic reviews of health economic evaluations. This is an initial step in the development of a new reporting guideline for systematic reviews of health economic evaluations.
Client group Who is the service or policy aimed at? Authors, editors, peer-reviewers and stakeholders that conduct or facilitate the reporting of systematic reviews of health economic evaluations.
Location Where is the service or policy located? Unrestricted
Impact What is the change in service or policy that the researcher is investigating? Findings from the scoping review will form the initial dataset of candidate reporting items, which is a key step in developing the PRISMA-EconEval guideline. The development of a common reference standard for reporting should enhance completeness, transparency and structure in the reporting of systematic reviews of health economic evaluations.
Professionals Who is involved in providing or improving the service or policy? PRISMA-EconEval Management Group, Independent Advisory Group, Delphi Panelists, Patient and Public Involvement Group, Pilot End-Users
Service What kind of service or policy is this? The development of a reporting guideline
Open in a new tab

Expected timeline for scoping review

August 2024 – December 2024.

Search strategy

We will conduct searches in PubMed MEDLINE, Embase, and Web of Science for the period from 2015 to 2024. Relevant papers published prior to this timeframe will be identified through supplementary searching, reference checking and citation searching.

For the grey literature, we will identify and agree upon a list of 10-12 websites of health technology assessment agencies or health economics associations that may publish methods papers, guidance, and recommendations for reporting systematic reviews of health economic evaluations.

Search terms and strings

An experienced information specialist (ABo) will search databases of the published literature, applying guidance on filters at the intersection of “systematic reviews” and “economic evaluations” to develop and pilot our search strategies, including data sources, date limits, search terms, and syntax ( Box 1).

Box 1. Search string examples for PubMed.

“systematic review of economic evaluations”[tiab:~0] OR “systematic review of cost effectiveness”[tiab:~0] OR “systematic review”[PT] AND (economic*[Ti] OR (“economic evaluation*” OR cost [Ti] OR costs [Ti] OR “cost effectiveness” OR “cost benefit analysis” OR “Cost Utility Analysis” OR “Cost-Benefit Analysis”)

“systematic reviews of economic evaluations”[tiab:~0] OR “systematic reviews of cost effectiveness”[tiab:~0] OR ((“Systematic Reviews as Topic” OR “Review Literature As Topic”) AND (economics [SH] OR economic*[Ti] OR cost [Ti] OR costs [Ti] OR “cost effectiveness” OR “cost benefit analysis” OR “Cost utility Analysis” OR “Cost-Benefit Analysis”)) OR (“systematic reviews” AND (“economic evaluation*” OR cost [Ti] OR costs [Ti] OR “cost effectiveness” OR “cost benefit analysis” OR “Cost utility Analysis” OR “Cost-Benefit Analysis”) AND (Methods OR methodology OR methodological OR checklist* OR guideline*)) OR ((“Cost-Benefit Analysis” AND (Methods OR methodology OR methodological OR checklist* OR guideline*) AND (“Systematic Reviews as Topic” OR “Review Literature As Topic”))

For the grey literature, the scoping review team will search the websites of health technology assessment agencies and health economics associations, using similar search terms.

Supplementary search methods

Recognizing that methodological considerations are not always adequately reported in the titles and abstracts of systematic reviews of health economic evaluations, the research team will use innovative supplementary search methods to access the contents of full text articles and establish their eligibility for inclusion. Scite (open access alternative — Consensus) and Litsense will be interrogated for “systematic reviews of (health) economic evaluations”. Google Scholar will be searched using targeted search terms via the free open source Publish or Perish search tool to access full text contents of high currency and specificity. We will also use BASE, 8 one of the largest search engines for academic web resources, which captures both academic and grey literature that may have been missed. Finally, a list of 10-12 known “index” items of high relevance to the review question will be used as the starting point for both backwards citation chaining (reference checking) and forwards citation chaining (citation chasing) using the Citation Chaser open-source tool.

Inclusion and exclusion criteria

For methodological papers, we will include articles that describe methods, frameworks, guidance, or recommendations on the reporting of systematic reviews of health economic evaluations. For enhanced case studies, we will include systematic reviews of health economic evaluations that contain at least two paragraphs critically discussing methods, conceptual approaches, limitations (e.g., items that were not reported but should have been), and/or thorough categorization and reporting of health economic evaluation features that can inform the initial list of candidate reporting items. Both types of articles published in English will be included without date restriction. We will exclude conference abstracts and proceedings, except for those that specifically focus on reporting methodologies of systematic reviews of health economic evaluations. For these, we will conduct additional searches and contact the authors (if necessary) to determine if a full-text publication is available. If a full-text version cannot be identified, they will be excluded.

Article screening and selection

References for the articles retrieved will be uploaded to Covidence 9 (open access alternative — Rayyan) for deduplication and initial screening (Phase One) and, subsequently, for full-text screening and reference management (Phase Two).

The eligibility of each identified study will be assessed by two of the five reviewers (ABa, ABo, JK, PT, SP) independently. A two-stage standard protocol will be developed and followed via checklists for each of the two phases. The checklists will be piloted on 100 random articles and screening criteria adapted accordingly.

Phase One screening (Title and abstract screening): The following checklist will be used for the first screening of articles after deduplication ( Table 2). All articles will be classified accordingly. Each independently classified article by every two reviewers will be automatically compared by Covidence. Disagreements between reviewers will be resolved through consensus or by referral to a third reviewer.

Table 2. Phase One screening checklist.

Q Question Answer Action following answer Comments/examples
1 Based on title/abstract, is the article likely to focus on reporting methodologies for systematic reviews/evidence syntheses of health economic evaluations? Yes Classify as “Retained phase 1, methodological paper” The primary focus of the article should be on methods, frameworks, guidance, or recommendations on the reporting of systematic reviews of health economic evaluations.
No Go to Q2
Maybe Go to Q2
2 Based on title/abstract, is the article a systematic review of health economic evaluations: i.e., a ‘worked example’? Alternatively, is it likely to focus on such reviews, or at minimum, on health economic evaluations? Yes Go to Q3 The article should at minimum, focus on full health economic evaluations or systematic reviews of such studies. Review of reviews may also be helpful. Partial health economic evaluations (e.g., cost analyses, utility elicitation studies) should be excluded.
No Classify as “Exclude”
Maybe Go to Q3
3 Does the title/abstract discuss issues relevant to or contain useful information regarding the reporting of systematic reviews of health economic evaluations? Yes Classify as “Retained phase 1, enhanced case study” In the case of ‘worked examples’, the abstract may show hints of discussions related to the reporting of systematic reviews of health economic evaluations. (e.g., Studies demonstrated variation or heterogeneity in reporting; XX studies did not include the perspective of the analysis, and YY studies failed to specify the choice of discount rate.)
The abstract may also include thorough categorizations that could inform the list of candidate reporting items. (e.g., Cost-effectiveness was estimated using partitioned survival models in XX studies and state-transition models in YY studies. Across studies, XX reported utility values by health state and YY reported disease-specific HRQoL.)
No Classify as “Exclude”
Maybe Classify as “Retained phase 1, potential enhanced case study”
Open in a new tab

Phase Two screening (Full-text screening): For all articles classified as “Retained Phase 1”, the full article will be read and further classified as below ( Table 3). Disagreements between reviewers will be resolved through consensus or by referral to a third reviewer.

Table 3. Phase Two screening checklist.

Q Question Answer Action following answer Comments/examples
1 Does the article focus on reporting methodologies for systematic reviews/evidence syntheses of health economic evaluations? Yes Classify as “Include, methodological paper” The primary focus of the article should be on methods, frameworks, guidance, or recommendations on the reporting of systematic reviews of health economic evaluations.
No Go to Q2
No Classify as “Exclude”
2 Does the article contain at least two paragraphs critically discussing methods, conceptual approaches, limitations and/or recommendations for the structuring and reporting of systematic reviews of health economic evaluations. Alternatively, does it contain useful information regarding the reporting of systematic reviews of health economic evaluations? Yes Classify as “Include, enhanced case study” In the case of ‘worked examples’, the article may (e.g., in the discussion section) discuss issues relating to the reporting of systematic reviews of health economic evaluations. In the result section, they may also discuss items that were not reported but should have been. (e.g., Studies demonstrated variation or heterogeneity in reporting; XX studies did not include the perspective of the analysis, and YY studies failed to specify the choice of discount rate.)
The article may also include thorough categorizations that could inform the list of candidate reporting items. (e.g., Cost-effectiveness was estimated using partitioned survival models in XX studies and state-transition models in YY studies. Across studies, XX reported utility values by health state and YY reported disease-specific HRQoL.)
No Classify as “Exclude”
Open in a new tab

Data extraction and coding

Data from studies that fulfill the inclusion criteria will be entered (by ABa, ABo, JK, PT, SP) onto an Excel form. Data to be extracted will include: article PMID number, title of paper, authors, year of publication, journal or source, type of literature (i.e., published scientific article, grey literature), and type of paper (i.e., methodological paper, enhanced case study). Recommendations from included studies on the methods and reporting of systematic reviews of health economic evaluations will be extracted and organized by codes and sub-codes. The coding will be conducted using NVivo V.1.7 10 (open access alternative — QualCoder). The reviewers will work together using a shared open codebook. All domains described in the included studies will be gathered and tabulated. Conventional sections and sub-sections of a systematic review as informed by the PRISMA guideline (i.e., title, abstract, introduction, methods, results, discussion, other information) will be used to inform the coding by which the identified themes and topics are organized. The extraction process will follow a pilot round of coding, continual refinement and revision of the codebook, and frequent consensus meetings.

Data analysis and reporting

The objective of the scoping review is to produce an initial list of recommended reporting items for systematic reviews of health economic evaluations, to be used in subsequent stages of the PRISMA-EconEval project. Where necessary, items will be renamed/rephrased, similar items will be combined, and items will be split if they cover two or more distinct topics.

The initial list of candidate reporting items identified by the PRISMA-EconEval Management Group will be arranged according to the conventional sections of a systematic review: title, abstract, introduction, methods, results, discussion, and other information. This structure broadly aligns with other extensions to the PRISMA reporting guidelines. The initial list of candidate reporting items will have generic relevance to systematic reviews regardless of the types of health economic evaluation or the vehicles for health economic evaluation adopted by the individual studies they cover. We will also include specific reporting items relevant to different categories of health economic evaluations in this and subsequent stages of the PRISMA-EconEval project.

Dissemination

We plan to disseminate the findings of the scoping review by submitting the resulting manuscript to a suitable peer-reviewed journal and presenting the key findings at the 2025 International Health Economics Association Congress. The initial list of candidate reporting items will be used in the subsequent stages of the project; i.e., review of sample papers and the Delphi surveys.

Study status

The protocol has been registered on the EQUATOR Network database 11 and the Open Science Framework. 5 The database searches have been conducted and pilot screening will soon take place.

Ethical considerations

Ethical approval is not required for this aspect of the PRISMA-EconEval project as it does not involve individual or personal data.

Author contributions

The review was conceptualized collectively by the team (PT, JK, ABa, SP, ABo). PT developed the initial draft manuscript and created visualizations, with reviews and edits by the team. ABo and ABa provided expertise on the review methodology, databases and search strategy. PT was responsible for the revision and submission of the manuscript and is the guarantor of the protocol. SP, JK, ABo provided supervision throughout the entire process. All authors have reviewed and approved the final version of the manuscript.

List of members of the PRISMA-EconEval Management Group

  • 1.

    Prof Stavros Petrou, Nuffield Department of Primary Care Health Sciences, University of Oxford, United Kingdom

  • 2.

    Dr Joseph Kwon, Nuffield Department of Primary Care Health Sciences, University of Oxford, United Kingdom

  • 3.

    Dr Phuong Bich Tran, Nuffield Department of Primary Care Health Sciences, University of Oxford, United Kingdom

  • 4.

    Prof Andrew Booth, Sheffield Centre for Health and Related Research (SCHARR), University of Sheffield, United Kingdom

  • 5.

    Dr Anastasios Bastounis, Sheffield Centre for Health and Related Research (SCHARR), University of Sheffield, United Kingdom

  • 6.

    Prof Sophie Staniszewska, RCN Research Institute, Warwick Medical School, University of Warwick, United Kingdom

  • 7.

    Mr Richard Grant, Patient and Public Involvement Representative, United Kingdom

  • 8.

    Prof Sally Hopewell, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, United Kingdom

  • 9.

    Dr Matthew Page, Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Australia

Funding Statement

This study forms part of the 'Development of new reporting guidance for systematic reviews of health economic evaluations (PRISMA-EE)' project, funded by the National Institute for Health and Care Research (NIHR) Research for Patient Benefit Programme, grant number NIHR206833, hosted at the Nuffield Department of Primary Care Health Sciences, University of Oxford. SP receives support as a NIHR Senior Investigator (NF-SI-0616-10103) and from the NIHR Applied Research Collaboration Oxford and Thames Valley at the Oxford Health NHS Foundation Trust. The funders had no involvement in the development of the protocol.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 1; peer review: 2 approved

Data availability

Underlying data

No underlying data is associated with this article.

Extended data

Reporting guideline.

Open Science Framework. [Development of new reporting guidance for systematic reviews of health economic evaluations (PRISMA-EconEval)] DOI: https://doi.org/10.17605/OSF.IO/FSPG9. 5

This project contains the following data:

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

References

  • 1. Moosapour H, Saeidifard F, Aalaa M, et al. : The rationale behind systematic reviews in clinical medicine: a conceptual framework. J. Diabetes Metab. Disord. 2021 Apr 8;20(1):919–929. 10.1007/s40200-021-00773-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Page MJ, McKenzie JE, Bossuyt PM, et al. : The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021 Mar 29;372:n71. 10.1136/bmj.n71 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Luhnen M, Prediger B, Neugebauer EAM, et al. : Systematic reviews of health economic evaluations: A structured analysis of characteristics and methods applied. Res. Synth. Methods. 2019 Jun;10(2):195–206. 10.1002/jrsm.1342 [DOI] [PubMed] [Google Scholar]
  • 4. Shamseer L, Moher D, Clarke M, et al. : Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015 Jan 2;349:g7647. 10.1136/bmj.g7647 [DOI] [PubMed] [Google Scholar]
  • 5. Kwon J, Petrou S, Tran PB, et al. : Development of new reporting guidance for systematic reviews of health economic evaluations (PRISMA-EconEval). OSF. 2024 Jul 30 [cited 2024 Sep 26]. 10.17605/OSF.IO/FSPG9 [DOI] [Google Scholar]
  • 6. Tricco AC, Lillie E, Zarin W, et al. : PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018 Oct 2;169(7):467–473. 10.7326/M18-0850 [DOI] [PubMed] [Google Scholar]
  • 7. Wildridge V, Bell L: How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Inf. Libr. J. 2002 Jun 1;19(2):113–115. 10.1046/j.1471-1842.2002.00378.x [DOI] [PubMed] [Google Scholar]
  • 8. Bielefeld University Library: BASE - Bielefeld Academic Search Engine. [cited 2024 Oct 10]. Reference Source
  • 9. Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation;2024. Reference Source [Google Scholar]
  • 10. QSR International: NVIVO Release 1.7. 2022. Reference Source
  • 11. EQUATOR Network: Reporting guidelines under development for systematic reviews. 2024 [cited 2024 Oct 10]. Reference Source
F1000Res. 2025 Feb 26. doi: 10.5256/f1000research.172286.r363770

Reviewer response for version 1

Marta Trapero-Bertran 1

I would like to congratulate you on this outstanding piece of work. Your article is highly engaging, well-written, and clearly articulated, making it a valuable contribution to the academic field.

The only comment I would like to make relates to the search terms used in your bibliographic review. Rather than concatenating all the suggested search terms as an example of a search strategy in Medline, it might be more efficient to use the * symbol to avoid explicitly listing all possible variations of economic evaluation types. In this regard, I would suggest something like the following search strategy:

"systemat* rev*" AND ("cost*" OR "econ* eval*") AND ("checklist*" OR "method" OR "anal*")**

This adjustment could help broaden the search results while maintaining specificity, and shortening search terms used.

Once again, congratulations on this excellent article, and thank you for your valuable contribution.

Is the study design appropriate for the research question?

Partly

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Health Economics and Economic Evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2025 Feb 20. doi: 10.5256/f1000research.172286.r363774

Reviewer response for version 1

Vicki Brown 1

This paper presents the protocol for a scoping review of systematic reviews of health economic evaluations, with the aim of informing the development of a PRISMA "add-on" for reporting of SRs of health economic evaluations.  The manuscript is clear and well written.  The methods proposed are sound.  I have no comments or edits.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Health economics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2025 Jan 7. doi: 10.5256/f1000research.172286.r347340

Reviewer response for version 1

Ghislaine APG van Mastrigt 1

This study aims to investigate the existing literature through a scoping review to develop an alternative to the well-established PRISMA reporting guidelines, namely one to improve the reporting quality of health economic evaluations. Overall, this is a very well-written and clear protocol for performing this research. However, the authors were given a few suggestions to improve this scoping review protocol.

Regarding the questions of the review report form:

Is the rationale for, and objectives of, the study clearly described?

Partly

The authors report the three research objectives:

  • 1. To identify and review the methodological literature on reporting systematic reviews of health economic evaluations.

  • 2. To summarize and map methods, guidance, and recommendations for reporting such reviews.

  • 3. To identify and extract an initial dataset of candidate reporting items.

For objective 1, I would not solely focus on ‘reporting’ whether you can identify all the relevant literature; I would rephrase it as follows: To identify and review the methodological literature of systematic reviews of health economic evaluations to improve the reporting quality of these studies.

For objective 2: it would rephrase it as follows: To summarize and map methods, guidance, and recommendations for methodological literature of systematic reviews of health economic evaluations.

For objective 3: to identify and extract an initial dataset of candidate items to develop an extension to PRISMA guidelines for Health economic evaluations

In summary, what aspects do you consider as part of reporting economic evaluations?  This could be specified for the objectives, but this is also needed for the protocol.

Is the study design appropriate for the research question?

Yes

The authors propose a scoping review to investigate the existing literature on this topic, I agree this is an appropriate research design to investigate this research question.

Are sufficient details of the methods provided to allow replication by others?

Partly

Regarding the step of the selection of the databases: They state: ‘’ We will conduct searches in PubMed MEDLINE, Embase, and Web of Science for the period from 2015 to 2024’’. However, no rationale behind the choices of these bibliographic databases is given; for economic topics; Econlit could also be a relevant database to check.

Other topics

General topic: study designs

For the scoping review but also the PRISMA-EE development; what kind of types of economic evidence do the authors want to cover in their reporting guidelines? only full economic evaluations: like cost-utility or cost-effectiveness analyses or also partial economic evaluations like; cost of illness studies, resource use etc. This is explained in the phase one checklist but should also be more clearly mentioned in the protocol.

It is highly recommended to start the study after the registration of the protocol; so please adapt the start date of review to January 2025.

Sections: search strategy/search terms and strings

This section could be improved by adding additional information on search strategy development and databases selected (for details see comments below).

It is highly recommended to report the search strings for each of the bibliographic databases which will be used to identify relevant studies. In addition, it is also highly recommended to use validated and predefined search strings as much as possible. The ISSG website is a recommended source to look for, especially for the economic topics (table needs further specification). Furthermore, more details could be given on how the search strategies were developed. For this the Prisma extension ‘’ PRESS’’ is very helpful.

The search strategies listed in box 1 do not match the study objectives and could be made more specific.

Details of the 10-12 websites of health technology assessment agencies or health economics associations could be added in an appendix.

Please also specify the 10-12 ‘’index items’’ mentioned to be used as starting points for backward and forward citation chasing. ‘’Citation chasing’’ is mentioned twice, once briefly at the beginning of the section and more detail at the end, one of them could be deleted.

Section: Inclusion and exclusion criteria

Details regarding the type of economic evaluation included, approaches used (e.g. model/trial-based), as well as including full or also partial economic evaluations should be mentioned in the list of eligibility criteria.

Section: Article screening and selection

Will you use Ryan or Covidence to handle the study results?

In the sentence: ‘’The checklists will be piloted on 100 random articles and screening criteria adapted accordingly’’. You could mention that you use two separate checklists for screening and selection.

Add a sentence on the approach you will use to contact authors to obtain full-text papers if only abstracts are available.

  Section: data extraction and coding

Will you use NVivo V.1.7 10  or   QualCoder for coding?

A draft of the data extraction sheets could be added to the protocol.

In addition a list of example general or specific candidate items?

What do you mean by the last part of this sentence: ‘’We will also include  specific reporting items relevant to different categories of health economic evaluations in this and subsequent stages of the PRISMA-EconEval project’’? Could you please explain it?

Are the datasets presented in a usable and accessible format?

Not applicable

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Partly

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Health Economist, Methodologist, and Epidemiologist, in the field of systematic reviews of economic evaluations, quality asssessment and reporting of economic evaluations

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    Underlying data

    No underlying data is associated with this article.

    Extended data

    Reporting guideline.

    Open Science Framework. [Development of new reporting guidance for systematic reviews of health economic evaluations (PRISMA-EconEval)] DOI: https://doi.org/10.17605/OSF.IO/FSPG9. 5

    This project contains the following data:

    Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

    Articles from F1000Research are provided here courtesy of F1000 Research Ltd

    RESOURCES