Evidence-based recommendations for economic evaluations in spine surgery: study protocol for a Delphi consensus

Abstract

Introduction

Considering the rising global healthcare expenses, economic evaluations are more important than ever. Even though the number of studies regarding costs and cost-effectiveness is increasing, the quality of these studies remains relatively low. This is mainly caused by abundant heterogeneity in methods used for determining, calculating and reporting cost data, despite current general guidelines for the conduct of economic evaluations. Disease-specific recommendations for the conduct of economic evaluations in the field of spine surgery, as complement to existing general guidelines, will ameliorate overall research quality, comparability and interpretability and thus, the overall quality. We aim to provide expert-based recommendations for the design, conduct, and reporting of economic evaluations in spine surgery.

Methods and analysis

A modified Delphi study will be conducted to formulate expert-based recommendations. The following steps will be taken:

(1) The conduct of a systematic review to identify relevant publications and identify relevant authors. Formation of an expert group and a Delphi-panel. (2) Drafting of statements based on articles included in the systematic literature review. Validation of drafted statements by the expert group. Step 2 can be repeated up to three times, statements can be discarded and adjusted in these rounds. Statements with more than 75% agreement will be accepted as consensus statements. (3) Validation of statements by the Delphi-panel. (4) Final recommendations.

Ethics and dissemination

The underlying work is based on existing literature and published data and does not include participation of patients, and thus does not require ethical review approval. The final recommendations are intended for (clinical) researchers in the field of cost-effectiveness in spine surgery. The Delphi method ensures that the final output reflects the opinions of international participants and gives insight in the adherence level to the recommendations. The aim is to reach uniformity in design, conduct and reporting of these studies, as is currently lacking. This will provide a solid basis to determine cost-effectiveness of spine surgeries and consequently aid to limit the rising healthcare costs. The findings of this study and the final recommendations will be disseminated in conferences and seminars and will be published in an international peer-reviewed journal.

Strengths and limitations of this study.

• The multidisciplinary expert group and Delphi panel in this proposed study are formed by selecting authors from relevant publications identified by a systematic review, resulting in a representative panel with limited selection bias.

• The level of agreement to reach consensus and the maximum number of Delphi rounds are predefined to avoid bias in reaching consensus.

• A potential bias may occur as not all members invited will agree to participate.

• In case of live voting at a congress, results may be biased by the specific interest area of the congress, attendance of the voting cohort and presentation of the recommendations.

• The expert meeting may limit thorough discussion of complex problems because of time limits and large-group discussion.

Introduction

Economic evaluations are increasingly important considering the growing healthcare expenses. The number of people aged 60 years or older is expected to double by 2050.¹ As older individuals are more likely to require spine surgery, the amount of spine surgeries is also expected to increase. This, in turn, will result in higher healthcare-related costs.^2–4 To limit the increase of spine surgery-related healthcare costs, scarce healthcare resources should be allocated efficiently. Therefore, the most cost-effective surgical technique should be identified and implemented.^{5 6}

The value of economic evaluations is progressively renowned, as reflected by the observed increase in studies mentioning costs and cost-effectiveness in the last decade.^5–7 However, the variable quality and reporting of these economic evaluations limits their comparability and usefulness. This is mainly a result of heterogeneity in study design, study data and assumptions. An important factor, for instance, is the heterogeneity in determining, calculating and reporting cost data.⁸ Current outcomes of economic evaluations in spine-surgery vary largely in healthcare perspective and societal perspective costs due to differences in calculation methods of costs and/or charges, included costs, different inclusion and exclusion criteria and baseline characteristics.^{9 10} The Panel on Cost-Effectiveness in Health and Medicine in the USA recommends performing cost-effectiveness studies from both the healthcare and the societal perspective.¹¹ Nevertheless, only a minority of economic evaluations report on societal perspective costs.¹²

General guidelines and recommendations regarding the proper conduct of economic evaluations are available, including the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist, the series of Modelling Good Research Practices published by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the recommendations for Conduct, Methodological Practices and Reporting of Cost-effectiveness Analyses from the Second Panel on Cost-Effectiveness in Health and Medicine.^{7 11 13} A limitation of these general guidelines is that by their nature they do not incorporate disease-specific and topic-specific recommendations. As suggested by Carias et al: ‘it is not practical to adopt a single set of criteria for all public health CEAs; one size does not fit all’.¹⁴ Compared and in supplement to the generally accepted methodologic standards, it would thus be beneficial to have disease-specific guidelines to provide additional recommendations. For instance, this has been done for osteoporosis. Only a few disease-specific guidelines regarding the conduct of economic evaluations are available, however, and not in the field of spine surgery.^15–18

Kepler et al reviewed the existing economic evidence in spine surgery in 2012.¹⁹ This study portrays the lack of homogeneous reporting in terms of study design, study population, pathology studied, cost calculations and utility used. Moreover, they observed that only 12% of studies adhered to the recommendations of the US Second Panel of Cost-effectiveness in Health and Medicine. Subsequently, the lack of standardised costing methodology in spine surgery research is also extensively described by Alvin et al.⁸ The lack of several key aspects in cost calculations are described. First of all, the perspective of included costs should be considered. Second, the acquisition and definition of costs should be considered. Payments, charges, costs and expected reimbursements are separate entities that should not be used synonymously. Currently, there is an important difference in the (combined) use of Medicare reimbursements, case-costing databases, cost-to-charge ratios and national costing databases. Finally, the accurate calculation of costs is the time-frame in which costs are measured should be considered. Moreover, the need for appropriate discounting, and consideration of inflation and country is emphasised. They suggest future research could consider to include Net Monetary Benefit and/or Time-based activity-based costing to overcome several limitations in current cost-effectiveness.⁸

More recently, Radcliff et al and Droeghaag et al both described the current literature concerning cost-effectiveness research in cervical spine surgery and lumbar spine surgery, respectively.^{9 10}

In both studies, the authors note that the absence of uniformity in existing literature is apparent.

The aim of a disease-specific guideline, is not only to suggest optimal costing methods and utility measurements, but to incorporate specific disease-related components to these general recommendations.

The lack of homogeneity in economic evaluations regarding spine surgery impedes proper interpretation by healthcare professionals and financial decision-makers. Recommendations to conduct economic evaluations in this field, as a complement to the existing general guidelines, should ameliorate overall research quality, comparability and interpretability.

Therefore, this study has four objectives; (1) to create disease-specific recommendations for the design and conduct of economic-evaluations in spine surgery, (2) to construct recommendations for reporting of economic evaluations in spine-surgery as a complement to the CHEERS checklist, (3) to define a disease-specific reference as a minimum standard for all economic analyses in spine-surgery in order to reduce interstudy heterogeneity and (4) to discuss methodologic challenges and defining the need for future research.

Methods

This study will be conducted according to the RAND/UCLA Appropriateness Method, a modified Delphi Process.^{20 21} A four-step process will be followed to create and validate disease-specific statements and recommendations for the conduct and reporting of economic evaluations in spine surgery.

The authors will form a multicentre research group which will consist of a working group and an advisory board. The working group will consist of researchers who are in charge of conduct of the study (n=5). The advisory panel (between 5 and 10) will consist of experienced researchers, both clinical (eg, neurosurgeons, orthopaedic surgeons, clinical researchers) and health-economic experts. The role of the advisory board is to advise and supervise the conceptualisation of the study, the first drafts of the statements and the conduct of the study.

Systematic literature review

A systematic review is conducted in July 2021 to assess articles concerning general guidelines or recommendations on economic evaluations, or articles concerning economic evolutions in spine surgery. The systematic review is conducted in accordance with the preferred reporting items for systematic reviews and meta-analyses statement.^{22 23} The literature search is conducted using several terms, including, but not limited to: ‘economic evaluation’, ‘cost-effectiveness’ and ‘spine surgery’. The full-search strategy can be found in online supplemental appendix file A. The following databases will be searched: PubMed, Web of Science, Embase, Cochrane, CINAHL, EconLit and NHS-EED.

Identifying relevant studies

Relevant studies will be selected and reviewed based on titles and abstracts. Articles deemed appropriate for inclusion will be reviewed for further analysis. Reviews concerning economic evaluations in spine-surgery, published economic evaluations in spine-surgery, general guidelines for the conduct of economic evaluations and disease-specific guidelines for economic evaluations will be included. Included articles will be cross-referenced. Studies will be selected by two independent reviewers. Duplicates will be removed, potentially eligible studies will be screened on title and abstract and full texts will be assessed using abovementioned eligibility criteria. Level of evidence will be assessed for all relevant studies. Discrepancies between reviewers will be resolved through discussion and with the assistance of a third reviewer if needed.

Identifying relevant authors

Expert group formation

First and last authors will be identified in included articles deriving from the systematic literature search to form the expert panel. This will include those who are most active in publishing, but it may also exclude some of the experts in the field. To prevent this, the first and last identified authors will be asked to consider whether they can propose a more suitable expert in spine surgery and healthcare economics, after which the recommended individual may be included in the expert panel.

The expert group should at least include scientists of the following disciplines: neurosurgery, orthopaedic surgery and health-economics.

In addition, economic experts will be selected from general guidelines and disease-specific guidelines (n=5) by evaluating the number of publications and citations. Clinical experts will be selected from economic evaluations in spine surgery (n=10). Together with the advisory board of the research group, they will form the expert group. The role of the expert group will be to perform a primary validation of statements drafted by the research group. This step is essential to assess external (ie, international) validity of statements drafted by the research group, before approaching a larger group for validation. Finally, members of the research group can also propose potential experts.

All experts will be approached for participation in either the expert group or DELPHI panel through e-mail. This e-mail will include a summary of the study design, the objectives and a request for participation. Written consent will be obtained from all individual experts before participation in the procedure. To assure blinding, experts will not be informed about each other’s participation. We aim to include at least 15 experts. To ascertain an organised group discussion, we maintain a group maximum of 30 experts.

Delphi-panel formation

All authors of included articles will be asked to participate in the online survey. Experts will be asked to propose colleagues, researchers and residents. The number of participants in the DELPHI panel is unlimited.

If possible, attendees of relevant congresses (ie, cost-effectiveness in ortho/neuro/spine-surgery, health economics) will be asked to participate either by a real-time survey.

Drafting first statements

The research group will draft statements based on information provided by included studies in the systematic literature review. Recommendations will be made concerning, but not limited to, the following topics;

1. Design and conduct of trial-based economic evaluations.

   1. Type of economic evaluation.

   2. Method of conduct.

   3. Outcome measures.

      1. Costs.

      2. Utilities.

   4. Treatment characteristics.

      1. Surgical.

      2. Postoperative pharmaceuticals.

      3. Additional therapy.

2. Reporting of economic evaluations, as a complement to the CHEERS checklist.

   1. Outcomes.

   2. Setting.

3. Discussion on methodologic challenges and to define the need for future research.

Validation by expert group

Online survey

Statements drafted by the research group will be sent to the expert group to obtain a level of consensus and feedback. The receival of feedback will take place through a web-based questionnaire. Level of consensus is assessed on a 0–10 scale for each statement, in which 0 means ‘disagree’, 5 means ‘neutral’ and 10 means ‘agree’. Experts may provide comments or feedback on statements if desired. Furthermore, all experts will be given the opportunity to suggest additional statements and will be invited to leave further comments or advice for the research group. No discussion is allowed between the experts at this point of time.

Expert meeting

Subsequently, an expert meeting will be held to discuss statements and feedback provided by the expert group. The expert meeting will be led by a member of the research group. Statements will be accepted if the expert group reaches a consensus of more than 75% on the statement.^{21 24} If consensus cannot be reached on a proposed statement during the expert meeting, the statements can be discarded, adjusted or reformulated. Steps 2 a and 2b can be repeated up to three times.^{21 24}

Validation by DELPHI panel

The Delphi method is a structured process, commonly used to develop healthcare quality indicators and consists of four key components; iteration, controlled acquisition of feedback, aggregation of responses and anonymity. As anonymity might not always be applicable in our situation, we used the term modified.^{21 25}

All consensus statements are gathered and will be sent to the complete DELPHI panel for final evaluation and validation. Again, all statements reaching consensus of more than 75% will be accepted for the final report. The DELPHI panel will also have the possibility to comment on all statements. This process of evaluation and validation by the DELPHI panel can be repeated multiple times if deemed necessary. If possible, statements will be presented at a congress concerning cost-effectiveness in spine-surgery, to reach a broader audience. Attendees can then vote using a web-based tool to score level of agreement.

Final report on outcomes

The research group will report on all consensus statements in the form of final recommendations for economic evaluations in spine surgery. This is done preferably in an open-access peer-reviewed renowned scientific journal. A spine-specific checklist can be designed, which includes items to report when performing an economic evaluation. Encountered methodologic challenges and need for further research will be discussed.

Patient and public involvement

No patient involved.

Ethics and dissemination

The underlying work is based on existing literature and published data and does not include participation of patients, and thus does not require ethical review approval. The final recommendations are intended for (clinical) researchers in the field of cost-effectiveness in spine surgery. However, they can also serve as an example for other disease-specific guidelines. Considering the number of publications addressing the lack in standardised methodology and reporting of cost-effectiveness in spine surgery, the demand for disease-specific guidelines for cost-effectiveness research in spine surgery appears to be high. The Delphi process ensures that researchers in the field are informed of the existence of the project and expected guidelines. Moreover, the Delphi method ensures that the final output reflects the opinions of international participants and gives insight in the adherence level to the recommendations. The aim is to reach uniformity in design, conduct and reporting of these studies, as is currently lacking. This will provide a solid basis to determine cost-effectiveness of spine surgeries and consequently aid to limit the rising healthcare costs. The findings of this study and the final recommendations will be disseminated in conferences and seminars and will be published in an international peer-reviewed journal.

Ethics statements

Patient consent for publication

Not required.