Quality of Reporting in Economic Evaluations of Interventions to Prevent Dental Caries Needs Improvement

SUMMARY

Selection Criteria

The authors searched electronic databases (MEDLINE, EconLit, ISI, Cochrane Library, NHS Economic Evaluation Database (NHSEED), Centre for Reviews and Dissemination, EMBASE, LILACS and Scielo) for full economic evaluations written in English, Spanish, Portuguese, or Italian that were published between January 1975 and April 2012. Bibliographies of all retrieved articles were hand-searched and additional studies known to the authors were also included.

Key Study Factors

Among the economic evaluations included, 30 were cost-effectiveness analyses, 22 were cost-benefit analyses, 5 included both cost-effectiveness and cost-benefit analyses, 2 were cost-minimization analyses, and 4 were cost-utility analyses.

Main Outcome Measure

The major outcomes considered were frequency of study characteristics, percentage of studies meeting each individual quality criterion, and mean scores for groups of quality criteria. Quality criteria came from a 35-item checklist developed by Drummond et al.¹ Each checklist item fell into one of three groups – study design (7 items), data collection (13 items), or analysis and interpretation of results (15 items).

Main Results

Of the 206 publications located in the search, 63 unique evaluations were included in this review. The majority of studies were published after 2000. The most frequently occurring interventions in economic evaluations were dental sealants (n = 13) and community water fluoridation (n = 12).

Quality elements most commonly omitted from the economic analyses were information on adjustments for discounting and sensitivity analyses. The mean score across economic evaluations for study design was 4.9 (SD 1.4) out of a maximum score of 7, for data collection was 6.4 (SD 2.7) out of 13, and for analysis and interpretation of results was 8.7 (SD 3.0) out of 15. The total average score for all 3 sections was 19.9 (SD 6) out of 35. No study scored 35 points and only 6 studies scored 30 or higher.

Conclusions

The authors conclude that the quality of reporting in economic evaluations of interventions to prevent dental caries needs to be improved.

COMMENTARY AND ANALYSIS

This review employs a different perspective than the typical systematic review in that it does not synthesize findings of economic analyses of interventions but instead presents summary statistics for the quality of research reported in such syntheses. Quality criteria appear to be related to reporting as opposed to conduct of economic evaluation (e.g., quality criteria relate to whether items such as study perspective and discounting were discussed and justified as opposed to recommending a specific perspective or discount rate as was done in the 1996 recommendations of the Panel on Cost-effectiveness in Health and Medicine²).

Although there have been at least two systematic reviews of economic evaluations of interventions to prevent dental caries,^3,4 this is the first to have used quality of the study as the outcome measure. Given the increased attention to reducing healthcare costs, the increasing emphasis on the use of interventions with evidence of effectiveness and cost-effectiveness, and the increasing sophistication and application of meta-analysis, this review opens a crucial dialog about the need to improve the reporting of findings from economic evaluations of oral health interventions to increase their usefulness for continued scientific discovery. Because the findings of this review are not directly applicable to dental practice, they are likely to be more relevant to researchers, journal reviewers, and policy makers than to practitioners. This review provides useful information on the overall quality of economic evaluations, which could inform policy makers’ or program managers’ decision making and highlight pitfalls future researchers could avoid in reporting such evaluations. Indeed, both journal editors and policy makers have expressed concerns regarding the lack of transparency and structure in reporting economic evaluations of interventions to improve health.⁵

While this review found that the quality of reporting of results from economic evaluations of caries prevention interventions could be improved, this need does not appear to be unique to the field of dentistry.⁵ Although there is no gold standard evaluation tool for economic evaluations (at least 6 quality checklists exist⁶), new guidance to evaluate the quality of reporting of economic evaluations was released this year – the Consolidated Health Economic Evaluation Reporting Standards (CHEERS⁵). A task force, consisting of health economics journal editors and content experts from around the world, consolidated and updated previous guidelines into one current guideline. Although Marino et al's review of economic evaluations of caries prevention interventions was completed before the release of CHEERS, it used a checklist by Drummond et al¹ that is very similar to CHEERS. CHEERS, however, includes additional items relating to the authors’ disclosure of funding sources and conflicts of interest, sufficient information in article titles to facilitate study retrieval in literature searches for economic evaluations, and sufficient information in structured abstracts for readers to determine if the evaluation is relevant to their needs.

To make their work even more valuable, we encourage the authors of this review to make available a detailed bibliography of the studies that were the subject of their review, thus providing an excellent starting point for researchers interested in synthesizing such findings and eliminating duplication of search effort. The reader is asked to accept the validity of the current review without knowing the identity of the included studies.

The study also could be extended by reporting the quality findings by type of intervention, so users of this information understand the quality of the economic studies for the particular intervention they are considering. In addition, information on the overall quality of economic evaluations for a given intervention might result in more efficient allocation of resources, with researchers being more likely to conduct economic evaluations of those interventions where currently no high quality evaluations exist as opposed to duplicating findings for interventions with a strong evidence base for economic efficiency.

ARTICLE TITLE AND BIBLIOGRAPHIC INFORMATION.

Systematic review of publications on economic evaluations of caries prevention programs.

Marino RJ, Khan AR, Morgan M. Caries Res 2013;47(4):265-72.

REVIEWERS

Susan O. Griffin, PhD, Kari Jones, PhD

PURPOSE/QUESTION

The purpose of this study was to review the literature on economic evaluations of interventions for the prevention of dental caries with the intention of describing the characteristics (e.g., intervention evaluated, type of economic evaluation, and year of publication) of these studies and to assess their quality. The quality of economic evaluation was assessed using a checklist developed by Drummond et al.¹ This review did not present economic findings for individual studies or synthesize findings across studies.

TYPE OF STUDY/DESIGN

Systematic review of study characteristics and quality

LEVEL OF EVIDENCE

Not applicable

STRENGTH OF RECOMMENDATION GRADE

Not applicable