Application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis

Abstract

Background

The use of Multi‐Criteria Decision Analysis (MCDA) in health care has become common. However, the literature lacks systematic review trend analysis on the application of MCDA in health care.

Aim

To systematically identify applications of MCDA to the areas of health care, and to report on publication trends.

Methods

English language studies published from January 1, 1980 until October 1, 2013 were included. Electronic databases searches were supplemented by searching conference proceedings and relevant journals. Studies considered for inclusion were those using MCDA techniques within the areas of health care, and involving the participation of decision makers. A bibliometric analysis was undertaken to present the publication trends.

Results

A total of 66 citations met the inclusion criteria. An increase in publication trend occurred in the years 1990, 1997, 1999, 2005, 2008, and 2012. For the remaining years, the publication trend was either steady or declining. The trend shows that the number of publications reached its highest peak in 2012 (n = 9). Medical Decision Making was the dominant with the highest number published papers (n = 7). The majority of the studies were conducted in the US (n = 29). Medical Decision Making journal published the highest number of articles (n = 7). Analytic Hierarchy Process (n = 33) was the most used MCDA technique. Cancer was the most researched disease topic (n = 12). The most covered area of application was diagnosis and treatment (n = 26).

Conclusion

The review shows that MCDA has been applied to a broad range of areas in the health care, with the use of a variety of methodological approaches. Further research is needed to develop practice guidelines for the appropriate application and reporting of MCDA methods.

Background

The significance of decision making in health care cannot be stressed enough as many of these decisions are complex, involve uncertainties, and the elicitation of stakeholders' preferences and values. Several methods have been proposed to aid and support the decision‐making process in health care. Multicriteria decision analysis (MCDA) represents one of the most frequently used decision‐making frameworks.1, 2 MCDA is often described as a process utilizing a set of qualitative and quantitative approaches that simultaneously and explicitly take into account multiple and often conflicting factors.3 The use of MCDA is rapidly increasing because of its potential for improving the quality of decisions by making the decision process more explicit, rational and efficient than traditional deliberative processes.4

MCDA frameworks have been successfully applied to solve decision problems in many areas, including sustainable energy management,5, 6 energy planning,7, 8 transportation,9, 10 geographical information systems,11, 12 budgeting and resource allocation.13, 14 Details on conducting and using MCDA are discussed in other publications.1, 2, 15, 16, 17, 18, 19, 20, 21

MCDA is increasingly becoming a popular framework for aiding and supporting health‐care decision making. The literature includes some reviews of the application of MCDA in health care. Shim,22 provided a comprehensive bibliographical survey of studies on the analytic hierarchy process (AHP). Vaidya and Kumar,23 looked into research papers in an attempt to understand the spread of the AHP applications in different fields. Ho,24 surveyed the applications of the integrated AHPs through a literature review and classification of the international journal articles from 1997 to 2006. Liberatore and Nydick,25 presented a literature review of the application of the AHP to important problems in medical and health‐care decision making. Guindo et al.26 identified decision‐making criteria and its frequency in health‐care literature. Diaby et al.27 documented MCDA applications in health care and identified publication patterns, as well as the range of topics to which MCDA has been applied. Recently, Marsh et al.28 conducted a review of the literature to assess the value of health‐care interventions using MCDA. While these reviews have significantly contributed to the MCDA literature, a systematic review is needed. The aim of this study is twofold: (i) to systematically identify applications of MCDA to the areas of the health care; (ii) to report on the publication trends of MCDA in health care based on the identified bibliographical records.

Methods

Systematic review

Eligibility criteria

A search of the literature was conducted to identify studies applying MCDA techniques within a health‐care context between 1 January 1980 and 1 December 2013. This time frame was set to capture a wide range of studies. Specifically, the time frame will introduce earlier and more recent publications that are not known to have been included in existing reviews. Furthermore, the review seeks to be inclusive of all MCDA techniques, as opposed to focusing on a particular technique. The search strategy was restricted to English language studies. To be included in the review, studies had to contain a description of the MCDA method, answer a health‐care question and elicit stakeholders' (e.g. policymaker, provider and researcher) preferences and/or values.

Search strategy

Relevant studies were identified using the following electronic databases: Excerpta Medica Databases (EMBASE), Cumulative Index of Nursing and Allied Health Literature (CINAHL), Medical Literature Analysis and Retrieval System Online (MEDLINE), Public Medline (PubMed), Web of Knowledge and ProQuest. Databases were searched using search terms comprising ‘multicriteria decision analysis OR MCDA’, ‘multiple criteria decision aiding’, ‘multicriteria decision making OR MCDM’, ‘multicriteria analysis’, ‘multiattribute utility OR MAU’, ‘multiattribute utility theory OR MAUT’, ‘weighted product method’, ‘analytic hierarchy process OR AHP’, ‘analytic network process OR ANP’, ‘measuring attractiveness by a categorical based evaluation technique’, ‘goal programming’, ‘elimination and choice expressing reality OR ELECTRE’, ‘preference ranking organization method of enrichment evaluation OR PROMETHEE’, ‘technique for order preference by similarity to ideal solution OR TOPSIS’, ‘weighted product model OR WPM’ and ‘measuring attractiveness by a categorical based evaluation technique OR MACBETH’. This list of terms reflects the different terms used to refer to MCDA in the literature. The search terms were used in conjunction with health‐ and medical‐related words: ‘health’, ‘health care’, ‘healthcare’, ‘medical decision’, ‘medical decision making’, ‘medicine’, ‘medication’, ‘disease’ and ‘pharmacy’, using Boolean operators (AND, OR).

In addition to electronic databases, the following conference proceedings were hand searched: Society for Medical Decision Making (SMDM); Health Technology Assessment International (HTAi); International Society for Pharmacoeconomics and Outcomes Research (ISPOR); International Symposium on the Analytic Hierarchy Process (ISAHP); and International Conference on Operations Research and Enterprise Systems (ICORES).

The reference lists of previously published review of the literature and studies identified through the electronic databases search were also scrutinized for relevant citations. Search for the specified period was undertaken in relevant journals comprising International Journal of Multicriteria Decision Making (IJMCDM), Journal of Multicriteria Decision Analysis (JMCDA), European Journal of Operational Research (EJOR), American Journal of Operations Research (AJOR), International Journal of Operations Research (IJOR), decision support systems (DSS), International Journal of Technology Assessment in Health Care, BMC Medical Informatics and Decision Making, Medical Decision Making (MDM), Therapeutics and Clinical Risk Management and Operations Research for Health Care and Value in Health.

Identified studies were independently examined by two reviewers (G.A and A.A) to determine whether they met the pre‐specified eligibility criteria, and disagreements were resolved by a third party (V.D).

Bibliometric analysis

bibliometric analysis was carried out to describe and analyse the trends of MCDA applications in the areas of health care. The trends that were analysed include health care by year of publication, journal source, country of publication, MCDA technique, type of intervention and application area. The analysis was performed using Microsoft Excel 2007, (Microsoft Corporation, Redmond, Washington, US).

Results

Systematic review

The literature search identified 205 publications (Fig. 1). Of these, 23 duplicates were excluded. The screening of titles excluded 28, the screening of abstracts excluded 41, and the screening of the full texts further excluded 47. Full texts were excluded on the basis that they did not provide a description of the MCDA method, or elicit stakeholders' preferences and/or values. A total of 66 publications met the inclusion criteria and were reviewed, including 61 articles, four dissertations and one technical report.

Figure 1.

Flow diagram describing study selection.

All the publications included in the review were organized by the year of (starting with the most recent), journal, MCDA technique, intervention/decision, area of application and country as shown in Table 1.

Table 1.

Study characteristics

Author (s)/Year	Journal	MCDA Technique	Intervention/Decision	Application Area	Country
Diaz‐Ledezma and Parvizi (2013)	Clinical Orthopaedics and Related Research	AHP	Cam femoroacetabular impingement	Diagnosis and treatment	USA
Dionne et al. (2013)	Cost Effectiveness and Resource Allocation	Not specified	Physiotherapy practices	Diagnosis and treatment	Canada
Maruthur et al. (2013)	F1000 Research	AHP	Diabetes	Diagnosis and treatment	USA
Pecchia et al. (2013)	BMC Medical Informatics and Decision Making	AHP	Computed tomography (CT) scanner	HTA	UK
Ramli et al. (2013)	Therapeutics and Clinical Risk Management	MAST	Cardiovascular diseases/statins	Formulary management	Malaysia
Defechereux et al. (2012)	BMC Health Services Research	DCE	Values of the health policymakers	Priority setting	Norway
Erjaee et al. (2012)	Hong Kong Journal of Paediatrics (new series)	AHP	Helicobacter pylori	Diagnosis and treatment	Iran
Goetghebeur et al. (2012)	Medical Decision Making	EVIDEM	Medicines appraisal	HTA	Canada
Hummel et al. (2012)	The Patient: Patient‐Centered Outcomes Research	AHP	Antidepressant drug	Diagnosis and Treatment	Germany
Lin et al. (2012)	Journal of Multi‐Criteria Decision Analysis	AHP	Organ transplant	Organ Transplantation	USA
Marsh et al. (2012)	Journal of Public Health	DCE	Preventative health interventions	Public health and policy	UK
Miot et al. (2012)	Cost Effectiveness and Resource Allocation	EVIDEM	Cervical cancer screening	Diagnosis and treatment	South Africa
Sullivan (2012)	University of Otago	PAPRIKA	Prioritization criteria	Priority setting	New Zealand
Youngkong et al. (2012)	Value in Health	Not specified	Universal health coverage	Priority setting	Thailand
Cunich et al. (2011)	The Patient: Patient‐Centered Outcomes Research	Rating scale	Prostate cancer screening	Diagnosis and treatment	Australia
Danner et al. (2011)	International Journal of Technology Assessment in Health Care	AHP	Antidepressant treatment	HTA	Germany
Dehe et al. (2011)	Annual Production and Operations Management Society	Evidential Reasoning	Health care centre	Site selection	UK
Diaby et al. (2011)	Applied Health Economics and Health Policy	DCE	Drug reimbursement	Formulary management	Côte d'Ivoire
Tony et al. (2011)	BMC Health Services Research	EVIDEM	Tramadol for chronic non‐cancer pain	HTA	Canada
Chung et al. (2010)	American Journal of Health‐System Pharmacy	MAUT	Hypertension/dihydropyridine CCBs and ARBs	Formulary management	South Korea
Goetghebeur et al. (2010)	Cost Effectiveness and Resource Allocation	EVIDEM	Growth hormone for Turner syndrome	HTA	Canada
Nutt et al. (2010)	The Lancet	Scoring	Drugs (alcohol and tobacco products)	Public health and policy interventions	UK
Vidal et al. (2010)	Expert Systems with Applications	AHP	Anticancer drugs	Diagnosis and treatment	France
Young (2010)	Advances in Intelligent Decision Technologies	AHP	Health service	GIS	Canada
Filho et al. (2009)	Evolutionary Multi‐Criterion Optimization/Springer	ELECTRE IV	Alzheimer	Diagnosis and treatment	Brazil
Suehs et al. (2009)	The American Journal of Managed Care	MADM	Bipolar disorder/Mood‐Stabilizing Medications	Formulary management	USA
Zuniga et al. (2009)	Journal of Health and Human Services Administration	Ranking	Health disparities	Public health and policy	USA
Chang et al. (2008)	The Clinical Journal of Pain	MAUT	Epidural analgesia	Pain management	Taiwan
Enyinda (2008)	North Dakota State University	AHP	Management pharmaceutical global supply chain logistics	Supply chain	USA
Jehu‐Appiah et al. (2008)	Value in Health	DCE	Prioritization of interventions	Priority setting	Ghana
Pinheiro et al. (2008)	Computational Science and Engineering	MACBETH	Alzheimer	Diagnosis and treatment	Brazil
Van til et al. (2008)	The Patient: Patient‐Centered Outcomes Research	SMART	Cognitive impairment	Diagnosis and treatment	USA
Van Til et al. (2008)	Archives of Physical Medicine and Rehabilitation	AHP	Acquired equinovarus deformity	Diagnosis and treatment	the Netherlands
Bettinger et al. (2007)	The Annals of Pharmacotherapy	MAUT	Schizophrenia/Atypical antipsychotics	Formulary management	USA
Peacock et al. (2007)	Social Science & Medicine	MAU	PBMA	Priority setting	Australia
Hariharan et al. (2005)	Journal of Critical Care	AHP	Intensive care units	Performance measurement	Barbados, Trinidad, and India
Hummel et al. (2005)	Journal of Rehabilitation Research & Development	AHP	Upper limb in tetraplegia	Diagnosis and treatment	USA
Richman et al. (2005)	The Journal of Urology	AHP	Prostate cancer treatment selection	Diagnosis and treatment	USA
Anthony et al. (2004)	Quality Management in Healthcare	MAUT	Nursing practice	Professional practice	USA
Chang et al. (2004)	Expert Systems with Applications	AHP	Discharge planning	Performance measurement	Taiwan
Cho and Kim (2003)	International Journal of Health Planning and Management	AHP	Medical devices and materials	HTA	South Korea
Liberatore et al. (2003)	Computers & Operations Research	AHP	Prostate cancer screening	Diagnosis and treatment	USA
Dolan and Frisina (2002)	Medical Decision Making	AHP	Colorectal cancer screening	Diagnosis and treatment	USA
Zachry et al. (2002)	Formulary	MAUT	Anticonvulsants	Formulary management	USA
Rossetti et al. (2001)	Computers & Industrial Engineering	AHP	Hospital distribution services	Medical Automation	USA
Wenstøp and Magnus (2001)	Health Policy	Not specified	Aids	Public health and policy	Norway
Azar (2000)	Scholarly Commons at the University of Pennsylvania	SAW,WPM, TOPSIS	Breast Cancer	Diagnosis and treatment	USA
Bots and Hulshof (2000)	Journal of Multi‐Criteria Decision Analysis	SMART	Ranking of diseases/efficiency improvements	Priority setting	the Netherlands
De Bock et al. (1999)	Medical Decision Making	MAU	Sinusitis and rhinitis	Diagnosis and treatment	the Netherlands
Carter et al. (1999)	Medical Decision Making	AHP, ANP	Breast cancer	Diagnosis and treatment	USA
Lee and Kwak (1999)	Journal of the Operational Research Society	AHP	Health‐care system	Resource planning	USA
Nobre et al. (1999)	Statistics in Medicine	TODIM	Health technology procurement	HTA	Brazil
Singpurwalla et al. (1999)	Socio‐Economic Planning Sciences	AHP	Estrogen replacement therapy/cosmetic eyelid surgery	HTA	USA
Shaw et al. (1998)	International Journal of Obstetrics & Gynaecology	MAU	Menorrhagia	Diagnosis and treatment	UK
Stowers (1999)	ProQuest Dissertations and Theses	AHP, ANP	Abdominal pain	Diagnosis and treatment	USA
Min et al. (1997)	Socio‐Economic Planning Sciences	AHP	Cancer	Public health and policy interventions	USA
Peralta‐Carcelen et al. (1997)	Archives of Pediatrics and Adolescent Medicine	AHP	Neonatal group B streptococcal (GBS) sepsis.	Priority setting	USA
Koch and Rowell (1997)	Pediatric Nursing	AHP	Organ transplant eligibility	Organ transplantation	Canada
Kwak et al. (1997)	Journal of Medical Systems	AHP	Hospital laboratory personnel	Resource planning	USA
Weingarten et al. (1997)	Academic Medicine	AHP	Selection general surgery residents	Resource planning	USA
Dolan (1995)	Medical Decision Making	AHP	Colon cancer	Diagnosis and treatment	USA
Dolan et al. (1994)	Medical Decision Making	AHP	Tuberculosis	Diagnosis and treatment	USA
Dolan (1990)	Journal of Clinical Epidemiology	AHP	Idiopathic nephrotic syndrome	Diagnosis and treatment	USA
Gales and Moatti (1990)	International Journal of Technology Assessment in Health Care	ELECTRE IS	Hemoglobinopathies	Diagnosis and treatment	France
Dolan (1989)	Medical Decision Making	AHP	Acute pyelonephritis	Diagnosis and treatment	USA
Hannan et al. (1981)	Socio‐Economic Planning Sciences	AHP	Long‐term care facilities	Priority setting	USA

Bibliometric analysis

A bibliometric analysis was undertaken to present the publication trends of MCDA methods in health care by year of publication, journal source, country of publication, MCDA technique, type of intervention and application area.

The citations meeting the inclusion criteria were published from 1981 to 2013. Figure 2 suggests important fluctuations in the publication trend during that time. An increase in publication trend occurred in the years 1990, 1997, 1999, 2005, 2008 and 2012. For the remaining years, the publication trend was either steady or declining. The trend shows that the number of publications reached its highest peak in 2012 (n = 9). Furthermore, the correlation coefficient, which expresses the degree that the variables ‘number of published articles’ and ‘year’ change correspondingly, was 0.71 (P < 0.003). As a result, the coefficient of determination value R ² = 0.51. These indicators suggest a statistically significant and steady increase in the number of published articles over the review time horizon and that the exponential model derived from Fig. 2 explains 51% of the variation in the number of publication.

Figure 2.

Number of publications by year.

The 66 publications were distributed among 47 journals that covered a wide range of areas. Medical decision making was the dominant with the highest number published papers (n = 7), followed by cost‐effectiveness and resource allocation (n = 3), The Patient: Patient‐Centred Outcomes Research (n = 3), socio‐economic planning sciences (n = 3). The remaining thirty‐one journals had one or two publications each.

All retrieved documents were published from 20 countries (Fig. 3). It indicates that the largest number of publications was from the United States (n = 29), followed by Canada (n = 6), the UK (n = 5), the Netherlands (n = 3) and Brazil (n = 3).

Figure 3.

Number of publications by country.

The retrieved publications used a wide range of MCDA techniques with the AHP (n = 33) being the most used, followed by the MAU/MAUT (n = 8).

The retrieved publications covered a total of 60 interventions or disease areas. Cancer was the most researched disease topic, represented by 12 (18%) articles. The other most researched topic was depression, represented by 6 (9%), followed by Alzheimer 2 (3%).

The retrieved publications covered 14 areas of application (Fig. 4). The top four areas of applications covered were disease diagnosis and treatment (n = 26 or 39%), followed by priority setting (n = 8 or 12%), health technology assessment (n = 8 or 12%) and formulary management (n = 6 or 9%).

Figure 4.

Percentage of publications by area of application.

Discussion

The current systematic review and bibliometric analysis of studies applying MCDA to the area of health‐care spanning 33 years, evaluated a total of 66 studies. The systematic review identified a substantial number of publications, and the bibiometric evidence presented is very optimistic concerning the growth of MCDA. The retrieved publications addressed a wide range of decision problems and used various MCDA methods. A number of studies examined other attributes (criteria) in the MCDA decision‐making process beyond those that are traditionally typical to the health‐care domain. Traditional health‐care decision making tools are largely viewed as tools that inform health professionals' or health‐care organizations' decisions instead of stimulating patient involvement.

Involving patients in the decision‐making process could make a potentially significant difference in health outcomes and reduce cost of care. It is worth nothing that patients' involvement is not intended to transfer power to patients, but to endorse the decisions of clinicians and policymakers. As such, mechanisms to involve patients in decision‐making processes need to be established.

The finding about the majority of research published in scientific journals being in medical decision making is not a surprise given that it is the official journal of the Society for Medical Decision Making, thus represents the flagship journal of this particular research field. It is obvious that MCDA research became more global based on the fact that the publications covered different world regions. Findings about the United States ranked first in terms of number of publications can be attributed to several factors, including the priority that has been given to improving the quality of health care and increasing the value of health expenditure. Our study reveals a significant use of the AHP certainly because it is very flexible, helps capture both subjective and objective aspects of a decision, and countless software have been developed to suit this method. Cancer was the most researched disease topic because it is an important health problem globally. Based on the International Agency for Research on Cancer, there were an estimated 14.1 million new cancer cases and 8.2 million cancer‐related deaths in 2012, compared with 12.7 and 7.6 million, respectively, in 2008.30 The cancer burden is growing at an alarming pace and emphasizes the need for urgent implementation of efficient prevention strategies to curb the disease. In the past years, the advances in technologies, better understanding of the natural history of diseases have led to progress in diagnostic procedures and the refinement of treatment parameters. With these advances, increased attention is being paid to evidence‐based medicine and may explain why disease diagnosis and treatment were the most covered areas in the retrieved documents.

This study also confirms the findings of prior work in that MCDA has the potential to improve decision making. The current work differs from the prior bibliometric analysis27 and the review of the literature28 that have documented MCDA applications in health care. First, the present study consisted of a systematic review. In contrast to the prior reviews, the systematic review used a more rigorous and well‐defined approach to reviewing the literature.29 Indeed, the protocol developed prior to the review helped to minimize biases. Second, this study sought to comprehensively review the literature by setting a larger time frame to capture a larger variety of studies.

The application of MCDA has been considered by several public and private health‐care organizations and agencies including the US Agency for Healthcare Research and Quality's (AHRQ),31 the Canadian Agency for Drugs and Technologies in Health (CADTH),32 the UK Department of Health,33 the National Institute for Health and Care Excellence (NICE) in England and Wales,34 the UK Office for Health Economics (OHE),35 the German Institute for Quality and Efficiency in HealthCare (IQWiG),36 the International Health Policy Programme (IHPP) and the Health Intervention and Technology Assessment Programme (HITAP) in Thailand.37, 38 These organizations/agencies have used and proposed MCDA as an approach to: incorporate stakeholder preferences in comparative effectiveness research (AHRQ),31 assess new health technologies,32 prioritize investment in public health interventions,33 assess orphan drugs,35 support benefit risk assessment and weigh the multiple endpoints considered in the assessment of quality and efficiency in health care,36 develop universal coverage health benefit package.37

The findings from this study should be considered in light of potential limitations. First, the systematic review focused on English language publications; thus, relevant publications in other languages were not included. Although this may suggest that the review was not far‐reaching, it plausibly captured the majority of papers that met the inclusion criteria. Second, it cannot be presumed that the search strategy, despite being inclusive, identified all publications. Nonetheless, a great number of electronic databases were used, bibliographies were hand searched, and experts in the field were contacted. Third, the lack of standards for reporting on important aspects of MCDA may have undermined the quality of some publications. Even though the methodological quality of the publications included in the review was not appraised, at face value, the manner in which the studies were conducted appears to be relatively sound. Fourth, the publication bias favouring optimistic findings may not be underestimated.

Conclusions

The evidence presented in this review makes a valuable contribution to discussions about research methodology and best practices for decision making in health care. The evidence also tends to suggest that MCDA provides a sound and rigorous approach for decision making in health care. There is no definitive solution for improving the decision‐making process in health care; nevertheless, the use of tools such as MCDA will be a step further. MCDA offers the potential to overcome the challenges of traditional decision‐making tools, especially when making complex decisions that include multiple criteria, simultaneously consider quantitative and qualitative data, and involve multiple stakeholders. A suggestion for future research is to define how MCDA compares to other decision‐making support tools, generate a resource to select the most appropriate method depending on the research question and assess its external validity. There is also a need for initiating and developing guidelines on good practice for MCDA and on the use of this method in health‐care decision making.

Funding

This research received no funding.

Conflicts of interest

No conflict of interest has been declared.