Table 3.

Summary of the analysis of the pathway benchmarking projects

Author	Study design	Benchmarking model and/or steps	Indicators	Outcomes	Impact (improvements/improvement suggestions)	Success factors
Brucker [27]	Prospective interventional multi-centre feasibility study.	Partner: Industry Content: Performance Purpose: Collaborative Independent, scientific benchmarking system. Nine guideline-based quality targets serving as rate-based QIs (Quality Indicators) were initially defined, reviewed annually and modified or expanded accordingly. QI changes over time were analyzed descriptively	Quality outcome indicators derived from clinically relevant parameters.	The results from this study provide proof of concept for the feasibility of a novel, voluntary, nationwide system for benchmarking the quality of BC care	Marked QI (Quality Indicators) increases indicate improved quality of BC care.	The project was voluntary and all data was anonymized.
Chung [17]	Multi comparisons study and the development of core measures for colorectal cancer including a modified Delphi method.	N.A.	Quantitative structure, process and outcome indicators	Developing core measures for cancer care was a first step to achieving standardized measures for external monitoring, as well as for providing feedback and serving as benchmarks for cancer care quality improvement.	N.A.	N.A.
Hermann [18]	Multi comparisons study and indicator consensus development process (with elements of the Delphi method).	Partner: Industry/Global Content: Performance/Process Purpose: Collaborative Development of indicators for benchmarking.	Process and outcome indicators.	The bench mark was not performed, indicators were developed for a possible benchmark.	N.A.	N.A.
Mainz [19]	Multi comparisons study and the development of indicators based on consensus of a working group	N.A. The results that are available for the prioritized quality indicators cannot really be used for true comparisons and benchmarking	Outcome indicators	A major difference between the Nordic countries has been identified with regard for 5 years survival for prostate cancer.	N.A.	N.A.
Miransky [20]	Multi comparisons study with stakeholder consensus methods. Use of a specialized database for benchmarking outcomes for cancer patients. Conference calls and joint meetings between comprehensive cancer centers and possible benchmark vendors were used to develop this benchmarking database.	Partner: Industry Content: Performance Purpose: Collaborative	Development of a database containing outcome indicators. Benchmarking clinical outcomes and patient	The various databases developed by the collaborative provided the tools through which the group accomplished its goals.	Each consortium member is expected to participate in one quality improvement initiative annually	N.A.
Roberts [28]	Multi comparisons study on staffing and inpatient data at hospices. Study design drew extensively from a UK-wide nursing study (The UK Best practice Nursing Database).	N.A.	Mixture of indicators, both qualitative and quantitative and process and outcome indicators	A broader NHS ward data system, was successfully converted for hospice use. The resultant hospice and palliative care ward data show that, compared to NHS palliative care wards, charitable hospices: (i) look after fewer patients, but generate greater workloads owing to higher patient-dependency and acuity scores; (ii) are much better staffed; and (iii) achieve higher service-quality scores.	N.A.	N.A.
Setoguchi [24]	Retrospective and prospective cohort study.	Partner: Industry Content: Performance Purpose: Collaborative Defined benchmark measures for the quality of end-of-life cancer care previously developed by Earle et al. New measures were defined for the use of opiate analgesia, which included the proportion of patients who received an outpatient prescription for a long-acting opiate; a short-acting or a long-acting opiate; or both a short acting and a long-acting opiate.	Outcome indicators	Retrospective and prospective measures, including a new measure of the use of opiate analgesia, identified similar physician and hospital patterns of end-of-life care.	Findings suggest that the use of opiates at the end of life can be improved	N.A.
Stewart [21]	Multi comparisons study (clinical productivity and other characteristics of oncology physicians). Data collection by survey	Partner: Industry Content: Performance Purpose: Collaborative Established productivity benchmarks. The clinical productivity and other characteristics were reviewed of oncology physicians practicing in 13 major academic cancer institutions.	Outcome productivity indicators	Specific clinical productivity targets for academic oncologists were identified. A methodology for analyzing potential factors associated with clinical productivity and developing clinical productivity targets specific for physicians with a mix of research, administrative, teaching, and clinical salary support.	N.A.	N.A.
Stolar [22]	Multi comparisons study using a non-searchable anonymous data capture form through SurveyMonkey. Feedback from stakeholders and availability of information was used to develop indicators. A final questionnaire, containing 17 questions, was send to thirty pediatric surgery practices.	N.A.	Quantitative outcome indicators	A review of the clinical revenue performance of the practice illustrates that pediatric surgeons are unable to generate sufficient direct financial resources to support their employment and practice operational expenses.	The value of the services must accrue to a second party	N.A.
Van Vliet [23]	A retrospective comparative benchmark study with a mixed-method design	Partner: Industry/Global Content: Strategic Purpose: Collaborative The method comprised of 6 steps: (1) operational focus; (2) autonomous work cell; (3) physical lay-out of resources; (4) multi-skilled team; (5) pull planning and (6) elimination of wastes.	N/A	The environmental context and operational focus primarily influenced process design of the cataract pathways.	When pressed to further optimize their processes, hospitals can use these systematic benchmarking data to decrease the frequency of hospital visits, lead times and costs.	N.A.
Wallwiener [25]	Review of existing literature/data.	Partner: Industry Content: Performance Purpose: Collaborative Phase 1: Benchmarking; Phase 1a: proof of principle: Develop quality indicators; Phase 1b: analysis for a single specific specialty: to demonstrate the feasibility of subgroup analysis. Phase 2: certification of breast centres: to implement a quality management system to assess structural, process and outcome quality. Phase 3: nationwide implementation of certified breast centres.	Structural and process indicators	The voluntary benchmarking programme has gained wide acceptance among DKG/DGS-certified breast centres. The goal of establishing a nationwide network of certified breast centres in Germany can be considered largely achieved.	Improvements in surrogate parameters as represented by structural and process quality indicators suggest that outcome quality is improving.	N.A.
Wesselman [26]	Review of existing literature/data. Analysis of existing benchmarking reports of cancer centers.	Partner: Industry Content: Performance Purpose: Collaborative Analysis of benchmarking reports by the certified centers with the OnkoZert data which reflects the centers’ reference results over a period of 3 years. The data for these reports are collected by the centers using an electronic questionnaire and are submitted to OnkoZert. (an independent institute that organizes the auditing procedure on behalf of the DKG)	Respective and guideline-based outcome indicators	The present analysis of the results, together with the centers’ statements and the auditors’ reports, shows that most of the targets for indicator figures are being better met over the course of time.	There is a clear potential for improvement and the centers are verifiably addressing this.	N.A.

N.A. = Not applicable