Table 1.
Results of the questionnaire used to assess the status of outcome-based quality improvement in heart centres in the Netherlands
| Heart centres with thoracic surgery | Heart centres without thoracic surgery | Total | |||
|---|---|---|---|---|---|
| No. | Questions | Answer options | N = 11 | N = 9 | N = 20 |
| 1. Monitoring outcomes | |||||
| 1 | How frequent are health outcomes measured and discussed within the heart centre? (for one or more medical conditions) | Not | 0% | 0% | 0% |
| Less than once a year | 0% | 0% | 0% | ||
| Once a year | 0% | 11% | 5% | ||
| Between once a year and quarterly | 0% | 22% | 10% | ||
| Quarterly | 18% | 33% | 25% | ||
| Between quarterly and monthly | 46% | 0% | 25% | ||
| Monthly | 18% | 22% | 20% | ||
| More often than monthly | 18% | 11% | 15% | ||
| 2 | Which specialties are involved in the periodic measurement and discussions of health outcomes (in joint or separate meetings). More than one answer possible | Cardiology | 100% | 100% | 100% |
| Thoracic surgery | 100% | n.a. | 100% | ||
| Anesthesiology | 55% | n.a. | 50% | ||
| Multidisciplinary team | 82% | 22% | 52% | ||
| Management | 55% | 89% | 67% | ||
| These meetings do not take place | 0% | 0% | 0% | ||
| 3 | How many physicians take active knowledge of the health outcomes of your hospital (for instance through meetings in which the annually reported outcomes are discussed)? | 0% | 0% | 0% | 0% |
| 1–25% | 36% | 33% | 35% | ||
| 25–50% | 18% | 22% | 20% | ||
| 50–75% | 9% | 22% | 15% | ||
| >75% | 36% | 22% | 30% | ||
| 4 | At what level within the organization and/or care chain are health outcomes discussed? (more than one answer possible) | Not | 0% | 0% | 0 |
| Among physicians (doctor's units, departments) | 100% | 100% | 100% | ||
| Support staff | 82% | 0% | 45% | ||
| Multidisciplinary | 73% | 22% | 50% | ||
| Patients | 9% | 11% | 10% | ||
| Nursing ward | 18% | 22% | 20% | ||
| Hospital management | 45% | 67% | 55% | ||
| Medical board | 9% | 33% | 20% | ||
| Board of directors | 45% | 44% | 45% | ||
| General practitioner | 9% | 11% | 10% | ||
| Referring hospitals | 45% | 11% | 30% | ||
| 5 | Are quality dashboards (or other tools) used to monitor outcomes of heart care? | No | 18% | 44% | 30% |
| Yes | 82% | 56% | 70% | ||
| 2. Identification improvement potential | |||||
| 6 | When do outcome reports (such as national benchmarks) lead to improvement initiatives within your hospital? (more than one answer possible) | Never | 0% | 0% | 0% |
| Only when the hospital is performing significantly worse than the average of other hospitals | 64% | 44% | 55% | ||
| When the report leads to clinically relevant insights that can be starting point for improvements (for instance a negative trend in the data or performance of subgroups within the patient population) | 91% | 89% | 90% | ||
| When one or more (other) hospitals are performing significantly better than average | 64% | 33% | 50% | ||
| Other (please specify) | 18% | 0% | 10% | ||
| 7 | Does the heart centre look at trends in the data on health outcomes periodically (based on all available tables, figures, etc.)? | No | 0% | 0% | 0% |
| No, only the comparison between centres is looked at (using funnel plots) | 9% | 0% | 5% | ||
| Partially, the comparison between centre and dependencies of outcomes on risk factors is looked at | 55% | 33% | 45% | ||
| Yes, all figures and tables available are looked at | 36% | 67% | 50% | ||
| 8 | Have targets been set for all outcome measures provided in outcome reports (e.g. 30-day mortality < 0.5%)? | No | 55% | 78% | 65% |
| Yes, for one or some outcome measures for all heart care provided by the heart centre | 18% | 0% | 10% | ||
| Yes, for one or some outcome measures for each of the medical conditions for which outcomes are available (e.g. coronary artery disease, atrial fibrillation) | 27% | 11% | 20% | ||
| Yes, for all outcome measures | 0% | 11% | 5% | ||
| 3. Selection improvement initiatives | |||||
| 9 | Have additional data analyses been performed in 2018 or 2019 based on the outcome reports? (aiming to better understand results and possibly to suggest improvement initiatives) | None | 9% | 33% | 20% |
| 1 | 0% | 22% | 10% | ||
| 2–4 | 55% | 33% | 45% | ||
| 5 or more | 36% | 11% | 25% | ||
| 10 | How many improvement initiatives have monitoring of outcomes resulted in, in 2018 and 2019 (e.g. NHR report, internal quality dashboards, etc.)? | None | 18% | 33% | 25% |
| 1 | 18% | 33% | 25% | ||
| 2–3 | 46% | 11% | 30% | ||
| 4 or more | 18% | 22% | 20% | ||
| 11 | Which learning strategies are used to initiate improvement initiatives?* (more than one answers possible) | None | 0% | 0% | 0% |
| Best practice | 67% | 67% | 67% | ||
| Process analysis | 67% | 50% | 60% | ||
| File review | 56% | 50% | 53% | ||
| Scientific literature | 67% | 33% | 53% | ||
| Guidelines | 33% | 17% | 27% | ||
| (Ad-hoc) initiatives based on clinical experience | 44% | 0% | 27% | ||
| (Structural) learning environment with other hospitals | 33% | 0% | 20% | ||
| Inter-physician variability | 11% | 0% | 7% | ||
| Benchmarking | 78% | 17% | 53% | ||
| Other (please specify) | 22% | 0% | 13% | ||
| 4. Implementation improvement initiatives | |||||
| 12 | Is it standard practice to monitor the implementation of improvement initiatives? (e.g. do you check if improvements are implemented correctly and for all eligible patients?) | No, never | 0% | 11% | 5% |
| No, most of the time not | 27% | 11% | 20% | ||
| Yes, most of the time | 64% | 56% | 60% | ||
| Yes, always | 9% | 22% | 15% | ||
| 13 | Is the effect of improvement initiatives monitored? (impact on outcomes or intermediate outcomes) | No | 9% | 33% | 20% |
| Yes, annually using the outcome reports | 36% | 33% | 35% | ||
| Yes, more often than annually, during regular team meetings. | 55% | 33% | 45% | ||
| 1. Strategy | |||||
| 14 | To what extent is measuring and improving outcomes using outcome measures part of the strategy and annual plans of the heart centre? | The heart centre is now mainly focusing on registering outcome measures. | 27% | 56% | 40% |
| The heart centre has set clear targets in the annual plan (or annual plans of the individual departments) aiming to improve outcomes of specific patient groups. | 46% | 33% | 40% | ||
| Performance on outcomes is a central part of the long-term strategy of the heart centre. This results in specific annual targets that are monitored using outcome measures. | 27% | 11% | 20% | ||
| 2. Governance (Structure) | |||||
| 15 | Is there a multidisciplinary meeting of the involved specialties in which outcomes of care are discussed (e.g. involving cardiology, thoracic surgery, and anesthesiology for coronary artery disease)? | No | 18% | 78% | 45% |
| Yes | 82% | 22% | 55% | ||
| 16 | Who are involved in the regular meetings in which the outcomes of care and improvement initiatives are discussed?** (more than one answer possible) | This does not take place | 0% | 0% | 0% |
| Only physicians | 100% | 50% | 91% | ||
| Nurses | 33% | 50% | 36% | ||
| Team leaders | 44% | 50% | 45% | ||
| Specialist nurses | 33% | 50% | 36% | ||
| Physicians from referring hospitals | 0% | 0% | 0% | ||
| General practitioners | 0% | 50% | 9% | ||
| Data manager/Data analyst | 67% | 100% | 73% | ||
| Department management | 89% | 100% | 91% | ||
| Hospital management | 22% | 50% | 27% | ||
| Patients or patient representatives | 11% | 0% | 9% | ||
| Support staff from the quality department | 67% | 50% | 64% | ||
| Other (please specify) | 33% | 0% | 27% | ||
| 17 | Are outcomes discussed with and are joint improvement initiatives started with partners in the care chain? (e.g. referring hospitals, general practitioners) | No | 46% | 44% | 45% |
| Yes | 55% | 56% | 55% | ||
| 3. Culture (Shared values) | |||||
| 18 | What is the involvement of physicians in the measurement and improvement of outcome measures? | No involvement | 0% | 0% | 0% |
| Small. One physician has responsibility for data delivery to external stakeholders. Apart from that no physicians are involved. | 0% | 0% | 0% | ||
| Reasonable. Some physicians are involved. | 64% | 78% | 70% | ||
| Large. There is a wide involvement. | 36% | 22% | 30% | ||
| Very large. All physicians are involved. | 0% | 0% | 0% | ||
| 19 | What level of trust exists within specialties to discuss outcomes openly (e.g. variance between physicians)? | Poor | 0% | 0% | 0% |
| Moderate | 9% | 11% | 10% | ||
| Fair | 27% | 22% | 25% | ||
| Good | 36% | 44% | 40% | ||
| Very good | 27% | 22% | 25% | ||
| 20 | What level of trust exists between specialties to discuss outcomes openly (e.g. between thoracic surgery, cardiology, and anesthesiology)? | Poor | 0% | 0% | 0% |
| Moderate | 9% | 13% | 11% | ||
| Fair | 27% | 25% | 26% | ||
| Good | 36% | 50% | 42% | ||
| Very good | 27% | 13% | 21% | ||
| 4. Leadership (Style) | |||||
| 21 | How many physicians are ambassadors of measuring and using outcome measures? (i.e. physicians with a leadership role to stimulate development of the hospital in this area and who are able to get colleagues along) | None | 9% | 44% | 25% |
| 1 | 9% | 0% | 5% | ||
| 2 | 9% | 0% | 5% | ||
| 3 | 18% | 22% | 20% | ||
| More than 3 | 55% | 33% | 45% | ||
| 22 | At which level(s) in the organization is initiative taken to realize an outcome-based improvement cycle within the heart centre? (more than one answer possible) | Physicians | 100% | 89% | 95% |
| Management of the department or heart centre | 73% | 56% | 65% | ||
| Hospital management | 18% | 56% | 35% | ||
| Nurses | 27% | 11% | 20% | ||
| Hospital quality department | 55% | 56% | 55% | ||
| Board of directors | 18% | 22% | 20% | ||
| Medical board | 9% | 11% | 10% | ||
| Other (please specify) | 18% | 0% | 10% | ||
| 5. Infrastructure (Systems) | |||||
| 23 | How is outcome data for external reports collected (excluding follow-up data)? Please select what best matches the current situation. | Not | 0% | 0% | 0% |
| Retrospectively by combing several sources. Involving still a lot of manual work | 18% | 44% | 30% | ||
| Prospectively build in a separate quality database | 18% | 11% | 15% | ||
| Prospectively build in a separate quality database and connected to the EHR | 27% | 22% | 25% | ||
| Prospectively build in the EHR | 9% | 22% | 15% | ||
| Other (please specify) | 27% | 0% | 15% | ||
| 6. Staff | |||||
| 24 | Who gets time to work on realization of an outcome-based improvement cycle? (more than one answer possible) | Physicians (FTE) (average) | 0.19 | 0.12 | 0.15 |
| Quality managers (FTE) (average) | 0.28 | 0.43 | 0.36 | ||
| Internal advisors (FTE) (average) | 0.50 | 0.02 | 0.20 | ||
| Medical management (FTE) (average) | 0.17 | 0.25 | 0.21 | ||
| Department management (FTE) (average) | 0.00 | 0.28 | 0.18 | ||
| Others (FTE) (average) | 1.20 | 0.36 | 0.68 | ||
| 7. Skills | |||||
| 25 | Are there employees within the hospital with the explicit task as part of their job to work on the realization of an outcome-based quality improvement cycle (e.g. manager value-based healthcare, advisor) | Yes (FTE) | 73% (3.00 FTE average) | 67% (1.27 FTE average) | 70% (2.26 FTE average) |
| No | 27% | 33% | 30% | ||
| 26 | How many physicians in the heart centre have expertise and affinity with data management and data analysis? | 0 | 9% | 0% | 5% |
| 1 | 0% | 33% | 15% | ||
| 2–3 | 55% | 56% | 55% | ||
| 4–5 | 18% | 0% | 10% | ||
| More than 5 | 18% | 11% | 15% | ||
EHR, Electronic Health Record; FTE, Full time equivalent ; n.a., Not applicable; 1 FTE in Dutch healthcare equals 36 h/wk.
Percentages might not add up to 100 due to rounding or in case of multiple choice.
*applicable if one or more improvement initiatives were initiated (question 10)–9 heart centres with thoracic surgery, 6 heart centres without thoracic surgery.
**applicable if regular meetings in which the outcomes of care and improvement initiatives are discussed are organised (question 15)–9 heart centres with thoracic surgery, 2 heart centres without thoracic surgery.