The costs and value of modelling-based design in healthcare delivery: five case studies from the US

ABSTRACT

In the nineties and noughties, Hollocks surveyed the use of Discrete Event Simulation (DES) in industry and listed (although he could not quantify the value of) benefits. This paper explores how DES is now used to design healthcare facilities and services, developing a value-for-money case with a protocol on collecting information. We present a set of five DES case studies from the US care system and, following Hollocks, focus on modelling as part of a rigorous design process, capturing as many of the benefits as possible. Healthcare offers the possibility of ascribing value to health improvement, but in these cases it is primarily the operational benefits of a better service that are reported and monetarised. By estimated the cost of modelling and the value of the operation gains, this paper contributes significantly to the literature. We conclude with a protocol for collecting information and a discussion of methods by which different types of benefit may be captured.

1. Introduction

Operational researchers began to apply computer simulation – and very quickly discrete event simulation (DES) – around the end of World War II (Goldsman, Nance, & Wilson, 2010; Nance & Sargent, 2002) as the computer age dawned. For several decades, developments in simulation modelling were closely linked to advances in hardware and programming languages (Nance, 1995; Robinson, 2005). Computing was expensive and most of the early modelling was undertaken by central groups in large corporations, where an early focus was work scheduling and cost-based planning models (Hollocks, 2008). By the mid-90s DES had spread across manufacturing and logistics, including airline scheduling, clustering in five categories (Hollocks, 1995):

• Facilities

• Productivity

• Resourcing

• Training

• Operations

Although centred on the UK, Hollocks’ surveys of DES practise (Hollocks, 2006) are important in tracing its corporate roots, in assessing and listing the benefits, and in following the ways in which standard practise emerged. Specifically, he reported on two aspects of modelling in use:

• The choice of runs and run parameters – selection of experiment, number of runs, and initialisation

• Sensitivity analysis

He concluded that running models relied on the judgement of modellers, although there was evidence in places of the emergence of formal process, such as using statistical criteria to set up the experiment to be modelled. With advances in computing power some restrictions, such as run-time, receded. The migration of computing from mainframes to personal computers, with graphical and interactive interfaces, and from central planners to local teams also created a narrative around the problem modeller and the problem owner. By the turn of the millennium, problem owners and modellers were often part of the same group once again, and the emerging standard practise centred on the means of selecting the modelling experiments, warm-up runs, and sensitivity analysis, as noted above (Hollocks, 2001).

Although able to list the benefits of simulation modelling (Hollocks, 1995), it was not possible to monetarise them systematically. From the first, the value-for-money argument was implicit rather than explicit: the expense of developing code internally and of building and maintaining planning teams was justified because critical questions could not be answered in any other way. Even now, the price of licences may be cited as evidence of value (Robinson, 2005).

The problem of value for money in healthcare modelling projects is nearly 40 years old: 200 papers on healthcare modelling revealed only 16 papers which reported that the recommendations had been acted upon, of which only 7 reported follow up (Wilson, 1981), of which 3 reported financial measures of success.

Meanwhile, records of DES studies in healthcare date back to the 1960s (Williams, Covert, & Steele, 1967) and literature reviews consistently identify an impact gap (Brailsford, Harper, Patel, & Pitt, 2009), noting a lack of reported outcomes affecting delivery and calling for research into the value of modelling in healthcare (Fone et al., 2003). Meanwhile, a comparison of the maturity of modelling usage in healthcare, showed that it lagged military and aerospace on the one hand, and manufacturing on the other (Jahangirian et al., 2012).

An historical perspective, therefore, encourages us to ask two questions:

• How far along the trajectory followed by other sectors has healthcare come?

• What progress has been made in articulating the value-for-money case?

These questions require an exploration of “standard practice” and the ways in which the value of modelling is interpreted within healthcare communities. As part of a wider sabbatical research project, one of us (TY) has collected narratives of modelling and has been analysing them with another of the team of authors (SS) to assess the value-for-money case. This collection is small and purposive but provides a ranging shot on DES practise today.

The present set of narratives comes from that larger collection and the studies share a number of common features. First, they are linked to the flow of patients and staff through built environments and so align with the facilities and productivity categories noted above. Second, the modelling has been performed by experts for whom modelling is a tool in reaching a specific goal: architectural practise in one case and hospital engineering and operations in another. Third, they are based in the US, and so the appeal to health economics in valuing health benefits is greatly weakened. In terms of the modellers and problem owners, in these cases the modellers are, or are closely linked to, problem owners, rather than being a separate community.

This paper provides insight into healthcare modelling, as did Hollocks in his reports from industry nearly two decades ago but goes further in estimating the value of modelling. Finally, from the analysis of these case studies, a protocol is proposed for collecting information to simplify value-for-money judgements in future. Note: the purpose of this paper is not to establish that modelling can replicate important aspects of service performance to appropriate levels of accuracy – after decades of application in many sectors that is a given – but to report on current practise in healthcare and to attempt to put a monetary value on the exercise of modelling ahead of service change.

The rest of the paper is structured as follows: the next section describes how the modelling was undertaken, followed by a section presenting the case studies. The main findings are discussed next, followed by a section exploring how full cost-effectiveness may be integrated into such studies. The last section presents the conclusions.

2. Conceptual framework, design, and modelling approach

2.1. Methods for case studies 1–3

These studies were undertaken by an engineering team within a hospital system, where the prevailing philosophy was Lean/Six Sigma, structured as DMAIC (define & measure, analyse, improve, and control). This use of simulation is a recognised, if niche, element of lean applications (Bhamu & Sangwan, 2014). In general terms, the approach fits generally within the recommendation of building a model to replicate the existing system before modifying or developing it to study options (Hoad & Kunc, 2018). Figure 1 shows the process within which the modelling was undertaken.

Figure 1.

Design process within which case studies 1–3 were conducted.

Queuing analysis is useful for investigating capacity and the utilisation of expensive equipment, such as CT scanners, where the Design of Experiment approach was used to determine the major variables which impacted on CT turnaround time. The team used brainstorming techniques to identify possible interventions. To inform these and the models through which they would be studied, one-to-one meetings were held with front line staff, clinical specialists, and Emergency Department and CT management.

Initial scoping and root cause analysis utilised such Lean Six Sigma tools as regression analysis, Voice of the Customer, and 5 Whys analysis (Andersen & Fagerhaug, 2006). Typically, during the analysis phase a decision was made as to whether to use a simulation to ensure that the project would be implemented successfully. Once interventions were chosen, a computer simulation model was built to test the proposed solution and the effects on the process and data collection plans would be modified to meet the requirements of the modelling system.

Data would be input into the model while it was being built to match current state of the service and this model would be confirmed with subject matter experts and other stakeholders, as well as run to get good agreement with the existing data. This process seeks to meet the needs of all service users and ensure that such input is incorporated into the design of each simulation. Moreover, the team aimed to establish a 95% confidence level between the simulated existing process and its modelled performance.

Once the team was satisfied that the model represented the existing behaviour and could replicate the outputs given appropriate input data, models for the options would be built and tested.

2.2. Methods for case studies 4 and 5

For these cases, the modelling was undertaken by an architectural, engineering, planning, and consulting practise which lacked access to the facilities afterwards to be able to measure the benefit. That said, the basic features were similar:

1. Consult with stakeholders to identify the key constraints of the existing system and the requirements of any new system.

2. Build a model of the existing system in consultation with stakeholders using client data.

3. Once validated, build further models to explore options.

4. Decide on what new processes, equipment, technology, staffing models, scheduling protocols, and spatial additions or modifications might be needed on the basis of (3) above.

2.3. Comparison with other reported methods

As retrospective studies, all five cases reflect practise rather than theory. However, by comparing the process outlined in Figure 1 with the more academically led Lean simulation in the literature [eg, (Robinson, Radnor, Burgess, & Worthington, 2012) (Baril, Gascon, Miller, & Côté, 2016)], we note three critical differences:

1. The academic literature emphasises stakeholder engagement, often as a “Lean event”, whereas the overall process here is owned by a department or organisation that draws in Voice of the Customer and stakeholder meetings in a unified process. Also, there is a strong focus in these studies on ensuring that the stakeholder validation is backed by excellent quantitative agreement.

2. The design team here has access to the information systems and intermediate measures or proxies (eg, cost of a patient per hour in each service). Getting suitable data for these case studies was therefore not a problem, and validation of the model could be especially rigorous, as noted.

3. Finally, the team is able to assess afterwards how well the selected design worked in practise.

3. Case studies and analysis

Five case studies are presented, three by an internal design team within a US health system and two by an architectural practise serving healthcare users. The benefits and savings are estimated on the basis of the models where there is no further evidence of impact, but where there is, this is cited.

3.1. Case study 1: extra elevator for OR patient flows

This project analysed the floor usage and flow for all aspects of a design for a two-story operating room expansion costing $31million, in 2012, including pre-op admission, transport to OR, OR time, and post-anaesthesia care units (PACU) for admitted and outpatient surgery in a Level I Trauma Centre/Tertiary Urban Operating Room (see Figure 2).

Figure 2.

DES model of elevators and floor layout for OR operations and capacity planning.

DES was used to assess the facilities and layout for the expansion, for which a 15% increase in volume over the next 5 years was anticipated. The key finding, which was implemented in the final design, was that the addition of a third elevator would improve the flow of patients and reduce movement and waiting time by 30 min per case, with the potential to raise OR capacity by up to 10 cases per day and increase the access for providing vital surgical services to the community. The model now created can serve to test incremental and evolutionary changes to OR processes, facilities, and staffing, thus contributing to the cumulative knowledge of a “process literate” organisation for the future.

The new system was built with the lift, which was estimated to have paid for itself (and the modelling) within two weeks. Table 1 provides a breakdown of the costs and benefits.

Table 1.

Costs and benefits for third lift simulation.

Measured costs	($k)	Measured benefits	($k)
Modelling (60 h of a modellers time @$48/h)	2.9	30minutes for 40,400 patients operated on/year @ 54$/min	65,500.0
Internal decisions (2 senior managers for a total of 10 h @$600/h)	12.0
Third elevator	75.0
Total	89.9	Annual saving	65,500.0
Unmeasured costs		Unquantified benefits
Stakeholder meetings		Health benefits of the patients who were operated on more quickly.

Table 1 shows the costs of the modelling, the decision, and the implementation, together with the savings calculated. The savings are mainly operational, since the hospital system can readily monetarise the extra throughput. In simple terms, it works out at >700:1, although the precise value would depend upon the timing of the investment and the internal discount rates applied by the finance office.

3.2. Case study 2: redesign of patient flows to CT

This 2014 study was to reduce patient waits for a CT in the Emergency Department (ED). Initial requests included the addition of a second CT in the ED to increase the throughput. Leadership requested an analysis to support such a significant investment in equipment, facilities, and staffing. A simulation showed that a new CT was not cost effective but that significant improvements in wait time were possible if all patients requiring pre-CT lab testing received their tests, and if all patients were ready, by the time they reached the exam. The plan was to assign a radiology technologist assistant to the Emergency Department’s CT service from 11:00 to 19:30 to ensure that pre-checked point of care tests were appropriately applied for creatinine and in cases of pregnant women.

The CT pre-testing was simulated as part of a Lean improvement programme and the implementation of the extra radiology technologist assistant was also tested using simulation. In the latter case, the model simulated the care environment for both a single day and a full week, based on data from May to July. The model compared the current state (1 technologist assistant) against the proposed intervention (2 technologist assistants) and showed a 27% decrease in the longest patient wait over an elapsed day, with a 33% decrease over an elapsed week. This improvement was piloted for 4 days a week, 11:00–19:30, and then rolled out 7 days/week, with the additional radiology technologist assistant assigned to the ED CT service during peak demand hours (11:00–19:30).

The CT Tech Assistant now preps patients to ensure they are ready for their CT exam. This preparation includes communication of delays, information about jewellery and clothing, consent, screening forms, confirmation of lab results, and patient transport. Point of care testing was implemented on the Abdominal Pain power plan. Post-intervention data showed a reduction in average time from 80.75 min (8/1/148/14/14) to 72.56 min (8/15/14–9/7/14), representing a 10.15% reduction in CT Abdomen/Pelvis exams requiring pre-labs. Requested vs. started time for a CT in the ED from 11:00 to 19:30 dropped from an average of 96.81 min (9/1/2014–9/30/2014) to 70.60 min (11/1/14–12/14/14). This is statistically significant (p-value 0.0000) at a 95% confidence level. The intervention continued to show improvement dropping to an average of 65.41 min (2/1/15–2/28/15) – a 32% overall reduction from the time the CT is requested until start time. From a modelling perspective, the simulation predicted a 33% decrease in request to start time and actual performance achieved a 32% reduction. This represents a difference in predicted verses actual performance of under 1 min (57.81 s).

The costs and benefits are shown in Table 2 where the return on investment in terms of savings alone is >9:1 on the basis of annual savings and just under 30:1 for the one-off decision not to instal another CT scanner. The precise total RoI would depend upon the internal financial processes.

Table 2.

Costs and benefits for simulation and redesign of route to the CT scanner.

Measured costs	($k)	Measured benefits	($k)
Modelling (28 h @ $48/h)	1.3	Annual savings of 30 min each for 22,200 patients at $1/min*	688.0
Decision (10 senior staff for 3 h each @ $600/h)	18.0	One-off saving on not buying a new CT ($1M) and installing it ($1M)	2,000.0
Implementation (extra staff member, 25 h/week, internal rates)	51
Total	70.3	Annual	688.0
		One off	2,000.0
Unmeasured costs		Unquantified benefits
Stakeholder engagement		Value of quicker scanning. Worth most to those with brain trauma or stroke.

*Cost estimated from Crimson Clinical Advantage database (Crimson Cinical Advantage, n.d.)

However, we have also identified a health benefit for people who need urgent scanning, and who would benefit from the speedier service, but have not monetarised it.

3.3. Case study 3: redesign of patient flows from ED to wards

This 2015 programme was designed to reduce the delay in placing a patient from ED in an appropriate inpatient bed, by setting a target of 2 h from request to occupation. A Lean Six Sigma transformation was set up that used DES to explore the following interventions:

• (4) Bed Requests processed using First In-First Out (FIFO) system

• (5) Elimination of “held” beds

• (6) Elimination of unnecessary placement priorities and

• (7) Patient assigned to clean and un-occupied bed only

The modelling provided leaders and staff with quantitative evidence of the viability of FIFO operation and other improvement interventions. This evidence was necessary given the inability to test the theory in a live environment, since once it was turned on, the process would impact patient care directly. In 2014, pre-implementation (January-October), compared to the previous year, the turnaround time for a bed request in Teletracking until a Bed is assigned had risen in both the ED (30 min on average) and the PACU (10 min on average). Both areas represent the largest source for patients to place. Patient Placement is responsible for the placement of these patients onto the inpatient unit and to address the issue, we must look at the Patient Placement process as a whole. Delays in placing a patient on a unit decreases staff satisfaction, can cause harm to patients, decreases patient satisfaction, and creates quality concerns including increasing the chances of mortality.

The project applied advanced virtual DES of a first-in, first-out (FIFO) industry best practise algorithms to a dataset comprising over 530,000 data points. The computer model calculated simultaneous queuing distributions for 14 clinical product lines by four points of service origins, adjusted for actual historical volume, service time, room cleaning time, and staffing variables for every patient placement from January through April 2015. The simulation model predicted a 30% improvement opportunity. Baseline patient placement time was 5:13, exceeding the Joint Commission recommendation of 4 h.

The Patient Placement FIFO process also yielded results in ICU downgrades of care, overall bed request to occupy, and inter-unit bed request to occupy times. Post-intervention data shows a reduction in ICU downgrades of care from the time of bed request until occupied from a mean of 1,306.03 min to a mean of 780.98 min (40.27% reduction) at $2,411.62/day in variable cost of care, yielding savings of $880.91 per ICU patient (annualised savings of $3,065,556.80), reduction in overall turnaround times from the time of bed request until occupy from a mean of 327.01 min to 277.736 min (15% reduction). Inter-unit bed request to occupy pre-intervention performance mean of 1148.61 min to 490.103 min (57% reduction). All were statistically significant (p-value < 0.0005) at a 95% confidence interval. Globally speaking, actual delay was decreased 37% compared to the 30% prediction which was expected given the complexity of the model.

As shown in Table 3, the RoI is >50:1.

Table 4.

Costs and benefits of GP clinic replacement.

Measured costs	($k)	Measured benefits	($k)
Modelling	58.9	Predicted 1-off saving	3,500.0
Stakeholder engagement (2 staff, 6 weeks, 2 h/week @ $55/h)	2.6	Predicted annual operating costs saved	540.0
Decision (6 staff, 8 h @ $75/h)
Training costs to use new facility (80 staff, 8 h, @ $45/hr).	28.0
Total	89.5	Annual	540.0
Redesign of the clinics		One off	3,500.0

Table 5.

Costs and benefits for redesigned OR pathway.

Measured costs	($k)	Measured benefits	($k)
Modelling	77.5	No extra ORs required	3,000.0–5,000.0
Stakeholder engagement (2 staff, 4 hrs/wk for 6 weeks @ $75/hr	3.6
Decision (3 staff, 2 h @ $75/hr)
Total	81.1	Annual
		One off	3,000.0–5,000.0

Table 3.

Costs and benefits for redesigned pathway from ED to wards.

Measured costs	($k)	Measured benefits	($k)
Modelling (325 h @ £48/h)	15.6	Savings of 70 min/patient @ $1.22/min for 24,000 discharges/year*	2049.6
Decision (1 person, 21 h @ $600/h)	12.6
Command Centre training (5 people for 40 h each @ $60/h)	12.0
Total	40.2	Annual	2,049.6

*Cost estimated from Crimson Clinical Advantage database (Crimson Cinical Advantage, n.d.)

3.4. Case study 4: general practise clinic replacement

This study was undertaken for an affiliated practise of 8 Geneal Practitioners (GPs) whose offices were collocated on three adjacent floors of a building, with a total combined clinic space of more than 50,000 square feet (4,700 m²), not including circulation and other common spaces.

The operational goal of a new design was to reduce the operating costs by $165k pa for the community of 8 GPs as a whole, while the clinical objectives were for a more streamlined care delivery environment. This required an understanding of the way in which staff and patients would interact, given a range of possible office layouts in a new facility.

As part of this design a modelling study was undertaken as described above, not simply an engineering design (with rules-of-thumb to hand) but a design-and-business model where total operational costs were tracked for the existing and proposed designs. The key outcomes from this 4-month programme modelling were that:

• (8) The number of patients seen per exam room was increased by 56%

• (9) The amount of non-value added waiting time per patient was decreased by 66%

• (10) The amount of value added time, spent with a provider, increased by 49%

• (11) The area per provider decreased by 23.3%

• (12) The non-provider FTE count was reduced by six

• (13) First costs were reduced by $3,500,000.

• (14) Annual operating costs, including staff and lease expenses, was reduced by $540,000 per year.

On the basis of the modelling, the RoI is >6:1 on the recurring gains and around 40:1 on the 1-off gains. Again, local accounting practise would determine how this was reported.

3.5. Case study 5: operating room (OR) expansion

This 6-month study was commissioned when the hospital concerned was facing an increase of 29.4% in the volume of surgical cases and wanted to know whether its 12 ORs with a total of almost 8,000 square feet (740m²) of surgical procedure space (not including corridors, storage, air circulation, etc.) would suffice or whether new ORs would be required. There was also a question of the capacity of the existing central sterile processing department equipment’s ability to accommodate the increase in instrument sets occasioned by the increased surgical case volume, or, if additional equipment would be needed.

Previous evaluation using traditional assessment methods had determined that the client would require two new ORs at a cost of $3–5M. Moreover, the PACU would also have to be relocated, which would have violated the hospital’s long-term master plan.

A DES model of the future state operational concepts within the existing set of OR rooms was used to model the throughput capacity of the Central Sterile Processing’s existing equipment. It showed that by making process and scheduling improvements, by converting a poorly utilised, dedicated, cystoscopy room to a general purpose OR, and by moving some activity from the OR to procedure rooms, the existing 12 ORs were sufficient for the increased case volume without the need for new construction or even renovation. This saved the client $3–5M, based on the estimates of the first contractor.

The analysis of the Central Sterile Processing facility, accounting for up- and down-stream departments, showed that the throughput capacity would not be adequate for the new volumes and so new sterilising equipment was purchased and installed by the client.

The most basic RoI would be estimated at between 37:1 and >60:1.

4. Discussion

Despite its limitations, there are common features in this set of studies. First, sizeable returns on investment are possible from simulation used in design, most obviously when service can be boosted but without buying an experience piece of equipment (scanner) or facilities (ORs). Second, significant operational benefits in perpetuity are possible, so the internal discount rate used by the Finance department will determine what RoI is actually and how many years of benefit contribute to it. Without knowing what this will be, we can report returns from 6:1 to >700:1 for efficiency gain.

Third, the cost of modelling is generally a small part of the total cost of the redesign and implementation cycle, so that RoIs calculated on the modelling alone are potentially huge.

Fourth, the availability of data is key, especially where freely available and where costs per patient, etc., have been estimated for all departments.

The cost of modelling is higher for the latter studies than for the first three, for which we offer the following:

• (15) There is a difference between cost and price (which includes commercial risk).

• (16) The start-up costs for an internal project and a design-for-business project will differ because of the data and information available to each at the start.

• (17) In the engineering studies, the business factors can be address as rule-of-thumb conversion factors, whereas business models must be built for the latter pair.

5. Towards a protocol for capturing the full cost-benefit balance

Healthcare differs from other sectors in being able to monetarise health benefits using a simple graphical tool, the cost-effectiveness plane (CE-plane), with cost increases or decreases along one axis and health gains or losses along the other (Black, 1990). By centring the axes on the current best solution, one may evaluate where new designs stand in terms of cost and health benefits. Figure 3, then, may be applied in a general way.

Figure 3.

A simple model of the cost-effectiveness (CE) plane for new service development.

Thus, service developments that save money and increase health – a green dot in quadrant Q3 – should be adopted (such as case study 2 which was more operationally efficient and for which the quicker access to the CT scanner would represent significant life- saving or health-saving for some patients. These issues are discussed in the case of stroke (Soorapanth & Young, 2015; Soorapanth & Young, 2018). New services that cost more and reduce health – a red dot in Q1 – should be avoided: they are generally mistakes, although there is a relatively recent analysis that shows that trying to provide 7-day services can reduce everyone’s health if the weekend staff is taken from the weekday resource (Meacock, Doran, & Sutton, 2015).

Services in the second and fourth quadrants may or may not be cost-effective, depending upon how much health is generated or lost for how much is invested or saved. In the UK, there is a maximum threshold for which, in the case cited above, even if it were successful, the benefits of weekend services were deemed far too small to justify expense.

If one wanted to collect the information to make the health case as well as the operational case, one must address the following issues. The following protocol was adapted based on the health economic analysis framework (Drummond, Sculpher, Claxton, Stoddart, & Torrance, 2015; Drummond & McGuire, 2001) and shall be used to guide the data collection for the assessment of a modelling project.

5.1. Data this paper has reported as being collected and used

The cost of the modelling may be estimated as shown above (staff hours plus an hourly rate) or the cost of the external modeller reported.

The cost of stakeholder engagement and of the decision-making can also be estimated as shown above. Note that an organisation that has internal rates may include much higher overheads on senior managers’ time since there is a greater opportunity cost associated with their time.

Time saved can be collected in the manner shown, and having rules-of-thumb to turn that into money is exceptionally useful.

Modelling that shows that expensive new facilities are not needed or may be substituted for lower-cost facilities make the case very readily. In such cases, the modelling is usually aims at determining whether increased flow can overcome capacity limitations of the existing facilities.

5.2. Outcomes data that this paper has not reported on

We have identified benefits which it is now possible to monetarise, since when health systems work better patients often experience direct health gains. If you are scanned sooner, for instance, your prognosis after a stroke is better; if the OR suite handles more patients, you are likely to get your operation sooner, with better chances for your rehabilitation. However, benefits are currently difficult to quantify:

First, not every system recognises the same value of health utility, or attributes any formal value to utility at all. The UK leads the world in this but the US, for instance, does not adopt this approach in assessing provision.

Second, key data are not readily available. Nobody estimates the health state of every patient entering and leaving a healthcare facility or follows their ongoing improvement. Moreover, the same equipment and facilities will be used to support patients with different conditions and, again, the value of optimising their use within a service will vary by condition. One must, therefore, break down patient streams by condition and estimate the typical utility boost offered by healthcare engagement before integrating the entire picture together again. For stroke, for instance, there is some published data, but for most pathways there is nothing useful to hand. In principle, these benefits can be monetarised, and the sooner the health benefits are recognised alongside operational improvement, the easier it will be to design better care.

For those interested in more background, a formal protocol in the form of key questions is presented in Appendix A based on (Gold, Siegel, Russell, & Weinstein, 1996).

6. Conclusions

Let us return to the two questions we posed at the start:

1. How far along the trajectory followed by other sectors has healthcare come?

Although we have focused on “industrialised” healthcare environments, we can, still identify many features that Hollocks observed in earlier ecosystems (Hollocks, 2001): modelling is being applied in operations, to facilities and to productivity (Hollocks, 1995); it is being used by central teams or by groups with a responsibility to a central team, and for related reasons (a combination cost and risk); selection of experiments is still a matter of judgement and there is evidence of design frameworks within which such choices are being made. However, where the cost of modelling itself was an issue 30–40 years ago, it is now minimal. Also greater than the cost of modelling are the costs of making decisions and then, on a different scale altogether, the cost of implementing the preferred options modelled. The perceived cost of failure (buying an unnecessary scanner, or lacking elevator capacity) appears to be driving demand.

We turn now to the second question:

• (b) What progress has been made in articulating the value-for-money case?

On the immediate front, we have found cases where, with good hospital data and with good business models (eg, how much a patient costs per minute in the system) it is easy to compare options and to monetarise the value of the improvement. Sometimes, the business case is particularly clear: do not build the extra OR or instal the extra scanner. The value-for-money case in such examples is now exceptionally easy to monetarise.

However, we have also identified benefits which it is now possible to monetarise, but which we have not been assessed, principally because there is a lack of conversion factors to convert the benefit, for instance, of earlier scanning or swifter allocation to a ward (or even treatment at home) into a health premium for a return on investment calculation. We recommend the further development of economic evaluation theory and production of approximate conversation factors to fill this gap.

This type of analysis is really only available in healthcare, where the epidemiology provides precise information on the number of patients with each condition and their progression under different treatments, while health economics provides a way to convert the utility of different health states into cash. Whether it is possible to spend such currency or not, it has to be accounted for before we can have truly efficient outcomes-based healthcare.

Disclosure statement

No potential conflict of interest was reported by the authors.