Development and implementation of a computerised clinical decision support system for hospital‐in‐the‐home patient identification

ABSTRACT

Background

Identification of appropriate patients for hospital‐in‐the‐home (HITH) remains a significant challenge. We hypothesise that a human‐centred design (HCD) approach can assist in developing a computerised clinical decision support system (CDSS) to address this issue via leveraging data within the electronic medical record (EMR).

Aims

We describe the design, development, validation and implementation outcomes of the HITH Eligibility Identifier (HEI), a CDSS for identifying inpatients suitable for HITH in Melbourne, Australia.

Methods

Approaches include: (a) describing digital algorithmic design and workflow development via the iterative HCD process; (b) validation of the HEI algorithm, interface and performance using Message Understanding Conference and Miettinen‐Nurminen analyses; and (c) evaluation of system impact and outcomes post‐implementation.

Results

The HEI CDSS integrates algorithmic decision support with clinician judgement through an interactive user interface embedded within the EMR. It ranks hospitalised patients from most to least likely eligible for HITH using administrative data, clinical information and markers of illness severity. Validation of HEI performance found that high‐ranking patients were twice as likely to be eligible for HITH (improvement in eligibility probability of 46%; 95% confidence interval: 38%–54%). Post‐implementation metrics show improvements in HITH utilisation, service delivery and efficiency.

Conclusions

The HEI CDSS illustrates the importance of an HCD approach in ensuring local relevancy and clinical adoption of digital healthcare projects. Principles from its design and implementation can improve patient identification for HITH programs and be adapted for other CDSSs relevant to hospitalised patient cohorts.

Abbreviations

AI

artificial intelligence

CDSS

clinical decision support system

CI

confidence interval

ED

emergency department

EDD

estimated discharge date

EMR

electronic medical record

HCD

human‐centred design

HEI

hospital‐in‐the‐home eligibility identifier

HITH

hospital‐in‐the‐home

ICU

intensive care unit

RMH

Royal Melbourne Hospital

SD

standard deviation

TRIPOD

Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis

Introduction

Home‐based inpatient bed substitutive care, termed ‘hospital‐in‐the‐home’ (HITH), is increasingly utilised to deliver acute medical, surgical and rehabilitative treatments to patients in their own homes. The aging Australian population, increasing prevalence of chronic disease and advancements in digital health technologies contribute to HITH being seen as an alternate model of healthcare that can address barriers to emergency department (ED) access, optimise patient flow and provide a better healthcare experience with fewer complications compared with conventional ward‐based care. ¹ , ² , ³

However, identification of appropriate patients for HITH remains a significant challenge for health services. ⁴ , ⁵ , ⁶ Current approaches largely rely on manual processes where clinicians must remember eligibility criteria and actively consider HITH during busy ward rounds. ⁴ , ⁵ , ⁶ , ⁷ This creates cognitive burden and can lead to missed opportunities for home‐based care. Variations in HITH models among Australian hospitals further complicate this process, as healthcare professionals struggle to develop shared mental models about which patients might benefit. ⁵ , ⁸ , ⁹ These workflow and communication challenges contribute to poorer transitions of care, suboptimal patient and carer experience and adverse health outcomes. ¹⁰

Computerised clinical decision support systems (CDSSs) embedded within electronic medical records (EMRs) could potentially address these workflow challenges. However, reports of models using alert‐based systems have shown limited effectiveness, with implementation complicated by alert fatigue, potential biases and workflow disruptions. ¹¹ The challenge involves not only identifying HITH‐eligible patients but designing digital systems that support clinician workflows and enhance decision‐making processes.

Human‐centred design (HCD) approaches can assist in healthcare technology development by systematically exploring various complex stakeholder needs. ¹² The Double Diamond framework is a structured HCD methodology that involves four phases: Discover (understanding the problem), Define (synthesising insights into problem statements), Develop (prototyping solutions) and Deliver (testing and implementing). ¹³

Using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist, we report on the development and implementation of a CDSS named the ‘HITH Eligibility Identifier (HEI)’ to identify potential patients for HITH within a tertiary metropolitan hospital in Melbourne, Australia. ¹⁴ We describe the design process and outcomes following implementation, with particular attention to how the iterative HCD approach addressed common challenges in CDSS implementation. Though there are a small number of reports of predictive models for HITH patients, none have incorporated results from utilisation of an electronic tool in an Australian setting.

Methods

Setting

This study occurred at the Royal Melbourne Hospital (RMH) in Australia, an academic, tertiary hospital with over 112 000 admissions in 2022/2023, using the Epic EMR system. ¹⁵ , ¹⁶ The hospital's HITH program cares for ~200 patients monthly, with a daily census of seven new admissions within a total of 70 patients in the service.

Prior to HEI, patient selection occurred via manual ‘push’ workflow where ward clinicians had to recognise suitable patients and generate electronic consultation requests. However, there was a lack of visibility into the referral pipeline within the EMR and no mechanism for assessing likelihood of transfer to HITH. Insufficient referrals and excess HITH capacity required clinicians to perform time‐consuming manual screening across multiple wards.

The study was conducted from February to December 2024, with the project team comprising HITH clinicians, EMR informatics officers, data analysts and change management officers. The design process utilised Double‐Diamond methodology that involved: (a) stakeholder interviews, (b) collaborative design sessions, (c) building prototypes within the EMR development environment and (d) clinician testing with rapid refinement of the model.

This study has been approved by RMH's human research ethics committee (QA2024083).

Project stages

1. Digital interface design

Stakeholder interviews identified four key elements for the HEI CDSS:

1. utilisation of routinely available EMR data,

2. development of a ranking algorithm rather than binary classification,

3. integration of human‐computer feedback into the CDSS and

4. generation of a pragmatically sized list (~30 patients) to avoid information overload.

Inclusion and exclusion criteria were selected by HITH clinicians who identified key decision factors that could be translated into queryable structured data elements within the EMR (Table 1), consisting of administrative data, clinical information, markers of critical illness and functional status. Administrative data reflect geographical and psychosocial factors impacting suitability, while clinical criteria included diagnoses, medications and procedures common within HITH cohorts. Markers of critical illness excluded patients too unstable for HITH.

TABLE 1.

Inclusion and exclusion criteria, with key criteria allocated points within the ranking algorithm

Category	Inclusion criteria	Exclusion criteria
Patient demographics	RACF resident* Age >16 years Age >75 years*	Forensic patient/consumer History of carrying weapons
Administrative data	Admitted to inpatient acute ward Currently within ED/radiology department Current HARP patient	Admitted to the ICU, or day admission (dialysis or day medical unit), or subacute ward Current HITH patient Home address is interstate or international status Next hospital destination is a subacute ward, mental health or external health service Surgery scheduled for next day
Hospital (ED) presentation	Disposition being short stay unit >48 h Triage problem: genitourinary, gastroenterology, respiratory, collapse, syncope, low fall, fever/infection, trauma	Triage category 1 or 2 Triage problem: neurological, dental, chest pain, ear/nose/throat, social, palpitations/arrhythmia, oncology/haematology, immunocompromised, mental health or alcohol or other drugs. Section 232 (police guard)
Clinical diagnoses	Common HITH presentations: heart failure*, fluid overload, chronic obstructive pulmonary disorder*, cellulitis*, pneumonia, acute renal failure, gastroenteritis*, pyelonephritis, urinary tract infection, endocarditis*, septicaemia, bone/joint/muscle necrosis*, respiratory infection, hepatic encephalopathy, syncope, fever, COVID‐19 infection, anticoagulated on warfarin*, presentation related to prosthetic heart valve*
Treatment plans	Surgical plans: debridement, graft, flap, incision and drainage, washout, breast reconstruction*, hemicolectomy and formation of stoma VAC dressing*	HITH consultation request (active) Anaesthetics consultation Traction required Log Roll required Spinal management
Medication orders	Enoxaparin (daily dose >80 mg)* Common HITH antimicrobials or medications: amphotericin B, benzylpenicillin*, flucloxacillin, warfarin*, remdesivir, frusemide*, amoxicillin, clindamycin, piperacillin/tazobactam, cephazolin*, vancomycin, amikacin, tigecycline, teicoplanin, azithromycin, ceftriaxone, cefepime, cefoxitin, ceftazidime, meropenem, ertapenem, acyclovir, ganciclovir, amikacin	Medications unable to be delivered via HITH: intravenous – heparin, potassium, calcium, phosphate, adrenaline, noradrenaline, insulin, glucose, mannitol, amiodarone, magnesium, fentanyl, morphine. Intravenous fluid requirements >1 L/day
Procedural orders	PICC line insertion* Stomal therapy referral Nephrostomy insertion	Nuclear medicine ventilation perfusion study
Lab values		Last result – neutrophils <0.5 ×109/L, sodium <130 or > 155 mmol/L, potassium <3 or > 7 mmol/L, bicarbonate <10 or > 40 mmol/L, phosphate <0.5 or > 2.0 mmol/L, magnesium <0.5 or > 2.0 mmol/L, haemoglobin <70 g/L, white blood cell count <1.0 ×109/L
Clinical status	Last documented vital signs within normal adult physiological limits* Active wound present	Last documented Glasgow Coma Scale <10, unresponsive level of consciousness, sedation score 2 or 3, pain score >7, moderated or severe respiratory distress Last documented heart rate > 120 bpm, respiratory rate > 30 bpm, temperature > 38.5°C
Nursing care plans	Negative pressure dressings* Delirium referral and care Monitor wound drainage/discharge Wound care nurse referral Dermatology inpatient consult Fluid restriction* Daily weights requirements	Constant observation (Nursing Special) required Occupational violence and aggression service required Adult sepsis management Emergency theatre request Fresh blood products Adult anaphylaxis management Continuous bladder washout Adult acute stroke management
Other	Estimated discharge date within the next 8 days* Previous HITH patient*

Note: (*) Key criteria. Abbreviations: bpm, beats per minute; ED, emergency department; HARP, Hospital Admission Risk Program; HITH, hospital‐in‐the‐home; ICU, intensive care unit; PICC, peripherally inserted central catheter; RACF, Residential Aged Care Facility; VAC, vacuum‐assisted closure.

A major design decision led to creating a points‐based ranking algorithm that aligned with clinicians’ mental models of patient assessment (Fig. 1). Patients with points spread across multiple eligibility categories were ranked higher than those heavily weighted within a single category. When queried, the EMR interface provided further details about factors contributing to each patient's ranking that underpin the algorithm.

Figure 1.

HEI digital phenotype – inclusion and exclusion criteria and ranking algorithm. Abbreviations: EDD, estimated discharge date; HEI, hospital‐in‐the‐home eligibility identifier; HITH, hospital‐in‐the‐home; yo, years old.

The HEI interface incorporated human‐computer feedback allowing clinicians to screen and designate patients as ‘Eligible’, ‘Potentially Eligible’, or ‘Not Eligible’, creating a learning loop where clinician reviews continually refined the list. The CDSS was visible as a ‘Patient List’ within the inpatient EMR environment, which rendered it easily accessible to all hospital staff.

• 2Validation of interface and HEI CDSS performance

Validation was conducted by HITH clinicians not previously involved in its design using the Message Understanding Conference method that involved validating both the accuracy of data capture as well as the algorithm's ranking performance. ¹⁷ Four clinicians—two nurses and two consultant physicians—reviewed patient charts drawn from high‐ranking and low‐ranking cohorts within the HEI list twice daily, simulating usual clinical workflows. The top 10 patients (or top quintile if <20 patients) were defined as high‐ranking, with low‐ranked being bottom 10 or lowest quintile. Sampling occurred 21 times over 12 days.

We planned to validate a minimum sample size of 100 patients, with the primary outcome being a composite measure of patients identified as Eligible or Potentially eligible by clinician‐validators. A Pearson chi‐square test was used to analyse the global comparison of HITH eligibility versus ineligibility, with clinician‐validated outcomes collapsed into two categories: eligible (eligible now or potentially eligible) and ineligible (all ineligibility subcategories). ¹⁸ The Miettinen‐Nurminen method was used to demonstrate differences in HITH eligibility via corresponding two‐sided 95% confidence intervals (CIs). ¹⁹ Additional comparisons between HEI subcategories were performed using two‐sided Fisher exact test. ¹⁸ All tests assumed a two‐sided alpha of 0.05, and a P value <0.05 was considered statistically significant.

User interface and workflow development

Via design workshops, requirements for the interface were identified to:

1. provide decision support without disrupting workflows

2. integrate with existing referral management processes

3. be resource‐neutral

This resulted in the HEI CDSS built as an on‐demand, non‐interruptive tool without automatic chart alerts. Hospital staff accessed the CDSS based on self‐identified needs such as when HITH referral numbers were low. This decision also meant there were no requirements for additional staffing resources – as HITH staff would be redeployed from usual clinical responsibilities into utilising the HEI CDSS only when there was a need to do so (Fig. 2). The workflow empowered HITH staff to communicate with ward teams about HEI CDSS patients; a by‐product of this engagement process was ongoing education of ward staff about the HITH program with the secondary goal of increasing ward‐initiated referrals over time.

• 3Post‐implementation evaluation

Figure 2.

HEI workflow activation during excess HITH capacity, demonstrating communication points between HITH staff and ward clinicians in determination of patient eligibility for HITH. Abbreviations: HEI, hospital‐in‐the‐home eligibility identifier; HITH, hospital‐in‐the‐home.

We compared outcomes in the first 6 months after launch (July–December 2024) with outcomes in the same 6 months the preceding year, evaluating HITH utilisation, referral metrics and safety indicators such as the numbers of HITH patients who had unplanned representations to the ED during their HITH episode of care. HITH utilisation included measurements of the number of patients discharged from HITH and HITH occupancy rates, defined as the percentage of patients occupying an HITH ward bed at midnight (with 100% being the program's capacity of 70 beds). Referral metrics encompassed monthly count of patients referred to HITH and referral timeliness; defined as monthly averages of hours from admission to electronic lodgement of the HITH referral within the EMR. Statistical analyses utilised the Mann–Whitney U test to compare bed occupancy rates, two‐sample Poisson rate tests to estimate incidence rate ratios (IRRs) comparing monthly HITH patient numbers and referral counts and paired t tests for analyses of differences in referral timeliness. Comparison of the same 6 months during 2023 and 2024 controlled for seasonal variation in hospital workflow and patient cohorts. Analyses were performed using R version 3.6.3 (R Foundation for Statistical Computing).

Results

The HEI system launched within the hospital's EMR environment on 22 May 2024. Figure 3 shows the final user interface, including:

1. a dynamic patient list ranked by eligibility score with real‐time updates

2. clear visual indicators of ranking category (high, medium, low) showing relevant patient information with the ability to access additional details

3. interactive buttons for clinicians to update patient status and see system responses

FIGURE 3.

HEI user interface – ranked patient list with details accessed via hovering over icons. (copyright belonging to 2025 EPIC Corporation.) Abbreviations: HEI, hospital‐in‐the‐home eligibility identifier.

Validation of interface and HEI CDSS performance

Over 21 sampling episodes, 707 patients were identified by the HEI CDSS. Validation of the algorithm involved manual chart review and comparison of high‐ranking against low‐ranking groups. Table 2 displays the demographics and clinical characteristics of the validation cohort totalling 395 patients. Most (59.6% to 60.5%) patients identified by HEI had medical issues, with a subset presenting with miscellaneous clinical issues (noted as ’other’) such as drug and alcohol intoxication, social concerns and minor injuries.

Table 2.

Patient demographics and clinical characteristics of the HEI validation patient cohort

HEI patient cohort	Total, N = 395	High‐ranked patients, n = 210	Low‐ranked patients, n = 185
Sex, male	240 (60.8)	150 (71.4)	90 (48.7)
Age, y, mean (SD)	62.2 (22.9)	65.3 (22.1)	58.7 (23.9)
Primary clinical issue type
Medical	237 (60.0)	125 (59.6)	112 (60.5)
Surgical	107 (27.1)	54 (25.5)	53 (28.4)
Other	51 (12.9)	31 (14.9)	20 (11.1)
Pre‐hospital residence
Home	316 (80.0)	150 (71.4)	166 (89.9)
Supported residential	79 (20.0)	60 (28.6)	19 (10.1)
service
Estimated discharge date
< 48 h	251 (63.5)	142 (67.6)	109 (58.8)
> 48 h	144 (36.5)	68 (32.4)	76 (41.2)

Note: All variables are presented as n (%) unless otherwise indicated.

Abbreviations: HEI, hospital‐in‐the‐home eligibility identifier; SD, standard deviation.

There were no technical inaccuracies identified, confirming the robustness of the structured data extraction approach. One patient had a mismatch between their medication orders and clinical treatment plan within their (free text) clinical notes, leading to inaccurate data capture.

The HEI ranking algorithm's performance in comparison to clinician‐validated results is depicted in Table 3. Among HEI highly ranked patients (n = 210), 127 (60.5%) were clinician‐validated as eligible for HITH compared with 14.1% of low‐ranked patients. Using the Mettienen‐Nurminen method, the absolute difference in eligibility proportions between groups was 46.5%, with a 95% CI of 37.5% to 54.8%, indicating that high‐ranked patients were more likely to be classified as eligible for HITH. Global comparison in eligibility via Pearson chi‐square test showed a significant association between HEI ranking and clinician‐validated status (χ ² = 89.6, df = 1, P < 0.001). Common reasons for misidentification within HEI algorithmic ranking were patients imminently planned for discharge (41.1%) and cohorts with functional or behavioural complexities that limit HITH delivery (18.9%). This is potentially due to the algorithm selecting for patients with vital signs within normal limits as a marker of clinical stability. Similarly, a large subset of ineligible patients in the ED (38.5%) were awaiting investigations that would lead to a same‐day discharge, thus negating HITH.

Table 3.

Comparison of HEI ranking performance against clinician validation by chart review

	HEI patient cohort, n = 395		High‐ranked, n = 210	Low‐ranked, n = 185	P value
Clinician‐validated patients	Eligible
	Total		127 (60.5)	26 (14.1)
	Eligible now		29 (13.8)	3 (1.6)	<0.001
	Potentially eligible		98 (46.7)	23 (12.4)	<0.001
	Ineligible
	Total		83 (39.5)	159 (85.9)
	Discharge to primary care		32 (15.2)	76 (41.1)	<0.001
	Functionally ineligible for home‐based care		24 (11.4)	35 (18.9)	0.03
	Clinically inappropriate		20 (9.5)	33 (17.8)	0.02
	Awaiting transfer to another hospital		7 (3.3)	15 (8.1)	0.04
Difference in eligibility proportions between clinician‐validated patients and HEI ranked cohorts, % (95% CI range)			46.5 (37.5–54.8)

Note: All variables are presented as n (%) unless otherwise indicated.

Abbreviations: CI, confidence interval: HEI, hospital‐in‐the‐home eligibility identifier.

System impact post‐implementation

Outcomes 6 months following implementation demonstrated improved HITH utilisation without a corresponding increase in unplanned patient representations to the ED that was used as a surrogate marker of safety and inappropriate patient selection within HITH (Fig. 4). More patients (1189 patients) were treated between July to December 2024 compared with 2023 (1090 patients), with an IRR of 1.113 when comparing the 6 months in 2023 against 2024 (95% CI: 1.016–1.219; P = 0.02), thus indicating an 11.3% higher discharge rate post‐implementation. Median bed occupancy increased from 87.1% (2023) to 95.7% (2024), with the Mann–Whitney U test indicating a statistically significant shift towards higher occupancy rates as well (U = 24 928, P = 3.92 × 10⁻¹⁵, two‐sided).

FIGURE 4.

Hospital‐in‐the‐home activity before and after implementation; July to December data from 2023 and 2024. Abbreviation: ED, emergency department.

The 6 months post‐implementation in 2024 saw 1079 referrals compared with 991 in the same period of 2023, with a corresponding IRR of 1.089 (95% CI: 0.999–1.187; P = 0.053) (Fig. 5). Referrals occurred earlier, with the average time from admission to electronic lodgement of HITH referral decreasing from 11 437 min in 2023 to 10 419 min in 2024. The mean month‐paired comparison was −1018 min, i.e., approximating time‐savings of 17 h, (95% CI: −2578 to 541 min; P = 0.154).

FIGURE 5.

Referrals to HITH before and after implementation; July to December data from 2023 and 2024. Abbreviation: HITH, home‐in‐the‐hospital.

Discussion

This study describes the design, build and implementation of a CDSS for HITH patient identification using structured EMR data with a points‐based ranking algorithm. The iterative HCD process, particularly user feedback loops between Develop and Deliver phases, was crucial for increasing relevance, usability and adoption of the HEI CDSS. Comparison of outcomes within the same 6 months pre‐ and post‐implementation indicates improvements in service activity, referral quantity and timeliness without adverse impacts in patient safety. This was achieved without increased staffing or changes to rostering practices within the program.

Though CDSSs have been trialled in many clinical environments, existing reviews report minimal improvements in care processes and clinical outcomes due to a mix of technical and human‐related factors. ¹⁹ , ²⁰ End‐user involvement in the HEI CDSS design assisted in its integration with existing workflows, leading to design features whereby clinicians could flexibly access the CDSS in response to hospital throughput and avoidance of interruptive alerts within the EMR. In contrast, reports of CDSS models with contrasting design features, e.g., those utilising best practice advisories, have reported low rates of adoption and transfers into HITH. ¹¹ Explainability has been shown to be important to enhance clinician trust and uptake, and the HEI interface with a pop‐up box displaying additional information underpinning each patient's ranking helps clarify algorithmic reasoning. ²¹ , ²² Validation in our study indicates successful EMR data capture and automation of the previously manual chart reviews, which other studies have equated to a time‐savings of 310 min per enrolled HITH patient. ²³

The digital architecture of the HEI algorithm is similar to other published models in its reliance on a rules‐based algorithm instead of predictive artificial intelligence (AI) and machine learning. ¹¹ , ²³ , ²⁴ A rules‐based CDSS using structured data mapped to common EMR architecture enables scalability across healthcare settings, has high interpretability and is modifiable post‐implementation – this is particularly relevant for novel programs such as HITH where there is rapid evolution in clinical care models, staffing capabilities and patient eligibilty. ²⁴ , ²⁵ Incorporation of human‐computer feedback within the HEI interface leading to pruning of the list following clinician chart review reduces overfitting risk and enables future diversification of patient cohorts. ²⁵ , ²⁶ , ²⁷ In comparison, models solely based on machine learning and AI analytics risk bias from inadequate datasets and are limited by reliance on surrogate markers. ²⁶ , ²⁸ , ²⁹

Nevertheless, there is scope to improve on the HEI CDSS accuracy rate of 60%. The common factors for HITH ineligibility relate to free‐text information, particularly in relation to social complexities, diagnostic formulations and plans for further investigations that are currently not reliably captured within the EMR's structured datasets. Broadening the algorithm to incorporate data such as social drivers of HITH eligibility, home support networks and environmental safety considerations could enhance algorithm performance. ³⁰ , ³¹ , ³² Future work could also include hybridised elements from AI techniques such as natural language processing to capture information within clinical notes. ³³

Other limitations in our study include its pilot status with testing within a single health system with a specific HITH program and EMR platform. Nevertheless, implementation risk in different EMR or hospital environments could be reduced via utilising ³⁵ Clinical Quality Language or integrating via Substitutable Medical Apps and Reusable Technologies on Fast Healthcare Interoperability Resources applications. ³³ , ³⁴ , ³⁵ Another limitation relates to our evaluation's focus on service metrics, with limited usability measures obtained from patients and end‐users. A more comprehensive evaluation drawing on a larger dataset across multiple healthcare settings would be useful, as well as utilisation of implementation frameworks that evaluate both digital interventions and their adoption in practice. ³⁶

Conclusion

The HEI demonstrates how HCD can be applied to create effective clinical decision support systems via emphasising human‐computer collaboration rather than automation and offers a model where digital algorithmic support augments clinical decision‐making in healthcare. By focusing on workflow integration, transparency and clinician agency, we developed a system that improved service delivery without creating additional resource requirements. This suggests that careful design processes with a focus on end‐user involvement can overcome common barriers to integration of technology into clinical workflows within complex healthcare environments.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.