Abstract
Purpose
Our purpose is to study of outpatient services face unpredictable demand, periodic patient overload, manpower constraints, and inefficient facility and energy utilization. Existing digital healthcare solutions emphasize prediction but provide limited adaptive decision support under uncertainty.
Methods
This study develops and evaluates, entirely in simulation, a dynamic decision-support framework that integrates a Bayesian Markov decision process (Bayes-MDP) with Monte Carlo look-ahead simulation to balance patient traffic, specialist allocation, facility management, and energy use. The novelty lies in combining Bayesian demand forecasting, Random Forest–based overload-risk estimation, and Bayes-MDP planning within a single look-ahead evaluation framework. Using Weekly outpatient data (2007–2025) from Taiwan’s National Health Insurance (NHI) database and real-time outbreak and disease surveillance (RODS), aggregated by county and disease (influenza/influenza-like illness, acute diarrhea, acute upper respiratory and enteroviral infections; 88 county–disease series). A probabilistic time-series model forecast weekly demand; a Random Forest classifier converted demand history and seasonal features into overload-risk probabilities (held-out AUC = 0.940 under chronological validation). A Bayesian Markov decision process over quantile-based demand states, with Dirichlet–multinomial belief updating and 4-week look-ahead, selected among policies (manpower adjustment, rescheduling, telemedicine, and energy-saving strategies). Policies were evaluated over 
Monte Carlo replications with stochastic demand against a static scheduling baseline simulated on identical demand traces, alongside risk-threshold-rule and myopic one-step benchmarks.
Results
For the results, the bayes-MDP look-ahead policy reduced the mean simulated outpatient queue by 39% (95% CI 38.8–39.3) and estimated (proxy-based) energy consumption by 5.4% (95% CI 5.3–5.4) relative to the static baseline, with energy-saving actions concentrated in low-risk periods (51% of low-risk weeks). Simpler risk-treshold rule achieved by 12.1 and queue reduction, but slightly increase energy use; myopic one-step policy performed comparably to the look-ahead policy. No single fixed action performed best across all risk levels.
Conclusion
In conclusion, the simulation results suggest that adaptive, risk-aware decision support can improve service level, specialists workload balance, and facility and energy management compared with static scheduling. Because all findings are simulation-based and energy estimates depend on proxy variables, prospective validation into the real-world scenario would be in real clinical settings is required before implementation.
Keywords: Bayesian markov decision process, Monte Carlo simulation, Outpatient scheduling, Healthcare management
Introduction
Taiwan healthcare service is widely regarded as among the best healthcare service in the world. Its National Health Insurance (NHI) covers 99.8% of citizens and residents. The benefit can be widely covered treatments, such as, outpatient, inpatient, dental services, physical therapy and traditional Chinese medicine with equal accessibility to citizens.1 This accessibility, however, has contributed to chronic overload for physicians.
Unpredictable situations occur frequently, such as emergency sickness. Sudden shortage of manpower and sudden patient overload. Hospitals cannot reject these cases or ask patients to schedule in advance,2 while unpredictable workloads and consecutive days of overtime working increase burnout and reduce the work-life satisfaction of physicians relative to other workers.3 These can affect to service and treatment performance. Moreover, electricity load required to keep hospital and clinics processes on standby for incoming patients raises operating costs and produces the waste in energy utilization. Especially, some medical devices must remain on standby with high load with staff to operate. Together, walk-in demand prediction, manpower preparation, and medical equipment standby management constitute the major operational challenges addressed in this study.
Furthermore, Taiwan’s healthcare sector is undergoing digital transformation. Investment in medical artificial intelligence (Medical AI) rapidly grew to approximately NT$360 million in 2023 and is projected to reach NT$1.12 billion by 2030,4 with applications of robotics, medical imaging, precision medicines, and automated processes that reduce the operation time,5 and digital health platforms supporting the NHI.1 These Industry 4.0 technologies improve efficiency through automation. However, efficiency-focused automation itself does not address the human-centered and sustainability requirements, such as, patient quality, workforce well-being, and energy use. Therefore, we emphasized by the Industry 5.0 concept, which motivates decision support frameworks that treat all objectives together rather than optimizing prediction accuracy alone and manpower disruption.
How Can These Challenges Be Overcome?
Regarding previous studies of outpatient scheduling, Qu et al6 developed an MDP model to address high no-show and walk-in patient rates in primary care clinics.7–9 The present study extends this line of work to unexpected illness surges in Taiwan, focusing on the leading causes of emergency and walk-in outpatient visits in recent consecutive years: influenza, acute diarrhea, and acute upper respiratory infections. Moreover, existing studies focus mainly on time-based optimization and rarely consider the holistic waste of facilities; this study therefore develops an overall optimization that also includes energy sustainability.
Research Gap
Prior work on outpatient operations spans three streams.10 First, the appointment and scheduling models based on Markov decision processes and related dynamic approaches optimize booking, walk-in admission, and diagnostic scheduling under demand uncertainty,6,11–14 but in current they typically predict stationary, fully known transition dynamics and service-quality objective.
Second, simulation model and optimization studied quantify waiting time improvement15–19 also most evaluation of a fixed set of static policies rather than adaptive and dynamic, state dependent decision rules. Third, machine learning studies able to predict wait times, no-shows, and operational risk with increasing accuracy,20–23 but most of researches rarely connect to predictions to sequential decision-making, and sustainability objectives such as, energy usage are never represented within decision model.24 Furthermore, in Taiwan’s utilization NHI environment,25 pandemic-driven demand surges make these limitations operationally significant. This becomes the knowledge gap mainly addressed as following, no existing outpatient scheduling framework simultaneously (i) learns uncertain demand dynamics online, (ii) links predicted overload risk to sequential multi-objective decisions, and (iii) treats energy sustainability as a first-class decision objective.
Objectives and Contributions
This study develops and evaluates, in simulation, an integrated decision-support framework for outpatient management under epidemic-driven uncertainty. The four contributions are as follows. First, Bayesian Markov decision process (Bayes MDP) with Dirichlet–multinomial belief updating that learns transition dynamics online under uncertainty.26,27 Second, a Random Forest overload-risk estimator whose output probability augments the MDP state, explicitly linking prediction to decision. Third, Monte Carlo lookahead procedure that evaluates candidate policies (manpower adjustment, rescheduling, telemedicine, and energy-saving strategies) against a static baseline using 2007–2025 NHI and RODS surveillance data. Fourth, explicit incorporation of energy utilization into the reward function, so that sustainability is considered as a decision objective rather than metric reporting.
In summary, all findings are simulation-based, which prospective clinical implementation is beyond the scope of study.
Methodology
Data Sources for Analysis
This study contains the integration of real-world cases dataset of outpatient demand, operational risk and energy utilization within Taiwan healthcare system for 19 consecutive years (2007–2025).25 The data were obtained from Taiwan’s National Health Insurance (NHI) database, covering influenza, influenza-like illness, acute diarrhea and acute upper respiratory infections. We chose these datasets because these diseases caused to the outpatient visit situation in Taiwan healthcare system. Enhancing responsive analysis of the system, this study also used the dataset of emerging outbreaks, real-time syndromic surveillance and real-time outbreak and disease surveillance system (RODS). In addition, we gather data of the hospital and clinic information, including, type, size and locations. Those data are publicly available at National Health Insurance Administration, Ministry of Health and Welfare, Taiwan. All datasets were gathered to a weekly consistency. Disease incidence data reported by calendar week and were converted to standardized weekly timestamps using ISO week conventions. County-level aggregation was adapted to capture spatial variation in outpatient demand while maintaining sufficient details of data. For some missing data or errors were solved by using forward-fill methods for small gaps, for examples, some weeks with the insufficient information were not included for analysis. All data were de-identified and aggregated at the county-week level; no individual-level patient records were accessed.
Process of Work
The conceptual framework (Figure 1) is constructed by the Bayesian Markov Decision Process. Agent observes the healthcare environment to create States (
), Actions (
), which operational action and get the multi-objective rewards () positive and/or negative effects to service level, facilities consumption and workloads prediction under uncertainty of outpatient demand. When the environment transferred to a new state (
), and the agent receives a reward reflecting patient service performance, sustainability impact, and workforce burden. Moreover, Monte Carlo look-ahead simulation is used to evaluate policy performance under uncertainty and the workforce with system decision support which led to human and machine aligned to operate altogether.
Figure 1.
Conceptual framework of the Bayesian Markov decision process (Bayes-MDP) decision-support system. The agent observes the healthcare environment state (demand level, overload risk, energy utilization, and manpower load), selects a policy action, and updates its beliefs from observed state transitions.
Detailed Mathematical Transformation
Sets, Indices and Decision Epochs
: discrete decision epochs (by week)
: location index
: disease type and symptoms
: Policies actions to outpatient management
 |
Dataset Input and Observations
: observation of weekly outpatient demand by location (county) 
, disease 
From the data of disease_panel_weekly (were arranged from NHI+RODS database)
 |
From
 |
Combined to
 |
Bayesian Demand Forecasting Model
Shown as the probabilistic time series model, which is the predictive distribution output Generic probabilistic forecast as follows,
For horizon 
 |
Where 
is the information available at time 
and 
are model parameters.
Therefore, the output of Bayesian demand forecasting model (3.3.3.) as following
Predictive mean 
Predictive uncertainty 
Samples 
for Monte Carlo rollout
Overload Risk Assessment Model
This overload risk assessment pattern helps a compact state variable summarizing demand uncertainty which is always happens in the real-world scenarios.
Step 1: Define a disease-specific overload threshold as the formula,
 |
Step 2: Define overload label
 |
Step 3: Example features for Random Forest Classifier
 |
Step 4: Estimation of Random Forest Classifier
 |
Where:
: Weekly outpatient demand
: Specific disease overload threshold
: Demand gap and seasonal features
Overload probability and risk probability as following illustrations of Figures 2–4.
Figure 2.
Mean overload probability by disease category, estimated by the Random Forest classifier. Overload probability is the estimated probability that weekly demand exceeds the disease-specific overload threshold (historical mean + 1 SD).
Figure 3.
Mean overload probability over time by disease category (2007–2025, 13-week rolling mean). Higher values indicate a greater probability that weekly outpatient demand exceeds the disease-specific overload threshold.
Figure 4.
Heatmap of mean overload probability by area (county) and disease category. Counties are ordered by overall mean overload probability.
Bayesian Decision Modelling (Baye-MDP)
The state of the Baye-MDP model in this study is constructed by the summary of demand + risk assessment results + energy utilization + manpower load.
Arrange demand by using quantile per disease per area (county)
 |
Define the demand usage
 |
A compact state is defined as
 |
Then, the completed bayes-MDP is following
 |
Where 
is the belief distribution over unknown transition parameter 
 |
Belief update after observing transition 
 |
is modelled as Dirichlet-multinomial distribution over the next state per 
as
 |
Then, update counts
 |
Energy Utilization Modelling
We use the hybrid proxy to simulate the holistic of power consumption by analyze the environment factors which are direct and indirect effect to energy utilization in the hospitals. This model can predict and interconnect to the estimation of the energy consumption during on and off period, surrounding environment in outpatient operation. Furthermore, this simulation in energy consumption is beneficial to hospital service level performance, better decision support in hospital operation and energy management sustainably.
Illustrate a seasonal proxy to replace the weather data and simulate the hot and cold weather which are temperature effect to the energy usage,
 |
Statistical energy model in simulating the historical facility usage, by letting rooms 
as the active specialists’ rooms
 |
Where:
: Baseline of hospital usage (always on)
: Seasonal HVAC effects
: Energy consumption per patient
Monte Carlo Look Ahead Simulation Design
to evaluate the efficacy of the Bayes-MDP policy, we conducted a Monte Carlo simulation with 
(Experimental settings, validation and reproducibility) The simulation policy performance repeatedly to make sure the result of the forecasting is accurate by following steps below,
Step 1: Simulation and Decision Period Settings
Discrete weekly decision period 
of disease and surveillance data by area (county) 
and disease category. The system state 
to observe demand level and overload risk of current state.
Step 2: Stochastic Demand Creation
Bayesian forecasting generally create predictive distribution,
 |
Where,
: denote of forecasting outpatient demand at the next period.
: historical demand and seasonal indicators.
Then, adding the possible future actions as 
, this factor also helps for outbreak uncertainty or unpredicted situation into the decision process as follows,
 |
Step 3: State Transition and Belief Update
The system is developed by a Bayes-MDP. Based on an action 
, the next state is sampled from a transition model as follows,
 |
If 
as unknown transition parameters then, creation of the belief distribution as 
to update the new observation become the adaptive learning of outbreak dynamics
 |
Step 4: Service and Energy Response Models
The simulation is computed over demand 
and action 
Service performance proxy (backlog and queue)
 |
Energy utilization estimation
Utilization by using seasonal effect and demand dependent load as hybrid model to predict electricity usage of 
(kWh)
 |
Step 5: Reward Function and Policy Evaluation
For policy evaluation is created by using the multi-objective reward function of service, sustainability factor (energy consumption), hospital staffs workforce overloaded and operational disruption to estimate rewards or penalty with discounted factor of 
0.05
 |
Where,
: penalty of workforce service backlog
: Penalty of energy consumption
: Penalty of operational disruption or overload work
, 
: weighting coefficients
For each candidate policy 
, Monte Carlo can be estimated of expected discounted return as
 |
m: Indexing trajectories of dynamic simulation
λ ϵ (0,1): Discount factor
Step 6: Look-Ahead Action Selection for Simulation
For each action at can be estimated by integration of decision making process under uncertainty by simulating horizon H as follows,
 |
To be selected action under uncertainty decision making,
 |
Experimental Settings, Validation and Reproducibility
All models were implemented in Python (scikit-learn, statsmodels, NumPy/pandas). The dataset (2007–2025, weekly, county × disease of 88 series, 84,277 series-weeks) was split chronologically: the overload-risk classifier was trained on 2007–2019 (n = 18,035 observations) and evaluated on the held-out 2020–2025 period (n = 25,032), avoiding look-ahead bias. The Random Forest used 300 trees with balanced class weighting and features comprising ISO week and demand lags 1–2; held-out performance was AUC = 0.940, sensitivity = 0.631, and specificity = 0.966 at a 0.5 threshold (test-period overload prevalence 12.1%). Disease-specific overload thresholds were defined as the historical mean plus one standard deviation of weekly demand.
The Bayes-MDP used four quantile-based demand states (quartiles per series), a symmetric Dirichlet (1) prior over state transitions updated online after each observed transition, a look-ahead horizon of H = 4 weeks with discount factor 0.95, and reward weights of 0.02 per queued patient-week and 0.3 per kWh. Service capacity was calibrated per series to the historical mean weekly demand. Action effects were: staffing increase (+20% capacity, +5% energy), telemedicine (−15% on-site arrivals, −5% energy), rescheduling (10% of arrivals deferred one week), and energy-saving operation (−12% energy, −2% capacity). Policy performance was evaluated over N = 50 Monte Carlo replications, each applying mean-preserving lognormal demand perturbations (σ = 0.15) identically across all policies (common random numbers; base seed 42). Reported metrics are means across replications with 95% percentile confidence intervals; percentage improvements are computed pairwise within each replication against a forced Normal-operation baseline simulated over the identical timeline. Benchmarks comprised the static baseline, a risk-threshold rule, and myopic one-step policy (Table 1). Sensitivity of policy rankings to the energy penalty weight (0.1–0.5) is reported in Sensitivity Analysis. A complete parameter summary is provided in Table 2.
Table 1.
Policy Comparison Across N=50 Monte Carlo Replications (Means with 95% Percentile Confidence Intervals). Improvements are Computed Pairwise within Each Replication Against the Static Baseline; Queue and Energy are Better When Lower, Reward When Higher
| Policy | Mean Queue, Patient-Weeks (95% CI) | Mean Energy, kWh (95% CI) | Mean Reward (95% CI) | Queue Improvement, % (95% CI) | Energy Improvement, % (95% CI) | Reward Improvement, % (95% CI) |
|---|---|---|---|---|---|---|
| Static baseline (Normal) | 147,166 (145,705–148,265) | 5.700 (5.695–5.703) | −2945 (−2967 to −2916) | – | – | – |
| Risk-threshold rule | 129,352 (127,873–130,448) | 5.755 (5.750–5.758) | −2589 (−2611 to −2559) | 12.1 (12.0 −12.2) | −1.0 | 12.1 (12.0 −12.2) |
| Myopic one-step | 89,414 (88,143–90,362) | 5.393 (5.389–5.396) | −1790 (−1809 to −1764) | 39.2 (39.1–39.5) | 5.4 | 39.2 (39.0 −39.5) |
| Bayes-MDP look-ahead | 89,725 (88,443–90,664) | 5.394 (5.389–5.397) | −1796 (−1815 to −1770) | 39.0 (38.8 −39.3) | 5.4 (5.3–5.4) | 39.0 (38.8 −39.3) |
Table 2.
Summary of Model Parameters, Assumptions, and Hyperparameters Used in the Corrected Evaluation
| Component | Parameter | Value | Justification/Selection Method |
|---|---|---|---|
| Demand forecasting | Model form; horizon | SARIMAX(1,0,1)×(1,1,1)52 per county×disease; 8-week horizon | Probabilistic weekly forecasts with seasonal structure |
| Random Forest classifier | Trees; class weights; features; validation | 300 trees; balanced; ISO week + demand lags 1–2; chronological split 2007–2019/2020–2025 (AUC 0.940; sens 0.631; spec 0.966) | Grid-searched on training period; leakage-free validation |
| Overload threshold | Disease-specific definition | Mean + 1 SD of historical weekly demand per disease | Flags statistically unusual surges |
| Bayes-MDP | States, prior, horizon, discount | 4 quartile demand states per series; symmetric Dirichlet(1); H = 4 weeks; discount 0.95 | Online Dirichlet-multinomial belief updating |
| Reward function | Weights | 0.02 per queued patient-week; 0.3 per kWh; intervention cost 0 in this configuration | Multi-objective penalty; sensitivity in Sensitivity Analysis |
| Monte Carlo evaluation | Replications, noise, seeds | N = 50; mean-preserving lognormal demand noise σ = 0.15; common random numbers; base seed 42 | 95% percentile CIs; paired improvements vs baseline |
| Energy proxy model | Structure, coefficients | 0.7×statistical + 0.3×bottom-up; base 0.6 kW/room + 0.2 kW equipment; ≤6 rooms (1 per 50 patients); HVAC 1.2 + 0.12·seasonal + 0.02·√cases; 2-h blocks | Hybrid seasonal-volume proxy; limitation in Limitations and Future Work |
| Capacity & action effects | Calibration, multipliers | Capacity = historical mean weekly demand per series; AddStaff +20% cap/+5% energy; Tele −15% arrivals/-5% energy; Reschedule defers 10%; EnergySave −12% energy/−2% cap | Operational judgment, limitation in Limitations and Future Work |
Modeling Assumption and Rationale
The simulation deliberately abstracts several operational details. These idealizations should be interpreted in light of the study’s purpose, which is to isolate and evaluate the decision mechanism, online belief updating and look-ahead action selection rather than to reproduce hospital operations as following,
First, the service capacity was calibrated to the historical mean weekly demand of each county–disease series. This grounds capacity in observed data and ensures that congestion arises from genuine demand surges rather than from an arbitrary capacity choice.
Second, action effects were represented as fixed multipliers, for examples, 15% on-site arrivals reduction for telemedicine and 20% increment capacity for staffing. Conservative point values were used because reliable intervention-effect estimates are not available at the county-week level. The sensitivity analysis (Sensitivity Analysis) examines the robustness of the policy rankings to parameter variation, and estimation of these effects from real intervention data is identified as future work and limitation (Limitations and Future Work).
Third, demand uncertainty was modeled as mean-preserving multiplicative lognormal noise applied identically to all policies (common random numbers), so comparisons are paired within each replication. Conclusions therefore rest on relative differences between policies rather than on absolute values, and relative rankings are far less sensitive to the noise assumptions than absolute magnitudes.
Fourth, the reward function assigned no explicit intervention cost in this configuration. This prefers active policies over the passive baseline and may overstate the net benefit of intervention, which is acknowledged as future work and imitation (Limitations and Future Work).
Fifth, energy consumption was represented by the proxy model of Energy Utilization Modelling rather than by metered data. Because all policies were evaluated under identical assumptions, these idealizations affect the absolute magnitudes of the reported improvements more than the ordering of policies, which remained stable across the tested sensitivity range. A staged path toward operational realism is outlined in Practical Implementation Challenges as retrospective replay on hospital-level operational data, shadow-mode deployment alongside human schedulers, and prospective pilot evaluation.
Results
Baseline and Bayes-MDP Performance Comparison
We evaluate the comprehensive results of the Monte Carlo simulation conducted over 18 years (2007–2025) period. The analysis compares the performance of the proposed Bayes-MDP framework against a traditional static scheduling (baseline) approach. The primary objective is to evaluate the system’s resilience to stochastic disease outbreaks, this study used the most popular outpatient diseases such as, Influenza-like Illness, Enterovirus, and Acute Diarrhea symptoms.
Queue Performance
Regarding to the static Normal operation baseline on identical demand traces. Bayes-MDP look-ahead policy reduced the mean simulated queue length by 39% (95% CI 38, 8–39.3; N = 50 Monte Carlo replications as Figure 5 and Table 1). The improvement raised primarily from adaptive substitution of telemedicine capacity during moderate-risk and high-risk periods and staffing escalation ahead of peak congestion, which prevents backlog from building up week after week during surge periods. The simple risk-threshold rule reduced queues by 12.1% (95% CI 12–12.2), indicating that most of the additional benefits stem from look-ahead evaluation of queue dynamics rather than risk triggered thresholds alone.
Figure 5.
Mean simulated queue length (patient-weeks) under the static baseline, risk-threshold rule, myopic one-step policy, and Bayes-MDP look-ahead policy. Bars show means across N=50 Monte Carlo replications; error bars show 95% percentile confidence intervals.
Energy Utilization
The energy-aware framework reduced estimated hospital energy consumption by 5.4% (95% CI 5.3–5.4) relative to baseline (Figure 6 and Table 1) primarily by selecting energy-saving operation during low-risk, low-demand weeks (51.1% of low-risk weeks; Table 3) and avoiding unnecessary standby operation. Because energy use is estimated with a hybrid seasonal - volume proxy model rather than metered consumption, this value should be interpreted as a modeled estimate rather than a measured saving, the risk-threshold rule slightly increased energy use (by 1%), illustrating that congestion-focused intervention alone can raise energy costs when energy is not represented in the decision objective.
Figure 6.
Mean estimated (proxy-based) energy consumption (kWh) by policy. Bars show means across N=50 Monte Carlo replications; error bars show 95% percentile confidence intervals.
Table 3.
Risk-Adaptive Policy Distribution Summary, Showing the Share of Each Action Selected by the Bayes-MDP Policy at Each Risk Level
| Risk Level | Overload Probability | Recommended Action | Priority |
|---|---|---|---|
| Low (Green) | 0.00–0.25 | Normal operations | Low |
| Moderate (Yellow) | 0.25–0.50 | Reschedule/Telemedicine | Moderate |
| High (Red) | 0.50–0.75 | Add staff | High |
| Overload (Red) | 0.75–1.00 | Emergency escalation | Critical |
Combination of Queue Performance and Energy Consumption as Reward Function Improvement
The combination reward function; queue performance, energy consumption, and intervention penalties improved by 39% (95% CI 38.8–39.3) under the Bayesian-MDP look-ahead policy (Figure 7 and Table 1). The reward is defined as negative, cost-based penalty, so the values closer to zero indicate better performance; percentage expresses the reduction in mean total penalty relative to baseline, computed pairwise within each replication.
Figure 7.
Mean reward by policy. The reward is a negative, cost-based penalty; values closer to zero indicate better performance. Bars show means across N=50 Monte Carlo replications; error bars show 95% percentile confidence intervals.
Policy Behaviour by Risk Level Assessment
From risk probability level from the risk of the overloaded situation, the policy of the probability that the outpatient system can be divided into the main 4 stages and no single actions approach in Bayes-MDP when the result gets the moderate and high risk as reference to Table 4 and Table 5. While the decisions when in normal or low risk, no any actions need to approach to get more cost of operation; cost of manpower, bottleneck from rescheduling and unnecessary energy cut-off. The overall model of this study shows that adaptive decision not only works in fixed or static decision-making, also, decide to support the reliability under risk and uncertainty to become smart health decision process support.
Table 4.
Risk Distribution Assessment by Policy Action. Values Show the Distribution of Selected Actions by Assessed Overload-Risk Level
| Risk Level | Distribution | Action (s) |
|---|---|---|
| Low | 0-0.25 | Normal operations (no change) |
| Medium | 0.25–0.5 | Reschedule and Telemedicine |
| High | > 0.50 | Add staffs |
Table 5.
Risk Probability Level Distribution. Risk Levels are Derived from the Random Forest Overload Probability Estimates
| Risk Probability | Description |
|---|---|
| 0-0.25 | Safe |
| 0.25–0.5 | Moderate warning |
| > 0.50 | High risk |
| 0.75–1.00 | Overload |
Temporal Action Switching Over Time
The temporal action trajectory for Taipei City (Influenza) reveals structured and risk-dependent switching behaviour. During low-incidence periods, the policy predominantly maintains Normal operation. As weekly case counts begin to increase, the decision framework transitions to reschedule, indicating early preventive mitigation. When overload probability surpasses high-risk thresholds, the policy escalates to AddStaff, reflecting capacity expansion under anticipated surge conditions.
Notably, intervention shifts occur prior to observed peak congestion, suggesting that the Monte Carlo look-ahead component anticipates backlog propagation rather than reacting to realized overload. Furthermore, increased telemedicine utilization in the after 2020 period illustrates adaptive learning under structural healthcare transformation.
These findings confirm that the Bayesian-MDP framework created predictive and risk-sensitive outpatient management.
Pareto Sustainability Trade-Offs Result
This analysis reveals structural tension between service congestion and energy consumption. The adding manpower (AddStaff) policy achieves the lowest queue levels but incurs the highest energy cost, whereas the EnergySave policy minimizes energy usage at the expense of significant backlog accumulation. Intermediate policies (Normal and rescheduling) occupy transitional positions within the trade-off space. The model of Bayes-MDP leads to solve purposed problem by selecting policies based on posterior overload probability and future demand projections. As a result, system-level performance shifts toward the Pareto-efficient region, achieving meaningful congestion reduction without proportional increases in energy usage.
This multi-objective balance reflects the integration of sustainability considerations into operational decision-making as this consideration of finding the smart to balance the facilities below Table 6.
Table 6.
Effect of Each Single-Action Policy without Trade-Off Decision-Making
| Individual Actions | Advantages | Disadvantages |
|---|---|---|
| Add staff | Patients Reliability | Energy Consumption |
| Energy Saving | Sustainability Concern | Patients Convenience |
Model Robustness of Pre and Post 2020 (COVID-19 Pandemic)
To evaluate healthcare dynamics, we also examined outpatient demand patterns before and after 2020. Pre and post 2020 comparison reveals noticeable shifts in volatility and peak intensity, particularly for influenza-like illness (ILI). Then, after 2020 patterns exhibit altered baseline levels and more irregular peak structures compared with earlier years, suggesting that outpatient behaviour and disease transmission dynamics have incurred some structural transformation during pandemic.
Such rule shifts challenge static scheduling policies compared to historical seasonality. Under a fixed operational rule, capacity decisions derived from pre-2020 patterns may become misaligned with post pandemic demand volatility. In contrast, the proposed framework updates overload probabilities dynamically based on observed data and incorporates these probabilistic beliefs into action selection.
The temporal action-switching analysis demonstrates that the policy continues to escalate interventions appropriately under post-2020 volatility, particularly during renewed outbreak waves in 2023–2025. The adaptive belief update mechanism allows the system to remain responsive without relying on fixed historical thresholds.
These findings confirm that the proposed framework maintains operational coherence under structural regime changes, supporting resilience in evolving epidemiological environments.
Sensitivity Analysis
Regarding Figure 8 illustrates the variation of the energy penalty weight (γ), the relative ranking of policies remains consistent. This shows the high-impact interventions of adding manpower (AddStaff) and the new concept with technological-based like Telemedicine (Tele) maintain their structural hierarchy relative to low-cost actions for no actions (Normal) and reduce energy consumption (EnergySave). This study is providing a stable foundation for decision-making based on the real-world scenarios and remaining patient-oriented concept. While Figure 9 shows the change in expected reward as sustainability priority increases. The Red and orange lines show the Energy-intensive actions when adding manpower (AddStaff) experience sharper penalties as penalty weight (γ) rises, reflecting their higher energy cost. For the Blue and green slopes illustrate Energy-efficient strategies in reducing energy consumption (EnergySave) show minimal sensitivity, gaining relative advantage in high-sustainability scenarios. This linear relationship demonstrates that the framework functions as a decision-support tool, allowing hospital administrators to trade-off between Queue rearrangement (Service level factor) and energy conservation (Sustainability factor) without destabilizing the core operational logic.
Figure 8.
Sensitivity of mean reward to the energy penalty weight (0.1, 0.3, and 0.5) for the static baseline, risk-threshold rule, and Bayes-MDP look-ahead policies. Policy rankings remain unchanged across the tested range.
Figure 9.
Policy trade-off between simulated queue length and estimated (proxy-based) energy consumption. Small points show individual Monte Carlo replications (N=50); large points show policy means. The look-ahead and myopic policies dominate the static baseline on both objectives, whereas the risk-threshold rule reduces queues at the cost of higher energy use.
Overall Performance Summary
The comparative evaluation between the baseline Normal-operation rule and after approached by Bayes-MDP with Monte Carlo look-ahead simulation shows the consistent and predicable improvements in operational performance.
First, the proposed model reduced average backlog magnitude relative to baseline, indicating improved congestion management during surge periods. The reduction reflects proactive escalation to capacity-expansion policies under elevated overload probability, preventing queue accumulation from compounding over time.
Second, energy consumption was reduced relative to baseline under low and moderate-risk rules, while remaining controlled during high-risk situation. The system smartly trade-off the energy-intensive actions only when probabilistically justified, avoiding unnecessary standby load and excessive resource usage.
Third, the reward function in this study, capturing service backlog, energy usage, and intervention cost, showed overall improvement under the proposed policy. This confirms that multi-step Monte Carlo look-ahead planning enhances long-term system efficiency compared before implementation carefully.
Unless stated otherwise, all reported improvements are means across N = 50 Monte Carlo replications with mean-preserving stochastic demand perturbatuions (σ=0.15) applied identically to all policies (common random numbers) with 95% confidence intervals are percentile intervals across replications, and baseline versus policy comparisons are reported in Table 1.
In conclusion, policy distribution analysis further demonstrates structured risk-adaptive behaviour as shown at Table 3, the model divides different approach by prioritizing the seriousness of risk from green (low-risk) to red (high-risk) to avoid the patient-oriented support dynamically. Moreover, Pareto trade-off analysis reinforces these findings by illustrating the inherent tension between queue reduction and energy conservation.
Collectively, these results indicate that the model in this study improves outpatient system resilience, balances sustainability and service objectives, and adapts coherently under epidemiological uncertainty. This model provides an operational pathway toward Industry 5.0 for healthcare and smart health management decision support that is human-centered, sustainable with flexibility management.
Discussion
Implications for Healthcare Informatics
This study demonstrates that outpatient management under epidemic-driven uncertainty is fundamentally a decision-support problem, rather than a pure prediction or automation task. While existing healthcare informatics systems increasingly provide accurate forecasts of patient demand or disease incidence, such predictive outputs alone are insufficient for operational decision-making in highly dynamic environments. Our results show that integrating probabilistic forecasting with adaptive decision models enables healthcare systems to respond more effectively to demand variability.
From an informatics perspective, the proposed framework illustrates how heterogeneous healthcare data, such as, national insurance records, surveillance signals, and operational proxies can be transformed into decision intelligence. The overload probability serves as an interpretable intermediate representation that related to raw data and operational actions. This design aligns with the core objective of healthcare informatics in converting data into decisions that improve system performance and patient outcomes.
Importantly, the results highlight the real-time relevance of decision-support systems. The adaptive policies by the Bayesian MDP respond flexibly for changing demand and risk conditions, rather than relying on fixed rules. This capability is particularly valuable for outpatient services, where walk-in demand and epidemic outbreaks introduce substantial uncertainty that cannot be fully anticipated through planning alone.
Industry 5.0: Human-Centric and Sustainable Decision Support
The findings of this study align closely with the principles of Industry 5.0, which emphasize human-centricity, sustainability alongside technological advancement. Further from Industry 4.0 approaches that prioritize efficiency and automation, the proposed framework explicitly models trade-offs between patient service quality, workforce burden and energy consumption fairly. The Simulation results show that policies reducing patient backlog during high-demand periods often caused to higher energy usage, while energy-saving actions are most effective under low-risk conditions. This cause to implement sustainability in healthcare cannot be achieved through uniform cost-cutting measures,28,29 but instead requires context-aware decision-making that balances competing objectives. By incorporating energy utilization directly into the reward structure, the framework enables sustainability to function as a first-class system objective rather than an external constraint. Furthermore, the framework supports human-in-the-loop governance.
This integrative operation can be adapted into real-world scenarios than usage of a single optimal policy, it creates trade-off analysis that makes all factors to adjust priorities based on situational needs and ethical considerations. This flexibility is particularly important in healthcare long-term operations, also operational staffs well-being and patient trust.
Comparison with Previous Studies
Direct numerical comparison with previous outpatient scheduling studies is limited by differences in settings, data, and objective functions as nevertheless, the present framework can be positioned against three groups of studies. Traditional or classical MDP-based scheduling model6,12,13 optimize admission or booking decision under known transition dynamics, whereas the present framework updates its transition beliefs dynamically, which is advantageous under epidemic period shifts, such as, those observed post 2020 (Model Robustness of Pre and Post 2020 (COVID-19 Pandemic)). Simulation-based studies report waiting-time reduction roughly 15–40% from improved scheduling rules15–17 and our simulated mean queue reduction of 39% lies at upper end of this range, which is given that the evaluation covers epidemic surge periods in which adaptive policies have the greatest scope for improvement. Compared with prediction-only machine learning approaches,20–23 the contribution here is the closed loop between predicted overload risk and multi-objective action selection; relative to recent digital-twin sustainability work,24 it is the explicit inclusion of energy in the reward function. Benchmarking against a myopic one-step policy and a risk-threshold rule (Table 1) shows that the look-ahead policy substantially outperforms the threshold rule (39.0% versus 12.1% queue reduction) and performs comparably to the myopic policy in this configuration, suggesting that most of the benefit derives from adaptive, queue-aware action selection; benchmarking against model-free reinforcement learning methods remains an avenue for future work.
Practical Implementation Challenges
Computationally, the framework is practical for routine use, the complete 2007–2025 evaluation (50 Monte Carlo replications across 88 county-disease series) required approximately 12 seconds (0.2 minutes) on a standard laptop (Apple M2, 8 GB RAM), so the look-ahead computation for a single weekly planning decision is effectively instantaneous, and the more intensive model-fitting steps (seasonal forecasting and Random Forest training) are performed offline without affecting the weekly planning cycle. Integration with hospital information systems would require automated weekly data feeds from NHI/RODS-type surveillance, mapping of policy actions onto rostering and clinic-template systems, and governance procedures for reviewing and overriding model recommendations. Acceptance by clinicians and administrators is equally critical as the framework is designed for human-in-the-loop use, in which the model proposes actions and healthcare staff retain final decision authority, but prospective studies of usability, trust, and organizational readiness are needed before routine use. A staged evaluation path can mitigate adoption risk: retrospective replay of the policy on hospital-level operational data, followed by shadow-mode operation in which recommendations are generated but not executed, and finally a prospective pilot in selected clinics.
Limitations and Future Work
Several limitations of this study should be acknowledged. First, all findings are simulation-based; no prospective real-world implementation was conducted, so the reported improvements demonstrate the potential of the framework rather than confirmed operational outcomes. Second, energy utilization was estimated using a hybrid proxy model combining seasonal patterns and outpatient volume rather than direct measurements from hospital energy meters; the proxy assumptions (baseline load, seasonal HVAC effects, and per-patient consumption) directly influence the estimated 5.4% saving and require validation against metered data. Third, the state-transition and reward formulations embed modeling assumptions, weekly decision epochs, county-level aggregation, quartile-based state discretization, capacity calibrated to historical mean demand, fixed action-effect multipliers, and lognormal demand noise (σ=0.15), whose individual effects were only partially explored in the sensitivity analysis. Fourth, the action-effect magnitudes (eg, 15% of telemedicine substitution rate and 20% of staffing capacity gain) were specified from operational judgment rather than estimated from intervention data, and the reward assigned no explicit intervention cost in this configuration. Fifth, benchmarking covered rule-based and myopic policies but not model-free reinforcement learning approaches.
For the future work should prioritize prospective validation in actual clinical settings, estimation of action effects from real intervention data, benchmarking against reinforcement-learning baselines, integration of metered energy data, and evaluation of integration with hospital information systems and routine workflows. Despite these limitations, the modular design of the framework allows incorporation of additional data sources, refined energy models, and alternative decision criteria, and supports real-world environmental, social, and governance (ESG) reporting aligned with the United Nations 2030 Agenda30 and governance-oriented long-termism31 in sustainable healthcare development.
Conclusion
This study proposed and evaluated, in simulation, an integrated Bayes-MDP and Monte Carlo look-ahead framework for adaptive outpatient management, using 2007–2025 weekly data from Taiwan’s NHI and disease surveillance systems. Across 50 Monte Carlo replications, the adaptive risk-aware look-ahead policy reduced the mean simulated outpatient queue by 39% (95% CI 38.8–39.3) and estimated (proxy-based) energy consumption by 5.4% (95% CI 5.3–5.4) relative to a static scheduling baseline evaluated on identical demand traces, outperforming a risk-threshold rule (12.1% queue reduction) and performing comparably to a myopic one-step benchmark. No single fixed policy performed best across all risk levels: energy-saving operation dominated in low-risk weeks, while telemedicine substitution and staffing escalation dominated under elevated risk. These results indicate the potential of adaptive, human-in-the-loop decision support to balance service quality, specialist workload, and sustainability objectives simultaneously. Because the evidence is entirely simulation-based and the energy estimates rely on proxy variables, these findings should not be interpreted as confirmed operational outcomes. Prospective validation in actual clinical settings, benchmarking against alternative learning-based methods, and integration with hospital information systems are required before the framework can be implemented in routine hospital practice.
Acknowledgments
This study was supported by National Taiwan University of Science and Technology, Taipei, Taiwan and the database used in this study is available at National Health Insurance, Taiwan (NHI). All materials related to this study are available from the corresponding author upon reasonable request.
Funding Statement
This study was supported by the National Taiwan University of Science and Technology, Taipei, Taiwan.
Data Sharing Statement
The data analyzed in this study are publicly available from the National Health Insurance Administration, Ministry of Health and Welfare, Taiwan, and the Taiwan real-time outbreak and disease surveillance (RODS) system. The processed datasets and simulation code are available from the corresponding author on reasonable request.
Ethics Approval
This study did not involve any interaction or intervention with human subjects and did not access any individual-level or identifiable patient data. It used only de-identified, aggregated, publicly available secondary statistics released by the National Health Insurance Administration and the real-time outbreak and disease surveillance (RODS) system, Ministry of Health and Welfare, Taiwan (available at https://www.nhi.gov.tw/en/np-44-2.html), reported at the county-week level. Under Taiwan’s Human Subjects Research Act (人體研究法), Article 5, research falling within the exemption categories announced by the competent authority (the Ministry of Health and Welfare) is not subject to mandatory review by an institutional review board. These categories include research that uses legally obtained, publicly available information and de-identified data from which specific individuals cannot be identified. As this study relies solely on such publicly available, non-identifiable aggregate data, it is exempt from institutional review board review under this national legislation.
Disclosure
No conflicts of interest relevant to this article.
References
- 1.Lu J, Liang L. The role of digital health under Taiwan’s national health insurance system: progress and challenges. Health Syst Reform. 2024;10(2):2375433. doi: 10.1080/23288604.2024.2375433 [DOI] [PubMed] [Google Scholar]
- 2.Green LV, Savin S, Murray M. Providing timely access to care: what is the right patient panel size? Jt Comm J Qual Patient Saf. 2007;33(4):211–20. doi: 10.1016/s1553-7250(07)33025-0 [DOI] [PubMed] [Google Scholar]
- 3.Shanafelt TD, West CP, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2023. Mayo Clin Proc. 2024;100:1142–1158. [DOI] [PubMed] [Google Scholar]
- 4.InvesTaiwan, Ministry of Economic Affairs. Current status and trends in Taiwan’s digital healthcare industry. 2025. Available from: https://investtaiwan.nat.gov.tw/intelNewsPage202504250001eng?lang=eng&search=202504250001. Accessed September 09, 2026.
- 5.Huang WL, Liao SL, Huang HL, et al. A case study of lean digital transformation through robotic process automation in healthcare. Sci Rep. 2024;14(1):14626. doi: 10.1038/s41598-024-65715-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Qu X, Peng Y, Shi J, LaGanga L. An MDP model for walk-in patient admission management in primary care clinics. Int J Prod Econ. 2015;168:303–320. doi: 10.1016/j.ijpe.2015.06.022 [DOI] [Google Scholar]
- 7.Dantas LF, Fleck JL, Cyrino Oliveira FL, Hamacher S. No-shows in appointment scheduling – a systematic literature review. Health Policy. 2018;122(4):412–421. doi: 10.1016/j.healthpol.2018.02.002 [DOI] [PubMed] [Google Scholar]
- 8.Ferro DB, Brailsford S, Bravo C, Smith H. Improving healthcare access management by predicting patient no-show behaviour. Decis Support Syst. 2020;138:113398. doi: 10.1016/j.dss.2020.113398 [DOI] [Google Scholar]
- 9.Feldman SR, Martin B. Machine learning approaches to predict patient no-shows: systematic review. JMIR Med Inform. 2022;10(6):e34702. doi: 10.2196/34702 [DOI] [Google Scholar]
- 10.Brailsford SC, Vissers J. OR in healthcare: a European perspective. Eur J Oper Res. 2011;212(2):223–234. doi: 10.1016/j.ejor.2010.10.026 [DOI] [Google Scholar]
- 11.Gupta D, Denton B. Appointment scheduling in health care: challenges and opportunities. IIE Trans. 2008;40(9):800–819. doi: 10.1080/07408170802165880 [DOI] [Google Scholar]
- 12.Patrick J. A Markov decision model for determining optimal outpatient scheduling. Health Care Manag Sci. 2012;15(2):91–102. doi: 10.1007/s10729-011-9185-4 [DOI] [PubMed] [Google Scholar]
- 13.Geng N, Xie X. Optimal dynamic outpatient scheduling for a diagnostic facility with two waiting time targets. IEEE Trans Automat Contr. 2016;61(12):3725–3739. doi: 10.1109/TAC.2016.2523882 [DOI] [Google Scholar]
- 14.Creps J, Lotfi V. A dynamic approach for outpatient scheduling. J Med Econ. 2017;20(8):786–798. doi: 10.1080/13696998.2017.1318755 [DOI] [PubMed] [Google Scholar]
- 15.Homem-de-Mello T, Kong Q, Godoy-Barba R. A simulation optimization approach for the appointment scheduling problem with decision-dependent uncertainties. INFORMS J Comput. 2022;34(4):1511–1532. doi: 10.1287/ijoc.2022.1212 [DOI] [Google Scholar]
- 16.Rema V, Sikdar K. Optimizing patient waiting time in the outpatient department of a multi-specialty Indian hospital: the role of technology adoption and queue-based Monte Carlo simulation. SN Comput Sci. 2021;2(3):200. doi: 10.1007/s42979-021-00595-8 [DOI] [Google Scholar]
- 17.Pisharody VA, Yarmohammadi H, Ziv E, et al. Reducing wait times for radiology exams around holiday periods: a Monte Carlo simulation. J Digit Imaging. 2023;36(1):29–37. doi: 10.1007/s10278-022-00728-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jamjoom A, Abdullah M, Abulkhair M, Alghamdi T, Mogbil A. Improving outpatient waiting time using simulation approach. In: 2014 UKSim-AMSS 8th European Modelling Symposium. IEEE; 2014. doi: 10.1109/EMS.2014.85. [DOI] [Google Scholar]
- 19.Zhao L, Lie B. Modeling and simulation of patient flow in hospitals for resource utilization. In: Proceedings of the 49th Scandinavian Conference on Simulation and Modeling. Oslo University College; 2008. [Google Scholar]
- 20.Ala A, Goli A. Incorporating machine learning and optimization techniques for assigning patients to operating rooms by considering fairness policies. Eng Appl Artif Intell. 2024;136:108980. doi: 10.1016/j.engappai.2024.108980 [DOI] [Google Scholar]
- 21.Joseph J, Senith S, Kirubaraj AA, Jino Ramson SR. Prediction of consultation wait time in outpatient clinic: an approach using neural network with optimized feature selection. Procedia Comput Sci. 2024;233:129–137. doi: 10.1016/j.procs.2024.03.202 [DOI] [Google Scholar]
- 22.Zahra SA, Irfan M. Machine learning techniques for predicting patient no-shows in hospitals. In: Al-Fuqaha A, Khan MA, Jan MA editors. Machine Learning for Healthcare Systems. Elsevier; 2023:285–300. doi: 10.1016/B978-0-443-13701-3.00058-9 [DOI] [Google Scholar]
- 23.Liu C, Wang H, Liu Y. A novel ensemble learning framework for proactive healthcare risk prediction. Future Gener Comput Syst. 2023;141:210–223. doi: 10.1016/j.future.2023.01.021 [DOI] [Google Scholar]
- 24.Liu Y, Zhang Y, Chen X, Yang M. Optimizing sustainable healthcare delivery using a digital twin-based simulation approach. J Clean Prod. 2025;436:140950. doi: 10.1016/j.jclepro.2025.140950 [DOI] [Google Scholar]
- 25.Wang C, Cheng SF, Hung JL, Tang PL. Highly frequent utilization of outpatient services in a national health insurance system – analysis of associated factors and underlying co-morbidity. Curr Med Res Opin. 2020;36(11):1761–1767. doi: 10.1080/03007995.2020.1832057 [DOI] [PubMed] [Google Scholar]
- 26.Puterman ML. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley; 2014. [Google Scholar]
- 27.Hauskrecht M, Fraser H, Greenes RA. Modelling and using probabilistic patient state transitions in medical decision making. Artif Intell Med. 2000;18(3):221–247. doi: 10.1016/S0933-3657(00)00070-1 [DOI] [PubMed] [Google Scholar]
- 28.Maruthappu M, Hasan A, Zeltner T. Developing sustainable health care systems. J R Soc Med. 2015;108(11):439–443. [Google Scholar]
- 29.Kringos DS, Jansen T. Sustainable health systems: a policy brief. Health Policy. 2019;123(6):545–550. [Google Scholar]
- 30.United Nations. Transforming our world: the 2030 agenda for sustainable development. United Nations; 2015. Available from: https://sdgs.un.org/2030agenda. Accessed September 09, 2026. [Google Scholar]
- 31.CFA Institute. Certificate in ESG Investing Curriculum 2025. CFA Institute; 2025. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data analyzed in this study are publicly available from the National Health Insurance Administration, Ministry of Health and Welfare, Taiwan, and the Taiwan real-time outbreak and disease surveillance (RODS) system. The processed datasets and simulation code are available from the corresponding author on reasonable request.