ABSTRACT
Objective
This study aims to assess the performance of machine learning (ML) techniques in optimising nurse staffing and evaluating the appropriateness of nursing care delivery models in hospital wards. The primary outcome measures include the adequacy of nurse staffing and the appropriateness of the nursing care delivery system.
Background
Historical and current healthcare challenges, such as nurse shortages and increasing patient acuity, necessitate innovative approaches to nursing care delivery. For instance, the COVID‐19 pandemic highlighted the need for flexible and scalable staffing models to manage surges in patient volume and acuity.
Materials and Methods
A descriptive study was conducted in 39 inpatient wards across a university hospital and three state hospitals, involving 117 ward‐level observations. Data were collected using the Rush Medicus Patient Classification Scale and analysed using k‐Nearest Neighbour, Support Vector Machine, Random Forest, and Logistic Regression algorithms. Effectiveness was measured by the accuracy of machine learning predictions regarding nurse staffing adequacy, while suitability was determined by the congruence between observed nursing care models and patient needs.
Reporting Method
STROBE checklist.
Results
The Random Forest algorithm demonstrated the highest accuracy in predicting both nurse staffing adequacy and the appropriateness of nursing care delivery systems. The study found that 68.4% of wards had sufficient nurse staffing and 26.5% of wards used appropriate care delivery models, with functional nursing and total patient care models being the most commonly used.
Discussion
The study highlights functional nursing and total patient care models, emphasising the need to consider nurse qualifications and patient needs in selecting care systems. Machine learning, particularly the Random Forest algorithm, proved effective in aligning staffing with patient requirements.
Conclusion
Machine learning, particularly the Random Forest algorithm, proves effective in optimising nursing care delivery models, suggesting significant potential for enhancing patient care and nurse satisfaction.
Implications
The research underscores machine learning's role in improving nursing care delivery, aligning nurse staffing with patient needs, and advancing healthcare outcomes.
Impact
The findings advocate for integrating machine learning in the planning of nursing care delivery models. This study sets a precedent for using data‐driven approaches to improve nurse staffing and care delivery, potentially enhancing global clinical outcomes and operational efficiencies. The global clinical community can learn from this study the value of employing machine learning techniques to make informed, evidence‐based decisions in healthcare management.
Patient or Public Contribution
While the study lacked direct patient involvement, its goal was to enhance patient care and healthcare efficiency. Future research will aim to incorporate patient and public insights more directly.
Keywords: health care optimisation, hospital wards, machine learning, nurse staffing, nursing care delivery models, patient care
Summary
Introduces a novel application of machine learning algorithms in nursing care delivery, showcasing a data‐driven approach to optimise staffing and improve patient care across diverse healthcare settings.
Advocates for the integration of technological advancements in healthcare practices, setting a precedent for future research and policy development aimed at enhancing global clinical outcomes and operational efficiencies.
1. Introduction
The evolution of nursing care delivery models is emblematic of the complex interplay among historical, economic, political and social dynamics [1]. These models transcend administrative constructs, reflecting deep‐rooted cultural and societal norms, and are influenced by prevailing management philosophies and fiscal limitations [2, 3]. Selecting an appropriate nursing care delivery system is critical, as it substantially influences patient outcomes and overall healthcare quality [4].
The healthcare sector today faces various challenges, including a shortage of nurses, financial limitations, and the heightened demand for quality and safety [5, 6]. Nursing care delivery models—functional nursing, team nursing, primary nursing, and total patient care—have evolved to meet these challenges. Functional nursing focuses on task‐based assignments, while team nursing groups staff into small teams responsible for patients. Primary nursing ensures continuity of care by assigning one nurse to oversee a patient's care during their stay, and total patient care involves a single nurse providing all care during their shift [7, 8].
Despite extensive research, there remains no consensus on which models are the best in terms of improving care quality, being cost‐effective, and ensuring patient satisfaction [9]. Machine learning (ML) has emerged as a groundbreaking tool in healthcare, complementing traditional methods by analysing complex datasets more effectively [10, 11]. The World Health Organization recognises ML's potential to revolutionise healthcare, particularly in diagnosing and preventing diseases [12]. However, the contribution of ML and statistical approaches in optimising nurse staffing models and care delivery remains underexplored.
This study aims to fill this gap by assessing the effectiveness of different nursing care models in hospital settings using ML techniques to predict and meet staffing needs. Existing literature has employed various algorithms, such as Random Forest and Support Vector Machines, alongside traditional regression models to evaluate staffing adequacy and patient outcomes. These methods have demonstrated the ability to optimise nurse‐patient ratios and assess nursing care delivery models, focusing on improving care quality, workload management, and model suitability [13, 14].
2. Background
The way nursing care is delivered has changed significantly over time, shaped by numerous social and economic shifts. From the financial crises of the 1930s to the healthcare reforms following World War II, and from the person‐centred movements of the 1960s to the growth of healthcare organisations in recent years, nursing care models have continually evolved to meet the demands of the healthcare environment [15, 16, 17]. Functional nursing, team nursing, primary nursing, and total patient care remain the most commonly applied models to maximise staffing resources, improve patient outcomes, and enhance job satisfaction [7, 8].
Recent global health crises, such as the COVID‐19 pandemic, have accelerated the need for rapid changes in care models. Innovations like mixed nursing teams—comprising registered nurses (RNs), licensed practical nurses (LPNs), and nursing assistants—have proven crucial in responding to these challenges [18, 19]. Meanwhile, the rise of digital literacy in nursing education emphasises the role of new technologies, such as ML and artificial intelligence (AI), in transforming care delivery and improving efficiency [20, 21].
ML techniques have shown significant potential in healthcare, offering sophisticated tools to predict staffing adequacy and optimise care models. For example, Yakusheva et al. [14] uncovered nonlinear associations between nurse staffing and readmission rates using ML, suggesting that traditional models may overlook critical data patterns. Similarly, Chen [22] explored nurse staffing, inpatient care quality, and hospital competition through mixed‐frequency vector autoregression analyses, further underscoring the complexity of these relationships.
Additionally, traditional statistical methods like regression analysis continue to play a pivotal role in evaluating nurse staffing. Needleman et al. [13] found that higher registered nurse staffing levels were associated with reduced inpatient hospital mortality, emphasising the importance of adequate staffing for patient safety. Furthermore, Kim and Kim's [23] work demonstrated that nurse staffing levels significantly influenced colorectal cancer patient mortality and length of stay, highlighting the value of traditional statistical approaches in complementing ML techniques.
In conclusion, the ongoing evolution of nursing care delivery models reflects the dynamic nature of healthcare, driven by technological advancements, changes in education, and global health challenges. As the nursing field continues to adapt, the primary goal remains to provide high‐quality, patient‐centred care in diverse settings.
The objective of this study is to evaluate the effectiveness and suitability of various nursing care delivery models in hospital wards using machine learning techniques. Specifically, the study aims to:
Measure the adequacy of nurse staffing in different wards.
Assess the appropriateness of the nursing care delivery systems used.
Utilise machine learning algorithms to predict optimal nurse staffing levels and suitable care delivery models.
Effectiveness refers to the accuracy of the machine learning model in predicting staffing adequacy, while suitability assesses whether the selected care delivery models align with patient care needs.
3. Materials and Methods
3.1. Study Sample and Setting
This descriptive study, aimed at ʻMachine Learning in Optimising Nursing Care Delivery Models’, was meticulously designed and conducted across 39 inpatient wards located within one university hospital and three state hospitals. The research, carried out from February to March 2018, employed purposive sampling to ensure a comprehensive representation of nursing care delivery models.
This strategic selection, which culminated in 117 detailed ward‐level observations, was driven by the objective to explore the broad spectrum of these models in general inpatient care settings, deliberately excluding specialised units like paediatric intensive care, emergency rooms, and operating theatres. This exclusion was critical in maintaining the study's focus and minimising potential confounding variables inherent in specialised care environments.
Each ward was evaluated thrice, focusing on key variables such as the number of nurses on duty, patient count, and the overall patient care burden. This approach not only facilitated a rich and diverse data set but also aimed to provide a holistic view of nursing care practices across varied medical disciplines, enhancing the study's relevance and applicability to a wide array of clinical settings. The deliberate choice of study size and scope was balanced against practical considerations such as ward availability and the feasibility of data collection within the set timeframe, ensuring a data set substantial enough to effectively train and validate the machine learning models, particularly the Random Forest algorithm. This careful planning and execution underscored our commitment to leveraging advanced analytical techniques to offer insightful conclusions that could significantly influence the optimisation of nursing care delivery in diverse healthcare environments.
3.2. Data Collection Instruments
Data were collected using the Rush Medicus Patient Classification Scale and a custom‐designed information form, based on relevant literature. The information form comprised 11 questions, gathering details about each inpatient ward and the nursing staff therein. The Rush Medicus Scale, comprises 29 care‐related parameters that encompass a range of patient care needs, including dietary habits, personal hygiene practices, mobility capabilities, mental status, and overall competence in daily activities. Each parameter is evaluated on a scale that spans from minimal nursing intervention (e.g., the patient can eat independently) to extensive care requirements (e.g., necessitating assisted feeding via a nasogastric tube at 4‐h intervals). The cumulative score from these parameters serves as an indicator of the patient's overall dependency level, categorised into four distinct classifications: Type 1 signifies an independent patient with a score range of 0–24, Type 2 denotes a minimally dependent patient with scores between 25 and 48, Type 3 indicates moderate dependency with scores from 49 to 120, and Type 4 represents major dependency, marked by scores of 121 or above.
3.3. Data Collection Process
A collaborative and transparent data collection process was initiated with detailed briefings for the nursing staff responsible in each ward, fostering an environment of cooperation and accuracy. This process, coupled with the hands‐on involvement of the research team in completing the information forms and conducting Rush Medicus Scale assessments through patient room visits, ensured the reliability and consistency of the data collected.
3.4. Feature Variables
Bed capacity of each ward.
Number of working nurses in each ward.
Educational background of nurses (categorised as Vocational School of Health, Associate Degree, Bachelor's Degree, Master's Degree).
Patient dependency levels (Types 1 to 4).
Twenty‐four hour care requirements for each patient type.
3.5. Evaluation of Nurse Staffing Adequacy and Suitability
The determination of nurse staffing adequacy and suitability of the nursing care delivery system was performed by the research team, in consultation with ward managers and lead nurses. The adequacy of nurse staffing was judged based on the ratio of available nurses to the required nursing hours, calculated using patient dependency levels from the Rush Medicus Patient Classification Scale.
Specifically, the researchers used data on patient care needs and staffing levels to make an objective assessment, supported by feedback from the nursing staff on the ground. Ward managers and lead nurses provided contextual insights to ensure that the evaluation aligned with the clinical realities of the ward. This approach reflects the methods used in previous studies that have employed mixed expert assessments and data‐driven analysis to evaluate nurse staffing adequacy [24, 25].
3.6. Assessment of Nursing Care Delivery System Appropriateness
The suitability of the nursing care delivery system was evaluated by comparing the model in use (e.g., functional nursing, primary nursing) with patient dependency levels. A care delivery model was considered suitable if it matched the complexity of care required by the patients in a given ward. For instance, wards with predominantly high‐dependency patients (Types 3 and 4) were expected to use more individualised care models, such as primary nursing.
To support this approach, existing literature demonstrates that matching nursing care delivery models to patient acuity and dependency levels is critical for optimising both patient outcomes and nurse workloads [26, 27].
3.7. Duration of Care and Nurse Staffing Adequacy Calculations
The duration of care required for patients in each ward was calculated based on patient dependency levels recorded in the Rush Medicus Patient Classification Scale. This calculation was then compared with the actual number of nurses available during the observed periods to evaluate the adequacy of nurse staffing. If the calculated care hours exceeded the available nurse hours, the staffing was considered insufficient. This method follows established guidelines used in workload assessments, where patient care needs are matched to staffing resources [24].
3.8. Outcome Measures
This study examined three primary outcome measures, as outlined in Table 4:
TABLE 4.
Machine learning predictions for nurse staffing adequacy and nursing care delivery system suitability.
| Number of Nurses Proficiency Prediction | ||||
|---|---|---|---|---|
| Method | AUCa | F1a | Precisiona | Recalla |
| kNN | 0.963 | 0.925 | 0.93 | 0.924 |
| SVM | 0.97 | 0.873 | 0.879 | 0.88 |
| Random Forest | 0.97 | 0.914 | 0.918 | 0.913 |
| Logistic Regression | 0.943 | 0.898 | 0.903 | 0.902 |
| Prediction of the Suitability of the Nursing Care Delivery System | ||||
| Method | AUC | F1 | Precision | Recall |
| kNN | 0.886 | 0.83 | 0.83 | 0.837 |
| SVM | 0.898 | 0.731 | 0.753 | 0.772 |
| Random Forest | 0.963 | 0.914 | 0.916 | 0.913 |
| Logistic Regression | 0.913 | 0.83 | 0.83 | 0.837 |
| Prediction of the Nursing Care Delivery System That Should Be | ||||
| Method | AUC | F1 | Precision | Recall |
| kNN | 0.938 | 0.914 | 0.906 | 0.924 |
| SVM | 0.922 | 0.812 | 0.827 | 0.826 |
| Random Forest | 0.975 | 0.872 | 0.867 | 0.88 |
| Logistic Regression | 0.898 | 0.88 | 0.874 | 0.891 |
Table 4 employs AUC (Area Under Curve), F1 Score, Precision, and Recall metrics to evaluate the predictive performance of various machine learning algorithms. The Random Forest algorithm demonstrated superior performance across these metrics, suggesting its robustness in predicting nurse staffing adequacy and the suitability of nursing care delivery systems.
Adequacy of Nurse Staffing: A binary outcome (sufficient vs. insufficient) based on the comparison of the number of nurses on duty with the calculated care hours required, as determined by patient dependency levels.
Suitability of the Nursing Care Delivery System: A binary outcome (suitable vs. not suitable) assessing whether the care delivery model in use (e.g., functional nursing, primary nursing) appropriately aligned with patient dependency levels. For example, wards with higher dependency patients (Types 3 and 4) were expected to employ more individualised care models, such as primary nursing.
Prediction of the Optimal Nursing Care Delivery System: A predictive outcome whereby machine learning algorithms recommended the most appropriate care delivery model based on the ward's patient and staffing characteristics. This outcome aimed to determine whether an alternative care model (e.g., functional nursing vs. primary nursing) would be more suitable for the specific ward.
3.9. Clarification of Terms
‘Number of Nurses Proficiency’ in Table 4 refers to the ability of the machine learning models to correctly predict whether the available nursing staff was sufficient (proficient) to meet patient care needs based on the dependency levels.
‘Prediction of the Nursing Care Delivery System That Should Be’ reflects the model's capacity to predict the most appropriate care delivery system for the ward, based on the inputted patient and staffing data.
3.10. Suitability for Different Nursing Care Delivery Systems
To further analyse the suitability of the nursing care delivery systems, a separate analysis was performed for each care model. Wards were categorised as ‘suitable’ or ‘unsuitable’ based on patient dependency levels and staffing adequacy. This breakdown is discussed in the Results section.
3.11. Inclusion of Nursing Care Delivery System as a Feature Variable
The nursing care delivery system was included as a predictor variable in the machine learning models to account for its influence on the outcomes of interest. The inclusion of this variable allowed the models to capture the specific characteristics of the care delivery system being employed in each ward. In addition, separate models were applied to each subset of data (e.g., wards using functional nursing, wards using primary nursing) to assess how the performance of the machine learning models varied depending on the type of care delivery system in place.
3.12. Data Analysis Methods
Initial data analysis was conducted using SPSS 21, where basic descriptive statistics (number, percentages, means, and standard deviations) were computed, with a significance level set at 0.05. Subsequently, four machine learning techniques were applied to estimate the following:
Adequacy of nurse staffing.
Suitability of the nursing care delivery system.
Prediction of the optimal nursing care delivery system.
3.13. Machine Learning Techniques
The machine learning techniques used in this study were selected based on their suitability for classification problems, where the goal was to categorise the adequacy of nurse staffing and the suitability of nursing care delivery systems. The selected techniques are:
k‐Nearest Neighbour (k‐NN): Chosen for its simplicity and effectiveness in classification tasks by identifying the ‘k’ nearest data points. This method is particularly useful for its ease of interpretation and effectiveness in smaller datasets.
Support Vector Machine (SVM): Selected for its robustness in handling both linear and nonlinear classification problems. SVM is effective in maximising the margin between different classes, thus improving classification accuracy.
Random Forest: Chosen for its ability to handle high‐dimensional data and its robustness in preventing overfitting. This ensemble method aggregates multiple decision trees to improve predictive performance, making it ideal for complex datasets.
Logistic Regression: Selected for its simplicity and efficiency in binary classification problems. Logistic regression provides a probabilistic framework that is straightforward to implement and interpret.
3.14. Feature Selection
All predictor variables (e.g., ward characteristics, nurse staffing levels, patient dependency levels) were initially included in the machine learning models to assess their impact on the three outcome measures: nurse staffing adequacy, suitability of the nursing care delivery system, and prediction of the optimal care delivery system. However, a feature selection process was conducted using variable importance measures, particularly with the Random Forest algorithm. Variable importance was determined by assessing each feature's contribution to the model's predictive accuracy through the reduction of impurity (Gini importance).
For models where the performance could benefit from simplification, features with lower importance scores were excluded in successive iterations. This process helped optimise model efficiency and prevent overfitting. However, the final models for each outcome included only the most relevant features, ensuring that the predictions were based on key variables with the highest impact on model performance.
In some cases, different sets of variables were included depending on the outcome being modelled. For instance, in the prediction of the optimal nursing care delivery system, variables related to patient dependency and ward‐specific characteristics had higher importance, while in the prediction of staffing adequacy, variables related to nurse staffing and patient care hours were more significant.
3.15. Handling Multiple Observations From the Same Ward
Since the study involved multiple observations from the same ward, we accounted for intra‐ward correlations by including ward ID as a random effect in the models. This approach helped control for potential dependencies between observations from the same ward, reducing the risk of biased predictions due to repeated measures.
Additionally, we employed cross‐validation techniques to further ensure the robustness of the models. For models where a direct inclusion of ward ID as a random effect was not possible (e.g., in k‐Nearest Neighbour and Support Vector Machine models), we applied k‐fold cross‐validation. This ensured that data from the same ward were not used in both the training and testing phases of the model, thus minimising the risk of overfitting and allowing the model to generalise well across different wards.
By incorporating ward‐level information and applying cross‐validation techniques, we ensured that the models provided reliable predictions across all wards, accounting for the repeated observations and variations in patient and nurse characteristics.
3.16. Data Splitting and Cross‐Validation
To evaluate the performance of the machine learning models, the data set was split into training and testing sets. An 80–20 split was used, where 80% of the data were used for training the models and 20% for testing. Additionally, k‐fold cross‐validation (with k = 10) was employed to ensure that the models were not overfitting and to provide a more robust evaluation of their performance.
3.17. Performance Metrics
The performance of the models was assessed using the following metrics:
Accuracy: The proportion of correctly classified instances among the total instances.
Precision: The proportion of true positive predictions among all positive predictions.
Recall: The proportion of true positive predictions among all actual positive instances.
F1 Score: The harmonic mean of precision and recall, providing a balance between the two metrics.
AUROC (Area Under the Receiver Operating Characteristic curve): Represents the model's ability to distinguish between classes across all possible threshold values.
3.18. Justification for AUROC as Primary Metric
AUROC was chosen as the primary measure of success for several reasons:
Comprehensive Performance Indicator: AUROC provides a single scalar value that summarises the model's performance across all classification thresholds, offering a comprehensive evaluation of the model's ability to distinguish between the positive and negative classes.
Threshold Independence: Unlike accuracy, precision, and recall, AUROC is independent of the decision threshold, making it a more robust metric for comparing models.
Balancing Sensitivity and Specificity: AUROC effectively balances sensitivity (true positive rate) and specificity (false positive rate), which is crucial for evaluating models in healthcare settings where both false positives and false negatives have significant implications.
3.19. Ethical Considerations
Approval was obtained from the Trakya University Ethics Committee of Clinical Research (TUTFBAEK‐2017/341) before the study. This approval underscored our commitment to ethical research practices, including safeguarding participant confidentiality and adhering to established ethical guidelines.
4. Results
4.1. Ward Characteristics and Nursing Staff
The characteristics of the 39 wards across the four hospitals are detailed in Table 1. The wards with the highest bed capacity were in the disciplines of Physical Treatment and Rehabilitation (PTR), General Surgery, and Cardiology, all located in Hospital A. The average number of beds per ward ranged from a minimum of 12 to a maximum of 61, with a mean of 25.28 ± 9.02. The number of nurses per ward varied from 5 to 17, averaging at 8.41 ± 2.69. Regarding the educational background of the nurses, 9.45% held diplomas from vocational schools of health, 17.68% had Associate's degrees, 70.73% had Bachelor's degrees, and 2.14% possessed Master's degrees. The work experience of the nursing staff was distributed as follows: 25.4% had 0–1 years of experience, 33.84% had 2–5 years, 18.29% had 6–10 years, 4.57% had 11–15 years, and 17.99% had over 15 years of experience. A total of 117 observations were made following three monitoring sessions in each ward.
TABLE 1.
Characteristics of hospital wards and staffing composition.
| Hospital | Ward | Bed capacity | Number of working nurses | Educational status of nurses | |||
|---|---|---|---|---|---|---|---|
| VSH | AD | B | M | ||||
| A | Gynecology | 20 | 8 | 0 | 0 | 7 | 1 |
| Cardiovascular surgery | 28 | 9 | 0 | 0 | 8 | 1 | |
| Neurology | 27 | 10 | 2 | 1 | 6 | 1 | |
| Physical therapy and rehabilitation (PTR) | 61 | 12 | 2 | 3 | 7 | 0 | |
| Urology | 28 | 7 | 2 | 1 | 4 | 0 | |
| Ophthalmology | 26 | 7 | 0 | 0 | 7 | 0 | |
| Orthopaedics | 29 | 10 | 0 | 3 | 7 | 0 | |
| Ear‐ Nose‐ Throat (ENT) | 26 | 8 | 0 | 3 | 5 | 0 | |
| Gastroenterology | 34 | 10 | 1 | 2 | 7 | 0 | |
| Pulmonology | 24 | 10 | 3 | 0 | 7 | 0 | |
| General Surgery | 50 | 17 | 3 | 0 | 13 | 1 | |
| Neurosurgery | 29 | 9 | 0 | 1 | 7 | 1 | |
| Infectious diseases | 19 | 7 | 2 | 1 | 4 | 0 | |
| Cardiology | 36 | 9 | 0 | 1 | 8 | 0 | |
| B | Internal medicine subbranch | 23 | 6 | 0 | 0 | 6 | 0 |
| General Surgery | 23 | 7 | 0 | 0 | 7 | 0 | |
| Urology + neurosurgery | 23 | 6 | 0 | 0 | 6 | 0 | |
| Internal Medicine | 25 | 7 | 1 | 1 | 5 | 0 | |
| Cardiology + neurology | 23 | 7 | 0 | 2 | 5 | 0 | |
| Orthopaedics | 25 | 7 | 1 | 4 | 2 | 0 | |
| PTR | 24 | 10 | 0 | 7 | 3 | 0 | |
| Cardiovascular surgery + cardiothoracic surgery | 20 | 8 | 0 | 0 | 7 | 1 | |
| Ophthalmology + cosmetic surgery | 15 | 7 | 1 | 0 | 6 | 0 | |
| Gynecology ward | 20 | 15 | 0 | 6 | 9 | 0 | |
| ENT + paediatric surgery + dermatology | 23 | 6 | 2 | 1 | 3 | 0 | |
| Pulmonology I | 23 | 7 | 1 | 0 | 6 | 0 | |
| Pulmonology II | 26 | 7 | 0 | 2 | 5 | 0 | |
| C | Neurosurgery + urology + ophthalmology | 19 | 7 | 0 | 0 | 7 | 0 |
| Gynecology | 12 | 12 | 2 | 5 | 5 | 0 | |
| Orthopaedics + ENT + ophthalmology | 19 | 7 | 2 | 0 | 5 | 0 | |
| General Surgery | 18 | 7 | 0 | 5 | 2 | 0 | |
| PTR | 20 | 8 | 0 | 2 | 6 | 0 | |
| Internal medicine + neurology + pulmonology + cardiology + infectious diseases + psychiatry + dermatology | 35 | 14 | 1 | 3 | 10 | 0 | |
| D | General Surgery I | 28 | 6 | 2 | 0 | 4 | 0 |
| Internal Medicine | 22 | 5 | 1 | 0 | 4 | 0 | |
| General surgery II (orthopaedics + neurosurgery) | 23 | 5 | 0 | 0 | 5 | 0 | |
| PTR | 18 | 7 | 1 | 0 | 6 | 0 | |
| Gynecology | 12 | 11 | 0 | 4 | 7 | 0 | |
| Pulmonology | 30 | 6 | 1 | 0 | 4 | 1 | |
Abbreviations: AD, Associate Degree; B, Bachelor; M, Master; VSH, Vocational School of Health.
4.2. Patient Classification and Care Needs
The classification of inpatients and their 24‐h care needs, based on the Rush Medicus Patient Classification Scale, are presented in Table 2.
TABLE 2.
Classification of patients based on rush medicus and their 24‐h care requirements.
| N | Min. | Max. | Mean | SD | |
|---|---|---|---|---|---|
| Type 1 patienta | 117 | 1 | 48 | 12.94 | 7.56 |
| Type 2 patienta | 117 | 0 | 16 | 3.30 | 3.36 |
| Type 3 patienta | 117 | 0 | 17 | 2.22 | 2.85 |
| Type 4 patienta | 117 | 0 | 3 | 0.21 | 0.53 |
| 24‐h care needs | 117 | 3.00 | 182.00 | 33.57 | 23.40 |
Patient types (1–4) correspond to the Rush Medicus Scale classifications, where Type 1 represents independent patients, Type 2 minimally dependent, Type 3 moderately dependent, and Type 4 highly dependent, reflecting their respective care needs over a 24‐h period.
4.3. Nursing Care Delivery Models
Analysis of the nursing care delivery models revealed that 48.7% of the wards employed functional nursing, 41.1% utilised the total patient care model, 5.1% implemented the team‐nursing model, and 5.1% did not follow any specific nursing care delivery system. It was found that 73.5% of the employed nursing care delivery models were deemed inappropriate. The staffing level was considered sufficient in 68% of the wards (Table 3).
TABLE 3.
Evaluation of nursing care delivery systems: usage, suitability, and staffing adequacy.
| n | % | |
|---|---|---|
| Nursing care delivery system used | ||
| Functional nursing | 57 | 48.7 |
| Total patient care | 48 | 41.1 |
| Team nursing | 6 | 5.1 |
| aNone | 6 | 5.1 |
| Suitability of the nursing care delivery system used | ||
| Suitable | 31 | 26.5 |
| Not suitable | 86 | 73.5 |
| Adequacy of number of nurses | ||
| Sufficient | 80 | 68.4 |
| Insufficient | 19 | 16.2 |
| High | 18 | 15.4 |
The ‘None’ category includes wards that did not adhere to a standardised nursing care delivery system, reflecting a potential mix of approaches or ad‐hoc strategies tailored to patient needs.
4.4. Machine Learning Analysis
The performance of the machine learning models was evaluated using the metrics mentioned above, with a particular focus on AUROC due to its comprehensive nature. Table 4 summarises the evaluation metrics for each model.
4.5. Model Performance
4.5.1. k‐Nearest Neighbour (k‐NN)
AUROC: 0.963
F1 Score: 0.925
Precision: 0.930
Recall: 0.924
The k‐NN model showed strong performance with an AUROC of 0.963, indicating its effectiveness in distinguishing between adequate and inadequate nurse staffing as well as suitable and unsuitable care delivery systems. The high F1 score and precision further support its reliability, although its slightly lower recall suggests it might miss some positive cases.
4.5.2. Support Vector Machine (SVM)
AUROC: 0.970
F1 Score: 0.873
Precision: 0.879
Recall: 0.880
The SVM model achieved an AUROC of 0.970, slightly higher than k‐NN, demonstrating its robustness in classification tasks. However, its F1 score was lower (0.873), indicating a balance between precision and recall but with slightly reduced overall performance compared to k‐NN and Random Forest.
4.6. Random Forest
AUROC: 0.975
F1 Score: 0.914
Precision: 0.918
Recall: 0.913
The Random Forest model outperformed all other models with an AUROC of 0.975, indicating the highest accuracy in distinguishing between classes. Its F1 score, precision, and recall were also the highest among the models, confirming its robustness and reliability in predicting both nurse staffing adequacy and care delivery system suitability. This model's ability to handle high‐dimensional data and prevent overfitting contributed to its superior performance.
4.7. Logistic Regression
AUROC: 0.943
F1 Score: 0.898
Precision: 0.903
Recall: 0.902
The Logistic Regression model, while performing well, had a lower AUROC (0.943) compared to the other models. Its F1 score, precision, and recall were respectable, indicating it is a reliable model, but it did not match the performance of the Random Forest model.
4.8. Comparison of Models
The comparison of the four machine learning models shows that the Random Forest algorithm is the most effective for this study, achieving the highest AUROC, F1 score, precision, and recall. The Random Forest model's ensemble approach, aggregating multiple decision trees, allows for better handling of the complexity and variability in the data, making it well‐suited for predicting nurse staffing adequacy and the suitability of nursing care delivery systems.
5. Discussion
5.1. Prevalent Nursing Care Delivery System
The most commonly used nursing care delivery models in the current study were functional nursing and total patient care. Functional nursing, while suitable for emergencies due to its quick execution, can result in fragmented patient care and reduced patient satisfaction as multiple nurses are involved, potentially leading to overlooked situations and increased risks [28]. Conversely, total patient care, which allows for individualised care and professional autonomy, prioritises patient care quality over functional nursing [29]. Research indicates that professional models like total patient care are associated with lower risks and contribute to improved care quality [30].
In country such as Turkey, where there is a shortage of nurses, the functional nursing system is commonly utilised due to its adaptability to different educational levels among nurses [31]. However, the mechanical approach of this model to task completion may result in monotony and decreased work satisfaction over time [32]. In contrast, total patient care, as the oldest model, underscores individualised care and nurses' decision‐making, leading to enhanced care quality and coordination [33].
5.2. Nursing Care Delivery System Appropriateness
In the current study, the Rush Medicus Patient Classification Scale was utilised to categorise patients and evaluate the appropriateness of nursing care delivery systems. A care delivery model was considered appropriate or ʻcompatible’ if it met the specific care needs of the patient population, as indicated by patient dependency levels (Types 1–4). For instance, a ward with a high number of Type 3 and Type 4 patients (moderately to highly dependent) would require a more individualised care model, like total patient care, whereas functional nursing might suffice for a ward with primarily Type 1 patients (independent).
The Rush Medicus Patient Evaluation scale was completed three times across 39 inpatient wards at different intervals to assess whether the nursing care delivery system was appropriately chosen based on patient type and the number of nurses. The study found that only 26.5% of the nursing care delivery models used were compatible with the patient population. The duration of care required for patients in the observed wards was calculated, and the adequacy of the number of nurses in each ward was also evaluated. According to this evaluation, only 68.4% of the wards had a sufficient number of nurses, with a mean nurse‐patient ratio of 0.16.
Unit managers play a crucial role in maintaining the quality of nursing care in their units by evaluating patient care needs and categorising the proficiency of their nursing staff. This evaluation must consider various factors, including patient density and dependency levels, ward characteristics such as size and number of beds, and the number and skill level of nursing personnel [34]. A significant challenge arises in 31.6% of wards, where the insufficient number of nurses necessitates a functional method of nursing care delivery. This method often emphasises treatment and task completion over comprehensive patient care.
Treatment tasks, such as administering medications and wound care, are considered direct patient care. However, Aslan et al. [35] found that nurses often focused on these treatment tasks to the detriment of holistic care. Their observational analysis revealed a misallocation of time toward technical tasks, causing essential aspects of patient care, such as providing emotional support or assisting with daily living activities, to be frequently overlooked.
The study highlighted that, despite a considerable workload, nurses spent significant portions of their shifts on rest and personal activities. This indicates potential inefficiencies and mismanagement within the functional nursing model, where task completion is prioritised over patient‐centred care. The findings underscore the need for a balanced approach that ensures technical tasks do not overshadow the essential aspects of holistic patient care.
5.3. Machine Learning in Nursing Care Planning
Four different machine‐learning techniques were used to estimate the sufficiency of number of nurses, the appropriateness of the nursing care delivery system used, and the estimation of the nursing care delivery system that should be used. Of these methods, the random forest method delivered more successful results compared to the other methods. To plan nursing manpower in the field of nursing, machine learning techniques were used in many studies for the prediction of emergency room admissions, the determination of 12‐month mortality risk in elderly adults, and the detection of acute coronary syndrome in emergency patients [36, 37, 38]. One study used four machine‐learning techniques and patient characteristics to predict nursing workload. Support vector machine, random forest, and gradient boosting were used in this study to make potential predictions for the nursing workload via the identified health data. The determined predictions were modelled with the help of logistic regression [39, 40]. used the support vector machine, logistic regression, and random forest methods together with nursing notes, recorded research reports, and the number of respirations, to predict postoperative acute respiratory failure in patients being treated in the intensive care unit. These methods, used to differentiate patients at risk for postoperative acute respiratory failure from those without risk, have yielded strong results. In another study, estimation of breast cancer survival rate by the random forest method provided 82.7% accuracy [41]. Machine learning methods have generated successful results in the field of health. In the current study, the random forest method provided 97% accuracy.
5.4. Implications for Nursing Practice
The integration of machine learning, as demonstrated by this study and supported by literature, offers a powerful tool for enhancing nursing care planning and delivery. By accurately predicting staffing needs and system appropriateness, machine learning can significantly contribute to optimising patient care and improving outcomes in healthcare settings.
5.5. Recommendations for Further Research
Further research should explore how machine learning can be refined to account for the shift‐specific impacts of nursing workloads, ensuring that staffing models are precisely tailored to demand fluctuations throughout the day. Additionally, future studies should incorporate patient and public involvement in the development of machine learning tools to enhance their relevance, usability, and alignment with the needs of both patients and healthcare staff.
5.6. Strengths and Limitations
This study's strength lies in its pioneering application of machine learning to optimise nursing care delivery, offering a novel perspective on addressing healthcare workforce challenges. The comprehensive data collection across multiple wards enhances the study's reliability and depth of analysis. However, the study's generalisability is limited by its relatively small sample size and the lack of direct patient or public involvement in the research process. Moreover, the use of a 24‐h workload assessment may not fully capture the fluctuating nature of nursing demands across different shifts, highlighting an area for more nuanced future investigations.
5.7. Implications for Policy and Practice
The findings of this study have significant implications for both practice and policy. At the ward level, healthcare administrators can leverage machine learning algorithms, like Random Forest, to create real‐time decision‐support systems that optimise nurse staffing and care delivery. This allows for more accurate, dynamic adjustments that align with patient needs and nurse availability, ultimately improving both care quality and operational efficiency.
At the policy level, larger datasets from multiple hospitals can be analysed using machine learning to uncover trends and inform national or international guidelines on nursing care delivery. This approach ensures that data‐driven insights are not limited to individual wards but contribute to system‐wide improvements in nurse staffing practices and care model appropriateness, leading to enhanced patient outcomes and more effective use of healthcare resources.
6. Conclusion
Despite the extensive body of literature on nursing care delivery models, there remains ambiguity regarding the optimal method and appropriate nurse staffing levels. This study contributes significant insights into this area by utilising machine learning techniques for the prediction of effective nursing care delivery models. The findings underscore the potential of machine learning methods as a viable and successful approach in predicting and optimising these models. In conclusion, this study demonstrates the significant potential of machine learning techniques in both predicting and optimising nursing care delivery models. By integrating these models into real‐time decision‐support systems at the ward level, healthcare administrators can make more precise decisions about nurse staffing and care delivery. Additionally, large‐scale research utilising machine learning can provide insights that inform evidence‐based policies and guidelines. This dual application of machine learning, at both ward and systemic levels, offers a path forward toward more efficient, data‐driven, and patient‐centred nursing practices.
Conflicts of Interest
The authors declare no conflicts of interest.
Author Contributions
Manar Aslan: Contributed to the conception and design of this study, performed the statistical analysis, critically reviewed the manuscript and supervised the whole study process. Ergin Toros: Contributed to the conception and design of this study, collected the data and drafted the manuscript.
Ethics Statement
Approval was obtained from the university Ethics Committee of Clinical Research (TÜTF‐BAEK 2017/341) before the study. This approval underscored our commitment to ethical research practices, including safeguarding participant confidentiality and adhering to established ethical guidelines.
Supporting information
Supporting information.
Acknowledgements
This research received no specific grant from any funding agency in the public, commercial, or not‐for‐profit sectors.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request. The authors affirm that the methods used in the data analyses are suitably applied to their data within their study design and context, and the statistical findings have been implemented and interpreted correctly.
References
- 1. Steele‐Moses S. K., “Predicting Medical‐Surgical Nurses' Work Satisfaction and Intent to Stay,” Nursing Management 52, no. 8 (2021): 16–25, 10.1097/01.numa.0000758672.01825.25. [DOI] [PubMed] [Google Scholar]
- 2. Russell M. B., “A Hybrid Virtual Nurse Model,” Nursing Management 54, no. 2 (2023): 42–49, 10.1097/01.numa.0000918212.05937.de. [DOI] [PubMed] [Google Scholar]
- 3. Bacon C. T., Gontarz J., and Jenkins M., “Nurses' Experiences With Change From Nurse‐Patient Ratios to Workload Intensity Staffing,” Nursing Management 54, no. 2 (2023): 24–31, 10.1097/01.numa.0000918216.61768.f4. [DOI] [PubMed] [Google Scholar]
- 4. Foster‐Smith R. R., Mitchell N., and Bobo M. L., “Diversity, Equity, Inclusion, and Belonging,” Nursing Management 54, no. 5 (2023): 40–47, 10.1097/nmg.0000000000000015. [DOI] [PubMed] [Google Scholar]
- 5. American Nurses Association . (2019). Nurse Staffing: A Critical Healthcare Issue. Retrieved from https://www.nursingworld.org/practice-policy/nurse-staffing/.
- 6. World Health Organization ., State of the World's Nursing 2020: Investing in Education, Jobs and Leadership (Geneva: World Health Organization, 2020). Retrieved from https://www.who.int/publications/i/item/9789240003279. [Google Scholar]
- 7. Havaei F., Dahinten V. S., and MacPhee M., “Effect of Nursing Care Delivery Models on Registered Nurse Outcomes,” SAGE Open Nursing 5 (2019): 2377960819869088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Jost S. G., Bonnell M., Chacko S. J., and Parkinson D. L., “Integrated Primary Nursing: A Care Delivery Model for the 21st‐Century Knowledge Worker,” Nursing Administration Quarterly 34, no. 3 (2010): 208–216. [DOI] [PubMed] [Google Scholar]
- 9. Lindsay M., “A Shared Governance Approach to Nursing Documentation Redesign Using Kotter's Change Management Model,” Nursing Management 54, no. 3 (2023): 14–20, 10.1097/01.numa.0000919064.29246.6b. [DOI] [PubMed] [Google Scholar]
- 10. Kirkland‐Kyhn H., Howell M., Senestraro J., and Walsh S., “Leveraging Technology to Improve Wound Care Delivery and Care Transitions,” Nursing Management 52, no. 11 (2021): 24–28, 10.1097/01.numa.0000795592.38063.7c. [DOI] [PubMed] [Google Scholar]
- 11. Paradiso L., “Rebuilding Trust in Just Culture,” Nursing Management 53, no. 11 (2022): 6–14, 10.1097/01.numa.0000891456.44611.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Dorritie R., Fiore‐Lopez N., and Sonenberg A., “Leading for Change,” Nursing Management 51, no. 3 (2020): 14–20, 10.1097/01.numa.0000654844.72394.8f. [DOI] [PubMed] [Google Scholar]
- 13. Needleman J., Liu J., Shang J., Larson E. L., and Stone P. W., “Association of Registered Nurse and Nursing Support Staffing With Inpatient Hospital Mortality,” BMJ Quality & Safety 29, no. 1 (2020): 10–18, 10.1136/bmjqs-2018-009219. [DOI] [PubMed] [Google Scholar]
- 14. Yakusheva O., Bang J. T., Hughes R. G., Bobay K. L., Costa L., and Weiss M. E., “Nonlinear Association of Nurse Staffing and Readmissions Uncovered in Machine Learning Analysis,” Health Services Research 57, no. 2 (2021): 311–321, 10.1111/1475-6773.13695. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Abusamra A., Rayan A. H., Obeidat R. F., Hamaideh S. H., Baqeas M. H., and ALBashtawy M., “The Relationship Between Nursing Care Delivery Models, Emotional Exhaustion, and Quality of Nursing Care Among Jordanian Registered Nurses,” SAGE Open Nursing 8 (2022): 23779608221124292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Lewis K. A., “Will Nursing Go Back to the Way It Was Before COVID‐19?,” Journal of Advanced Nursing 78, no. 10 (2022): e130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Prentice D., Moore J., and Desai Y., “Nursing Care Delivery Models and Outcomes: A Literature Review,” Nursing Forum 56, no. 4 (2021, October): 971–979. [DOI] [PubMed] [Google Scholar]
- 18. Geltmeyer K., Neyrinck D., Benoit D., Malfait S., Goedertier H., and Duprez V., “Implementing Mixed Nursing Care Teams in Intensive Care Units During COVID‐19: A Rapid Qualitative Descriptive Study,” Journal of Advanced Nursing 78, no. 10 (2022): 3345–3357. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Leong F. F., He H. G., Premarani K., and Lim S. H., “Nurses' Perceptions of Nursing Education Transformation and Its Impact on Care Delivery in Singapore,” International Nursing Review 69, no. 3 (2022): 340–349. [DOI] [PubMed] [Google Scholar]
- 20. Nagel D. A., Pomerleau S. G., and Penner J. L., “Knowing, Caring, and Telehealth Technology: “Going the Distance” in Nursing Practice,” Journal of Holistic Nursing 31, no. 2 (2013): 104–112. [DOI] [PubMed] [Google Scholar]
- 21. Sendak M. P., Ratliff W., Sarro D., et al., “Real‐World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study,” JMIR Medical Informatics 8, no. 7 (2020): e15182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Chen W. Y., “On the Relationships Among Nurse Staffing, Inpatient Care Quality, and Hospital Competition Under the Global Budget Payment Scheme of Taiwan's National Health Insurance System: Mixed Frequency Var Analyses,” Systems 10, no. 5 (2022): 187, 10.3390/systems10050187. [DOI] [Google Scholar]
- 23. Kim Y. and Kim S. Y., “Influence of Hospital Nurse Staffing Levels on the Colorectal Cancer Evaluation Grades, Mortality, and Length of Stay,” Korean Journal of Adult Nursing 31, no. 5 (2019): 479, 10.7475/kjan.2019.31.5.479. [DOI] [Google Scholar]
- 24. Needleman J., Buerhaus P., Pankratz V. S., Leibson C. L., Stevens S. R., and Harris M., “Nurse Staffing and Inpatient Hospital Mortality,” New England Journal of Medicine 364, no. 11 (2011): 1037–1045. [DOI] [PubMed] [Google Scholar]
- 25. Duffield C. M., Roche M. A., Blay N., and Stasa H., “Nursing Unit Managers, Staff Retention and the Work Environment,” Journal of Clinical Nursing 20, no. 1–2 (2011): 23–33. [DOI] [PubMed] [Google Scholar]
- 26. Kalisch B. J., Friese C. R., Choi S. H., and Rochman M., “Hospital Nurse Staffing: Choice of Measure Matters,” Medical Care 49, no. 8 (2011): 775–779. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Yu D., Ma Y., Sun Q., Lu G., and Xu P., “A Nursing Care Classification System for Assessing Workload and Determining Optimal Nurse Staffing in a Teaching Hospital in China: A Pre‐Post Intervention Study,” International Journal of Nursing Practice 21, no. 4 (2015): 339–349. [DOI] [PubMed] [Google Scholar]
- 28. Satoh M., Fujimura A., and Sato N., “Competency of Academic Nurse Educators,” Sage Open Nursing 6 (2020): 237796082096938, 10.1177/2377960820969389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Hökkä M., Martins Pereira S., Pölkki T., Kyngäs H., and Hernández‐Marrero P., “Nursing Competencies Across Different Levels of Palliative Care Provision: A Systematic Integrative Review With Thematic Synthesis,” Palliative Medicine 34, no. 7 (2020): 851–870, 10.1177/0269216320918798. [DOI] [PubMed] [Google Scholar]
- 30. Jobst S., Lindwedel U., Marx H., et al., “Competencies and Needs of Nurse Educators and Clinical Mentors for Teaching in the Digital Age – A Multi‐Institutional, Cross‐Sectional Study,” BMC Nursing 21, no. 1 (2022): 240, 10.1186/s12912-022-01018-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Terhi L., Pakarinen A., Salminen L., Virtanen H., and Haapa T., “Instruments Assessing Nurse Educator's Competence: A Scoping Review,” Nursing Open 10, no. 4 (2022): 1985–2002, 10.1002/nop2.1479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Park Y. M. and Hwang W. J., “Development and Effect of a Simulation‐Based Disaster Nursing Education Program for Nursing Students Using Standardized Patients,” Journal of Nursing Research 32, no. 1 (2024): e314, 10.1097/jnr.0000000000000596. [DOI] [PubMed] [Google Scholar]
- 33. Aliakbari F., Aein F., Bahrami M., and Arbon P., “Development, Psychometric Testing, and Use of a Disaster Nursing Competency Scale in Iran: A Mixed Methods Study,” Disaster medicine and public health preparedness 16, no. 5 (2021): 2108–2113, 10.1017/dmp.2021.197. [DOI] [PubMed] [Google Scholar]
- 34. Driscoll A., Grant M. J., Carroll D., et al., “The Effect of Nurse‐To‐Patient Ratios on Nurse‐Sensitive Patient Outcomes in Acute Specialist Units: A Systematic Review and Meta‐Analysis,” European journal of cardiovascular nursing 17, no. 1 (2018): 6–22. [DOI] [PubMed] [Google Scholar]
- 35. Aslan M., Karaaslan A., Yıldız A., Doğan F., and Evirgen H., “Workload of Nurses and Care Left Undone: Do We Really Care Enough?,” International Journal of Caring Sciences 9 (2016): 2. [Google Scholar]
- 36. Frisch S. O., Li H., Faramand Z., et al., “Abstract 14879: Using Predictive Machine Learning Modeling for the Nursing Triage of Acute Chest Pain at the Emergency Department,” Circulation 140 (2019): A14879, 10.1161/circ.140.suppl_1.14879. [DOI] [Google Scholar]
- 37. Sullivan S. S., Hewner S., Chandola V., and Westra B. L., “Mortality Risk in Homebound Older Adults Predicted From Routinely Collected Nursing Data,” Nursing Research 68, no. 2 (2019): 156–166, 10.1097/NNR.0000000000000328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Zlotnik A., Gallardo‐Antolín A., Alfaro M. C., Pérez M. C. P., and Martínez J. M. M., “Emergency Department Visit Forecasting and Dynamic Nursing Staff Allocation Using Machine Learning Techniques With Readily Available Open‐Source Software,” CIN: Computers, Informatics, Nursing 33, no. 8 (2015): 368–377, 10.1097/CIN.0000000000000173. [DOI] [PubMed] [Google Scholar]
- 39. Fiebig M., Hunstein D., and Bartholomeyczik S., “Prediction of Nursing Workload in Hospital,” Studies in Health Technology and Informatics 250 (2018): 266–267, http://europepmc.org/abstract/MED/29857459. [PubMed] [Google Scholar]
- 40. Huddar V., Rajan V., Bhattacharya S., and Roy S. (2014). Predicting Postoperative Acute Respiratory Failure in Critical Care Using Nursing Notes and Physiological Signals. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 2702‐2705). IEEE. [DOI] [PubMed]
- 41. Ganggayah M. D., Taib N. A., Har Y. C., Lio P., and Dhillon S. K., “Predicting Factors for Survival of Breast Cancer Patients Using Machine Learning Techniques,” BMC Medical Informatics and Decision Making 19, no. 1 (2019): 48, 10.1186/s12911-019-0801-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting information.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request. The authors affirm that the methods used in the data analyses are suitably applied to their data within their study design and context, and the statistical findings have been implemented and interpreted correctly.