Development and evaluation of a data-driven integrated management app for perioperative adverse events: protocol for a mixed-design study

Abstract

Introduction

A patient record review study conducted in 2006 in a random sample of 21 Dutch hospitals found that 51%–77% of adverse events are related to perioperative care, while Centers for Disease Control and Prevention data in USA in 2013 estimated that the medical error is the third-leading cause of mortality. To capitalise on the potential of apps to enhance perioperative medical quality, there is a need for interventions developed in consultation with real-world users designed to support integrated management for perioperative adverse events (PAEs). This study aims: (1) to access the knowledge, attitude and practices for PAEs among physicians, nurses and administrators, and to identify the needs of healthcare providers for a mobile-based PAEs tool; (2) to develop a data-driven app for integrated PAE management that meets those needs and (3) to test the usability, clinical efficacy and cost-effectiveness of the developed app.

Methods and analysis

We will adopt an embedded mixed-methods research technique; qualitative data will be used to assess user needs and app adoption, while quantitative data will provide crucial insights to establish the demand for the app, and measure the app effects. Phase 1 will enrol surgery-related healthcare providers from the West China Hospital and identify their latent demand for mobile-based PAEs management using a self-designed questionnaire underpinned by the knowledge, attitude and practice model, as well as expert interviews. In phase 2, we will develop the app for integrated PAE management and test its effectiveness and sustainability. In phase 3, the effects on the total number and severity of reported PAEs will be evaluated using Poisson regression with interrupted time-series analysis over a 2-year period, while users’ engagement, adherence, process evaluation and cost-effectiveness will be evaluated using quarterly surveys and interviews.

Ethics and dissemination

The West China Hospital of Sichuan University’s Institutional Review Board authorised this study after approving the study protocol, permission forms and questionnaires (number: 2022-1364). Participants will be provided with study information, and informed written consent will be obtained. Study findings will be disseminated through peer-reviewed publications and conference presentations.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The purpose of this prospective study is to verify that the newly developed integrated management tool for perioperative adverse events (PAEs) is successful and cost-efficient in real-world clinical practice.

• This protocol will employ a combination of qualitative and quantitative methodologies, including demand surveys, app development and pilot evaluation to provide insights for future perioperative information technology research.

• Electronic record data and self-reported data over a 2-year follow-up period will be comprehensively reviewed for outcomes.

• It will not be possible to infer any causality between the mobile-based PAE management tool with clinical outcomes because this is an observational study and associations with patient safety improvement may be confounded by other interventions, even using interrupted time series model.

• The findings from a single site, the second-largest hospital in China, may not be generalisable to other hospitals.

Introduction

Perioperative adverse events have high incidence and mortality in hospital settings

The use of medical services may result in adverse events (AEs), which are unplanned and undesirable occurrences that can have a negative impact on patient safety and quality of care in hospitals.^{1 2} Various consequences for patients and organisations can result from AEs. In terms of patient outcomes, as the WHO estimated in 2019, approximately 134 million AEs due to unsafe care occur per year in low-income and middle-income countries, contributing to approximately 2.6 million in-hospital deaths annually.³ A 2007 systemic review of available AE data in the literature suggested that although mild or temporary impairment is the most common outcome of AEs, a sizeable proportion (median 14%) also cause permanent disability (7%) or death (7%).^{4 5} In terms of organisational consequences, AEs are linked to high direct medical expenses as a result of extended hospital stays or readmissions, negative media coverage and legal action.

The incidence rate of AEs may be twofold and fourfold higher among surgical and critical patients, respectively.⁶ For surgical patients, doctors, nurses, technicians and other healthcare workers from multiple departments are involved in the perioperative settings. This may lead to an increase in perioperative AEs (PAEs), which refer to medical errors related to perioperative care. The perioperative period is used to describe the three phases of any surgery, comprising the preoperative phase, the intraoperative phase and the postoperative phase. In practical terms, this usually lasts from the time the patient arrives at hospital or doctor’s office for surgery until the time the patient goes home.⁷ Perioperative care is responsible for 51%–77% of all in-hospital AEs,⁸ and 1 in 6 patients undergoing elective surgery have at least 1 PAE.⁹ Over 310 million surgeries are conducted worldwide every year,¹⁰ and a recent study by the Lancet Commission on Global Surgical Burden reported that at least 4.2 million people worldwide die within 30 days of surgery each year, accounting for 7.7% of all deaths globally, and making surgery the third greatest contributor to deaths after ischaemic heart disease and stroke.¹¹ Moreover, the growing annual surgical volume, increasingly complicated techniques and innovations, and the increasing complexity of comorbid conditions in patients having surgery¹² have all contributed to a rise in PAEs.^{13 14} Further research is needed to provide insight into the scale, nature, causes and preventability of PAEs, as well as to identify feasible prevention strategies to reduce incidence.

Current barriers to AE management

Perioperative care priorities include preventive treatment because 30%–70% of clinical AEs are preventable.¹⁵ Examining AEs that have already happened, and attempting to trace back the issue to pinpoint a root cause, is the most common strategy now used. Despite the dedication of significantly increased financial and human resources at the local, state and national levels to decrease mistakes and harm to patients, studies indicate that safety has not improved and AEs continue to occur.¹⁶ Many obstacles and constraints exist in various sectors of endeavour that make the adoption of voluntary AE reporting systems unsatisfactory; only approximately 1%–10% AEs are reported.^{17 18} Fear of blame, sanctions or dishonour; threats to peer relationships; lawsuits;¹⁹ lack of AE reporting knowledge; lack of support or expectation from colleagues; and reporting system issues such as tedious processes, lack of confidentiality and anonymity for reporting, and lack of efficient and timely feedback have been identified as the most significant barriers to voluntary reporting of AEs.²⁰ Traditional AE management systems may bias clinicians towards reporting only specific types of events (eg, sentinel or predefined event categories), missing numerous instances of suboptimal care that could facilitate understanding deficiencies in healthcare system performance and opportunities to improve patient safety.²¹ A gap in integrated management for AEs, which includes detection, evaluation, disclosure, action, communication and follow-up, would result from these aforementioned barriers. A prospectively validated management instrument is needed to significantly reduce the occurrence of PAEs²² because they may lead to severe adverse outcomes or death.^23–25

Information-based AEs management

Recent studies have investigated the potential advantages of using mobile technologies in healthcare delivery and have found them to be promising.²⁶ Various forms of healthcare information technology have been employed in the detection, management and follow-up of AEs.^{27 28} Text mining has been used to detect AEs based on data-driven methods, including spontaneous warning and reporting systems.²⁹ Commonly, AEs are characterised as a text classification problem, where a piece of text, either an entire document or its part, is mapped to one or more predefined classes that correspond to a type of AE or its property. In addition, the availability of large-scale patient databases, advances in computing power and the emergence of machine learning algorithms provides us with the capability to integrate diverse data sources and analyse the complex inter-relationships between risk factors and outcomes before the point of care.³⁰ For instance, regarding adverse drug events (ADEs), artificial intelligence (AI) algorithms and proposed tools based on patient medication data have the potential to inform clinical decision-making in real time to reduce the frequency, duration and severity of ADEs.³¹ Before drugs are prescribed, AI could provide fast and reliable predictions of which patients are likely to experience ADEs. Nevertheless, currently, data-driven AE management focuses mainly on ADEs rather than PAEs, and research-led interventions and evaluations are rarely publicly available, especially for the Chinese population. Due to the complexity and fast evolution of medical treatment during the perioperative period, it is difficult to reliably establish PAEs from patient data. In addition, the actual efficiency of data-driven AEs management tools in clinical contexts remains poorly understood.

Methods

Research aim and objectives

The overall goal of this mixed-methods study is to develop and test an app that incorporates evidence-informed management strategies and provides monitoring, warning, reporting and handling follow-up for PAEs. The app will focus on lowering the incidence of PAEs via two approaches: preventing PAEs through prompt intervention through risk warning; and avoiding concealment of PAEs through automatic and collaborative reporting of PAEs to uncover hidden medical flaws, enhance medical quality and fundamentally prevent the occurrence of PAEs. Moreover, it will provide an online platform for comprehensive PAE treatment and response, as well as maintenance PAE follow-up. The app is different from other assessable AE apps in several ways: first, features in the app are designed to support integrated management for PAEs, including detection, computerised and manually combined reporting, comprehensive processing, continuing follow-up; second, it is specific to PAEs; third, it is based on real-time perioperative patient data that support the app’s function.

The study is divided into three phases based on the three overall objectives of this pilot study: phase 1 will investigate the latent demand for app-based PAEs management among surgery-related medical professionals, nurses, and administrators using a knowledge, attitude and practices (KAP) survey; phase 2 will design and develop a data-driven app for integrated PAEs management; phase 3 will evaluate the usability and clinical efficacy of the developed app. An overview of these three phases is presented in figure 1. Findings from phase 1 will inform phase 2, and findings from phase 2 will provide the tool for phase 3; evaluation during phase 3 will lead to further refinement prior to phase 2. This study’s design incorporates a mixed-methods research methodology.

Figure 1.

The three-phase development and assessment process for the PAEs management app. Phase 1 aims to define users’ unmet needs and outline the prototypes; phase 2 aims to construct and refine the app and phase 3 aims to evaluate the app’s feasibility. KAP, knowledge, attitude and practices; PAEs, perioperative adverse events.

To determine the percentage of participants who are willing to use the app, user requirements for the app, and the efficacy of the result, quantitative approaches such as the KAP survey and interrupted time series (ITS) analysis will be used. We will employ qualitative techniques such as semistructured interviews and group discussions to improve the overall design and to better comprehend the quantitative data.

Study setting

The study will be performed in the Department of Anesthesiology of West China Hospital (WCH), which performs 500 surgeries per day and is ranked second among all hospitals in China according to the China’s Best Hospital Ranking released by Fudan University.^{32 33} Currently, a web-based retrospective AE reporting system is used for AE management. The Department of Anesthesiology is in charge of organising the recruitment of survey participants and the mobile app pilot, which will both take place at WCH and its affiliated hospitals.

Study design

Phase 1: use-centred design thinking

The significance of the user experience is constantly emphasised throughout the design phase because if a product does not appeal to people, it will not be used. In order to create a context design for this phase, we will collect a variety of data from doctors, nurses and administrators. The stages that follow will be used to better understand the requirements of potential users.³⁴

KAP survey

Programmes may be tailored to the requirements of the client by accounting for their levels of knowledge, personal attitudes and behaviours.³⁵ Box 1 describes the key elements for this KAP survey, while the related questionnaire based on the literature is supplied in online supplemental material table S1. In this phase, we will recruit all full-time healthcare professionals and administrators at WCH who provide surgical-related services (including anaesthetists, surgeons, operating room nurses and anaesthesiology nurses) and obtain their consent to participate. Because the app should be used by every WCH employee, no exclusion criteria are established in order to gather as much data as possible. The study team will use email invitations to undertake the recruiting on a voluntary basis.

Box 1. Key questions for knowledge, attitude and practices survey.

Key points regarding knowledge, attitudes and practices for perioperative adverse events (PAEs) among healthcare providers:

• Basic information, including age, sex, educational attainment, department and type of job.

• Participants’ level of knowledge regarding AEs and PAEs.

• Attitudes towards information technology-based PAE management (general perceptions, perceived benefits and concerns), as well as individual information technology proficiency.

• Current AEs and PAEs practice experience and attitudes.

• Willingness to use a mobile app for PAEs, as well as perceived barriers and facilitators.

Focus discussion

The research team will then have 4–6 discussions based on the results of the KAP survey before determining the app’s design and content. The members will analyse the questionnaire and form preliminary content for the app. This will offer insights into the present hurdles throughout the response to AEs and also provide an inventory of latent user needs. For instance, the app will have a latent requirement to automatically detect AEs and send reminders to the healthcare practitioners if the survey found that being ignorant of PAEs is one of the major reasons for missing PAEs. The second step is to transform the explicit and latent demands into prototype specifications, which are then connected to metrics or indicators via expert discussion.

Afterward, 5–8 representatives of physicians, nurses and PAEs administrators will be chosen to participate in the discussion about their lived experience of PAEs to further identify the latent needs of the topic. Throughout the study, focus group discussions will be conducted in a private environment and facilitated by a research team member (PL). Based on the findings of the KAP survey, the focus discussion will be used as a complement to the survey to observe all facets of the users’ objectives, needs and preferences without depending on extrapolations from quantitative measurements. Topics that must be covered include: (1) experience of engaging with the PAEs, (2) perceptions of the app (eg, attitudes towards specific features and content) and (3) prediction of potential barriers to the app’s use and suggestions for facilitating. All focus discussions will be recorded and transcribed for later analysis.

Finally, the research team will gather these prototypes for each iteration of the app, with potential indicators including the following: real-time monitoring of PAEs, automated warning and sending reminder messages to healthcare providers for reactions, automated reporting, root-cause analysis for the PAEs that occur, patient follow-up and data recording, information sharing. The development team (the research team and the technology team) will review those prototypes based on the stated requirements and use design thinking methodologies in an iterative manner, which entails redesigning on the basis of subsequent user testing. Sketch modelling (in two dimensions or three dimensions) will be used during design thinking sessions (2 or 3) to develop a model that represents how the participants imagine the app. The committee will next finalise the prototype’s requirements, which will subsequently be included in the app. Notably, the user experience will also be a priority for these versions. The technical team will proceed to phase 2 and implement the prototypes.

Phase 2: app development

In this stage, a data-driven interactive wireframe of the app will be developed based on the particular prototypes from phase 1. App development is overseen by a seven-person advisory council with expertise in PAE reporting, process and management, as well as user interface design, interaction, visual design and patient-engaged research. On a monthly basis, the council will convene to discuss the application’s development. The alpha version of the app will be compatible with the Android and iOS smartphone platforms, as well as the embedded computer in the operation room. It will be developed using the open-source Maslo platform, which integrates many technologies. The Maslo platform employs React Native, Therr.js, and Javascript or Typescript for its frontend. The EMR (electronic medical record, preoperative and postoperative), (anaesthesiology surgery system, intraoperative), and bedside monitor devices (postoperative) preserve all perioperative patient data. Connecting this app’s backend server to the previously described system enables real-time data exchange.

All data entered via the app will be saved with an anonymised identifier inside the app and on a secure server for data management. The backend will implement unified identity authentication and authority management for the doctor side (anaesthesiologists, surgeons and psychologist), the nurse side (anaesthesia nurses, operating room nurses and ward nurses), and the management side (department AE management and hospital AE management). As envisioned, there will be six key modules added to the app’s primary tabs: real-time monitoring and risk warning, reminder notification, automatic reporting, patient follow-up, psychological assistance and information sharing. These will be used in the following sequence for typical scenarios:

1. After the surgery is scheduled, all essential information, such as the patient’s diseases, medication history, and examination findings is promptly incorporated into the app system through the EMR, and perioperative monitoring information is generated. The associated physicians’ contacts are included. Each user is available to review the baseline information.

2. Once a PAE warning rule is triggered, all associated stakeholders of this patient will receive a phone or SMS message and have the possibility to provide timely intervention.

3. Once a PAE reporting rule is triggered, all associated stakeholders of this patient will receive a phone or SMS message and begin processing. Specifically, surgeons, anaesthesiologists and nurses will be required to complete a personal description of the PAE and a Second Victim Experience and Support Tool (SVEST) scale within 24 hours, while administrators will begin assessing the severity of AEs, organising multidisciplinary patient care, and monitoring discharged patients to prevent sequelae. After identifying the multifaceted causes of the PAE, key stakeholders will receive an analysis report with a summary of improvement efforts, which they must electronically sign to indicate accomplishment of those improvements. An individual who notices a PAE could log into the system, click ‘report’, and other healthcare providers involved in this surgery will receive the notification as well.

4. Psychiatrist users will receive a call or SMS notification to provide psychiatric assistance depending on the results of the SVEST evaluation.

5. Once a PAE occurs, the app will activate long-term patient follow-up; the data from this follow-up will be stored on the backend server.

6. Monthly anonymised reports on PAE incidence data, root analysis, prevention and improvement strategies will be given through Enterprise WeChat to each employee for self-reflection and self-correction. Figure 2 depicts the architecture of the app.

Figure 2.

A conceivable use scenario of the app component. Participants will be asked to think thoroughly regarding the appearance and functionality of each component. PAEs, perioperative adverse events.

Phase 3: pilot evaluation

The application will be accessible in Chinese and requires little training to operate. After the 4-week test and adjustment period, all employees in WCH will be asked to download the app with its instruction manual; all computers in the operation room will have the app installed. The evaluation study is proposed as an ITS analysis, regarded as a powerful quasi-experimental design for determining the efficacy of an intervention by collecting and comparing outcome measures before, during and after the intervention steps.³⁶ ITS analysis is intended to record the level and trend changes in several outcomes over time.

During the specific assessment phase, mixed approaches will be used and classified into two sections: Using data retrieved from the back-end server, section 1 will concentrate on the quantitative study of the app’s effect on increasing the reporting of PAEs, and its subsequent impact on reducing the incidence of PAEs, using the ITS model. Moreover, all healthcare practitioners in WCH will be required to answer web-based surveys through WeChat at four time points: 3 months, 6 months, 9 months and at the end of the trial. Section 2 will aim to evaluate the app’s acceptability via quarterly meetings, as well as the user’s experience with the intervention, focusing on success factors, problems and reasons for drop-outs, and determine how the use of technology can influence motivation for PAE integrated care management. Each interview will be audiotaped, verbatim transcribed and anonymised. Table 1 provides a summary of the evaluation’s outcomes, processes and tools. Described here are the data-gathering methods for each component of the process assessment.

Table 1.

Summary of outcomes data to be collected and evaluated

Data type	Outcomes	Scale or sources	Delivery method (frequency)
Primary outcome (app impact)	Total no of weekly reported PAEs Weekly incidence proportion of PAEs (calculated by dividing the reported total no of PAEs by the total no of surgeries that underwent general anaesthesia) Severity of reported PAEs	In-app record of each event and saved in backend server	In-app (real time) Analysed weekly
App adherence and engagement	Adherence (no of log-ins per month) App use (no and frequency of pages accessed; time spent on the app per session and overall; time spent on specific pages; no and length of time of unique sessions; length of time between unique sessions) Engagement (focused attention, perceived usability, aesthetic appeal and reward)	In-app record of each event and saved in backend server	In-app (real time) Analysed monthly
		UES Short Form Scale	Sojump (monthly)
Process evaluation	Subject app engagement and process evaluation, experiences	Focus discussion	Telephone or Zoom (monthly)
Cost-effectiveness	Cost-effectiveness evaluation	Healthcare cost	Real-time data

PAE, perioperative adverse event.

Evaluation 1: app impact

The weekly number of reported PAEs, weekly incidence proportion of PAEs (calculated by dividing the reported total weekly number of PAEs by the total weekly number of surgeries using general anaesthesia), and the weekly incidence of postoperative complications will be used to evaluate the app’s influence on enhancing perioperative medical quality. These indications will be extracted weekly from the backend of the app, and ITS modelling with random intercepts and slopes will be used to track the immediate change and weekly trend slope change in these measures across the 1-year intervention, and compared with the 1 year of the preintervention period. In addition, the severity of each PAE will be categorised according to the rule and compared before and after the application intervention (for PAE category, PAE severity classification and postoperative complications definitions, see online supplemental tables S2,S3 and S4).

Evaluation 2: app adherence and engagement

Another purpose of the evaluation during the 1-year study period is to assess the level of adherence and use of the application. For the purposes of this research, adherence, which is based on usage behaviours (eg, frequency and duration of app use), will be defined as the percentage of individuals who have begun using the app and will continue to do so.³⁷ The number of total uses, the usage frequency (frequently used, sometimes used and never used), and the amount of time spent on the app during each use will be evaluated periodically. The participant usage profiles will then be used to categorise people into use clusters, such as often used, occasionally used and never used, in order to investigate use patterns. Additional utilisation data, including the number and duration of follow-ups for each patient with PAEs, will also be collected.

User engagement has been identified as a crucial aspect in determining the success of an app,^{38 39} because it is correlated with the user’s desire to continue using an app.⁴⁰ Therefore, researchers must move beyond user continuance behaviour and examine user engagement as a broader concept.⁴¹ The User Engagement Scale (UES) form will be our quantitative scale for measuring subjectively perceived engagement. As box 2 shows, the UES Short Form (UES-SF) consists of 12 questions divided into 4 domains: focused attention (FA, feeling absorbed in interaction with the system and losing track of time), perceived usability (negative affect experienced because of effort expended to use the system), aesthetic appeal (AE, visual appeal of the interface) and reward (RW, perceived benefits and interest experienced because of using the app).⁴² Researchers in academia and industry have found the UES to be a reliable, valid and sensitive instrument for measuring engagement with a variety of technologies, including mHealth applications.⁴³ Both subscale and overall engagement scores may be calculated as the average of the included items (range 1–5), with higher scores indicating higher levels of app engagement. The form will be delivered to WCH employees as a Sojump survey every 3 months and at study completion. On a Likert scale ranging from 1 to 5, respondents will be asked to rate their degree of satisfaction and future use intent. The measure produces a total summed score, which is translated to a standard t score for analysis. Based on the nature of the behaviour management mechanisms, we anticipate a rise in the frequency and intensity of active participation over time.

Box 2. Questionnaire items for the User Engagement Scale short form.

FA-S.1 I lost myself in this experience.

FA-S.2 The time I spent using Application X just slipped away.

FA-S.3 I was absorbed in this experience.

PU-S.1 I felt frustrated while using this Application X.

PU-S.2 I found this Application X confusing to use.

PU-S.3 Using this Application X was taxing.

AE-S.1 This Application X was attractive.

AE-S.2 ThisApplication X was aesthetically appealing.

AE-S.3 This Application X appealed to my senses.

RW-S.1 Using Application X was worthwhile.

RW-S.2 My experience was rewarding.

RW-S.3 I felt interested in this experience.

Similarly, in-depth qualitative interviews (approximately 1 hour) will be conducted with a subsample (roughly n=30) of participants every 3 months and at the completion of the trial. Potential interviewees will be invited primarily based on their engagement patterns (assessed quantitatively). Specifically, we will seek to identify significant differences in adherence and engagement by sampling people according to their usage cluster (eg, regularly used, intermittently used and never used). To assure the subsample’s representativeness in terms of gender, age and departmental background, a systematic sampling method will be used.⁴⁴

Evaluation 3: process evaluation

We will concurrently conduct a process evaluation to identify and document user obstacles and programme implementation facilitators. All WCH participants will be invited to participate in the assessment of the process. At the end of the research, semistructured interviews with open-ended questions will be administered to volunteer surgeons, nurses, anaesthetists and administrators in order to understand participant experiences, satisfaction and acceptance of the app. They will be encouraged to discuss the app’s advantages, disadvantages, potential obstacles and suggestions. With the participant’s agreement, interviews will be recorded to enable accurate analysis of replies. The audiotapes will be transcribed and deidentified; following transcription, the tapes will be destroyed. In addition, technicians who design and maintain the system will be interviewed in order to discover ways to improve the app’s content delivery. This process assessment will contribute to the analysis of the PAE technology intervention by identifying several layers of obstacles and facilitators throughout the research.

Evaluation 4: cost-effectiveness

Lastly, an exploratory cost-effectiveness evaluation will be conducted from a healthcare payer’s perspective to estimate the cost of the development and maintenance of the app, with the hypothesis that patients managed by the integrated PAE management app will experience fewer postoperative complications, and hospitals will incur lower costs, compared with the previous management system.⁴⁵ The majority of anticipated expenditures will be allocated to app development and maintenance, data servers and administration. These estimates will be summarised using an incremental cost-effectiveness ratio and an incremental net benefit statistic.⁴⁶ The uncertainty of the estimates will be characterised using cost-effectiveness acceptability curves and 95% CIs.

Patient and public involvement

Healthcare providers in WCH will actively participate in the research phases, including the assessment of latent requirements and evaluation of the app’s practicality. The methodology of the research will not include patients and/or public enrolment.

Statistical analysis

Sample size

We benchmarked a prior study of a survey of concept mobile app for AE reporting after influenza vaccination, which revealed an acceptance rate of 86% for physicians.⁴⁷ Taking into account a 20% attrition rate, 90% power and an effect size of 0.5 SD, the projected sample size for achieving goal 1 is 185 participants. We aimed to survey more than 500 physicians and 1000 nurses, which should be a large enough sample to estimate parameters such as the SD, which in turn can be used to estimate the sample size for an appropriately powered full-scale randomised clinical trial (RCT).^{48 49} As a rule of thumb for an ITS analysis, 10 measurement points before and 10 measurement points after an intervention provide 80% power to detect a change in level of 5 SDs (of the predata) only if the autocorrelation is greater than 0.4.⁵⁰ The time points before and after the app intervention in this protocol research will each be 12 months. All surgeries performed in WCH will be collected to observe if PAEs occur, and the primary outcomes resulting from the app (tabulated in table 1) will be collected in each observation point. We aim to collect data on at least 300 000 operations (based on 500 surgeries daily at WCH) between 2023 and 2024 before and after the mobile app intervention, with 52 weekly measurement points before and 52 measurement points after the app intervention.

Regarding the sample size for the qualitative interviews, there are no definitive guidelines about the exact number of participants to include; instead, the sample size is determined by a variety of characteristics linked to the study objectives, pragmatic considerations and the researcher’s own expertise.⁵¹ Given our expansive objectives (objectives 1–3) and our earlier experience using qualitative techniques to investigate the use of digital health technologies, we believe that a sample size of roughly 30 participants for the interviews will be adequate to allow significant theme analysis.

Plans for statistical analysis

Two types of analyses will be used to explore the outcome trajectories of the mixed study: (1) quantitative analyses of the longitudinal impact and engagement data (see Evaluation 1-4) and (2) qualitative analyses of the interview data.

Regarding the KAP survey and process assessment, all quantitative data, including selections, logs and data, will be input twice into an Excel (Microsoft, Redmond, USA) data set and Python (Python Software Foundation, New York City, USA) will be used for analysis. The majority of the statistical analyses will be descriptive, with the goal of estimating feasibility parameters and informing power calculations for a future definitive trial. Categorical baseline characteristics will be reported as numbers and percentages, normally distributed continuous variables will be presented as means with SD, and non-normally distributed variables will be presented as medians with IQRs. For obtained survey data, the t-test or analysis of variance test will be used to assess mean differences. To investigate determinants of knowledge and attitude score, a multivariate linear regression model will be constructed. For the UES form, Pearson’s correlation coefficients will be calculated between long-term involvement and the score for each domain individually, as well as the total score. Additionally, we will use Student’s t-tests to examine the mean UES-SF score and related variables. P values less than 0.05 will be considered statistically significant.

As the majority of main outcomes will be regularly and consistently evaluated, we will estimate the trend difference using a segmented Poisson regression with the ITS model and test for changes in both the level and trend of the app’s effect. The propensity score match will be used to weight the severity of the surgical patient’s condition.⁵² Time, app intervention, the interaction between time and app intervention, and a random intercept will be included in the model. The differences between unadjusted and adjusted rate ratios will be shown with 95% CIs. Thematic analysis will be used to uncover, evaluate and summarise patterns (themes) within qualitative data. By comparing, contrasting and categorising interview material, themes will be identified. Data will be managed and analysed using NVivo V.12 (QSR International, Cambridge, USA).

Results

In May 2022, specific searches were undertaken to inform the creation of a complete search strategy for electronic database searches. The concluding search of MEDLINE, PubMed and Web of Science produced a total of 278 results. Throughout the study, relevant papers will continue to be identified. In September 2022, iterative adjustments to the scoping review procedure and the formalisation of methodologies were accomplished. The research is anticipated to commence in November 2022, with a total project length of 2 years spanning 2023 and 2024. The app is expected to be finished in October 2023, and the assessment of its efficacy will be undertaken 1 year later, by the end of 2024. Based on our goal of decreasing the occurrence of PAEs by about 40% and raising the reporting of PAEs that have happened by 70%, we aim to gather 24 months of surgical data on 300 000 surgical procedures in WCH. Open access peer-reviewed publishing is anticipated between 2023 (KAP survey) and 2025 (outcome evaluation). This article is based on protocol V.21 as of September 2022, and any modifications to the procedures described will be recorded and published. An overview of the study processes is shown in figure 3.

Figure 3.

Study time point schematic. ITS, interrupted time series; KAP, knowledge, attitude and practices; UES, User Engagement Scale.

Discussion

Value of the study

The purpose of this project is to build an integrated management tool for PAEs and assess its efficacy and cost-effectiveness in practice. The study is significant because it will address gaps in the current research on app-related PAEs. First, while past research has shown that AEs can be effectively managed through internet technology, the effects of such technology on productivity remain uncertain, particularly in China where the prevalence of PAEs has historically been under-reported. The purpose of this research is to identify the significance of mobile health in the management of PAEs in Chinese healthcare. Second, because surgeons, nurses and administrators are being recruited as participants for the creation of this app, the generalisability of the study’s results will be strengthened due to the consideration in the app development process of the associated participants’ desire for an integrated AE app. Third, this research will evaluate the efficacy of a PAE app using real-world data. Because cumulative knowledge on the effect of mobile AEs management is lacking, the findings of this research will contribute to a greater understanding of the impact and sustainability of app-based AE management tools in the workplace. This is particularly important in the contemporary context, where mHealth and eHealth have been integrated into healthcare; however, few practices regarding PAEs have been implemented owing to their complexity. In addition, we will assess the efficacy of PAE management with and without the app over a 2-year period using ITS analysis. Even though this is not a direct comparison based on an RCT, we may analyse the differences in the effects of the AEs app. Finally, we will provide an ongoing web-based questionnaire for app engagement and process evaluation assessment. In order to evaluate the sustainability and applicability of our product, economic efficiency will also be considered. This research will conclude with a guide to the architecture and framework for constructing an integrated care system for PAEs that may be used for the creation of comparable systems in other clinical disciplines.

Expected findings

We anticipate that this study will show how the interface of a technological PAE tool can be accepted and useful in the clinical setting. The use of technology as a means of improving AE management has been explored to some extent in ADE management, but few studies have investigated the use of mHealth for improving PAEs, especially for the Chinese population. The quantitative measurements are expected to find a decrement in PAE incidence, and an increment in reporting of occurred PAEs. In addition, we expect to detect a decrease in time commitment for physicians, nurses and administrators when handling PAEs, as well as favourable attitudes towards the app and willingness to use it in the future. The study will, therefore, offer insight into the inclusions of data-based mHealth tools in perioperative medical quality improvement. This study will have potentially important implications for both clinical practice and support trials. The results will be presented at conferences and in peer-reviewed journals.

Limitations

The following are this study’s limitations: As the app is dependent on the EMR, surgery and bed-monitoring system, we are unable to link it to other hospitals, which may limit its generalisability. Likewise, sample size may affect the representativeness of quantitative data. Second, as we lack an active control group, it may be difficult to prevent external bias in the study of ITS. However, given that the primary goal of this study is to evaluate the efficacy of the PAE app compared with that of conventional AE treatment, we deem the research design adequate for this purpose. Third, most evaluations are self-reported, leaving uncertainty about objectives. Lastly, although the PAE app would identify the AE using a mix of machine and human analysis, certain PAEs may nonetheless be overlooked. Despite these limitations, we feel that our work will contribute to future clinical studies and practice in this sector.

Future directions

If it is determined that the app is an effective intervention for reducing PAEs and improving reporting, we will attempt to distribute the app as an evidence-based alternative for clinical use by clinicians in more hospitals. Its enthusiastic adoption in one hospital will not necessarily lead to its adoption in other hospitals; thus, a survey and assessment study will be required for each hospital and its respective healthcare professionals. In order to undertake larger studies to further evaluate its impact, it is vital to explore barriers and enablers to its use in the longer term. Consideration will be given to applying for a software innovation grant to assist with the app’s distribution.

Conclusion

In conclusion, we have provided a unique method for developing mobile-based integrated management for PAEs. This research aims to enhance understanding of how an interactive software might improve perioperative care in the Chinese healthcare community. It is anticipated that the results of this study will reveal that this app is both practical and well received by physicians, nurses and hospital administrators. Depending on the availability of research funds, this approach may be implemented nationwide in China and contribute to the enhancement of medical quality for the population undergoing surgery.

Ethics and dissemination

The people being questioned will provide their informed permission before the questionnaire is administered. To safeguard individual privacy, all AE data will be examined anonymously only. Private information such as patients’ names and ID numbers will not be recorded during any analysis steps. The authors confirm that all procedures comply with the principles of the 2013 revision of the 1975 Declaration of Helsinki, as well as any additional amendments that may have been required, along with the ethical standards of the relevant national and institutional committees on human experimentation. The West China Hospital of Sichuan University’s Institutional Review Board authorised this experiment (ethical number: 2022-1364).

Ethics statements

Patient consent for publication

Not applicable.