An electronic patient-reported outcome symptom monitor: the Chinese experience with rapid development of a ready-to-go symptom monitor

Abstract

Background

Monitoring symptoms is crucial for the early detection of disease progression and timely intervention, which is essential for reducing severe cases and mortality rates in rapidly spreading pandemics, such as COVID-19. Therefore, during infectious disease pandemics, the rapid development of real-time symptom monitoring platforms is essential. This study aimed to explore the urgent development process of an electronic system for patient-reported outcome monitoring in emergency situations.

Methods

The development of the electronic patient-reported outcome COVID-19 symptom monitoring platform (ePRO-CoV-SM) included the following steps: (1) modifying an electronic patient-reported outcome symptom-reporting platform to assess patients with COVID-19 and validating its feasibility and sensitivity for longitudinal symptom measurement; (2) updating the system to accommodate the newly emerged severe acute respiratory syndrome coronavirus 2 BA.2.2 variant; and (3) applying it in real-world settings. Literature review, expert consultation, and subject-group discussions were used to develop symptom items. Response rate and missing item rate were used as validation indicators for ePRO-CoV-SM.

Results

The ePRO-CoV-SM (2.0) consists of a core set of symptom items, a WeChat mini program, an online project design backend, a management and communication front, and a database. During the 2020 verification, the response rate of ePRO symptom monitoring reached 89.47% and the item missing rate was 0.33%, the monitoring revealed that a considerable number of asymptomatic patients were experiencing undesirable symptoms during the isolation period. In its real-world application in 2022, the response rate was 85.93% and the item missing rate was 4.84%, the monitoring found the symptom burden was higher in the younger group (18–40 years old) than in the older group (40–67 years old), and over 30% of patients reported symptoms such as cough (36.08%), dry mouth (35.67%), sleep disorders (32.27%), appetite loss (32.17%), and sputum (30.79%) during the isolation period.

Conclusions

Electronic patient-reported outcome measurement was demonstrated to be sensitive and feasible for monitoring symptoms in patients with COVID-19. By integrating smartphone-based data collection with real-time online data transmission and secure data storage using Secure Sockets Layer encryption, an electronic platform for monitoring critical symptoms can be rapidly established in emergency situations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-024-20518-5.

Background

The COVID-19 pandemic, which has been associated with over 6 million deaths globally, has greatly affected global health management [1]. Infected individuals exhibit a range of symptoms, including fever, cough, fatigue, dyspnea, sore throat, headache, and nausea [2–4]. Effective monitoring of symptoms is crucial for the early detection of disease progression, and timely interventions can help reduce severe cases and mortality [5, 6]. However, the significant genetic diversity of viruses often leads to a sudden emergence of new strains and symptoms that are not defined by the existing guidelines, as exemplified by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). With the rapid surge in the number of patients infected with these novel strains and the resulting constraints in time and resources, particularly the shortage of healthcare professionals, innovative solutions are required for the prompt implementation of effective symptom management for this large population of patients [7–9].

Patient-reported outcome (PRO) is a measure that directly originates from the patient, with no subsequent interpretation of patient responses [10]. Therefore, an effective PRO instrument would help patients identify symptoms, determine impact on the quality of life, and alert clinicians to symptom worsening or disease progression. The inapplicability of the classic paper-and-pencil data collection method in infectious disease management has driven the development of electronic patient-reported outcome (ePRO) measurement. ePRO-based symptom management is effective and feasible in cancer patient care [11, 12]; however, further practical experience and exploration in the construction of a rapid symptom monitoring platform for infectious disease outbreaks are required. Therefore, this study aimed to report our experience in the rapid development and revision of an automatic symptom monitoring platform for urgent implementation during the COVID-19 pandemic, which can serve as a template for urgent implementation of symptom monitoring systems for infectious diseases in the future.

Methods

In response to the sudden outbreak of COVID-19 in 2020, an ePRO measurement platform for COVID-19 symptoms (ePRO-CoV-SM) was urgently developed, drawing on the framework of an existing electronic data collection platform for lung cancer surgery patients. This framework has been verified as feasible and effective in randomized controlled trials [11], and the development involved three phases: (1) modifying the reporting platform to suit the requirements of patients with confirmed COVID-19 and validating the feasibility and sensitivity of longitudinal symptom measurement; (2) updating the system to accommodate the emerging wave caused by SARS-CoV-2 BA.2.2; and (3) applying this system in real-world settings.

Development of the ePRO-CoV-SM

ePRO-CoV-SM construction

ePRO-CoV-SM was developed based on the existing ePRO symptom data collection platform for patients who underwent lung cancer surgery [11]. The ePRO symptom self-monitor, a WeChat mini program-based patient client, was connected to Redcap via an application program interface (API). At the management backend of Redcap, the project administrator defined the standard process of symptom monitoring, which encompassed symptom measurement scales, assessment schedules, and follow-up plans. Once the project design was finalized or updated, the internal intelligent engine could automatically process all settings. When a patient’s phone number was entered into the system by field coordinators, the system would automatically push electronic questionnaires to the patient in accordance with the preset monitoring schedule. Patients could easily complete their symptom self-assessment by accessing the WeChat mini program on their smartphones. Patient data were transmitted directly to the platform server using Secure Socket Layer (SSL) network encryption technology. All collected data were stored on the database server and backed up on another peer server. All data transmission or communication requests from the public network were forwarded through the implementing agency network proxy machine to safeguard the backend system and data.

Development of the electronic symptom measurement items

Patient-reported symptoms were extracted from the Protocol for COVID-19 Diagnosis and Treatment (5th revision ) issued by the National Health Commission of China [13]. Extraction was independently performed by two trained symptom researchers, and a senior symptom researcher was consulted when a disagreement arose between the two reviewers. A consensus on the final included symptom and its measurement properties (items, recall period, and response scale) was reached through discussion involving a study group, which included a senior expert on PRO symptoms and two medical staff who either treated or cared for patients with COVID-19. All symptom scale items were entered into ePRO-CoV-SM through the management backend of Redcap and presented to patients as an electronic questionnaire on the WeChat mini program of self-reporting (Fig. 1). During symptom monitoring, patients used the WeChat mini program to complete the symptom assessment and submit a report. Patients were reminded by the investigator via telephone according to the monitoring schedule. All submitted PRO data was transferred from the ePRO-CoV-SM system to the Redcap database through an integrated API.

Fig. 1.

The WeChat mini program of self-report. *WeChat mini program is in Chinese, the figure is the English translation version

Validation of ePRO-CoV-SM

The validation study was a cohort study designed to monitor symptoms of patients with COVID-19. This study was conducted between January and May 2020 in Wanzhou [14], a city located in the Chongqing municipality of China with a population of 1.74 million people. This study received approval from the Institutional Review Board of Chongqing Medical University. According to the prevention and control policy applicable at that time [13, 15], patients with COVID-19 who were diagnosed and asymptomatic were required to be isolated in a centralized isolation site.

We included individuals (1) aged ≥ 18 years, (2) diagnosed with asymptomatic SARS-CoV-2 infections between January and February 2020, and (3) receiving isolation management at centralized isolation sites in Wanzhou. We used the ePRO-CoV-SM platform to conduct daily symptom monitoring for 4 weeks in isolated patients from February 17, 2020, with subsequent symptom-monitoring follow-up in the 6th and 8th weeks. Demographic information, disease details, and symptom severity were collected through the WeChat mini program. In addition, electronic informed consent was obtained from all participants using the same WeChat mini program.

Development of ePRO-CoV-SM (2.0)

In February 2022, a new wave of SARS-CoV-2 BA.2.2 variant emerged rapidly in Shanghai, China. To address the modified symptom profile of this new strain and enhance the automation of symptom monitoring, we developed an updated version of ePRO-CoV-SM, designated as ePRO-CoV-SM (2.0).

ePRO-CoV-SM (2.0) construction

The ePRO-CoV-SM (2.0) platform utilizes the same WeChat mini program integrated with a commercially developed data management system, ePRO Vision (Beijing) Health Technology Co., Ltd., and the Real-World Data Management Platform (RWDMP). ePRO-CoV-SM (2.0) encompassed a browser-based project design backend, a browser-based patient management front, and a WeChat mini program. The project design backend serves as an online platform for project design and modification. The management front comprised a web and a smartphone application for patient management, providing a user interface for patient management, encompassing patient enrollment, withdrawal, symptom-reporting status, group assignment, and symptom-trajectory curve. The WeChat mini program served as a patient-reporting client, enabling patients to report their symptoms and personal information through their personal smartphones.

In the project design backend of ePRO-CoV-SM (2.0), medical professionals preset symptom monitoring frequencies for patients based on the disease characteristics and clinical needs. They also preset monitoring reminder messages and sent times and time limits for answering based on the on-site implementation. For example, if the patient was in a centralized quarantine site, to help doctors quickly grasp the symptoms during ward rounds, the reminder time could be set before the ward rounds. If the patient was being monitored at home, reminder messages could be sent in the morning or evening for the convenience of the patient. Moreover, the time limit for patient response was usually adjusted according to the frequency of monitoring. If it was daily monitoring during the acute phase, the response time limit was often 24 h, and if it was weekly routine monitoring during the stable phase, it could be extended to 7 days.

In addition, automatic registration and electronic reminders were added to this platform to address the practical difficulties of medical staff shortages. Patients can be directly registered by scanning the recruitment code and signing electronic informed consent through WeChat. During the monitoring period, ePRO-CoV-SM (2.0) sent an electronic reminder message to the patient at each predetermined time point. Patients can click on the reminder message from WeChat to access the mini program to complete the self-symptom assessment report. After the patient completes the symptom report, ePRO-CoV-SM (2.0) will send a message to the supervising physician, reminding them to pay attention to the patient’s condition. Upon receiving the information, the physician will review the situation on the management and communication front and make a judgment on the next step based on the patient’s condition and clinical guidelines. The design of ePRO-CoV-SM (2.0) is shown in Fig. 2.

Fig. 2.

The structure of ePRO-CoV-SM (2.0). RWDMP: Real world data management platform; ePRO: electronic patient-reported outcome

Modification of the symptom items

We revised the list of COVID-19 symptoms through a literature review. In this process, two symptom researchers independently extracted COVID-19-related symptoms from the literature. Based on the consensus, a draft symptom list was developed. The draft symptom list was evaluated by clinicians who had experience in diagnosing, treating, or caring for patients with COVID-19. The assessment was conducted using an electronic questionnaire, and the clinicians reported the frequency of these symptoms based on their practical experience as “never,” “occasionally,” “frequently,” or “almost constantly.” Finally, an expert panel discussion was conducted to finalize the list of symptoms for ePRO-CoV-SM (2.0).

Validation of ePRO-CoV-SM (2.0)

In February 2022, a surge of SARS-CoV-2 Omicron BA.2.2 infection rapidly emerged in Shanghai, China [16]. The validation of ePRO-CoV-SM (2.0) was conducted between May 2022 and June 2022 in Shanghai, using a cohort design for monitoring symptoms of asymptomatic carriers. This study received approval from the Ethics Committee of the Institute of Basic Clinical Medicine of the China Academy of Chinese Medical Sciences. Case definitions were based on the Prevention and Control Protocol for Novel Coronavirus Pneumonia (version 8) of the National Health Commission of China [17]. According to the policy in place at the time [17], asymptomatic carriers of SARS-CoV-2 were required to be isolated at a centralized isolation site. We included individuals (1) aged ≥ 18 years, (2) diagnosed as asymptomatic carriers of SARS-CoV-2, and (3) isolated in centralized isolation sites in Shanghai. We used the ePRO-CoV-SM (2.0) platform for one symptom survey during isolation and for up to four symptom follow-ups in the 1st to 4th weeks after release from isolation.

A flexible real-world application of ePRO-CoV-SM (2.0)

In November 2022, an outbreak of SARS-CoV-2 Omicron BA.5.2 occurred in Chongqing, China. Based on the policy in effect at that time [18], individuals infected with Omicron BA.5.2 were required to be quarantined in centralized isolation sites. To ensure effective monitoring of symptoms in isolation, ePRO-CoV-SM (2.0) was rapidly implemented. In order to enhance its practical application in real-world settings, an expert panel consisting of two symptom management experts and five COVID-19 clinical experts was convened. The panel objective was to select a symptom list that was both clinically meaningful and concise from the ePRO-CoV-SM (2.0) symptom item bank. This study received approval from the Ethics Committees of the Second Affiliated Hospital of Chongqing Medical University. We included individuals (1) aged ≥ 18 years, (2) diagnosed with COVID-19 with a positive reverse transcriptase polymerase chain reaction (RT-PCR) test, and (3) isolated in centralized isolation sites in Chongqing.

Statistical analysis

Asymptomatic infections refer to patients who test positive for SARS-CoV-2 nucleic acid through RT-PCR but lack characteristic clinical symptoms or signs, with no observable abnormalities in medical images, including lung computed tomography [15, 17, 19]. In ePRO-CoV-SM validation, the response rate was defined as the proportion of recruited patients who completed at least one symptom report through the WeChat mini program; the missing item rate was defined as the proportion of missing items in the total number of items reported by all patients. In the analysis of symptom burden, only the data of individuals who had completed the symptom assessment at least once were included. For patients with several assessments, we included the assessment with the highest single symptom score and recorded the highest total symptom score as the patient’s highest symptom burden. Moderate-to-severe symptom severity was defined as a symptom score ≥ 4 points on a 0–10 scale [11, 20, 21]. Participant data were presented in frequencies and percentages, while their age was presented as the mean and standard deviation (SD) obtained from their age range. Symptom items were presented as mean, SD, median, interquartile range (IQR), and percentages. T-tests are used to compare continuous variables. Statistical results were considered significant when the P-value was < 0.05. All statistical analyses were performed using JMP Software (version 13.0; SAS Institute Inc., Cary, NC, USA).

Results

PRO symptom items in ePRO-CoV-SM

A total of 21 symptoms were extracted from the protocol for COVID-19 diagnosis and treatment. The severity of these symptoms was measured on a numerical scale of 0–10 points (0 = not present, 10 = as bad as you can imagine), with all self-report items prefixed with “In the last 24 hours, you…. at its WORST.” The final list of symptoms (21 items) is presented in Table 1.

Table 1.

List of the symptom items included in ePRO-CoV-SM

ePRO symptom items a
1. Your fever at its WORST? c	0	1	2	3	4	5	6	7	8	9	10
2. Your cough at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
3. Your sputum at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
4. Your chest tightness at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
5. Your shortness of breath at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
6. Your chest pain at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
7. Your racing heartbeat at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
8. Your fatigue at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
9. Your muscle or joint pain at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
10. Your lack of appetite at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
11. Your stuffy nose at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
12. Your running nose at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
13. Your sore throat at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
14. Your throat itch at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
15. Your dry mouth at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
16. Your dizziness at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
17. Your headache at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
18. Your nausea at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
19. Your vomiting at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
20. Your bloating at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
21. Your abdominal pain at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
22. Your diarrhea at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
23. Your constipation at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
24. Your sweat at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
25. Your chills at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
26. Your shiver at its WORST?b, c	0	1	2	3	4	5	6	7	8	9	10
27. Your taste changes at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
28. Your smell changes at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
29. Your sleep disturbance at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
30. Your drowsiness at its WORST?c, d	0	1	2	3	4	5	6	7	8	9	10
31. Your rash at its WORST?c	0	1	2	3	4	5	6	7	8	9	10
32. Your anxiety at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
33. Your sadness at its WORST?b, c, d	0	1	2	3	4	5	6	7	8	9	10
34. Your red eyes at its WORST?b	0	1	2	3	4	5	6	7	8	9	10

^a All ePRO self-report items prefixed with ‘In the last 24 hours …

Response options “0 = symptom has not been present; 10 = symptom was as bad as you can imagine it could be”

^b Symptom items for asymptomatic SARS-CoV-2 infections in Wanzhou in 2020

^c Symptom items for asymptomatic SARS-CoV-2 BA.2.2 infections in Shanghai in 2022

^d Symptom items for COVID-19 patients in Chongqing in November 2022

Validation of results of ePRO-CoV-SM

A total of 57 asymptomatic patients with SARS-CoV-2 infections were recruited to participate in symptom monitoring, and 51 patients completed 1460 symptom self-reports with a response rate of 89.47% (51/57) and an item missing rate of 0.33% (100/30660), the most missed items were red eyes (1.16% [17/1460]) and chest tightness (0.48% [7/1460]). The age of the participants ranged from 22 to 75 years, with an average age of 41.39 ± 12.59 years, and females accounted for 60.78%. The monitoring compliance was 95.42% (1460/1530) in total, 95.52% (1364/1428) in the 4-week daily compliance, 96.08% (49/51) in the 6th week, and 92.16% (47/51) in the 8th week. During symptom monitoring, 69 (4.73%) severe symptom reports were detected, with one or more symptoms scoring ≥ 4 points. Table S1 presents the detailed symptom reports. During the isolation period, the primary symptoms were sleep disturbance (19.61%), anxiety (5.88%), lack of appetite (5.88%), diarrhea (5.88%), and nausea (5.88%). Table 2 presents the symptom burden of participants in Wanzhou.

Table 2.

Symptom burden in asymptomatic patients with COVID-19 in Wanzhou

Symptom	N	Mean ± SD	Median[P25,P75]	≥ 4	%
Sleep disturbance	51	1.65 ± 2.35	0 [0, 3]	10	19.61
Anxiety	51	0.69 ± 1.50	0 [0, 0]	3	5.88
Lack of appetite	51	0.61 ± 1.23	0 [0, 0]	3	5.88
Diarrhea	51	0.53 ± 1.42	0 [0, 0]	3	5.88
Nausea	51	0.37 ± 1.06	0 [0, 0]	3	5.88
Running nose	50	0.48 ± 1.58	0 [0, 0]	2	4.00
Headache	51	0.37 ± 1.28	0 [0, 0]	2	3.92
Vomiting	51	0.24 ± 0.89	0 [0, 0]	2	3.92
Sputum	51	0.49 ± 0.90	0 [0, 1]	1	1.96
Sadness	51	0.25 ± 0.82	0 [0, 0]	1	1.96
Shortness of breath	51	0.20 ± 0.72	0 [0, 0]	1	1.96
Sore throat	51	0.20 ± 0.69	0 [0, 0]	1	1.96
Chest pain	51	0.18 ± 0.68	0 [0, 0]	1	1.96
Muscle pain	51	0.14 ± 0.69	0 [0, 0]	1	1.96
Shiver	51	0.12 ± 0.59	0 [0, 0]	1	1.96
Cough	51	0.47 ± 0.81	0 [0, 1]	0	0.00
Stuffy nose	51	0.27 ± 0.70	0 [0, 0]	0	0.00
Chest tightness	50	0.10 ± 0.36	0 [0, 0]	0	0.00
Red eyes	51	0.06 ± 0.42	0 [0, 0]	0	0.00
Abdominal pain	50	0.06 ± 0.42	0 [0, 0]	0	0.00
Fatigue	51	0.06 ± 0.31	0 [0, 0]	0	0.00

Symptom items in ePRO-CoV-SM (2.0)

A total of 38 symptoms were extracted from the literature, and 101 clinicians participated in assessing these symptoms. Among these experts, 28 were doctors (27.72%), 73 were nurses (72.27%), 53 (52.48%) were intermediate-level doctors, 12 (11.88%) were senior doctors, and 43 (42.57%) had treated or nursed more than 50 patients with COVID-19. Among the 38 symptoms, four symptoms (“confusion” 35.64%, “unsteadiness” 37.62%, “memory loss” 40.59%, and “hemoptysis” 43.56%) were excluded from the analysis due to the lack of assent involving more than 35% of the experts (Table S2). After internal discussion, the research team eliminated the symptoms of “red eyes” (disagreement rate of 34.65%) and “sneezing” (closely related to runny nose), while adding “bloating” (a digestive symptom), resulting in a final list of 33 symptoms, as shown in Table 1.

Validation of ePRO-CoV-SM (2.0)

A total of 106 asymptomatic SARS-CoV-2 carriers participated in symptom monitoring and completed 312 self-reports, with an item missing rate of 1.02% (105/10296), the most missed items were racing heartbeat (1.60% [5/312]), fatigue (1.60% [5/312]) and diarrhea (1.60% [5/312]). The age of the participants ranged from 18 to 68 years, with an average age of 39.14 ± 13.18 years, and females accounted for 34.91%. Only 4.76% of the participants reported never being vaccinated for COVID-19; 39.05% had received two doses of the vaccine, and 51.43% had received three doses of the vaccine.

A total of 102 (96.23%) asymptomatic carriers completed one symptom assessment during the isolation period. The analysis revealed that more than 10% of the patients experienced dry mouth (12.12%) and cough (10.78%). A total of 91 (85.85%) asymptomatic carriers completed one of the four symptom assessments after release, 41.76% were reported from the 1st week after release and 47.25% from the 2nd week. More than 5% of the patients experienced sleep disturbance (10.99%), lack of appetite (5.56%), dry mouth (5.49%), or anxiety (5.49%) within 1 month after release. Figure 3 and Table S3 present the symptom burden of asymptomatic carriers in Shanghai.

Fig. 3.

Symptom reports of asymptomatic patients with COVID-19 in Shanghai

A flexible real-world application of ePRO-CoV-SM (2.0)

Through the expert opinion method, 18 symptoms were chosen from the ePRO-CoV-SM (2.0) symptom item bank to monitor the isolation symptoms of patients with COVID-19. The specific symptom items are listed in Table 1. A total of 405 patients scanned the recruitment code and signed electronic informed consent through WeChat, and 85.93% (348/405) patients completed at least one symptom assessment, resulting in a total of 692 symptom reports in isolation, with an item missing rate of 4.84% (603/12456), the most missed items were sleep disturbance (5.35% [37/692]), dizziness(5.20% [36/692]), headache(5.20% [36/692]), stuffy nose(5.20% [36/692]). Among these reports, 92.82% (323/348) included detailed demographic information, while 7.18% (25/348) chose not to disclose this information despite completing the symptom assessments. Data from 323 patients were available for analysis. Their ages ranged from 18 to 67 years old, with an average age of 39.77 ± 12.95 years. The proportion of female patients was 46.44% (150/323). A minority of participants (2.79% [9/323]) reported not completing the COVID-19 vaccine program, while 97.21% (314/323) had completed the vaccine program by receiving at least two doses of the vaccine.

Out of 323 symptom reports, cough (36.08%), dry mouth (35.67%), sleep disturbance (32.27%), lack of appetite (32.17%), and phlegm (30.79%) were the top five uncomfortable symptoms reported by the patients. Moreover, the symptom burden in the younger age group (18–40 years old) was generally higher than that in the older age group (40–67 years old), with statistically significant differences in six symptoms, including cough (3.18 ± 2.47 vs. 2.50 ± 2.22, P = 0.012), dry mouth (3.13 ± 2.65 vs. 2.39 ± 2.37, P = 0.011), lack of appetite (2.90 ± 2.87 vs. 1.93 ± 2.25, P = 0.001), phlegm (2.95 ± 2.46 vs. 2.38 ± 2.24, P = 0.035), stuffy nose (2.56 ± 2.47 vs. 1.69 ± 2.29, P = 0.002), and sweat (2.23 ± 2.57 vs. 1.64 ± 2.00, P = 0.026). Table 3 displays the symptom burden of patients with COVID-19 in Chongqing.

Table 3.

Symptom burden in patients with COVID-19 in Chongqing

Symptom	All (N = 323)					18–40 years (N = 154)		40–67 years (N = 157)		T	P
	N	Mean ± SD	Median [P25, P75]	≥ 4	%	N	Mean ± SD	N	Mean ± SD
Cough	316	2.77 ± 2.35	2 [1, 5]	114	36.08	150	3.18 ± 2.47	154	2.50 ± 2.22	2.53	0.012
Dry mouth	314	2.72 ± 2.53	2 [0, 5]	112	35.67	150	3.13 ± 2.65	152	2.39 ± 2.37	2.55	0.011
Sleep disturbance	313	2.38 ± 2.68	1 [0, 5]	101	32.27	149	2.70 ± 2.82	152	2.24 ± 2.55	1.49	0.138
Lack of appetite	314	2.37 ± 2.60	1 [0, 4]	101	32.17	149	2.90 ± 2.87	153	1.93 ± 2.25	3.27	0.001
Phlegm	315	2.61 ± 2.36	2 [0, 4]	97	30.79	150	2.95 ± 2.46	153	2.38 ± 2.24	2.12	0.035
Drowsiness	314	1.90 ± 2.38	0 [0, 4]	82	26.11	149	2.19 ± 2.53	153	1.73 ± 2.25	1.65	0.099
Stuffy nose	311	2.10 ± 2.41	1 [0, 4]	80	25.72	149	2.56 ± 2.47	150	1.69 ± 2.29	3.19	0.002
Sweat	313	1.90 ± 2.30	1 [0, 3]	75	23.96	149	2.23 ± 2.57	152	1.64 ± 2.00	2.24	0.026
Fatigue	311	1.78 ± 2.35	0 [0, 3]	69	22.19	148	2.10 ± 2.54	151	1.57 ± 2.16	1.95	0.052
Muscle or joint pain	313	1.76 ± 2.34	1 [0, 3]	68	21.73	149	2.00 ± 2.59	152	1.61 ± 2.10	1.45	0.148
Sore throat	314	1.85 ± 2.35	1 [0, 3]	66	21.02	149	2.02 ± 2.45	153	1.75 ± 2.27	0.99	0.324
Throat itch	313	1.81 ± 2.19	1 [0, 3]	65	20.77	149	1.98 ± 2.37	152	1.75 ± 2.05	0.90	0.368
Taste changes	313	1.59 ± 2.20	0 [0, 3]	60	19.17	149	1.68 ± 2.31	152	1.58 ± 2.15	0.41	0.682
Dizziness	311	1.65 ± 2.18	0 [0, 3]	57	18.33	148	1.91 ± 2.38	151	1.49 ± 2.00	1.63	0.104
Chest tightness	313	1.46 ± 2.09	0 [0, 3]	54	17.25	149	1.71 ± 2.28	152	1.30 ± 1.91	1.68	0.094
Headache	310	1.56 ± 2.34	0 [0, 3]	52	16.77	149	1.83 ± 2.57	149	1.38 ± 2.12	1.65	0.101
Anxiety	313	1.40 ± 2.14	0 [0, 2]	50	15.97	149	1.54 ± 2.32	152	1.36 ± 2.00	0.70	0.483
Sadness	313	1.02 ± 1.83	0 [0, 2]	33	10.54	149	1.06 ± 1.97	152	1.05 ± 1.74	0.04	0.971

Discussion

This study comprehensively introduces the urgent development, testing, and deployment process of an ePRO-based symptom monitoring system for COVID-19 (ePRO-CoV-SM). The high response rate reported by the ePRO-CoV-SM platform during the implementation phase indicates its feasibility in patient applications. As we continue to face the challenges posed by rapidly evolving pandemics like COVID-19, the development of this platform provides a response template for the rapid implementation of patient symptom monitoring during infectious disease outbreaks. Furthermore, the lessons learned from the development of these symptom-monitoring systems can be applied to address the needs of emerging diseases in the future.

In response to the first outbreak in 2020, we implemented a registration system initiated by Short Message Service in the first iteration of ePRO-CoV-SM with support from a team of researchers and clinicians. During the second wave of the COVID-19 outbreak in Shanghai, we transitioned from a clinician-led recruitment process to an automatic recruitment system using a scanning code, which allows patients to easily complete the recruitment process themselves, freeing up more medical resources. This self-recruitment method has been proven to be feasible and effective in the Shanghai and later Chongqing outbreaks.

The ePRO-CoV-SM (2.0) platform incorporates a core set of symptom items that patients experience, a WeChat mini program for patients to rate and report their symptoms, an online project design backend, a management and communication frontend, and a database to store information from both patients and medical records. The implementation of ePRO-CoV-SM (2.0) encompassed four key steps: (1) defining the process of symptom monitoring and informed consent on the backend of the project design; (2) automatic patient registration by scanning the recruitment code and signing electronic informed consent through WeChat; (3) completing the symptom assessment on the WeChat mini program; and (4) managing patients on the management front. These core structures enable ePRO-CoV-SM (2.0) to be immediately implemented during the outbreak of new strains of COVID-19 without significant modifications, while also serving as a structural template for the rapid development of electronic symptom monitoring platforms in response to other public health emergencies.

In real-world applications in 2022, the ePRO-CoV-SM platform allowed the monitoring and differentiating of symptom burden between different age groups. Compared with that in older patients, the symptoms of young patients were more pronounced, which is consistent with previous studies [22, 23]. Moreover, we observed that a subset of patients diagnosed as asymptomatic by clinicians experienced undesirable symptoms during isolation. In the 2020 validation study, more than 10% of patients reported sleep disturbances, and over 5% reported anxiety, lack of appetite, diarrhea, and nausea. In the 2022 validation, over 10% of patients were observed to experience dry mouth and cough, while more than 5% reported symptoms such as sputum, muscle or joint pain, headache, and fatigue. Previous studies have also confirmed that the chief complaints, which serve as the basis for clinicians’ diagnosis, cannot always account for potential symptoms. Nevertheless, PRO can help identify symptoms that may be overlooked by clinicians [24, 25].

The ePRO-CoV-SM platform had some limitations. First, COVID-19 has returned to social normalization now, but the learnings derived from the implementation of ePRO monitoring during this period of public health emergency deserve more reference. Second, as the purpose of this study was to explore the urgent development process of an electronic system for PRO monitoring in emergency situations, identify changes in patient status through monitoring items at different time points, and assist in patient management, we did not conduct traditional reliability and validity verification for the monitoring items, although most of these have been verified in previous tool development processes. The accuracy of items in assessing symptoms used in the emergency application will be further verified and optimized as needed. Third, when analyzing the symptom burden of patients, due to the limitations posed by urgent implementation, we had to adopt the symptom score ≥ 4 on the 0–10 scale recommended by previous studies and guidelines [11, 20, 21] as the classification threshold for moderate to severe symptoms, rather than setting a COVID-19-specific threshold through the standard development approach. This will be further studied in the development of standard assessment tools. Finally, it may be difficult to distinguish whether these symptoms are caused by COVID-19 or other reasons, but in practical applications, every sudden aggravated symptom should be medically attended to.

Conclusions

Electronic PRO measurement represents a sensitive tool for monitoring symptoms in patients with COVID-19. By integrating smartphone-based data collection with real-time online data transmission and SSL-encrypted secure data storage, we demonstrated that an electronic platform for monitoring critical symptoms can be rapidly established in COVID-19 emergency situations. This study explored the experience of developing the ePRO-CoV-SM platform, which provides a feasible template that can be used to establish an electronic symptom monitoring system in future public health crises, enabling timely interventions and optimized patient care.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Abbreviations

ePRO-CoV-SM

Electronic Patient-Reported Outcome COVID-19 Symptom Monitoring Platform

SARS-CoV-2

Severe acute respiratory syndrome coronavirus 2

PRO

Patient-Reported Outcome

ePRO

Electronic Patient-Reported Outcome

API

Application Program Interface

SSL

Secure Socket Layer

RWDMP

Real World Data Management Platform

RT-PCR

Reverse Transcriptase Polymerase Chain Reaction

IQR

Interquartile range

Author contributions

QS, JL and ZY conceived the idea for the study. YL, JY and JS obtained the data. JC, RX and DK cleared up the datasets; JZ and RX performed the data analyses. XW, JQ, XT and YN interpreted the results of the data analyses. JZ and QG wrote the manuscript. All authors read and approved the final manuscript.

Funding

This study was supported by the Fundamental Research Funds for the Central public welfare research institutes (NO. ZZ15-WT-05), the Science and Technology Innovation Key R&D Program of Chongqing (No. CSTB2022TIAD-STX0013) and the Postdoctoral Fellowship Program of CPSF (No. GZC20233357).

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author Qiuling Shi on reasonable request.

Declarations

Ethics approval and consent to participate

Ethics approval was obtained from the Institutional Review Board of Chongqing Medical University, the Ethics Committee of the Institute of Basic Clinical Medicine of the China Academy of Chinese Medical Sciences, and the Ethics Committees of The Second Affiliated Hospital of Chongqing Medical University, respectively. Electronic Informed consent was obtained from the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.