Comparing response rates between mobile web and telephone surveys for patient experience: a randomised experimental study in South Korea

Abstract

Objective

Healthcare quality improvement increasingly relies on patient experience data, yet traditional survey modes face declining response rates and rising costs. Mobile web surveys have emerged as a promising alternative for improving response rates. The primary aim of this study was to investigate the effectiveness of mobile web surveys in improving response rates in South Korea’s Patient Experience Assessment. We also aimed to assess the impact of a mixed-mode approach integrating mobile web and follow-up telephone surveys across different demographic groups.

Design

A randomised experimental design was employed to compare response rates as well as contact and cooperation rates among survey modes. A total of 4800 patients from four general hospitals were randomly allocated to telephone, mobile web or mixed-mode survey, with 1600 patients per mode. Each mode allowed five contact attempts through calls or mobile survey links. The mixed-mode survey included follow-up calls for mobile non-respondents.

Setting

The survey was conducted between October and November 2022 among patients discharged from four general hospitals in South Korea.

Participants

A total of 4800 patients aged 19 years or older who were hospitalised for more than 1 day and discharged within 2–56 days from four general hospitals were included in this study. Exclusion criteria included patients in day clinics, palliative care, paediatrics and neuropsychiatry, as well as those without personal information consent forms during hospital admission.

Primary and secondary outcome measures

The primary outcome measure was the response rate for each survey mode. Secondary outcome measures included the contact rate and the cooperation rate.

Results

The mobile web survey yielded an overall higher response rate (32.5%) than the telephone survey (22.4%), with the mixed-mode survey achieving the highest response rate (39.3%). Decomposing response rates revealed that while contact rates were comparable for both telephone and mobile web surveys, the cooperation rate was considerably higher for the mobile web survey (73.2%) compared with the telephone survey (52.2%). Substantial gender-age subgroup differences were found.

Conclusions

Adopting mobile web surveys for patient experience assessments, which aligns with the public’s preference for information and communication technologies, could significantly improve response rates in patient experience surveys.

Trial registration number

KCT0011374 (post-results).

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Randomised experimental design enabled robust causal inference about survey mode while maintaining real-world relevance.

• Detailed subcategory analysis that decomposed response rates into contact and cooperation rates and divided older adults at age 70 revealed important variations in response patterns.

• Exclusion of smaller hospitals (100–299 beds) limits generalisability to all South Korean hospitals.

• Prior patient experience surveys at participating hospitals may have influenced familiarity with mobile web formats, though this impact remains unclear.

Introduction

As patient-centredness is recognised as a key domain of healthcare quality, incorporating patients’ perspectives in healthcare quality assessment is increasingly highlighted,¹,⁴ and measuring patient experience has received greater attention in the past two decades.⁵,⁷ Against this backdrop, patient experience surveys are now conducted in many countries for measuring and improving patient experience.⁵⁸,¹⁰ Most patient experience surveys have used mail and telephone modes, though these traditional methods show declining response rates and increasing costs.⁷¹¹,¹³ Alternative modes include interactive voice response^{14 15} and web-based administration with survey links sent by mail, email, text or patient portal.¹⁴¹⁶,¹⁹ Earlier studies in the 2000s found these alternatives were not more effective than traditional methods. However, given the widespread adoption of smartphones over the 2010s and evolving public preferences for information and communication technologies (ICT),²⁰ particularly among younger populations,^{11 12} the performance of these alternative survey methods could now yield different results.

The traditional telephone survey, administered during typical working hours, faces several challenges in securing response rates. These include interference with respondents’ work schedules,^{21 22} inability to respond in shared environments and increasing reluctance to answer calls from unrecognised numbers, often viewed as potential spam.⁶ Self-administered mobile web surveys using smartphones can be a promising alternative, offering advantages such as low cost, rapid data collection, absence of interviewer effects, reduced social desirability bias and improved response quality for sensitive questions.^{23 24} For these multiple reasons, introducing mobile web mode in patient experience surveys warrants serious attention.

South Korea offers a relevant setting for investigating the effectiveness of the mobile web mode in patient experience surveys. The Patient Experience Assessment (PXA) was initially conducted on patients discharged from general hospitals, using traditional telephone surveys. Starting with general hospitals with 500 or more beds in 2017, the PXA gradually expanded to hospitals with 300 or more beds (2019) and to those with 100 or more beds (2021). The PXA results are provided to each of the participating hospitals and publicly reported. While contributing to increased societal interest in patient experience and catalysing hospitals’ improvement efforts, the PXA has shown methodological issues including low response rates (below 15%) and increasing administrative costs,⁶ along with suspected social desirability bias and limited response time.^{25 26} Consequently, there has been a growing need to adopt innovative modes of survey administration for the PXA. South Korea’s technological environment, with 99% adult internet usage and 98% smartphone penetration rates as of 2022,²⁷ makes it an ideal setting for transforming the current telephone survey to a mobile web survey for a more efficient and sustainable PXA.

The primary aim of this randomised experimental study was to investigate whether using a mobile web survey (vs the telephone survey as the then status quo) is effective in improving response rates in a real-world situation as of 2022, 1 year before the fourth PXA scheduled for 2023. The results of this investigation were to provide evidence on whether the Korean government could proceed with the new mobile web survey, instead of the telephone survey. As with other technological changes, however, it is important to identify population subgroups that would be affected by the new mobile web survey in the PXA and to address potential, unintended consequences of the shift when necessary. Therefore, three secondary aims of our study comprised: (1) to assess the extent to which the mixed-mode survey that integrates both mobile web and follow-up telephone surveys could increase response rates, (2) to examine gender and age-group differences in response rates by survey mode and (3) to gain further insights on the mechanism of changes in response rates by decomposing response rates into contact and cooperation rates by survey mode and age-gender subgroup.

Methods

Patient experience assessment

The PXA is administered by the Health Insurance Review and Assessment Service (HIRA), an organisation responsible for assessing and improving healthcare quality within the National Health Insurance in South Korea. The HIRA selects survey samples for each participating hospital based on medical claims data and requests the respective hospitals to provide mobile phone numbers of the selected patients. A survey company, which is contracted by the HIRA, then contacts the selected patients to conduct the survey. The number of hospitals participating in the third PXA in 2021 was 359, involving a total of 398 781 patients. Until 2021, the PXA was conducted using the single, telephone survey mode.

Study design and sample

This study employed a randomised experimental design to compare response rates primarily between telephone and mobile web surveys and secondarily between mobile web and mixed-mode surveys. The sample consisted of patients who had been discharged from four general hospitals participating in this study. These patients were randomly assigned to one of the three survey modes—telephone, mobile web or mixed-mode.

To ensure adequate statistical power (80%, α=0.05, effect size=5% points), we calculated a required sample size of 686 patients per group. To accommodate subgroup analyses, we included 1600 patients per mode (total 4800), distributed proportionally by hospital bed capacity: two tertiary hospitals (1500 each, >1000 beds), one mid-sized tertiary hospital (1000 patients, 500–999 beds) and one non-tertiary hospital (800 patients, <500 beds).

Following PXA criteria, we included adults aged 19+ who were hospitalised for >1 day and discharged within 2–56 days, excluding day clinics, palliative care, paediatrics and neuropsychiatry patients. We also excluded patients without personal information consent forms. This consent form, required by the Personal Information Protection Act, is an essential document obtained during hospital admission procedures. Administrative staff explained the purpose, specific items and duration of personal information collection, use and provision to patients before requesting their voluntary signatures on this form.

Hospital-specific sampling followed HIRA’s PXA framework, which uses a 12-cell classification system based on gender (Male/Female), age (19–39, 40–59, 60+) and field of care (Medical/Surgical) to ensure patient representativeness and hospital comparability. For this study, we modified the framework by subdividing the 60+age group into 60–69 and 70+ categories to examine age-related heterogeneity in older groups. Hospital patient experience staff conducted the sampling and securely transmitted patient information (including demographic data and contact details) to our research team. The contracted survey company, which regularly conducts PXA surveys for HIRA, then performed computerised randomisation. The research team maintained independence by not participating in the randomisation or data collection processes.

Survey questionnaire and procedures

We used HIRA’s 24-question PXA questionnaire covering six domains: nursing care, physician services, medication and treatment processes, hospital environment, patient rights and patient characteristics. The mobile format underwent usability testing with 14 individuals. We checked domain-specific screens with scroll-down navigation and clear progression indicators and made modifications based on the test results.

Before the survey was initiated, the selected 4800 study participants received an information message via phone or a mobile messenger app and were informed about the upcoming survey. The survey took place between October and November 2022.

Survey procedures varied by mode. For the telephone survey, trained interviewers made daily contact attempts (maximum five) following a standardised protocol of identity verification, information provision and questionnaire administration. For the mobile web survey, participants received mobile messages containing survey links through the PXA’s official channel in a messenger app (maximum five times, 2–3-day intervals). They accessed the survey through a ‘Start Survey’ button, completed identity verification, answered the questionnaire and submitted responses. The mixed-mode approach began with the mobile web survey protocol and was followed by telephone calls for non-respondents.

Outcome measures

Following the standard definitions of the American Association for Public Opinion Research,²⁸ we defined outcome measures for telephone surveys and modified them to suit mobile web surveys. Figure 1 illustrates the categorisation of the entire sample based on the level of survey participation.

Figure 1. Outcome measures: contact, cooperation and response rates. In the telephone surveys, ‘Contact’ refers to the respondent answering the phone call when an interviewer calls; ‘Refusal’ or ‘Break Off’ pertains to the respondent rejecting participation in the survey or simply hanging up the phone; 'Partial’ indicates that the respondent began the survey but did not complete it; ‘Complete’ denotes that the respondent answered all the questions in the survey fully. For mobile web surveys, ‘Contact’ is defined as the respondent clicking the ‘Start Survey’ link in a mobile message; ‘Refusal’ or ‘Break Off’ refers to the respondent accessing the first page of the survey but choosing not to participate at all; ‘Partial’ means that the respondent stops participating and exits the survey before completion; ‘Complete’ pertains to the respondent finishing the entire survey and clicking the ‘Submit Survey’ button at the end.

The primary outcome measure of our study is the response rate, which is to be compared first between the telephone and mobile web surveys, and subsequently between the mobile web and mixed-mode surveys. Since the response rate is a numerical product of the contact rate and the cooperation rate (conditional on successful contact), the response rate can be decomposed into the contact rate and the cooperation rate, for useful insights on possible ways to increase the response rate. In this study, the contact rate is defined as the percentage of participants successfully reached by the survey mode, while the cooperation rate is the proportion of contacted participants who completed the survey. In addition to the response rate, these two outcome measures were also compared among the three survey modes, both in the whole sample and in subgroups.

Statistical analysis

Statistical analysis included χ² tests comparing baseline attributes (gender, age, field of care, hospital type) and response rates across survey modes. We conducted separate analyses comparing telephone versus mobile web surveys, and mobile web versus mixed-mode surveys. Additional subgroup analyses examined response patterns by gender-age categories and hospital characteristics. All analyses used SPSS V.26, with significance at p<0.05. All randomised participants were included in the analysis, with no missing data for the primary and secondary outcome measures.

Patient and public involvement

None.

Results

Study participants

The flow of study participants is presented in figure 2. A total of 4800 patients discharged from four general hospitals were randomly assigned to the telephone, mobile web or mixed-mode groups (1600 per group). The distribution of gender, age group, field of care and hospital of origin was comparable across survey modes (table 1).

Figure 2. Participant flow chart. All 4800 participants were randomly allocated to one of three survey modes (n=1600 each).

Table 1. Characteristics of the study participants by survey mode.

	Telephone		Mobile web		Mixed-mode		P value*
	n	%	n	%	n	%
Total	1600	100.0	1600	100.0	1600	100.0
Gender
Male	799	49.9	784	49.0	815	50.9	0.55
Female	801	50.1	816	51.0	785	49.1
Age group
19–39	189	11.8	225	14.1	202	12.6	0.50
40–59	525	32.8	505	31.6	497	31.1
60–69	439	27.4	439	27.4	436	27.3
70+	447	27.9	431	26.9	465	29.1
Field of care
Medical	612	38.3	656	41.0	657	41.1	0.18
Surgical and other	988	61.8	944	59.0	943	58.9
Hospital of origin
A	500	31.3	500	31.3	500	31.3	1.00
B	502	31.4	500	31.3	498	31.1
C	334	20.9	332	20.8	334	20.9
D	264	16.5	268	16.8	268	16.8

^*The χ2 test was conducted to determine whether the three groups under different survey modes were similar for each of the respondent characteristics.

Response rates by survey mode and respondent characteristics

Table 2 shows response rates across the three survey modes, both overall and by subgroup of gender, age group and field of care. The overall response rate in the telephone survey was 22.4% (=358/1600), while the mobile web survey had a response rate of 32.5% (=520/1600). This difference of 10.1% points (=32.5%–22.4%) was statistically significant (p<0.001). The mixed-mode survey showed a response rate of 39.3% (=629/1600), which was 6.8% points higher than that of the mobile web survey. This difference was also statistically significant (p<0.001).

Table 2. Response rates by survey mode and respondents’ characteristics.

	Telephone (A, %)	Mobile web (B, %)	Mobile web vs telephone		Mixed-mode (C, %)	Mixed-mode versus mobile web
			(B–A, % points)	P value		(C–B, % points)	P value
Total
	22.4	32.5	10.1	<0.001*	39.3	6.8	<0.001*
Gender
Male	24.4	31.9	7.5	<0.001†	39.0	7.1	<0.001†
Female	20.3	33.1	12.7		39.6	6.5
Age group
19–39	16.4	37.3	20.9	<0.001†	48.0	10.7	<0.001†
40–59	24.4	45.7	21.4		49.1	3.4
60–69	26.4	31.2	4.8		37.8	6.6
70+	18.6	15.8	–2.8		26.5	10.7
Field of care
Medical	20.1	28.5	8.4	<0.001†	35.5	7.0	<0.001†
Surgical and other	23.8	35.3	11.5		42.0	6.7

^*P value from the χ2 test for testing whether the response rates were similar between the telephone and mobile web surveys and between the mobile web and mixed-mode surveys.

^†P value from the χ2 test for testing whether the response rates were similar by the characteristic in question between the telephone and mobile web surveys andbetween the mobile web and mixed-mode surveys.

While response rates were generally higher in the mobile web survey than in the telephone survey (reflected as positive differences (B–A, %p) in table 2), the magnitude of these differences varied by gender and age group. Women showed a larger positive difference (12.7% points) than men (7.5% points). Patients in the 19–39 and 40–59 age groups revealed substantially larger positive differences (more than 20% points) than those aged 60–69 (4.8% points). Among patients aged 70+, the response rate was lower for the mobile web survey than for the telephone survey, resulting in a negative difference of –2.8% points.

Response rates in the mixed-mode survey were higher than those in the mobile web survey. Again, the magnitude of the differences was not uniform across subgroups. Differences exceeding 10% points were observed among patients aged 19–39 and those aged 70+ (detailed mixed-mode survey results are presented in online supplement 1).

Contact, cooperation and response rates by survey mode and subgroup

Table 3 presents the findings of contact and cooperation rates as well as response rates for further exploration of important differences by survey mode and subgroup. When the telephone and mobile web surveys were compared, the overall contact rates were similar (telephone 42.9% and mobile web 44.4%). However, a considerable difference was observed in the cooperation rates between the two modes. The mobile web survey demonstrated notably higher cooperation rates (73.2% vs 52.2% in telephone), which primarily drove its higher overall response rate.

Table 3. Contact, cooperation and response rates by survey mode and subgroup (%).

	Telephone			Mobile web			Mixed-mode
	Contact rate	Cooperation rate	Response rate	Contact rate	Cooperation rate	Response rate	Contact rate	Cooperation rate	Response rate
Total
	42.9	52.2	22.4	44.4	73.2	32.5	58.5	67.2	39.3
Male
19–39	45.8	59.3	27.1	46.6	61.0	28.4	60.5	69.6	42.1
40–59	37.0	61.1	22.6	51.5	82.9	42.7	60.3	76.9	46.4
60–69	47.5	57.9	27.5	45.3	80.2	36.3	57.7	68.1	39.3
70+	52.9	42.2	22.3	29.8	61.4	18.3	58.2	53.6	31.2
Subtotal	45.8	53.3	24.4	42.6	74.9	31.9	58.9	66.3	39.0
Female
19–39	23.1	50.0	11.5	51.8	83.1	43.1	68.3	75.6	51.6
40–59	38.7	67.0	25.9	55.8	86.5	48.2	63.5	81.2	51.5
60–69	44.0	56.8	25.0	40.0	63.4	25.4	53.3	67.6	36.0
70+	48.8	29.0	14.1	34.7	36.8	12.8	49.5	41.0	20.3
Subtotal	40.0	50.9	20.3	46.1	71.8	33.1	58.1	68.2	39.6

This pattern of higher cooperation in the mobile web survey was consistent across all gender-age subgroups, with four groups achieving particularly high rates (above 80%): males aged 40–59 and 60–69, and females aged 19–39 and 40–59. Only females aged 70+ had a relatively lower cooperation rate (36.8%) than other gender-age subgroups (all above 50%), although it was still higher than the cooperation rate in the telephone group (29.0%).

Unlike the cooperation rate, the contact rate exhibited a distinct pattern of age group differences between the telephone and mobile web surveys: Males and females who were relatively younger (19–39 and 40–59) showed higher contact rates in the mobile web survey, whereas males and females who were older (60–69 and 70+) showed higher contact rates in the telephone survey.

The combination of higher contact and cooperation rates in mobile web surveys led to substantially higher response rates among younger groups (19–39 and 40–59), particularly females. For example, females aged 19–39 showed much higher contact, cooperation and response rates in the mobile web survey (51.8%, 83.1% and 43.1%) than in the telephone survey (23.1%, 50.0% and 11.5%). The 40–59 age group provides another such example. They showed the highest contact rates (males 51.5%, females 55.8%) and cooperation rates (males 82.9%, females 86.5%) among all age groups, resulting in significantly higher response rates in mobile web (males 42.7%, females 48.2%) compared with telephone (males 22.6%, females 25.9%).

For patients aged 70 and above, the mobile web survey showed lower contact rates (males 29.8%, females 34.7%), although their cooperation rates among those successfully contacted were higher than in the telephone survey. This combined effect led to slightly lower overall response rates in the mobile web survey (males 18.3%, females 12.8%) than in the telephone survey (males 22.3%, females 14.1%) for this age group. However, the 60–69 age group showed more favourable response rates: despite slightly lower contact rates in the mobile web survey, their high cooperation rates (males 80.2%, females 63.4%) were sufficient to produce higher overall response rates (males 36.3%, females 25.4%).

When comparing the mixed-mode survey with the mobile web survey, the additional gain in overall response rate was largely due to its higher contact rate (58.5% vs 44.4% in mobile web). This is particularly evident among males aged 19–39 and 70+. However, follow-up calls to mobile web survey non-respondents did not necessarily lead to the ultimate completion of the survey, as shown by the lower cooperation rate (67.2%) in the mixed-mode than in the mobile web (73.2%).

Discussion

Summary of findings

The primary findings of this study indicate that the mobile web survey yielded overall higher response rates than the telephone survey. We also found that the mixed-mode survey can achieve higher response rates than what the mobile web survey alone can obtain. Considerable gender and age-group differences existed in the extent to which response rates were higher in the mobile web survey than in the telephone survey. Decomposing response rates into contact and cooperation rates by age-gender subgroup helped provide richer insights on the mechanisms linking survey mode and response rates as well as on heterogeneous effects by subgroup. Most notably, our initial concern regarding significantly lower response rates in the mobile web survey among older individuals turned out to be less of an issue, particularly for those in the 60–69 category.

Findings in relation to other evidence

This study has generated interesting findings within the existing literature regarding the response rate of alternative modes in patient experience surveys. First, our study demonstrating higher response rates in the mobile web survey (vs telephone survey) is in contrast to the earlier studies that reported lower response rates in web-based surveys (vs telephone or mail surveys).¹⁴¹⁶,^{18 29 30} This inconsistency could be explained by three main factors: advances in internet and smartphone penetration, evolution in survey access methods and enhanced trust through sender identification. Higher internet and smartphone penetration in our study context, compared with previous studies in earlier periods or less digitally connected regions, likely contributed to improved response rates. Specifically, South Korea recorded 99% adult internet usage and 98% smartphone penetration rates as of 2022.²⁷ Additionally, the methods employed to access survey sites have evolved by the advancement in ICT in more recent years. In the previous studies, survey invitations and links to the web survey were sent via mail,^{14 18} text,¹⁹ email¹⁷,^{1929 30} or patient portal.²⁹ Respondents were then required to either enter the website address on a computer^{14 17 18} or access the survey by clicking a link in an email, text message or patient portal.^{18 19 29 30} The difficulty of accessing survey sites is likely to have posed a substantial barrier. As an illustrative example, web surveys that required respondents to input a website address and login credentials previously provided were associated with particularly low response rates.^{14 18 30} A slight rise in response rates was observed when respondents were able to access the survey directly via a URL link sent through text or email, but even then, these rates still remained lower than those from traditional telephone or mail surveys.^{18 29} In this case, the key issue impeding respondents’ willingness to participate in the web survey may be related to their distrust towards unfamiliar senders, along with their tendency to equate unsolicited survey links with potential spam or phishing attempts.³¹ In contrast, our study implemented a more user-friendly approach by disseminating the survey link through a popular messenger application in South Korea, while also using the official PXA survey channel within the application. This approach enabled participants to promptly receive notifications about the survey, easily identify the sender, read the survey invitation message and simply begin the survey by clicking the link. In addition to removing various access barriers to the survey website, the provision of sender’s information along with the survey link has the potential to enhance respondents’ trust in the sender, thereby increasing their willingness to participate in patient experience surveys.

Second, our results highlight the effectiveness of mobile web survey administration in improving response rates in a broad patient population, particularly younger patients. These demographic subgroups have often shown very low participation rates in interviewer-led telephone surveys,^{11 12 23} possibly due to their inability to answer calls during work hours or aversion to answering calls from unknown numbers.^{6 31 32} The higher response rate for patients under 70 in the mobile web survey resulted from both higher contact rates and higher cooperation rates compared with the telephone survey, though their relative contribution varied considerably by gender and age group. The higher contact and cooperation rates in the mobile web survey in these groups suggest their preference for this self-administered mode to participate in patient experience surveys at their convenience, in terms of both time and place.

Third, despite our initial concern that patients in the 60+ age group might perform poorly in the mobile web survey, this was not necessarily the case. Patients in their 60s exhibited higher response rates in the mobile web survey than in the telephone survey, while those aged 70+ did show lower response rates due to their lower contact rates in the mobile web survey. This finding is in line with the general pattern that response rates for web-based surveys decrease with increasing age.^{14 19 33} The variations in contact, cooperation and response rates among older men and women observed in this study were also similar to the disparities in levels of digital literacy in the literature,³⁴ specifically identifying women aged 70+ as the group with the lowest engagement in ICT.

Fourth, our results show that the mixed-mode survey was effective to some extent in further improving response rates and particularly in reaching specific population subgroups that would be less likely to be surveyed in the mobile web mode alone. Since the transition from traditional mail or telephone surveys to a mobile web survey raises concerns about lower response rates and reduced representation of survey respondents, there is often the need for a mixed-mode survey that combines web-based survey with traditional mail or telephone survey. Research has shown that such mixed-mode surveys often achieve higher response rates compared with single-mode surveys.^{1429 35},³⁷ In line with these findings, our study also observed higher response rates with a sequential approach of an initial mobile web survey followed by a phone survey.

Policy implications

The results of this study offered direct evidence for the policy decision regarding the transition of the survey mode for South Korea’s PXA. In August 2023, the HIRA’s fourth PXA started with the new mobile web survey mode, replacing the previous phone survey mode that had been used in the first through third PXA (2017–2021). The policy decision on this mode shift was informed by the timely, relevant and plausible evidence produced by our study. The evidence-based policy making was facilitated by the strengths of our study, which will be described below.

While our study showed that the mixed-mode survey achieved higher response rates than the mobile web mode, the mobile web mode was ultimately adopted by the HIRA. A number of other factors should be taken into account when choosing the mode of patient experience surveys, including the magnitude of the marginal improvement of response rates due to the additional survey mode, improved representativeness of the target population, administrative costs, and other logistical challenges. One specific methodological challenge relates to the potential mode effect on patient experience responses. Interviewer-led surveys often yield more positive responses to scale questions than self-administered surveys.^{25 38} Thus, when combining results from both conversation-based phone surveys and text-based mobile surveys, it is imperative to measure the mode effect and prepare a distinct adjustment plan.⁶ This process is anticipated to demand considerable preparation time and cost.³⁵

Strengths and limitations

The primary strength of this study lies in its randomised experimental design, enabling robust causal inference about survey mode effectiveness while still maintaining the real-world relevance to the extent possible. By randomly assigning participants into different modes of survey, we were able to improve causal inference on the effect of survey modes on response rates, as we had eliminated possible alternative explanations that could be easily made when using observational data. We carefully ensured our experimental conditions reflected the actual PXA environment, from patient sampling to survey implementation, thereby achieving strong internal validity without compromising external validity.

Another major strength is our detailed analysis that went beyond basic survey mode comparisons. By examining age-gender subcategories and decomposing results into contact and cooperation rates, we gained deeper insights into response patterns. Notably, our decision to split the 60+ age group into 60–69 and 70+ categories revealed important variations in digital literacy and survey preferences among older populations.

Our study has limitations. First, while we included four hospitals of varying sizes, we did not include smaller hospitals with 100–299 beds. Since patients from these facilities may show different response patterns, particularly in mobile web surveys, our findings should not be considered representative of all South Korean hospitals. Second, the participating hospitals had previously implemented their own mobile web patient experience surveys, potentially familiarising some patients with mobile web formats. However, the impact of this prior exposure remains unclear. Given these limitations, it must be noted that the results of this study are best interpreted as comparing response rates between survey modes along with individual characteristics within our sample, rather than as being nationally representative of patients or general hospitals.

Conclusions

Compared with the telephone survey, the mobile web survey shows a higher response rate in South Korea’s patient experience survey. A mixed-mode survey approach, which involves an additional follow-up telephone survey, could augment the likelihood of capturing the experiences of patients who might otherwise decline to participate in the mobile web survey. Despite considerable subgroup differences, the mobile web survey achieves significantly higher cooperation rates and subsequent response rates than the telephone survey, particularly in younger groups, suggesting evolving preferences of the public for ICT. The mobile web survey is less likely to be effective in improving contact and response rates among patients in their 70s and above than other, younger age groups, while patients in their 60s show promising potential for using mobile technology in reporting their patient experience. Taken together, adopting the mobile web survey for patient experience surveys, which aligns with the public’s preference for ICT, could significantly improve the response rate of patient experience surveys.

The funders did not influence the results/outcomes of the study despite author affiliations with the funder. The views expressed are those of the authors and not necessarily those of the HIRA.

Data availability statement

Data may be obtained from a third party and are not publicly available. Data requests should be directed to the Health Insurance Review and Assessment Service (HIRA).