Abstract
Background
This study aimed to identify key priorities for the development of guidelines for information and communication technology (ICT)-based patient education tailored to the needs of patients with rheumatic diseases (RDs) in the Republic of Korea, based on expert consensus.
Methods
A two-round modified Delphi study was conducted with 20 rheumatology, patient education, and digital health literacy experts. A total of 35 items covering 7 domains and 18 subdomains were evaluated. Each item was evaluated for its level of importance, and the responses were rated on a 4-point Likert scale. Consensus levels were defined as “high” (interquartile range [IQR] ≤ 1, agreement ≥ 80%, content validity ratio [CVR] ≥ 0.7), "Moderate" (IQR ≥ 1, agreement 50–79%, CVR 0.5–0.7), and "Low" (IQR > 1, agreement < 50%, CVR < 0.5).
Results
Strong consensus was reached for key priorities for developing guidelines in areas such as health literacy, digital health literacy, medical terminology, user interface, and user experience design for mobile apps. Chatbot use and video (e.g., YouTube) also achieved high consensus, whereas AI-powered platforms such as ChatGPT showed moderate-to-high agreement. Telemedicine was excluded because of insufficient consensus.
Conclusion
The key priorities identified in this study provide a foundation for the development of ICT-based patient education guidelines for RDs in the Republic of Korea. Future efforts should focus on integrating digital tools into clinical practice to enhance patient engagement and improve clinical outcomes.
Keywords: Rheumatic Diseases, Information and Communication Technology (ICT), Patient Education
Graphical Abstract
INTRODUCTION
Rheumatic diseases (RDs) are a diverse group of disorders that primarily affect the joints, connective tissues, and muscles, often involving chronic inflammation and immune system dysregulation. These conditions pose considerable challenges in clinical management owing to their progressive nature and complexities of treatment regimens.1,2 Effective disease management requires active patient participation, including medication adherence, lifestyle modifications, and monitoring of symptoms.3,4 Although many patients prefer to actively manage their condition, they require personalized support from healthcare professionals, which varies according to the disease stage and symptom severity.5 This highlights the critical importance of ongoing patient education in empowering individuals to effectively manage their condition while adapting to changing symptoms and disease progression.
Patient education is crucial for improving clinical outcomes, physical function, psychosocial health, pain management, and health behaviors in patients with RDs.6,7 However, the brief nature of consultations in clinical settings, which are often completed shortly, limits the effectiveness of comprehensive education, particularly in the Republic of Korea.8,9
Information and communication technology (ICT) provides a transformative opportunity to overcome these barriers. ICT-based interventions may deliver tailored education, facilitate real-time communication, and enhance patient engagement.10,11 ICT-based interventions, such as web portals, mobile apps, telehealth, and chatbots, have shown promise in delivering personalized education and real-time communication, as well as in keeping patients engaged.
Research from the U.S. has shown that regular monitoring and feedback helps patients with RDs manage their symptoms more effectively.12 Additionally, even in groups with low literacy, customized multimedia applications improve patients’ confidence, engagement, and quality of life in managing chronic diseases.13 These technologies can bridge the gap between patients and healthcare providers by offering continuous support and enhancing patients’ ability to manage their conditions.10
However, in the Republic of Korea, about 75% of rheumatologists face challenges in providing sufficient patient education owing to time limitations and a lack of support staff, highlighting the need for ICT-based solutions.14 Despite the potential benefits, the absence of standardized guidelines for ICT-based education has led to inconsistent outcomes and inefficiencies.15 Countries such as the U.S. and the U.K. have implemented ICT-based health literacy guidelines, whereas the Republic of Korea lacks practical guidelines to support the integration of digital health into patient education.16,17
Therefore, this study aimed to achieve expert consensus on the key priorities for developing guidelines for ICT-based patient education tailored to the needs of patients with RDs in the Republic of Korea.
METHODS
Study design
We conducted a two-round modified Delphi study to improve ICT-based patient education for RDs in the Republic of Korea. The first and second rounds occurred from April 2 to April 25, 2024, and from June 12 to June 27, 2024, respectively. The Delphi method was used to achieve expert consensus through iterative surveys based on predefined items selected by a steering group by literature review.18 Surveys were sent via e-mail, and responses were collected and analyzed in accordance with the Guidance on Conducting and Reporting Delphi Studies.19
Survey development
The Delphi study was conducted in five phases following the steps outlined in Fig. 1.
Fig. 1. Delphi study stages for developing guidelines.
Phase 1: preparation of Delphi survey
To develop survey items, a scoping review was conducted to identify RD-specific ICT-based patient education needs and key themes to guide the Delphi process, the results of which have been published.20 These themes were prioritized to ensure that the guidelines reflected unique challenges faced by patients, including fluctuating symptoms, long-term medication adherence, and effective self-monitoring of disease activity, and were used to define core concepts and frame the initial survey questions.
Phase 2: development of survey questions
The survey items for the first round of the Delphi study were meticulously developed through a multistep process. The scoping review findings identified and prioritized key themes related to patient education and digital health literacy within the context of RDs to ensure compliance with guidelines. These themes were used as the foundation for item generation. The development process involved a series of internal discussions with a panel of experts, comprising two rheumatology physicians, one nurse researcher with extensive experience in patient education, and one expert in digital health literacy. This interdisciplinary team ensured that the survey items captured the most critical aspects of patient education, particularly in relation to digital tools and platforms, while addressing the specific educational needs of patients with RDs.
Phase 3: selection of experts
We identified three expert groups that need to be included in the panel as follows: healthcare professionals in rheumatology, including those with > 10 years of patient education; individuals with > 3 years of experience in developing patient education materials or conducting related research; and experts in digital health literacy and health literacy with at least 3 years of experience in patient education. Twenty experts were invited by e-mail to participate in the study; however, those with a potential financial interest in the study’s outcomes and those who did not consent to participate were excluded (Supplementary Table 1).
Phase 4: conduct round 1 (R1)
We conducted two rounds of the Delphi process because a consensus is typically reached within two rounds in most Delphi studies.18 In the first round, we explored the participants’ experiences in developing educational materials, including the use of guidelines for ICT-based patient education. During this process, experts were instructed to prioritize RD-specific considerations including supporting patients in managing fluctuating symptoms, adhering to long-term medication regimens, and using digital tools to facilitate self-monitoring of disease activity. We also assessed their overall perspectives on the guidelines and their level of consensus regarding the 20 specific domains. Participants rated these domains using a 4-point Likert scale, ranging from 1 for “not important” to 4 for “very important.” The data were analyzed to discern common themes, consensus areas, and points of divergence.
Phase 5: conduct round 2 (R2)
In the second round, participants re-evaluated the necessity of the domains identified in the first round using a 4-point scale, with 1 meaning “not necessary” and 4 meaning “very necessary,” additionally, they rated the sub-items based on their content validity, with 1 indicating “low validity” and 4 indicating “high validity.”
Statistical analysis
Descriptive statistics were used in each round of the Delphi consensus process. We calculated the mean, standard deviation (SD), interquartile range (IQR), agreement, and content validity ratio (CVR). Agreement with a statement was defined as the combined proportion of participants who rated it 3 or 4 on the Likert scale.18
We categorized the level of consensus into three groups as follows: “high consensus” was defined as having an IQR ≤ 1, agreement percentage ≥ 80%, and CVR ≥ 0.7; “moderate consensus” had an IQR ≤ 1, agreement percentage between 50% and 79%, and a CVR between 0.5 and 0.7; and “low consensus” was characterized by an IQR > 1, an agreement percentage < 50%, and CVR < 0.5.19,21 R version 4.2.3 (R Foundation for Statistical Computing, Vienna, Austria) was used to analyze the exported dataset.
Ethics statement
This study was approved by the Institutional Review Board of Hanyang University Seoul Hospital (HYUH IRB) on March 20, 2024 (approval No. HYUH IRB 2024-01-070-002). Informed consent was obtained from all participants involved in the study.
RESULTS
Study population
Twenty experts with a mean age of 48 years (SD = 9.1) participated in the study. The majority were physicians specializing in rheumatology (80.0%, n = 16), and 20.0% (n = 4) were nurses or researchers. Most participants had over 10 years of experience in rheumatology, and all had at least 3 years of experience in patient education. Additionally, 25% (n = 5) of the participants were experts in digital health literacy. All participants were financially compensated for their time (Table 1).
Table 1. Basic information on experiences of experts (N = 20).
| Demographic characteristics | Experts (N = 20) | ||
|---|---|---|---|
| Age, yr | 48 ± 9.1 | ||
| Job | |||
| Physician | 16 (80.0) | ||
| Nurse, researcher | 4 (20.0) | ||
| Field of worka | |||
| Rheumatology-related healthcare work | |||
| > 10 yr | 16 (80.0) | ||
| Patient education (or material development) | |||
| > 3 yr | 20 (100.0) | ||
| (Digital) health literacy expertise | |||
| > 3 yr | 5 (25.0) | ||
Values are presented as mean ± standard deviation or number (%).
aOverlapping responses.
Essential domains for RD-specific ICT education guidelines
In this Delphi study, consensus was achieved on two major components of ICT-based patient education tailored to patients with RDs. The first component, focusing on guidelines for patient education content and materials development, incorporated domains including health literacy, guidelines for developing existing patient education materials, and online education strategies. The second part, emphasizing digital tool and interface utilization, encompassed mobile apps, chatbots, videos (e.g., YouTube), telemedicine, and AI-powered platforms (e.g., ChatGPT). We separated chatbots and AI-powered platforms (e.g., ChatGPT) into key domains to reflect their distinct roles. Chatbots, often rule-based, are used for tasks such as consultations, whereas AI platforms, such as ChatGPT, require patients to ask well-structured questions to effectively utilize their broader capabilities.
Experience of using guidelines
We assessed the participants’ experiences using guidelines to develop educational materials specifically for patients with RDs (Table 2). Most participants (85%) reported not having formal guidelines and instead relied on personal knowledge, patient feedback, and expert consultations. Among the 15% with access to formal guidelines, the resources included the Assessment in Ankylosing Spondylitis International Working Group Patient Education Resources, materials from the Korean College of Rheumatology, and translated versions of the guidelines from the American College of Rheumatology and the European League Against Rheumatism.
Table 2. Experience of using the guidelines for developing patient education.
| No. | Q1. Is there a guideline for developing patient education materials? | Q2. If no, on what basis do you develop and evaluate appropriateness? |
|---|---|---|
| 1 | NO | Frequently asked questions by patients, internet questions; developed materials based on these without appropriateness evaluation |
| 2 | Yes (ASAS Patient Education Resources, Korean College of Rheumatology education materials) | Not applicable |
| 3 | NO | Developed based on books, papers without appropriateness evaluation |
| 4 | Yes (ACR and EULAR guidelines, reviewed by experts, translated into Korean, and simplified terminology) | Not applicable |
| 5 | NO | Did not conduct appropriateness evaluations |
| 6 | NO | Patient interviews and expert consultations |
| 7 | NO | Asked if patients understood the educational content and addressed their questions |
| 8 | NO | Discussion with fellow medical specialists and patient interviews; revised based on feedback and pilot tests |
| 9 | NO | Collected patient feedback on actual understanding post-education |
| 10 | NO | Used textbooks as backbone; medical websites, pharmaceutical company resources, and foreign hospital materials; did not evaluate appropriateness |
| 11 | NO | Developed and ensured appropriateness through peer review |
| 12 | NO | No systematic appropriateness evaluations; used patient feedback to update materials |
| 13 | NO | Patient interviews to check understanding and ease of following instructions; expert opinions gathered |
| 14 | NO | Developed based on evidence-based medicine; no appropriateness evaluation experience |
| 15 | NO | Needs assessment through patient and internal staff interviews and surveys |
| 16 | NO | Evaluated educational outcomes through patient interviews and keyword comprehension |
| 17 | NO | Real-world usage evaluation |
| 18 | NO | Developed based on personal experience and knowledge |
| 19 | NO | Development: textbook-based disease information, treatment guided by CPG, Q&A from frequently asked questions during consultations. Appropriateness evaluated through patient interviews |
| 20 | NO | Adjusted materials based on foreign resources, ensuring appropriate content level |
ASAS = Assessment of SpondyloArthritis International Society, ACR = American College of Rheumatology, EULAR = European Alliance of Associations for Rheumatology, CPG = clinical practice guidelines.
Delphi study results
Consensus on key domains for ICT-based patient education
In this Delphi study, consensus was reached on several key domains related to ICT-based education for patients with RDs (Table 3). This includes ICT tools and strategies that emphasize the need to continuously encourage health information seeking and self-management, and address RD-specific challenges including symptom variability, medication adherence, and self-monitoring.
Table 3. Delphi survey results of initial domains.
| Domains | Subdomains | Round 1 | Round 2 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean ± SD | IQR | Agreement | CVR | Decisions after Round 1 | Mean ± SD | IQR | Agreement | CVR | |||
| Part 1. Guidelines for patient education content and materials development | |||||||||||
| Health literacy (health information understanding ability) | A1: Concept of health literacy | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.6 ± 0.5 | 1.0 | 1.00 | 1.00 | ||
| A2: Concept of digital health literacy | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.7 ± 0.5 | 1.0 | 1.00 | 1.00 | |||
| A3: Guidelines for explaining professional (medical) terminology | 3.6 ± 0.5 | 1.0 | 1.00 | 1.00 | 3.9 ± 0.4 | 0.0 | 1.00 | 1.00 | |||
| Guidelines for developing existing patient education materials | B1: Overview of patient education | 3.3 ± 0.6 | 1.0 | 0.90 | 0.80 | 3.4 ± 0.7 | 1.0 | 0.90 | 0.80 | ||
| B2: Principles of developing patient education materials | 3.5 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | |||
| B3: Evaluation methods for developing educational materials (focusing on standard tools) | 3.6 ± 0.5 | 1.0 | 1.00 | 1.00 | 3.6 ± 0.5 | 1.0 | 1.00 | 1.00 | |||
| Online | C1: Differences between developing traditional paper materials and web page materials | 3.3 ± 0.6 | 1.0 | 0.85 | 0.80 | 3.2 ± 0.7 | 1.0 | 0.85 | 0.70 | ||
| C2: Examples of effective online educational materials | 3.4 ± 0.8 | 1.0 | 0.85 | 0.70 | 3.3 ± 0.7 | 1.0 | 0.85 | 0.70 | |||
| C3: Considerations for developing online educational materials | 3.7 ± 0.5 | 1.0 | 1.00 | 1.00 | 3.7 ± 0.5 | 1.0 | 1.00 | 1.00 | |||
| Part 2. Patient education utilizing digital tools and interfaces | |||||||||||
| Mobile apps | D1: UI/UX principles for educational material apps | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | ||
| D2: UI/UX design for educational material apps | 3.3 ± 0.6 | 1.0 | 0.90 | 0.80 | 3.4 ± 0.7 | 1.0 | 0.90 | 0.80 | |||
| D3: Considerations for utilizing mobile app educational materials | 3.5 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | |||
| Chatbots | E1: Overview of chatbots (understanding conversational interfaces) | 3.6 ± 0.6 | 1.0 | 0.95 | 0.90 | 3.5 ± 0.7 | 1.0 | 0.90 | 0.80 | ||
| E2: Examples of chatbot utilization in patient education | 3.4 ± 0.7 | 1.0 | 0.90 | 0.80 | 3.4 ± 0.7 | 1.0 | 0.90 | 0.80 | |||
| E3: Considerations for introducing chatbots for patient education purposes | 3.6 ± 0.7 | 1.0 | 0.90 | 0.80 | 3.5 ± 0.5 | 1.0 | 1.00 | 1.00 | |||
| Videos (e.g., YouTube) | F1: Considerations for producing educational videos for patient education | 3.7 ± 0.5 | 1.0 | 1.00 | 1.00 | 3.6 ± 0.5 | 1.0 | 1.00 | 1.00 | ||
| Telemedicine | G1: Concept of telemedicine | 3.1 ± 0.9 | 1.2 | 0.75 | 0.50 | Deleted | |||||
| G2: Strategies for developing effective content | 3.1 ± 0.9 | 1.2 | 0.75 | 0.50 | Deleted | ||||||
| AI-powered platforms (e.g., ChatGPT) | H1: Examples of utilizing AI-powered platforms (e.g., ChatGPT) for patient education | 3.3 ± 0.7 | 1.0 | 0.85 | 0.70 | 3.3 ± 0.7 | 1.0 | 0.85 | 0.70 | ||
| H2: Considerations for utilizing AI-powered platforms (e.g., ChatGPT) for patient education | 3.5 ± 0.7 | 1.0 | 0.90 | 0.80 | 3.4 ± 0.7 | 1.0 | 0.90 | 0.80 | |||
SD = standard deviation, IQR = interquartile range, CVR = content validity ratio, UI = user interface, UX = user experience.
Part 1: guidelines for patient education content and materials development
High consensus was achieved in health literacy and digital health literacy (agreement rates over 95% and CVR between 0.9 and 1.0 in R1), with both subdomains receiving 100% agreement in R2, reflecting their critical importance in developing patient education materials. The requirement for clear guidelines on the explanation of medical terminology achieved complete agreement, emphasizing the need for patients to understand complex medical information.
For online educational materials, a strong consensus was achieved for the development of online educational materials, with broad expert alignment (agreement rates: 85.0–100.0%; IQR: 1.0). CVR was also high, ranging from 0.7 to 1.0, indicating a strong level of agreement regarding the relevance of the items. In the second round, this strong consensus was reaffirmed (agreement rates: 85–90%; IQR: 1.0), highlighting the recognition of online educational materials as critical in patient education.
Part 2: patient education utilizing digital tools and interfaces
Regarding digital tools and interfaces, moderate-to-high consensus was observed for mobile app-based educational materials, particularly in relation to user interface (UI)/user experience (UX) design principles, with agreement rates of approximately 90–95%, highlighting the importance of user-friendly digital tools in digital education. Similarly, the educational videos received strong endorsements with 100% agreement, indicating their value in enhancing patient engagement and comprehension. Chatbots, particularly in terms of conversational interfaces, also achieved high consensus (agreement rates: 90–95%; CVR: 0.8–1.0), demonstrating their value in digital patient education strategies. Although telemedicine was excluded after R1 owing to insufficient consensus (IQR: 1.2; agreement rate: 75%; CVR: 0.5), the use of chatbots and emerging AI-powered platforms (e.g., ChatGPT) achieved moderate-to-high consensus (agreement rates: 85–90%; CVR: 0.7–0.8), suggesting their potential as innovative educational tools in the future.
Key subdomains and items forming the foundation of guidelines
Table 4 presents the key subdomains and items that form the foundation of the guidelines aimed at assisting healthcare providers in ICT development.
Table 4. Preliminary guideline components by domain and subdomain: Delphi survey results.
| Subdomains | Items | Mean ± SD | IQR | Agreement | CVR | |
|---|---|---|---|---|---|---|
| Part 1. Guidelines for patient education content and materials development | ||||||
| A1: Concept of health literacy | 1. Definition of health literacy | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | |
| 2. Importance of health literacy | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| 3. Principles of patient education considering health literacy | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | ||
| A2: Concept of digital health literacy | 4. Definition of digital health literacy | 3.6 ± 0.5 | 1.0 | 1.0 | 1.0 | |
| 5. Importance of digital health literacy | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | ||
| 6. Introduction to digital health literacy assessment tools and methods of utilization | 3.8 ± 0.4 | 0.0 | 1.0 | 1.0 | ||
| A3: Guidelines for explaining professional (medical) terminology | 7. Introduction to easy-to-read materials (plain language) | 3.9 ± 0.4 | 0.0 | 1.0 | 1.0 | |
| 8. Principles of developing easy-to-read materials | 3.9 ± 0.4 | 0.0 | 1.0 | 1.0 | ||
| 9. Examples of easy-to-read material development (medical terminology, numbers, dates, measurements) | 3.7 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| B1: Overview of patient education | 10. Importance of patient education | 3.5 ± 0.6 | 1.0 | 1.0 | 0.9 | |
| 11. Goals of patient education | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| B2: Principles of developing patient education materials | 12. Core elements of health information: 3As (accurate, accessible, and actionable) | 3.6 ± 0.5 | 1.0 | 1.0 | 1.0 | |
| 13. Principles of developing patient education materials | 3.8 ± 0.4 | 0.0 | 1.0 | 1.0 | ||
| B3: Evaluation methods for developing educational materials | 14. Suitability assessment of materials evaluation method | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | |
| 15. U.S. Centers for Disease Control and Prevention’s clear communication index | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | ||
| C1: Differences between developing traditional paper materials and web page materials | 16. Three important elements of online web design | 3.2 ± 0.8 | 1.0 | 0.8 | 0.6 | |
| 17. Characteristics of online educational materials | 3.3 ± 0.7 | 1.0 | 0.9 | 0.8 | ||
| C2: Examples of effective online educational materials | 18. Design principles for developing online websites | 3.3 ± 0.8 | 1.0 | 0.8 | 0.6 | |
| 19. Examples of effective online patient education materials | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| C3: Considerations for developing online educational materials | 20. Guidelines for writing online content | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | |
| 21. Health literacy strategies for developing webpage content | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| Part 2. Patient education utilizing digital tools and interfaces | ||||||
| D1: UI/UX principles for educational material apps | 22. Overview of the patient education app development process | 3.3 ± 0.7 | 1.0 | 0.9 | 0.7 | |
| 23. Considerations for developing patient education apps (e.g., costs) | 3.3 ± 0.7 | 1.0 | 0.9 | 0.7 | ||
| D2: UI/UX design for educational material apps | 24. UI/UX guidelines for underserved populations (e.g., seniors) | 3.6 ± 0.7 | 1.0 | 0.9 | 0.8 | |
| 25. Examples of effective patient education apps | 3.4 ± 0.7 | 1.0 | 0.9 | 0.8 | ||
| D3: Considerations for utilizing mobile app educational materials | 26. Guidelines for writing mobile app content | 3.7 ± 0.6 | 1.0 | 1.0 | 0.9 | |
| 27. Health literacy strategies for developing mobile app content | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | ||
| E1: Overview of chatbots (understanding conversational interfaces) | 28. Definition of chatbots and the need for conversational interfaces in patient education | 3.4 ± 0.7 | 1.0 | 0.9 | 0.8 | |
| E2: Examples of chatbot utilization in patient education | 29. Examples of patient education materials using chatbots | 3.3 ± 0.7 | 1.0 | 0.9 | 0.7 | |
| E3: Considerations for introducing chatbots for patient education purposes | 30. Health literacy strategies for developing patient education chatbots | 3.5 ± 0.6 | 1.0 | 1.0 | 0.9 | |
| 31. UX writing for patient education | 3.4 ± 0.7 | 1.0 | 0.9 | 0.8 | ||
| F1: Considerations for producing educational videos for patient education | 32. Technical considerations for creating patient education videos | 3.6 ± 0.6 | 1.0 | 1.0 | 0.9 | |
| 33. Health literacy strategies for developing patient education video content | 3.7 ± 0.5 | 1.0 | 1.0 | 1.0 | ||
| H1: Examples of utilizing AI-powered platforms (e.g., ChatGPT) for patient education | 34. Examples of patient education using AI-powered platforms (e.g., ChatGPT) | 3.3 ± 0.7 | 1.0 | 0.9 | 0.7 | |
| H2: Considerations for utilizing AI-powered platforms (e.g., ChatGPT) for patient education | 35. Health literacy strategies for utilizing AI-powered platforms (e.g., ChatGPT) | 3.4 ± 0.7 | 1.0 | 0.9 | 0.8 | |
SD = standard deviation, IQR = interquartile range, CVR = content validity ratio, UI = user interface, UX = user experience.
Part 1: guidelines for patient education content and materials development
A high consensus was achieved across most subdomains for patient education content and material development. In the area of health literacy, 100% strong agreement was observed on defining health literacy and its importance, with high CVR values of 1.0 and 0.9, respectively. A similarly strong consensus was observed for all items of digital health literacy (IQR: 0.0; agreement rate: 100%; CVR: 1.0), indicating the importance of integrating digital health literacy into patient education. Additionally, 100% agreement was achieved for all items regarding the need for clear guidelines for explaining professional medical terminology. Considerations for writing online content and health literacy strategies for web-based content development both received moderate-to-high consensus (agreement rates: 80–100%; CVR: 0.6–1.0).
Part 2: patient education utilizing digital tools and interfaces
For mobile app-based educational materials, the consensus for all items was strong (agreement rates: 90–100%; CVR: 0.7–0.9), particularly in relation to UI/UX design principles, reflecting the importance of user-centered design in digital patient education tools. Chatbot utilization and considerations for producing educational videos for patient education showed high consensus on the need for conversational interfaces (agreement rates: 90–100%; CVR: 0.7–1.0), with strong support for the use of health literacy strategies in chatbot design. Finally, the subdomain utilizing AI-powered platforms (e.g., ChatGPT) for patient education showed strong agreement.
DISCUSSION
In this study, we explored the most important and urgent components for developing practical guidelines for healthcare professionals to design ICT-based patient education tailored to the specific needs of patients with RDs in the Republic of Korea, encompassing 7 domains, 18 subdomains, and 35 items. A modified Delphi survey was conducted on the components of the guideline with a panel of experts in rheumatology, patient education, and digital health literacy. Through a consensus process, the subdomains and items were formulated to be practical and feasible for providing evidence-based patient education and refining a more tailored approach for RDs, all within the context of digital health and health literacy.
This study provides an overview of the current guidelines for healthcare professionals in the development of patient education, particularly in the field of RDs. Most healthcare professionals developed educational materials without formal guidelines. This practice aligns with those of previous studies indicating that clinicians often lack the time and resources for comprehensive patient education.22,23 However, owing to the chronic nature of their condition, patients must manage their health daily and actively participate in healthcare to ensure healthy behaviors and day-to-day functioning over time. International guidelines emphasize24 that informed patient participation is crucial to ensure that decisions align with accurate information and individual needs. Hence, future approaches should focus on providing accessible and user-friendly educational resources. In this context, ICT-based tools present an opportunity to deliver continuous and tailored information without placing an additional burden on healthcare providers. We believe that our guidelines may be clinically significant because they address both the accessibility and practicality of ICT for enhancing patient education and engagement.
In this study, we integrate various innovative digital tools, including mobile applications, chatbots, YouTube videos, and AI-powered platforms such as ChatGPT, into the guideline subdomains. Previous studies have largely focused on the readability of written online patient education resources and video quality.25,26,27 Although existing evaluation criteria primarily emphasize content quality, reliability, and balance of information, they do not fully capture the diverse demands of ICT-based patient education platforms, including mobile applications, chatbots, and interactive digital environments. Our study addresses a broader range of digital platforms, underscoring the need for practical user-centered guidelines, including user engagement, tailored education delivery, and real-time feedback to enhance patient engagement and optimize the use of diverse digital tools. As ICT tools aim to offer personalized, patient-focused learning, we ensured that these critical adaptive components were reflected in the evaluation process.
The highest level of expert consensus emerged from the need for clear definitions of health literacy and digital health literacy, especially for strong technical support for mobile UX, UI, and YouTube. These findings align with those of previous studies that have highlighted mobile apps and YouTube as key platforms for patient education.28,29,30 However, a review of mobile apps for RDs has revealed deficiencies in the UX/UI, including limited UX, poor alignment with clinical guidelines, and inadequate tools for disease tracking and management.29 Although YouTube video content can effectively convey complex medical information, enhance patient understanding, and improve adherence to treatment plans, the absence of standardized guidelines for developing video-based educational materials limits its integration into clinical practice. The strong consensus among experts regarding the need for video content guidelines highlights a critical area for improvement in clinical practice. In addition, while healthcare professionals play a central role in patient education, specialized teams within hospitals often manage the development and editing of YouTube videos. These teams handle critical aspects, including content tailoring, optimizing video length, and ensuring patient-friendly language. Expanding the future guidelines to include these teams and other healthcare workers involved in content creation could enhance the consistency and effectiveness of video-based patient education materials. This emphasizes the urgent need for standardized guidelines to ensure the proper use of YouTube by patients and promote the development of appropriate content.
Conversely, we excluded the telehealth subdomain after the first round owing to a low level of expert consensus. Although previous studies have highlighted telehealth as an effective tool for managing RDs, offering benefits such as enhanced access to care and real-time monitoring,31 the Republic of Korea’s adoption rate is noticeably lower than that of other countries.32 Legal and regulatory barriers along with concerns over accountability in telehealth may be the primary reasons. These factors may have contributed to low agreement regarding telehealth in the Delphi study. Nevertheless, as the demand for telehealth increases in the Republic of Korea, particularly in chronic disease management, future research should focus on overcoming these challenges. Exploring how telehealth can be integrated into patient education strategies for RDs to meet the needs of both patients and healthcare professionals better is also vital.
This study had several limitations. First, most participating experts were affiliated with urban areas and university hospitals, which may limit the generalizability of the results to the broader healthcare context in the Republic of Korea. However, considering that smaller hospitals in rural areas are likely to face greater challenges in implementing ICT-based patient education, the insights provided by this study are valuable. In addition, rural populations face challenges such as lower digital literacy, limited internet access, and fewer healthcare resources, which may worsen disparities in ICT-based education. Future guidelines should address these issues by improving digital literacy, enhancing infrastructure, and providing tailored support. Second, the ICT tools and platforms included in the guidelines may be subject to rapid technological advancements, potentially limiting their long-term applicability. Nevertheless, we focused on platforms that were highly relevant in clinical practice and widely used by patients. Nonetheless, considering the rapid pace of technological advancement, it is crucial to ensure that these guidelines remain relevant and up to date. Establishing a regular review cycle (e.g., every two to three years) would allow for timely updates based on new technologies and practices. Future studies should explore strategies including incorporating a flexible framework to accommodate emerging digital tools and platforms, which could enhance adaptability and sustain the efficacy of the guidelines in improving patient education outcomes. Third, although patient perspectives were critical, the differing methodologies required to integrate patient and expert viewpoints led to adoption of the Delphi method, which focused on achieving expert consensus. This methodological choice represents a limitation, as patient perspectives are thus not included in this study.
Finally, an inherent limitation of the Delphi methodology is the potential for consensus bias in which minority opinions may be overshadowed by majority views. Although we sought to address this issue by involving a diverse expert panel, this remains a challenge. Nonetheless, our findings provide a strong basis for future research and the practical applications of ICT-based patient education in rheumatology.
In conclusion, this study establishes a strong foundation for developing ICT-based patient education guidelines tailored for RDs. By integrating digital tools such as mobile apps, chatbots, and video platforms, along with the principles of digital health literacy, these guidelines may enhance patient engagement and clinical outcomes. However, legal, regulatory, and technological challenges remain, and further research is required to explore the role of various digital tools and address the barriers to their implementation. Ensuring the effective integration of these tools into clinical practice is essential for maximizing their impact on patient education and disease management.
Footnotes
Funding: This research was supported by a grant of the Korea Health Technology R&D Project through the Patient-Doctor Shared Decision Making Research center (PDSDM), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI23C1762).
Disclosure: The authors have no potential conflicts of interest to disclose.
- Conceptualization: Yoon J, Cho SK, Sung YK.
- Data curation: Yoon J.
- Formal analysis: Yoon J, Lee SB.
- Funding acquisition: Sung YK.
- Investigation: Sung YK.
- Methodology: Yoon J, Cho SK, Choi SR, Lee SB, Cho J, Jeon CH, Kim GT, Lee J, Sung YK.
- Software: Yoon J, Lee SB.
- Visualization: Yoon J, Lee SB, Cho SK.
- Writing - original draft: Yoon J, Cho SK, Lee SB, Sung YK.
- Writing - review & editing: Yoon J, Cho SK, Choi SR, Lee SB, Cho J, Jeon CH, Kim GT, Lee J, Sung YK.
SUPPLEMENTARY MATERIAL
Details of information on experiences of experts (N = 20)
References
- 1.Smolen JS, Aletaha D, Barton A, Burmester GR, Emery P, Firestein GS, et al. Rheumatoid arthritis. Nat Rev Dis Primers. 2018;4:18001. doi: 10.1038/nrdp.2018.1. [DOI] [PubMed] [Google Scholar]
- 2.Goldblatt F, O’Neill SG. Clinical aspects of autoimmune rheumatic diseases. Lancet. 2013;382(9894):797–808. doi: 10.1016/S0140-6736(13)61499-3. [DOI] [PubMed] [Google Scholar]
- 3.Bokhari SF, Mushtaq A. Psychosocial aspects of rheumatic disease management: addressing mental health and well-being. Cureus. 2023;15(11):e49267. doi: 10.7759/cureus.49267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nikiphorou E, Santos EJ, Marques A, Böhm P, Bijlsma JW, Daien CI, et al. 2021 EULAR recommendations for the implementation of self-management strategies in patients with inflammatory arthritis. Ann Rheum Dis. 2021;80(10):1278–1285. doi: 10.1136/annrheumdis-2021-220249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Been-Dahmen JM, Walter MJ, Dwarswaard J, Hazes JM, van Staa A, Ista E. What support is needed to self-manage a rheumatic disorder: a qualitative study. BMC Musculoskelet Disord. 2017;18(1):84. doi: 10.1186/s12891-017-1440-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wu Z, Zhu Y, Wang Y, Zhou R, Ye X, Chen Z, et al. The effects of patient education on psychological status and clinical outcomes in rheumatoid arthritis: a systematic review and meta-analysis. Front Psychiatry. 2022;13:848427. doi: 10.3389/fpsyt.2022.848427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Senara SH, Abdel Wahed WY, Mabrouk SE. Importance of patient education in management of patients with rheumatoid arthritis: an intervention study. Egypt Rheumatol Rehabil. 2018;46:42-7. [Google Scholar]
- 8.Marques A, Santos E, Nikiphorou E, Bosworth A, Carmona L. Effectiveness of self-management interventions in inflammatory arthritis: a systematic review informing the 2021 EULAR recommendations for the implementation of self-management strategies in patients with inflammatory arthritis. RMD Open. 2021;7(2):e001647. doi: 10.1136/rmdopen-2021-001647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kim HA, Kim MG. A survey study on rheumatologist consultation time in Korean hospitals. J Rheum Dis. 2020;27(1):45–50. [Google Scholar]
- 10.Tahsin F, Armas A, Kirakalaprathapan A, Kadu M, Sritharan J, Steele Gray C. Information and communications technologies enabling integrated primary care for patients with complex care needs: scoping review. J Med Internet Res. 2023;25:e44035. doi: 10.2196/44035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Randine P, Sharma A, Hartvigsen G, Johansen HD, Årsand E. Information and communication technology-based interventions for chronic diseases consultation: Scoping review. Int J Med Inform. 2022;163:104784. doi: 10.1016/j.ijmedinf.2022.104784. [DOI] [PubMed] [Google Scholar]
- 12.Mollard E, Pedro S, Schumacher R, Michaud K. Smartphone-based behavioral monitoring and patient-reported outcomes in adults with rheumatic and musculoskeletal disease. BMC Musculoskelet Disord. 2022;23(1):566. doi: 10.1186/s12891-022-05520-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ownby RL, Simonson M, Caballero J, Thomas-Purcell K, Davenport R, Purcell D, et al. A mobile app for chronic disease self-management for individuals with low health literacy: a multisite randomized controlled clinical trial. J Ageing Longev. 2024;4(2):51–71. doi: 10.3390/jal4020005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.National Evidence-based Healthcare Collaborating Agency (NECA) Evidence-Based Study on the Clinical Effectiveness of Patient Education in Rheumatoid Arthritis. Seoul, Korea: NECA; 2016. [Google Scholar]
- 15.Seo MR, Kim G, Moon KW, Sung YK, Yoo JJ, Yoon CH, et al. Quality indicators for evaluating the health care of patients with rheumatoid arthritis: a Korean expert consensus. J Korean Med Sci. 2021;36(17):e109. doi: 10.3346/jkms.2021.36.e109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Centers for Disease Control and Prevention. Health literacy Develop & Test Materials. [Updated 2024]. [Accessed April 21, 2024]. https://www.cdc.gov/healthliteracy/developmaterials/index.html .
- 17.NHS England. Digital Resources, standards and guidance. [Updated 2024]. [Accessed April 21, 2024]. https://digital.nhs.uk/about-nhs-digital/technology-suppliers/resources-standards-and-guidance .
- 18.Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol. 2021;11(4):116–129. doi: 10.5662/wjm.v11.i4.116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. doi: 10.1177/0269216317690685. [DOI] [PubMed] [Google Scholar]
- 20.Yoon J, Lee SB, Cho SK, Sung YK. Information and communication technology-based patient education for autoimmune inflammatory rheumatic diseases: a scoping review. Semin Arthritis Rheum. 2024;69:152575. doi: 10.1016/j.semarthrit.2024.152575. [DOI] [PubMed] [Google Scholar]
- 21.Spranger J, Homberg A, Sonnberger M, Niederberger M. Reporting guidelines for Delphi techniques in health sciences: a methodological review. Z Evid Fortbild Qual Gesundhwes. 2022;172:1–11. doi: 10.1016/j.zefq.2022.04.025. [DOI] [PubMed] [Google Scholar]
- 22.Claassen A. Strategies for Patient Education in Rheumatic Diseases. Nijmegen, Netherlands: Sint Maartenskliniek; 2019. [Google Scholar]
- 23.Mardon AK, Leake HB, Szeto K, Moseley GL, Chalmers KJ. Recommendations for patient education in the management of persistent pelvic pain: a systematic review of clinical practice guidelines. Pain. 2024;165(6):1207–1216. doi: 10.1097/j.pain.0000000000003137. [DOI] [PubMed] [Google Scholar]
- 24.Zangi HA, Ndosi M, Adams J, Andersen L, Bode C, Boström C, et al. EULAR recommendations for patient education for people with inflammatory arthritis. Ann Rheum Dis. 2015;74(6):954–962. doi: 10.1136/annrheumdis-2014-206807. [DOI] [PubMed] [Google Scholar]
- 25.Prabhu AV, Hansberry DR, Agarwal N, Clump DA, Heron DE. Radiation oncology and online patient education materials: deviating from NIH and AMA recommendations. Int J Radiat Oncol Biol Phys. 2016;96(3):521–528. doi: 10.1016/j.ijrobp.2016.06.2449. [DOI] [PubMed] [Google Scholar]
- 26.Kahlam JS, Sacher A, Singh C, Lo DF, Reilly JP. Assessment of YouTube videos in patient education for coronary artery disease: a DISCERN-based cross-sectional analysis. Cureus. 2024;16(6):e62986. doi: 10.7759/cureus.62986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Koo BS, Kim D, Jun JB. Reliability and quality of Korean YouTube videos for education regarding gout. J Korean Med Sci. 2021;36(45):e303. doi: 10.3346/jkms.2021.36.e303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Lopez-Olivo MA, Lin H, Rizvi T, Barbo Barthel A, Ingleshwar A, des Bordes JK, et al. Randomized controlled trial of patient education tools for patients with rheumatoid arthritis. Arthritis Care Res (Hoboken) 2021;73(10):1470–1478. doi: 10.1002/acr.24362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cozad MJ, Crum M, Tyson H, Fleming PR, Stratton J, Kennedy AB, et al. Mobile health apps for patient-centered care: review of United States rheumatoid arthritis apps for engagement and activation. JMIR Mhealth Uhealth. 2022;10(12):e39881. doi: 10.2196/39881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Madathil KC, Rivera-Rodriguez AJ, Greenstein JS, Gramopadhye AK. Healthcare information on YouTube: a systematic review. Health Informatics J. 2015;21(3):173–194. doi: 10.1177/1460458213512220. [DOI] [PubMed] [Google Scholar]
- 31.Jackson LE, Edgil TA, Hill B, Owensby JK, Smith CH, Singh JA, et al. Telemedicine in rheumatology care: a systematic review. Semin Arthritis Rheum. 2022;56:152045. doi: 10.1016/j.semarthrit.2022.152045. [DOI] [PubMed] [Google Scholar]
- 32.Kim HS. Towards telemedicine adoption in Korea: 10 practical recommendations for physicians. J Korean Med Sci. 2021;36(17):e103. doi: 10.3346/jkms.2021.36.e103. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Details of information on experiences of experts (N = 20)