User-testing of a combined dual-mode educational approach for chronic obstructive pulmonary disease

Abstract

Background

Individualized patient education, continuous monitoring and regular follow ups are very essential for Chronic Obstructive Pulmonary Disease (COPD) management. Educational interventions can improve patient knowledge and self-management skills. Current study aims to evaluate the effectiveness of a combined dual-mode educational approach developed for patients with COPD, in improving knowledge and assessing the acceptability of the educational materials.

Methodology

A single-group pre-test post-test quasi-experimental study was conducted for six months duration in a tertiary care hospital. Initially, reliability assessment of the questionnaires was performed using a test-retest method. Afterwards, patient knowledge was assessed using pre- and post- intervention method to evaluate the effectiveness of the materials in improving patient knowledge. Finally, user-opinion analysis was carried out to attain feedback from the patients.

Results

A total of 36 patients were enrolled for the study. The reliability of the questionnaires was established using Intraclass Correlation coefficient (ICC) and it scored between 0.88 and 0.98 fulfilling the reliability criteria. During the knowledge assessment following the educational interventions showed an improvement in the patient knowledge and attained a statistical significance throughout all the language versions (p < 0.05). The user-opinion analysis attained high acceptance from the participants and majority of them (88%) gave “Good” feedback for the design, layout and clarity of the educational materials.

Conclusion

The user- testing analysis proved that the combined dual-mode educational approach was well accepted by the patients and it significantly improved their knowledge on COPD.

Background

Chronic Obstructive Pulmonary Disease (COPD) is a progressive condition distinguished by chronic airflow limitation and with symptoms such as shortness of breath, severe cough and sputum production.¹ It is one of the leading causes of death worldwide, where it was reported largely in the lower- and middle-income countries. COPD affects gender differently in terms of global prevalence, i.e., 15.7% in men and 9.9% in women.² In 2023, around 3.4 million deaths occurred due to COPD and it is responsible for approximately 6% deaths globally.³ Appropriate management of COPD mainly depends on adherence to pharmacological and non-pharmacological treatments, including proper inhaler technique.4., 5., 6. Lack of knowledge regarding the disease, medicines and poor compliance with the treatment are the main factors that lead to poor disease control, frequent hospital admissions and exacerbation among COPD patients.⁷ Hence, frequent monitoring of inhaler uses and medication adherence is important for achieving a positive clinical outcome. Furthermore, The Global Initiative for Chronic Obstructive Pulmonary Disease (GOLD) 2025 recommendations state that individualized patient education, continuous monitoring of inhaler techniques and follow-up are required to be incorporated into the COPD management.⁸

Patient education is an essential component of COPD management. Previous studies have demonstrated that various educational approaches including structured written education, multimedia and virtually narrated video educations attained a positive effect on COPD-related knowledge and inhaler technique knowledge.9., 10., 11. These studies have primarily employed single-mode of education and their findings indicate that written and visual methods of education can serve complementary roles, where the written materials can be used by the patients for recalling the information according to their convenience and video can provide step-wise demonstration of inhaler techniques for enhanced understanding. In prior COPD research utilizing leaflet as an educational intervention provided a quick summary of the disease and medication management.¹² Whereas the booklet provides an in-detailed information including disease, medication, lifestyle modifications and inhaler techniques. The inhaler use involves a sequence of practical and coordinated steps that may be difficult for an elderly patient to follow. Prior research reveals that inhaler technique errors are highly prevalent among this population with COPD due to lack of inhaler training.¹³ Therefore, the video was incorporated as a complementary educational modality to address this issue. The educational video visually demonstrates the sequence, technique, timing, and coordination required for inhaler use, thereby providing patients with a clear and practical demonstration of the correct procedure rather than relying solely on written descriptions. Thus, the combined dual-mode educational approach was intended not to overcome an inability to understand written information, but to complement knowledge acquired through reading with visual and practical demonstration.

As medication experts, Pharmacist plays an important role in providing individualized patient counselling, promote treatment adherence and helps in self-management practices. Due to complexity of the disease and overcrowding in the out-patient departments, providing individualized counselling and monitoring would be challenging to the physicians.14., 15. Therefore, pharmacists can be considered as highly accessible health care workers for providing medication counselling, regular follow ups and can contribute in improving overall health outcomes.16., 17., 18. In recent years, pharmacist-led interventions such as personalized patient counselling, self-management practices and use of educational materials has showed improvement in medication adherence, symptom management, disease-related knowledge and overall quality of life among patients with certain chronic conditions like diabetes mellitus and hypertension.19., 20. These findings show the significant role of pharmacists in providing structured patient education however, the educational materials developed for COPD requires evaluation and feedback from the patients rather than scientific accuracy. Therefore, a pharmacist-led combined dual-mode educational approach, including a patient education booklet and a health education video was developed. Hence, the current study aims to evaluate the educational materials for better acceptability before implementing it in a larger population.²¹

Materials and methods

Study design, duration and site

The research was carried out in a tertiary care hospital from September 2025 to March 2026 using a single-group pre-test post-test quasi-experimental study. The patients who were diagnosed with COPD and hospitalized in the pulmonary medicine department were included. Those who were able to read and understand any of the language which includes English, Malayalam and Kannada were enrolled in the study. Patients with limited literacy levels and those who were not willing to participate in the study were excluded. Ethical approval was obtained before initiating the study and informed consent was collected from the participants.

Sample size

The sample size was determined based on the commonly used minimum standard size of 30 participants for user testing. To provide a margin above this minimum and to represent the three language versions, a total of 36 participants were included.22., 23. For test–retest reliability, a subsample of 18 participants (n = 6 for each language) from the total sample of 36 was selected to represent of all three language versions.

Reliability of the questionnaires

Two separate questionnaires for both booklet and video each containing 10 questions were developed based on the original versions of the educational materials. The questionnaires were created for assessing the patient's knowledge on COPD and to evaluate the effectiveness of the educational booklet and health educational video. The questions included were about disease, management, lifestyle modifications and inhaler techniques. A panel of ten members were identified for content validation to evaluate the relevancy and clarity of the questionnaires. They obtained scores of >0.8 for relevance and clarity, satisfying the content validity criteria. The evaluated questionnaires were then translated into two regional languages (Kannada and Malayalam). It was again back translated into the source language (English) to maintain a semantic and conceptual equivalence.

The questionnaires were subjected to test-retest reliability, with a seven days gap between the two administrations. The same questionnaires were administered to the patients at two different points to measure the consistency of responses over time.24., 25. If the responses show consistency across two administrations, it demonstrates that the questionnaires have acceptable reliability. No COPD-related education was given during the interval between test and retest.

Assessment of knowledge following the educational interventions

For the purpose of assessing the knowledge following the educational intervention a booklet was developed and validated that provides detailed knowledge on the disease, inhaler techniques and their cleaning methods, lifestyle modifications such as diet, sleep and exercise, pranayama practices, pacing, relaxed positions, preventive measures and a self-monitoring action plan. A validated video was also employed to demonstrate the correct use of inhalers and nebulizers, including their cleaning procedures, through self-demonstration to improve patient knowledge and proper inhaler technique.

A single group pre-test post-test method was used for evaluating the effectiveness of combined educational approach consisting of booklet and video. The pre-test was performed to assess the initial knowledge of the patients using the questionnaires and their responses were analysed. Simultaneously, the patients were given with the booklet to read and understand according to their language preferences. Sequentially, the patients were provided with health education video in their preferred language, which provides step-wise demonstration on inhaler and nebulizer techniques. Finally, a post-test was carried out on the same day using the questionnaires to reassess the change in COPD-related knowledge of the patients following the combined educational intervention.

The percentage of change in knowledge was calculated as follows:

$\text{Knowledge score}=\text{Total number of correct responses}/\text{Total number of responses}\times 100$

User opinion analysis

After the completion of the readability assessment, the patients were given with a user-opinion questionnaire for evaluating their feedbacks about the booklet and the video. The questionnaire contained four questions based on the content, layout and design of the booklet. The scoring of the instrument ranges from 4 to 20. And the educational materials were scored as good (14–20), average (9–13), poor (<9).²⁶

Statistical analysis

Categorical variables were expressed in frequency and percentage. The Continuous variables were stated in mean ± standard deviation (SD). For evaluating the knowledge following the educational interventions paired-t-test was performed using SPSS version 29 and a ‘p’ value less than 0.05 is considered as statistically significant. Reliability of the questionnaire was measured using Intraclass Correlation Coefficient (ICC). It is a statistical measure of consistency and the ICC values ranges between 0 and 1. The values less than 0.5 is denoted as poor reliability, 0.5 to 0.7 is indicated as moderate reliability, 0.8 to 0.9 is considered as good reliability and > 0.9 is excellent reliability.²⁷

Results

Among 36 patients enrolled for readability testing, 75% were males and 25% females, and the mean age was founded to be 64.44 ± 10.0 years. Majority of the population had a middle school qualification (72.2%). More than half of the patients had been living with the illness for above 5 years and majority of them were having co-morbidities. The Data is given in Table 1.

Table 1.

Demographics of the participants for readability testing:

Demographics	Frequency(n)	Percentage (%)
Age (in years)
<50	4	11.1
50–60	5	13.9
61–70	15	41.7
71–80	12	33.3
Gender
Male	27	75.0
Female	9	25.0
Educational status
Middle school (6–8)	26	72.2
High school (9–10)	4	11.1
Higher secondary (11−12)	6	16.7
Occupation
Unemployed	19	52.8
Employed	17	47.2
Language choice
Kannada	14	38.9
Malayalam	12	33.3
English	10	27.8
Comorbidities
Comorbidities absent	2	5.6
Comorbidities present	34	94.4
Social history
smoker	20	55.5
Non-smoker	16	44.4
Duration of illness
<5 years	17	47.2
≥5 years	19	52.8

To assess the reliability of the user-testing questionnaire, 18 patients (subgroup of 36) were chosen and assessed for Kannada(n = 6), Malayalam(n = 6) and English languages(n = 6). The Intraclass Correlation Coefficient (ICC) was used to measure the reliability of questionnaires by analysing the consistency of scores between two administrations. The ICC values obtained for the questionnaires of Kannada and Malayalam versions of booklet was 0.88 denoting good reliability, whereas the English version attained an excellent reliability score of 0.98. In the case of video, all the language versions obtained an ICC score of >0.9 showing excellent reliability. The data is summarized in the Table 2.

Table 2.

Reliability of user-testing questionnaire:

Type of questionnaire	Test (mean ± SD)	Re-test (mean ± SD	Mean difference ± SD (range)	95% CI	ICC value	p value
Booklet
Kannada (n = 6)	68.33 ± 19.40	65.00 ± 15.16	3.33 ± 4.71 (10,0)	0.20–0.98	0.88	0.17
Malayalam (n = 6)	75.00 ± 10.48	70.00 ± 6.32	5.00 ± 5.00 (10,0)	0.20–0.98	0.88	0.07
English (n = 6)	60.00 ± 15.49	58.33 ± 17.22	1.67 ± 3.72 (10,0)	0.78–0.99	0.98	0.36
Video
Kannada (n = 6)	61.66 ± 18.34	58.33 ± 17.22	3.33 ± 7.45 (10,0)	0.82–0.99	0.94	0.36
Malayalam (n = 6)	46.66 ± 10.32	45.00 ± 10.48	1.66 ± 3.72 (10,0)	0.71–0.99	0.96	0.36
English (n = 6)	58.33 ± 22.28	60.00 ± 20.97	1.67 ± 8.97 (20,0)	0.90–0.99	0.94	0.69

SD: Standard Deviation; ICC: Intraclass Correlation Coefficient; CI: Confidence Interval.

To assess the knowledge following the educational intervention, 14 patients were opted for Kannada, 12 for Malayalam and remaining 10 selected English language versions of the user testing questionnaires. After reading the booklet and understanding the contents of the video, the knowledge of the patients was showed a significant improvement in their mean scorings. The findings were statistically significant with a p < 0.05, indicating that the educational intervention improved the COPD-related knowledge of the patients. Detailed user-testing scores are presented in Table 3.

Table 3.

Evaluation of knowledge following the educational intervention:

Type of questionnaire	Pre-test (mean ± SD)	Post-test (mean ± SD)	Mean difference ± SD (range)	p value
Booklet
Kannada (n = 14)	50.00 ± 14.14	72.14 ± 11.45	22.14 ± 8.60 (40,10)	<0.001*
Malayalam (n = 12)	54.16 ± 11.64	76.66 ± 9.84	22.5 ± 10.89 (40,30)	<0.001*
English (n = 10)	50.00 ± 14.90	82.00 ± 9.18	32.0 ± 14.00 (50,10)	<0.001*
Video
Kannada (n = 14)	39.28 ± 9.60	61.42 ± 9.14	22.14 ± 12.20 (50,0)	<0.001*
Malayalam (n = 12)	40.83 ± 13.11	59.16 ± 13.78	18.33 ± 15.72 (50,0)	<0.001*
English (n = 10)	41.00 ± 14.49	78.00 ± 11.35	37.00 ± 14.17 (60,20)	<0.001*

SD: Standard Deviation; * denotes that p value <0.05, statistically significant; Statistical test used was paired t-test.

The user-opinion analysis found that both the educational materials were easy to understand, appropriate and informative for patients. And also, for the content, design and layout a strong level of acceptance was noticed among the patients. An overview of the user-opinion analysis is provided in Table 4.

Table 4.

User opinion analysis:

Language	Rating	No. of participants
Kannada (14)	Good	11 (78.6)
	Average	3 (21.4)
	Poor	0 (0)
Malayalam (12)	Good	12 (100)
	Average	0 (0)
	Poor	0 (0)
English (10)	Good	9 (90)
	Average	1 (10)
	Poor	0 (0)
Overall (36)	Good	32(88.9)
	Average	4 (11.1)
	Poor	0

Discussion

The rationale of the study was to evaluate the understandability and acceptability of the combined dual-mode educational intervention for COPD patients. Educational materials should be simple, clear and feasible to the patients, to achieve the best outcomes. Such a combined approach using booklet and video was chosen for the current research, can significantly improve patient knowledge and understanding. To evaluate whether the developed educational materials are suitable for the patients with COPD, the process of User testing is being employed. It is a critical process to confirm that educational interventions are practically acceptable before using it in a larger population.21., 28., 29.

The validated questionnaires for user testing were assessed for reliability using a measure of uniformity called ICC. The findings suggest that the questionnaires used in the study gives a consistent and reliable results denoting that it is appropriate for using in patients. Similar results were obtained in previous studies, where their findings showed acceptable reliability scorings for the developed questionnaire among patients with other conditions.25., 26., 30., 31. After reliability testing the educational materials were undergone knowledge assessment using pre-test/post-test method. In all the language categories, the post-test scored higher that the pre-test, showing that the educational materials attained valuable improvement in the patient knowledge following the combined educational the intervention. The result is supported by a previous study saying that structured educational programmes can improve patient knowledge and quality of life.32., 33.

Finally, User opinion analysis was conducted and attained high acceptance from most of the participants. The majority of them rated the clarity, design and understandability as “Good,” denoting that educational materials are legible, user-friendly and improve patient understanding to recall the health information. These findings are comparable with the study conducted by Mateti et al., who concluded that user opinion analysis is important in verifying the educational materials are acceptable and practical to use in everyday clinical practice.²⁴

Limitations

The study has few limitations while interpreting the results that is to be considered. The sample size is relatively small and it was performed in a single centre that can affect the generalizability and applicability of the findings to patients in other healthcare settings. As well as the study participants were the patients those who were hospitalized with COPD, they can differ from outpatients and other stable patients. The patients who cannot read and understand English, Malayalam and Kannada along with illiterate patients were excluded from the study. For this reason, the findings are not generalizable for some populations with low literacy levels and those who speak other languages. The video was administered after the booklet that may influence the prior booklet exposure. Patients chose their own language version rather than being randomized, so there is a chance for potential bias. Further studies including larger sample size, multicentred and broad population along with patient having different literacy levels are necessary for further evaluation of the effectiveness, understandability and acceptability of the combined dual-mode educational intervention.

Conclusion

The user-testing analysis proved that there was a significant improvement in the knowledge of the patients after using the patient education booklet and video. In addition, it also helped to verify that the information provided is accurate and appropriate for the study population. Hence, currently developed patient education materials can be used as a reliable tool for educating COPD.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the author(s) used ChatGPT in order to check synonyms and grammar. After using this tool, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

CRediT authorship contribution statement

Milagrin Xavier: Writing – review & editing, Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. B.H. Giridhar: Writing – review & editing, Writing – original draft, Validation, Supervision, Resources, Project administration, Methodology, Formal analysis, Conceptualization. Uday Venkat Mateti: Writing – review & editing, Writing – original draft, Validation, Conceptualization. C.G. Deepak: Writing – review & editing, Writing – original draft, Validation. B.S. Roopa: Writing – review & editing, Writing – original draft, Validation. Juno Jerold Joel: Writing – review & editing, Writing – original draft, Validation, Supervision, Resources, Project administration, Methodology, Formal analysis, Data curation, Conceptualization.

Informed consent

All participants provided written informed consent prior to enrolment.

Ethical approval

The study was initiated after obtaining approval from the Nitte (Deemed to be University) Central Ethics Committee (NU/CEC/2025/948) and was registered with the Clinical Trials Registry of India (CTRI/2025/09/095278).

Funding

The authors did not receive any funding support for the submitted work.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary material

Data availability

Data is available upon request from Dr. Milagrin Xavier Email: milagrinxavier98@gmail.com