Abstract
To prove the equivalence of the Korean version of the Foot and Ankle Outcome Score (FAOS) in the printed (PFAOS) vs the electronic (EFAOS) form in a multicenter randomized study.
Overall, 227 patients with ages ranging from 20 to 79 years from 16 dedicated foot and ankle centers were included. Patients were randomized into either a ‘paper first’ group (P-F group, n = 113) or an ‘electronic device (tablet computer) first’ group (E-F group, n = 114). The first evaluation either by paper (P-F group) or tablet (E-F group) was followed by a second evaluation the following day. The difference between the PFAOS and EFAOS results in each group was calculated and analyzed. To evaluate the benefit of each methodology, the time consumed per evaluation was compared and patients were asked which methodology they preferred and which was the easiest to use.
There were no significant differences in age or sex between the groups. An intraclass correlation coefficient (ICC) value of 0.934 (95% confidence interval [CI]: 0.912–0.950, P < .001) was confirmed in PFAOS and EFAOS, showing a significant correlation between the 2 methodologies. EFAOS was completed in a shorter amount of time than PFAOS. The majority of patients agreed that EFAOS was easier to complete than PFAOS.
The paper or electronic forms of the Korean adaptation of FAOS were considered equivalent. The shorter time of completion and the preference for the electronic version over paper by patients deems the electronic FAOS a promising option to consider in future.
Keywords: electronic data collection, foot and ankle outcome score, patient-reported outcome measures
1. Introduction
Patient-reported outcomes (PRO), defined by the United States Food and Drug Administration (FDA) as ‘any report of the status of a patient's health condition that comes directly from the patient without interpretation of the patient's response by a clinician or anyone else’,[1] is growing in its importance and is being considered as a more valid measure than clinician-reported outcomes.[2–4] Since the effectiveness of treatment is evaluated from the patients’ perspective, PROs are valuable in recording subjective outcomes such as pain, daily performance and general quality of life, which are difficult to quantify in an objective and measurable way.[2,3] A measurable amount of resources and time are needed for collecting and processing such data in a traditional printed form including the preparation of the printed questionnaires, distribution during office visits or by mail, completion by the patient with or without a physician's supervision, collection of the questionnaires, confirmation of missed responses and manual registration of each item into a digital database. Such burden for both the physicians and patients prevents the extensive use of this valuable tool.[5]
Electronic forms of PRO (EPRO) can greatly simplify this traditional process.[6,7] Questionnaire websites can be distributed to patients or otherwise dedicated digital devices can be employed for office visits instead as potential replacements to the paper forms. EPROs can eliminate missed answers and maintain response quality by requiring that all forms are filled out prior to submission. Better patient compliance can be expected by pertaining to patients’ needs such as using text-to-speech options, high visibility fonts and other language options. These advantages of EPROs can lead to more reliable and accurate data than its traditional counterpart.[8,9] Furthermore, in the advent of cloud storage systems, the collective gathering and analysis of results can provide immense assistance in studies performed by multiple centers and investigators.[10,11]
It is however of paramount importance to show that the traditional printed PRO can be efficiently be converted to an electronic format without altering its results. Test-retest comparison studies would need take place in order to support the hypothesis that no significant differences would exist between the scores derived from a printed form vs an electronic form.[12,13] In order to achieve this, 2 important confounding factors will need to be considered.[14] First, the way that the questions are presented to the patient, which can differ in both methods, such as the number of items within a page or a screen, or contents having to be split in order to fit in a page of a small handheld device, as well as the inability to skip questions. Secondly, the patients’ abilities to manipulate electronic devices should be taken into account especially in the elderly population where patients are more confident using a pen rather than a screen on an electronic device.
The present multicenter randomized study aimed to show the equivalence of the Korean version of the Foot and Ankle Outcome Score (FAOS)[15] in its printed vs electronic (EFAOS) form.
2. Methods
The purpose of the present investigator led, prospective, randomized, multicenter clinical study was to evaluate the equivalence of EFAOS and PFAOS. A total of 227 patients with ages ranging from 20 to 79 from 16 dedicated foot and ankle centers were included in this study. Patients who were unable to independently understand and respond to either questionnaire were excluded from the study. The current study was performed after obtaining approval from the Institutional Review Board of CHA Bundang Medical Center (registration number: 2016-11-030-001, Date of Issue: 6th Jan 2017). The informed consent was given to all involved patients.
Patients who consented to participate in this study were assigned randomly by means of a sealed envelope to either the ‘paper first’ group (P-F group, n = 113) or the ‘electronic device (tablet computer) first’ group (E-F group, n = 114). Prior to using the electronic device, a 1-minute introduction to familiarize patients with the electronic questionnaire was carried out. Measures were taken to minimize any memory or learning effects. The first evaluation either by paper (P-F group) or tablet (E-F group) was carried out in the day of admission, followed by a second evaluation the next day (Fig. 1) in order avoid differences in scores due to changes in the patients’ conditions. The difference between the PFAOS and the EFAOS results in each group were then calculated and analyzed.
Figure 1.
The first evaluation either by paper (Paper group) or a tablet (Electronic group) took place in the day of admission, followed by the second evaluation either by a tablet or paper the day after. EFAOS = electronic Foot and Ankle Outcome Score, PFAOS = printed Foot and Ankle Outcome Score.
The Korean adaptation of the FAOS was previously validated and shown to be equivalent to the original version in its psychometric properties.[15] The PADAS software (PADAS Co. Ltd, Seoul, Korea) was the chosen tablet application for testing EFAOS. In PADAS, queries are presented 1 at a time on the screen along with a possible set of responses. When an answer is selected, it automatically shows the patient the next question. To evaluate the benefit of EFAOS compared to PFAOS, the time utilized for each evaluation was compared and the patients asked which methodology was easiest to use.
A Per-protocol (PP) analysis was conducted for statistical calculations. PFAOS and EFAOS scores were compared by calculation of the intraclass correlation coefficient (ICC). ICC typically ranges from 0 to 1.00 and, in order to support internal consistency, the derived ICC should be at least 0.80 with an alpha value of 0.85 to 0.95, irrespective of which methodology of the questionnaire was completed first. Since we obtained an ICC value of 0.840 in our pilot study, the present study was deemed valid if its ICC fell within the error range of 5% of the pilot value. Patients’ preferences were assessed using the Mann–Whitney U test, a nonparametric statistical test. A P value below .05 was considered statistically significant.
3. Results
There were no significant differences in either age or sex between the 2 groups. An ICC value of 0.934 (95% CI: 0.912-0.950, P < .001) was confirmed in PFAOS and EFAOS showing a significant correlation between the 2 methodologies. A shorter amount of time was needed to complete EFAOS (PFAOS 6.3 ± 4.8 minutes vs EFAOS 4.1 ± 2.1 minutes, P = .001). The majority of patients found that it was easier to complete EFAOS than PFAOS, regardless of the group they were initially assigned to (EFAOS preference: P-F group 67.3%, E-F group 48.2% vs PFOAS preference: P-F group 10.6%, E-F group 23.7%) (Table 1).
Table 1.
Response to question “Which modality was easier to complete?”.
4. Discussion
This study confirmed the equivalence of the paper to the electronic version of the Korean translation of FAOS. The patients’ preferences for the electronic version of the questionnaire and the shorter amount of time that was needed for its completion makes EFAOS a promising option in recording PROs in the foot and ankle services. A previous single-center study with 42 patients compared an electronic to a paper version of the foot function index (FFI) and FAOS and showed similar results to our findings.[16] The present multicenter study strengthened their findings by validating this equivalence with a further 227 patients.
FAOS is a well-established PRO for foot and ankle conditions,[17] and its effectiveness was previously verified in the Korean language adaptation.[15] The importance of such PROs in evidence-based medicine is increasing. PROs not only complement the clinician's measured objective outcomes, but also in many cases act as an exclusive means of quantifying data such as frequency and severity of symptoms and how these affect the patients’ daily lives.[2,4,18] Any conditions in the foot and ankle area, major weight-bearing organs, greatly affect patients’ daily lives and recording its outcome is paramount in the clinic. Nevertheless, the widespread use of FAOS can be limited both in a clinical setting and in research as it takes considerable time for patients to fill the questionnaire and to process the raw data digitally for further evaluation.
An electronic version of FAOS was beneficial not only in reducing the patients’ burden but also in the process of converting their recorded answers to research data. It is important to mention that as the electronic questionnaire does not allow for unanswered items, no additional steps are necessary to fill those gaps, as with paper-based questionnaires. All answers, along with demographic data were automatically processed for further research.
There are however some methodology limitations to our study. The preference of paper vs electronic versions was only evaluated from the patients’ perspective. As a multicenter study, such preference could have been quantified with the led investigators and clinicians as well. Furthermore, it may be worthwhile to compare the time needed to process both paper and electronic responses by staff. In addition, the variation between individuals’ ability to use an electronic device were not taken into account when randomizing patients into the 2 groups. The 1-minute introductory session may have been unnecessary for some patients, whereas critical for individuals, especially in the elderly population, where further assistance was needed. Further investigation will be needed in the future regarding the safety and reliability of tablet applications and data servers.
5. Conclusions
The paper or electronic forms of the Korean adaptation of FAOS were considered equivalent in this study. The shorter time of completion and the patients’ preference for the electronic version over paper deems the electronic FAOS a promising option to consider in future in the foot and ankle services.
Author contributions
Conceptualization: Yu Mi Kim.
Formal analysis: Bom Soo Kim, SeongJu Choi.
Investigation: Jae Yong Park.
Resources: Hyun June Lee, SeongJu Choi.
Supervision: Hyong Nyun Kim.
Validation: Jae Ho Cho, Hwa Jun Kang.
Writing – original draft: Young Rak Choi.
Writing – review & editing: Young Rak Choi.
Young Rak Choi orcid: 0000-0003-2061-7270.
Footnotes
Abbreviations: CI = confidence interval, E-F group = electronic device (tablet computer) first group, EFAOS = electronic Foot and Ankle Outcome Score, EPRO = electronic forms of PRO, FAOS = foot and ankle outcome score, FFI = foot function index, ICC = intraclass correlation coefficient, P-F group = paper first group, PFAOS = printed or paper Foot and Ankle Outcome Score, PP = per-protocol, PRO = patient-reported outcomes.
How to cite this article: Park JY, Kim BS, Lee HJ, Kim YM, Kim HN, Kang HJ, Cho JH, Choi S, Choi Y. Comparison between an electronic version of the foot and ankle outcome score and the standard paper version. Medicine. 2019;98:40(e17440).
This study was supported by the Ministry of Education, National Research Foundation of Korea (Grant nos. NRF-2017R1C1B5017705).
The authors report no conflicts of interest.
References
- [1].Food and Drug Administration, US Department of Health and Human Services. Guidance for industry use in medical product development to support labeling claims guidance for industry. Clin Fed Regist 2009;1–39. [Google Scholar]
- [2].Basch E, Jia X, Heller G, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst 2009;101:1624–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Khanna G, Singh JA, Pomeroy DL, et al. Comparison of patient-reported and clinician-assessed outcomes following total knee arthroplasty. J Bone Joint Surg Am 2011;93:e117(1)–7). [DOI] [PubMed] [Google Scholar]
- [4].Basch E, Bennett A, Pietanza MC. Use of patient-reported outcomes to improve the predictive accuracy of clinician-reported adverse events. J Natl Cancer Inst 2011;103:1808–10. [DOI] [PubMed] [Google Scholar]
- [5].Saleh KJ, Radosevich DM, Kassim RA, et al. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res 2002;20:1146–51. [DOI] [PubMed] [Google Scholar]
- [6].Farr J, Verma N, Cole BJ. Validation study of an electronic method of condensed outcomes tools reporting in orthopaedics. J Knee Surg 2013;26:445–51. [DOI] [PubMed] [Google Scholar]
- [7].Salaffi F, Gasparini S, Ciapetti A, et al. Usability of an innovative and interactive electronic system for collection of patient-reported data in axial spondyloarthritis: comparison with the traditional paper-administered format. Rheumatol (Oxf ) 2013;52:2062–70. [DOI] [PubMed] [Google Scholar]
- [8].Stokes T, Paty J. Technology update: electronic diaries, part 2 the role of the clinical protocol. Appl Clin Trials 2003;12:46–56. [Google Scholar]
- [9].Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health 2009;12:419–29. [DOI] [PubMed] [Google Scholar]
- [10].Muehlhausen W, Doll H, Quadri N, et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual. Life Outcomes 2015;13:167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [11].White MK, Maher SM, Rizio AA, et al. A meta-analytic review of measurement equivalence study findings of the SF–36® and SF–12® Health Surveys across electronic modes compared to paper administration. Qual Life Res 2018;27:1757–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [12].Zbrozek A, Hebert J, Gogates G, et al. Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force. Value Health 2013;16:480–9. [DOI] [PubMed] [Google Scholar]
- [13].Bushnell DM, Reilly MC, Galani C, et al. Validation of electronic data capture of the irritable bowel syndrome--quality of life measure, the work productivity and activity impairment questionnaire for irritable bowel syndrome and the EuroQol. Value Health 2006;9:98–105. [DOI] [PubMed] [Google Scholar]
- [14].Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11:322–33. [DOI] [PubMed] [Google Scholar]
- [15].Lee KM, Chung CY, Kwon SS, et al. Transcultural adaptation and testing psychometric properties of the Korean version of the Foot and Ankle Outcome Score (FAOS). Clin Rheumatol 2013;32:1443–50. [DOI] [PubMed] [Google Scholar]
- [16].Kim J-B, Kwon M-S, Kim J-G, et al. The methods for foot function index and foot and ankle outcome score measurement: a comparison between paper-and-pencil method and electronic method. J Korean Foot Ankle Soc 2017;21:33–8. [Google Scholar]
- [17].Roos EM, Brandsson S, Karlsson J. Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int 2001;22:788–94. [DOI] [PubMed] [Google Scholar]
- [18].Deshpande PR, Rajan S, Sudeepthi BL, et al. Patient-reported outcomes: a new era in clinical research. Perspect Clin Res 2011;2:137–44. [DOI] [PMC free article] [PubMed] [Google Scholar]