With great interest, we read the comment by Gilbert and Wicks on our recent publication [1] testing the accuracy and usability of Ada’s symptom checker among medical students.
We fully agree with Gilbert et al. that the accuracy and usability of the Ada app is extremely user and use-case dependent. Whereas Ada’s accuracy was extremely high (98%) in the rheumatoid arthritis vignette, it was clearly lower for the other two vignettes (granulomatosis with polyangiitis 43%; systemic lupus erythematosus 51%). In a recent publication [2] with actual patients, we analyzed in detail the varying perceived usability regarding Ada compared to a similar rheumatology-specific system (Rheport). Usability of both tools was good, although usability of Ada was significantly lower compared to Rheport. Importantly, usability significantly decreased with age.
The authors of the comment state that the app was designed for layperson users to test what underlying disease might be causing their health issues and state that the app explicitly is not developed for health care professionals (HCP). It is noteworthy that the underlying Ada intelligence is identical to Ada’s HCP focused system. Remarkably, the same authors (Gilbert et al., Ada Health GmbH [3]) published a study in which primary care physicians (GPs) tested 200 clinical vignettes with Ada and other digital symptom assessment apps. Also urgency advice was analyzed. Inspired and analog to that study, we intentionally chose medical students and rheumatology case vignettes (source public online learning center and Rheum2Learn section American college of Rheumatology) with very typical disease symptoms over laypersons to create a “best-case scenario.” Gilbert and Wicks further argue that the used Ada app in our study is not a diagnostic decision support system (DDSS); however, the Ada app provides diagnostic terms upon symptom entry and intentionally recommends urgency advices to support appointments. In our opinion and on the basis of the work by Sutton et al. [4], apps interpreting or translating symptoms into diagnoses fulfill characteristics of DDSS, but could also be named symptom assessment apps, self-diagnosis tools, technology-supported clinical decision support tools, etc. Nevertheless, arguing over the exact description should not distract from proper utilization of this technology, since we believe that DDSS or similar tools could operate through patients or HCPs as a gatekeeper for optimization of patient flows. Unfortunately, these tools still lack accuracy in certain rheumatologic rare diseases.
Abbreviations
- DDSS
Diagnostic decision support system
- HCP
Health care professionals
Authors’ contributions
JK and AJH drafted the manuscript. All authors approved the final manuscript.
Funding
None.
Declarations
Ethics approval and consent to participate
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Knitza J, Tascilar K, Gruber E, Kaletta H, Hagen M, Liphardt A-M, et al. Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students. Arthr Res Ther. 2021;23:233. doi: 10.1186/s13075-021-02616-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Knitza J, Muehlensiepen F, Ignatyev Y, Fuchs F, Mohn J, Simon D, et al. Patient’s perception of digital symptom assessment technologies in rheumatology: results from a multicentre study. Front Public Health. 2022;10:844669. 10.3389/fpubh.2022.844669. [DOI] [PMC free article] [PubMed]
- 3.Gilbert S, Mehl A, Baluch A, Cawley C, Challiner J, Fraser H, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020;10(12):e040269. doi: 10.1136/bmjopen-2020-040269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y. [DOI] [PMC free article] [PubMed] [Google Scholar]