Abstract
Background: Psoriasis is a chronic inflammatory disease that affects around 100 million people worldwide. The burden of disease is high, but more recent therapies show promising results. Clinicians need, however, more training in the use of such therapies.
Introduction: Project ECHO® (Extension for Community Healthcare Outcomes) is structured around the promise of delivering medical education at a distance, empowering clinicians who operate in remote areas. The use of instant messaging services, such as WhatsApp® Messenger, has the potential to improve on the existing framework and bridge the existing gap of knowledge. This article reports on a study concerning the implementation of a WhatsApp discussion group in Project ECHO Psoriasis in Argentina.
Materials and Methods: One hundred thirty-two dermatologists in Argentina were invited to participate in the WhatsApp discussion group. After 1 year of participation, a follow-up questionnaire was used to assess the effectiveness of the project.
Results: Eighty dermatologists participated. All questions placed in the discussion were answered by a psoriasis specialist, 79% of which were answered within the first 5 min. Clinicians report significant improvement in diagnosis, comorbidities, and treatment with both conventional and biological therapies.
Discussion: Preliminary results are promising. This new cost-effective solution builds on the existing Project ECHO Psoriasis in Argentina and shows potential in bridging the gap of knowledge, promoting better clinical decisions through empowerment of medical doctors operating in remote locations.
Conclusions: Further research is needed to increase generalization of the results. Moreover, it would be interesting to match the data from the discussion group with follow-up questionnaires.
Keywords: dermatology, distance learning, teledermatology, m-Health, telemedicine, psoriasis, Argentina, continuing medical education
Introduction
Psoriasis is a chronic inflammatory disease that affects around 850,000 people in Argentina,1 while estimates point toward at least 100 million patients across the world (World Health Organization).2 Prevalence varies significantly across the globe, with genetic and environmental factors playing a significant role and with higher prevalence when further from the equator.2,3 The burden of disease is high, namely in terms of quality of life, with symptoms such as low self-esteem and social stigma leading to more severe complications such as depression.4,5 Recent developments in both diagnosis and treatment options regarding psoriasis have widened the gap between specialists and general practitioners, even with dermatologists without specific training in psoriasis. One way to close this gap is through continuous medical training. However, since specialists are mostly located in big city centers, frequently far from rural and other less developed regions, distance training presents itself as an interesting solution.6 Platforms such as the Project ECHO® (Extension for Community Healthcare Outcomes), which is based on telementoring, seem to be an appropriate means to approach the issue.7
In a big country such as Argentina (eighth largest country in the world), 70% of the population lives in or around four major cities, which highlights great asymmetry in terms of development and resource allocation. This is also very evident when it comes to healthcare, and the lack of medical doctors in rural regions is an issue.8 There are regions where there is an evident lack of dermatologists, and even in some regions where there are specialists, some may lack the adequate training to deal with complex and chronic diseases such as psoriasis.
Information and communication technologies (ICTs) are now a reality in most major areas of business, and healthcare is no exception. Social media is at the center of the healthcare ICT revolution, and the potential has been identified in recent studies.9,10 Despite concerns expressed by some experts and national health institutions, which are centered mostly on privacy issues, the use of such tools is rising and benefits communication in the clinical context.11,12
One of the most used instant messaging/social media apps is WhatsApp® Messenger (WhatsApp, Inc.). On July 2017, WhatsApp had 1.3 billion monthly active users, which makes it a relevant player in instant communication. Literature on the use of WhatsApp in medical education is increasing and results seem to highlight the promise of this solution in facilitating the learning process while increasing motivation and enabling instant communication at a distance and as a shared space for discussion with the possibility of sharing multimedia contents.13–16
Some applications of this model have been used to reach out to at-risk patients (HCIA Complex/High-Risk Patient Targeting: Third Annual Report) or provide clinicians, who are placed in remote locations, with help from specialists who work in regions distanced from where they are.17,18 Project ECHO has been applied in different contexts and countries19–22 and results of some studies showed the benefits of the initiative in clinical output assessments.23–26
Project ECHO Psoriasis Argentina is already being implemented and is structured around virtual meetings where each participant presents a clinical case.27 During implementation of Project ECHO Psoriasis Argentina, the mentors of the project have identified that the meetings occurring twice a month were not enough to answer bedside and urgent queries. The most pressing issues were not only concerning the need for immediate answers from dermatologists during their consultations but also, when presented with clinical situations, related to cases that require an answer in a matter of hours.
This article describes a study conducted in Argentina based on the Project ECHO framework and concerns the implementation of a new feature using WhatsApp Messenger as a way to increase the link between specialists and other dermatologists, while creating a common space for discussion, sharing knowledge and experience, as well as providing instant help in the clinical dermatology setting related exclusively to psoriasis. The period analyzed here is the first 12 months of the initiative.
Materials and Methods
Sample
Over a recruitment period of 2 months before the start of the WhatsApp Messenger feature, 132 random dermatologists (from 22 different provinces in Argentina) were invited to participate in this study. Participants were asked to answer a set of questions related not only to clinical practice but also their perception about the WhatsApp Messenger ECHO discussion group.
Measures/Instruments
This study comprised two distinct measures. First, content analysis of questions placed in the WhatsApp Messenger as well as characterization in terms of urgency, response time, and response rate (namely if the question was answered within the discussion forum) in addition to sociodemographic information characterizing the participants in the discussion group (e.g., sex, age, and location).
There was a need to systematize the data from WhatsApp Messenger. Considering that this was text-based information (free unstructured text), the authors performed a content analysis and divided the questions into several categories. To ensure that the content analysis of the discussion group was performed in an accurate way, this process was conducted by three experts on psoriasis.
Second, to assess the educational and clinical outputs of the discussion forum, a GoogleDocs® questionnaire divided into five parts was sent to every participant: (1) clinical questions; (2) ECHO Psoriasis Argentina; (3) virtual meetings; (4) use of WhatsApp chat group; and (5) clinical practices related to psoriasis in the last year. Both measures were developed specifically for this study and were not based on any previous study.
Procedure
After the enrollment of participants, the first meeting of this study was centered on the discussion of some rules and regulations, which led to a framework being devised to ensure that everyone knew how to use this feature without infringing any ethical standards (namely privacy and confidentiality). The rules were to ensure that patient confidentiality was at the top of the priorities. Therefore, the rules were (1) no patient names; (2) no other specific sociodemographic data from patients would be shared, except for gender and age; (3) only relevant clinical data should be shared (e.g., clinical history, family history, medication, and comorbidities); and (4) images shared within the group would not allow identification of the patient. This protocol was designed considering the standards of The Health Insurance Portability and Accountability Act (HIPAA). All participating medical doctors had to create a password for their mobile phones to ensure that both the information shared within the group and the identities of participants were preserved.
Beyond the rules concerning mostly privacy and confidentiality, other operational rules were established: (1) all the questions placed in the group had to be related to psoriasis; (2) to avoid mistakes and misinterpretation of questions, and to create an orderly forum of discussion, new questions could only be placed when the previous question was answered; and (3) in the most complex questions, at the end of the discussion, the group administrators (experts) would issue a set of comments, resuming the discussion, and the final clinical recommendation.
After 1 year of running the program (from July 2016 to July 2017), the participants were requested to fill in a GoogleDocs questionnaire with questions related to educational and clinical outputs of the project and efficacy of the project in delivering the expected results. This questionnaire was sent on August 2017 and participants had 1 month to send it back to the project coordinators.
Statistical Analysis
This study is mainly an exploratory descriptive study based on the outputs of questionnaires and content analysis from the discussion forum. Therefore, the analysis is based on descriptive statistics. For numerical variables, reported results use mean and standard deviation (SD), and categorical variables are reported with absolute frequencies and percentages whenever adequate. Missing data in each question were not taken into consideration, meaning that in the case of categorical variables, valid percentage is reported.
Results were computed using Excel 2013 and IBM SPSS, version 24.
Results
Analysis of the Discussion Group Data
Eighty (N = 80) dermatologists accepted to test this new feature using WhatsApp Messenger. Most of the participants were females (n = 68, 85%), averaging 38.34 years (SD = 7.22). Considering provincial distribution, results show that participants are dispersed in different regions of the country, even though Bs As (n = 10; 12.5%), Gran Bs As (n = 12, 15%), Santa Fe (n = 11, 13.75%), and Córdoba (n = 9, 11.25%) are more represented, in line with these being the most densely inhabited regions of Argentina.
Beyond the sociodemographic characterization of participants, the data obtained from content analysis of the discussion group are summarized in Table 1. The most frequent topics were treatment (n = 84; 43%) and adverse effects (n = 34; 17%), while the less frequent were funding management (n = 3; 2%) and consultation with specialists from a different field (n = 1; 1%).
Table 1.
Content Analysis of Questions Based on Categories (N = 197)
| CATEGORY | N (%) |
|---|---|
| Diagnosis | 27 (14) |
| Treatment | 84 (43) |
| Adverse effects | 34 (17) |
| Comorbidities | 8 (4) |
| Management of complementary examinations | 5 (3) |
| Vaccination | 17 (9) |
| Special clinical situations | 6 (3) |
| Geographical variation | 6 (3) |
| Funding management | 3 (2) |
| Consultation with specialists from a different field | 1 (1) |
| Other | 6 (3) |
A total of 197 questions were placed in the chat during a period of 12 months. All of the 197 questions (100%) placed in the discussion group got an answer from other participants and a final recommendation from the experts. Of the total questions, 33 (17%) were situations that required an answer in a relatively short period of time, while the rest were related to consultations and clinical cases. In each question, the number of involved participants in the discussion ranged from 1 to 12 (mean = 3.41, SD = 2.42). Considering the time to respond, most of the questions were answered 5 min after being placed in the group; 116 (59%) were answered within 2 min and 156 (79%) within the first 5 min. Only four questions (2%) took more than an hour to have a final decision from the experts, and the average response time was 7.40 min (SD = 22.66). All questions were supervised by the experts and they gave a final advice that would agree, adjust, or change the solution mentioned in the group.
One-Year Follow-Up Questionnaire
After 1 year of implementation of the WhatsApp Psoriasis discussion group, a questionnaire was sent to participants to assess the results of the project. Of the 80 participants in the WhatsApp discussion group, 51 answered the 1-year follow-up questionnaire. Therefore, the percentage values presented next are valid percentages to make results clearer. Ninety-four percent of the participants who answered were dermatologists (n = 48), while the others were resident doctors (n = 3; 5.9%); 43% work alone (n = 22); 39.2% work in a multidisciplinary team (n = 20); and 17.6% work in a team exclusively of dermatologists (n = 9).
The mains goals were to understand the extent to which this new feature could improve on the already existing meetings of the Project ECHO Psoriasis held in Argentina and if the ECHO framework was having a significant impact in the participants' clinical practice. First questions were related to Project ECHO Psoriasis as a whole program. Eighty-six percent (44) of the participants who answered knew about the ECHO Psoriasis program through an invitation sent by one of the coordinators of the Project ECHO Psoriasis Argentina, while another 10% (n = 5) knew about it through a representative of the pharmaceutical company.
From the most relevant resources available in Project ECHO Psoriasis, 100% (n = 51) of the participants who answered the questionnaire use the WhatsApp chat feature, while 78.4% (n = 40) participate in ECHO's teleclinics (which happen twice a month). However, only 69% (n = 35) use the Facebook page of the project.
Considering the virtual meetings of Project ECHO that occur twice a month, the use of this resource varies significantly between users, but the most significant result suggests that 50% (n = 25) of the participants report attending these virtual meetings less than half the time they occur (Fig. 1).
Fig. 1.
Approximate percentage of meetings participated.
Results concerning the perceived effect of the virtual meeting are represented in Figure 2. Virtual meetings seem to result in better clinical competences in the evaluation of psoriasis comorbidities (Fig. 2B), in which 89.4% of the participants consider that participating improved their skills a lot and only 6.4% reported no improvement. Similar results can be found concerning the diagnosis of psoriasis (Fig. 2A), with 85.1% reporting a lot of improvement and 4.3% reporting no improvement at all. The results were perceived as positive concerning systemic conventional therapies, with 72.3% acknowledging a lot of improvement in implementation of therapies, and also positive in the implementation of biological therapies, with 65.2% reporting that the meeting helped a lot.
Fig. 2.
Assessment of the impact of ECHO® virtual meetings: (A) Did you notice that virtual meetings helped improve the diagnosis of psoriasis? (B) Did you notice that virtual meetings helped you to improve the assessment of psoriasis comorbidities? (C) Did you notice that virtual meetings helped you to implement conventional systemic therapies with less difficulty? (D) Did you notice that virtual meetings helped you to implement biological therapies with less difficulty? ECHO, Extension for Community Healthcare Outcomes.
The results from the use of the WhatsApp discussion group were the main goal of this study and are summarized in Table 2. In terms of participation in the discussion group, results show that more than a quarter of the sample (27.5%) only reads the discussions in the group, having a more passive participation. However, 96.1% of the participants recognize the chat as a very useful tool to clear urgent doubts regarding patients' evaluation, and roughly the same (98%) perceive it as very useful in addressing the adverse secondary effects of the treatments. Last, 86.3% consider the discussion groups very useful regarding doubts about comorbidities.
Table 2.
Assessment of the Project ECHO Psoriasis WhatsApp Discussion Group
| QUESTION | N (%) |
|---|---|
| Have you actively participated in the chat? | |
| Asking questions | 10 (19.6) |
| Answering questions | 7 (13.7) |
| Asking and answering questions | 20 (39.2) |
| Read the chat, but has not answered or asked questions | 14 (27.5) |
| How useful do you think chat is as a tool to clear urgent doubts regarding the general evaluation of patients? | |
| Very useful | 49 (96.1) |
| Slight useful | 2 (3.9) |
| Not useful | 0 (0) |
| How useful do you think chat is as a tool to clear urgent doubts regarding adverse secondary effects of treatments? | |
| Very useful | 50 (98) |
| Slight useful | 1 (2) |
| Not useful | 0 (0) |
| How useful do you think chat is as a tool to clear urgent doubts regarding the evaluation of comorbidities? | |
| Very useful | 44 (86.3) |
| Slight useful | 7 (13.7) |
| Not useful | 0 (0) |
ECHO®, Extension for Community Healthcare Outcomes.
Also relevant is to identify the improvement after 1 year of participation in the ECHO Psoriasis project. Considering the level of satisfaction with the virtual meetings of the ECHO project (evaluated on a 1–10 scale, 1—not satisfied to 10—very satisfied), 95.8% of the participants are above the midpoint of the scale as somewhat satisfied (mean = 9.08, SD = 1.86). Similar results were found when assessing satisfaction with the discussion group of ECHO Psoriasis in WhatsApp (using the same scale), with 88.2% above the median point of the scale (mean = 9.02, SD = 1.99). The results concerning satisfaction with the Facebook page of the ECHO Psoriasis project were similar (mean = 8.6, SD = 1.96).
Discussion
This study highlights the usefulness of WhatsApp as a simple, instantaneous, and massively used communication tool to bridge the gap of knowledge among dermatologists without the need to have them all in the same location.
The development of this new feature using the WhatsApp discussion group shows great promise in bridging the gap between experts on psoriasis and dermatologists working away from the big centers. Clinicians who are further from the big cities and have limited access to experts in loco can use this resource as a means to not only discuss specific cases but also to increase their knowledge on the use of alternative or innovative therapies and treatments for very specific and complex diseases such as psoriasis.28 This contributes to an empowerment for doctors who work in remote locations with limited access to peers or consults with other specialties. Results highlight this empowerment since increased knowledge leads to more autonomy and, ultimately, a significant improvement in delivery of health for a particular disease such as psoriasis in Argentina.
These results were obtained since instant messaging allows for a quick and almost real-time answer to clinical questions while allowing for relevant information to be shared within a closed group. Moreover, WhatsApp is considered to be one of the safest instant messaging solutions since it incorporates encryption technology,29 which is essential to the success of a solution where privacy and confidentiality are as important as being able to discuss and find solutions swiftly for complex clinical issues. Still, there are issues concerning recommendations from National Health System officers pertaining to privacy and confidentiality that need to be addressed to increase the potential of such solutions in a more global perspective.
On the other hand, these medical education solutions are perceived as cost-effective in almost every context and even more so in this context. Argentina is a country in which most of the population lives in or around four major cities, which leads to an asymmetrical distribution of both the population and clinicians. The dispersion of population in rural areas leads dermatologists who work there to have a less specialized approach in complex diseases such as psoriasis.
Moreover, psoriasis is a disease that has seen an increasing body of literature and research in the last decade, especially concerning comorbidities and treatment options. Dermatologists who are not specialized in the disease may benefit greatly from working with experts within the Project ECHO Psoriasis, applying the best clinical practices to their patients while avoiding traveling big distances.
However, this study has some limitations. The methodology should allow a correspondence between the data from the WhatsApp discussion group and answers from the final assessment questionnaire so as to perform inferential analysis on the factors that may explain better use of this discussion group. In addition, in future studies with this feature, included in the ECHO framework, would be interesting to explore the increase of knowledge and clinical efficacy of participants who use WhatsApp and virtual meetings and compare it with the dermatologists who solely attend virtual meetings. This study highlights the gains in education from the perception of participating dermatologists, but future studies should focus on measuring clinical outputs.
Generalizability was not an objective due to the preliminary nature of this study and the lack of inferential analysis. However, the results provide good evidence of the potential of a tool such as the ECHO WhatsApp discussion group for medical education.
Acknowledgments
The authors would like to thank Carolina Marchesi, Jimena Lorenzo, María de los Milagros San Román, and Claudia Koch for their contribution to the study.
Authors' Contributions
L.D.M., M.N.E., and C.M.E. contributed to idea, design, execution, data analysis and review of the manuscript; M.N.E. contributed to data insertion and preparation; and P.C.L., S.S., and M.D. contributed to coordination of the study.
Disclosure Statement
No competing financial interests exist.
References
- 1. Chouela E, Amaya M, Londoño A, Romiti R, Valenzuela F, Duarte AA, Perez Alfonzo R, Borlenghi C. Psoriasis in Latin America. Dermatol Online J 2016;22. [PubMed] [Google Scholar]
- 2. World Health Organization. Global report on psoriasis. Geneva, Switzerland: World Health Organization, 2016 [Google Scholar]
- 3. Parisi R, Symmons DP, Griffiths CE, Ashcroft DM. Global epidemiology of psoriasis: A systematic review of incidence and prevalence. J Investig Dermatol 2013;133:377–385 [DOI] [PubMed] [Google Scholar]
- 4. Kimball AB, Jacobson C, Weiss S, Vreeland MG, Wu Y. The psychosocial burden of psoriasis. Am J Clin Dermatol 2005;6:383–392 [DOI] [PubMed] [Google Scholar]
- 5. Eghlileb AM, Davies EEG, Finlay AY. Psoriasis has a major secondary impact on the lives of family members and partners. Br J Dermatol 2007;156:1245–1250 [DOI] [PubMed] [Google Scholar]
- 6. Katzman JG, Comerci G, Boyle JF, Duhigg D, Shelley B, Olivas C, Daitz B, Carroll C, Som D, Monette R, Kalishman S. Innovative telementoring for pain management: Project ECHO pain. J Contin Educ Health 2014;34:68–75 [DOI] [PubMed] [Google Scholar]
- 7. Arora S, Thornton K, Jenkusky SM, Parish B, Scaletti JV. Project ECHO®: Linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico. Public Health Rep 2007;122(Suppl 2):74–77 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Borracci RA, Arribalzaga EB, Couto JL, Dvorkin M, Ahuad Guerrero RA, Fernandez C, Ferreira LN, Cerezo L. Factors affecting willingness to practice medicine in underserved areas: A survey of Argentine medical students. Rural Remote Health 2015;15:3485. [PubMed] [Google Scholar]
- 9. Gulacti U, Lok U, Hatipoglu S, Polat H. An analysis of WhatsApp® usage for communication between consulting and emergency physicians. J Med Syst 2016;40:130. [DOI] [PubMed] [Google Scholar]
- 10. Hall AK, Cole-Lewis H, Bernhardt JM. Mobile text messaging for health: A systematic review of reviews. Annu Rev Public Health 2015;36:393–415 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Gould G, Nilforooshan R. WhatsApp® doc? BMJ Innov 2016;2:109–110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. O'Sullivan DM, O'Sullivan E, O'Connor M, Lyons D, McManus J. WhatsApp® doc? BMJ Innov 2017;0:1–2 [Google Scholar]
- 13. Bere A. Using mobile instant messaging to leverage learner participation and transform pedagogy at a south African university of technology. Br J Educ Technol 2013;44:544.–561. [Google Scholar]
- 14. Hossain IT, Mughal U, Atalla B, Franka M, Siddiqui S, Muntasir M. Instant messaging—One solution to doctor-student communication? Med Educ Online 2015;20:30593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Raiman L, Antbring R, Mahmood A. WhatsApp® messenger as a tool to supplement medical education for medical students on clinical attachment. BMC Med Educ 2017;17:7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Boyinbode O, Agbonifo O, Ogundare A. Supporting mobile learning with WhatsApp® based on media richness. Circ Comput Sci 2017;2:37–46 [Google Scholar]
- 17. Arora S, Geppert CM, Kalishman S, Dion D, Pullara F, Bjeletich MB, Simpson G, Alverson DC, Moore LB, Kuhl D, Scaletti JV. Academic health center management of chronic diseases through knowledge networks: Project ECHO®. Acad Med 2007;82:154–160 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Arora S, Thornton K, Komaromy M, Kalishman S, Katzman J, Duhigg D. Demonopolizing medical knowledge. Acad Med 2014;89:30–32 [DOI] [PubMed] [Google Scholar]
- 19. Lewiecki E, Bouchonville M, Chafey D, Bankhurst A, Arora S. Bone health ECHO: Telementoring to improve osteoporosis care. Womens Health (Lond) 2016;12:79–81 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Raney L, Bergman D, Torous J, Hasselberg M. Digitally driven integrated primary care and behavioral health: How technology can expand access to effective treatment. Curr Psychiatry Rep 2017;19:86. [DOI] [PubMed] [Google Scholar]
- 21. Tahan V, Almashhrawi A, Mutrux R, Ibdah JA. Show me ECHO-hepatitis C: A telemedicine mentoring program for patients with hepatitis C in underserved and rural areas in Missouri as a model in developing countries. Turk J Gastroenterol 2015;26:447–449 [DOI] [PubMed] [Google Scholar]
- 22. Bouchonville MF, Paul MM, Billings J, Kirk JB, Arora S. Taking telemedicine to the next level in diabetes population management: A review of the Endo ECHO model. Curr Diab Rep 2016;16:96. [DOI] [PubMed] [Google Scholar]
- 23. Anderson D, Zlateva I, Davis B, Bifulco L, Giannotti T, Coman E, Spegman D. Improving pain care with project ECHO in community health centers. Pain Med 2017;18:1882–1889 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: Insights from a qualitative study. Pain Med 2017. [Epub ahead of print]; DOI: 10.1093/pm/pnx233 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Cheallaigh CN, O'Leary A, Keating S, Singleton A, Heffernan S, Keenan E, Robson L, Sears J, Moloney J, Arora S, Bergin C. Telementoring with project ECHO: A pilot study in Europe. BMJ Innov 2017;0:1–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Lewiecki EM, Boyle JF, Arora S, Bouchonville MF, Chafey DH. Telementoring: A novel approach to reducing the osteoporosis treatment gap. Osteoporos Int 2017;28:407–411 [DOI] [PubMed] [Google Scholar]
- 27. Mazzuoccolo L, Marciano S, Echeverria C. Implementation of a telementoring model of medical education in psoriasis. [In Spanish]. Medicina 2016;76:359–361 [PubMed] [Google Scholar]
- 28. Christophers E. Psoriasis: Epidemiology and clinical spectrum. Clin Exp Dermatol 2001;26:314–320 [DOI] [PubMed] [Google Scholar]
- 29. WhatsApp encryption overview. 2016. Available at www.whatsapp com/security (last accessed October17, 2017)


