Abstract
Background:
Soft-tissue fillers are increasingly being used for noninvasive facial rejuvenation. They generally offer minimal downtime and reliable results. However, significant complications are reported and patients need to be aware of these as part of informed consent. The Internet serves as a vital resource to inform patients of the risks and benefits of this procedure.
Methods:
Three independent reviewers performed a structured analysis of 65 Websites providing information on soft-tissue fillers. Validated instruments were used to analyze each site across multiple domains, including readability, accessibility, reliability, usability, quality, and accuracy. Associations between the endpoints and Website characteristics were assessed using linear regression and proportional odds modeling.
Results:
The majority of Websites were physician private practice sites (36.9%) and authored by board-certified plastic surgeons or dermatologists (35.4%) or nonphysicians (27.7%). Sites had a mean Flesch-Kincaid grade level of 11.9 ± 2.6, which is well above the recommended average of 6 to 7 grade level. Physician private practice sites had the lowest scores across all domains with a notable lack of information on complications. Conversely, Websites of professional societies focused in plastic surgery and dermatology, as well as academic centers scored highest overall.
Conclusions:
As the use of soft-tissue fillers is rising, patients should be guided toward appropriate sources of information such as Websites sponsored by professional societies. Medical professionals should be aware that patients may be accessing poor information online and strive to improve the overall quality of information available on soft-tissue fillers.
Currently, the Internet serves as one of the most widely used sources of patient information. Research indicates that 74% of all US adults use the Internet, and 61% have looked for health or medical information online.1 However, online information has its limitations, especially in the field of plastic surgery. It can be factually incorrect, biased by industry and marketing strategies, and/or lack scientific evidence and peer review.
From a patient perspective, the Internet is regarded as a tool for self-education that empowers patients to make important decisions about their healthcare. However, patients who are seeking quick and easy cosmetic procedures may be increasingly vulnerable to false advertising and biased information. Many consumer Websites are driven by financial profits and therefore omit important medical information such as procedural risks and complications so as not to deter clients. As such, efforts to standardize the creation of patient information have been undertaken by many organizations such as the International Patient Decision Aid Standards collaboration, the Geneva-based Health on the Net Foundation, and the British Medical Association.2,3
We chose to review and analyze the online patient information on soft-tissue fillers because of their growing popularity. Google Trends data indicate a substantial increase in the term “dermal filler” searched by Internet users worldwide.4 Furthermore, since 1997, there has been a 274% increase in the total number of cosmetic procedures performed, with injectables being the top overall.5 After botulinum toxin injections, soft-tissue fillers are the second most common with a 253% rise from 2000 to 2014 and a total of 2.3 million procedures performed.6,7 However, unlike botulinum toxin injections, soft-tissue fillers can have serious consequences, especially if administered by nonqualified practitioners.8,9 Although severe complications are rare in experienced hands, they can include nodules or granulomas, infection, skin necrosis, embolization, stroke, and vision loss.10,11 Therefore, it is important to ensure that users are well informed of the soft-tissue filler procedure, which includes a thorough understanding of both the risks and benefits.
Although there are no worldwide regulations, the Food and Drug Administration (FDA) advises patients to carefully read around the products, surgeon experience, and postoperative warning signs and symptoms.9 Furthermore, the FDA recommends that patients seek a board-certified plastic surgeon or dermatologist to perform the soft-tissue fillers procedure to minimize the risks of potentially devastating complications.9
Given that the Internet serves as an important patient education tool, coupled with the increase in popularity of soft-tissue fillers and the accompanying rise in potential complications, we sought to describe and systematically analyze the content of relevant online patient information.
METHODS
Eligibility Criteria and Website Selection
The search terms “soft-tissue fillers,” “dermal fillers,” “wrinkle fillers,” and “injectable fillers” were queried separately in Google, Yahoo, and Bing. These search engines currently represent the top 3 engines used by people on the Internet.12 The first 30 entries were screened as it has been shown that 90% of search engine users click on a link within the first 3 pages of results.13,14
The Websites from the 3 engines were downloaded in August 2015 in the order they appeared with location services turned off so as not to bias the search results. The data were downloaded on the same date to ensure that identical content was being analyzed by the reviewers.
Websites were excluded from the study if they were duplicates, were unrelated, or did not contain enough information for a full review. Scientific articles and journals were also excluded from the study as they are either inaccessible to the general public or do not qualify as a patient resource. Finally, YouTube videos were excluded as they are not within the scope of this analysis. Figure 1 illustrates the initial screening and selection process.
Fig. 1.
Screening and selection process through the phases of our systematic search.
Data Assessment
The Websites were characterized by type, location, credentials of the author, and intended target. Websites were also examined for the presence or absence of 2 Website certificates: The Information Standard and the Health on the Net Foundation Code of Conduct (HONCode). The sites are certified if they meet required standards of health information reliability.15,16
Three reviewers independently assessed all included Websites and were blinded to each other’s results. Blinding was only removed once all data had been submitted for analysis to the independent statistician. The content analysis focused on each Website as a whole and included all of the web pages within the site that referenced soft-tissue fillers.
Patient Information Evaluation Instruments
Readability Assessment
To assess the ease of which Website content could be read, 2 validated readability tools were used: the Flesch Reading Ease (FRE) and Flesh-Kincaid (FK) grade level scores.17,18 FRE and FK grade levels are inversely related; therefore, a higher FRE should correlate to a lower FK grade level.
Accessibility, Usability, and Reliability Assessment
The Minervation Validation Instrument for Health Care Websites (LIDA tool version 1.2) was used to assess the accessibility, usability, and reliability of all the Websites. LIDA is a validated instrument produced by Minervation, a company based in the United Kingdom.19 This tool has been used by researchers to test the quality of online information in a variety of specialties including cardiology,20 orthopedics,13 urology,17 and oncology.21 The tool consists of 41 questions scored on a 4-point scale (0 = never, 1 = sometimes, 2 = mostly, 3 = always) and divided into 3 domains: accessibility score (out of 60), usability (out of 54), and reliability (out of 27).22
(1)Accessibility criteria: “can users, including those with disabilities, access the information on the Website?” For this portion, an online component of the tool is used to check the Website’s compliance with legal accessibility standards. In addition, this section is used to check the Websites compatibility across multiple operating systems (Windows, Linux, and Macintosh) and browsers (Mozilla Firefox, Google Chrome, Internet Explorer, and Apple Safari).
(2)Usability criteria: “can users find what they need in an effective manner?” This assesses Website clarity, consistency, functionality, and engageability.
(3)Reliability criteria: “does the site provide comprehensive, relevant, and unbiased information?” This assesses Website currency, conflicts of interest, and content production methods.
Quality Assessment
The quality assessment was performed using 2 measures: the validated “Ensuring Quality Information for Patients” (EQIP36) tool2,23 and the Journal of the American Medical Association (JAMA) benchmarks.3
The EQIP36 tool is an instrument that has been expanded to meet the International Patient Decision Aid Standards criteria and the British Medical Association’s guidelines for patient information appraisal.24 This tool consists of 36 items that assess content, identification, and structure data of patient information. The modified EQIP36 is scored on a binary scale (yes vs no or N/A).2,23
To serve as an additional quality measure, the JAMA benchmarks were used. The benchmarks consist of 4 core standards scored on a binary scale of yes versus no or N/A and include criteria around authorship, attribution, disclosure, and currency.3
Accuracy Assessment
To assess accuracy, the 3 reviewers independently rated each Website on a scale of 1 to 4.25,26 A score of 1 represented agreement with 0% to 25% of the content; 2 represented agreement with 26% to 56%; 3 represented agreement with 51% to 75%; and 4 represented agreement with 76% to 100%. This was done following all other instruments and was a cumulative assessment. Reviewers compared the accuracy to information obtained from a review of the literature that was compiled in an excel document. This included referenced details on the soft-tissue fillers procedure and its risks and benefits.
Statistical Analysis
An independent statistician analyzed the data, including descriptive statistics. Where the outcome measurement was continuous, linear regression was performed. Where the outcome was ordinal, proportional odds modeling was used. All of the models were adjusted for the difference between the raters. All analyses were performed using the R statistical language (version 3.2.1) in the R-Studio GUI.
RESULTS
Initially, 360 Websites were screened. Of these, 295 Websites were excluded from the study: 239 duplicates, 49 creams or ointments, 5 scientific articles, and 2 YouTube videos. A total of 65 Websites underwent full content and quality analysis (Fig. 1).
Characteristics of the Website
Table 1 displays the characteristics of the Websites based on location, target, category, and credentials. The majority of Websites were based in the United States (84.6%). Of the 65 Websites, the majority (86.2%) were targeted toward patients.
Table 1.
Characteristics of the Websites
The Websites were characterized by category and credentials based on declared authorship. Most Websites were produced by physician private practice firms (36.9%), whereas a minority were produced by professional societies (9.2%), encyclopedias (6.2%), and academic centers (4.6%).
In terms of credentials, the majority of Websites were authored by board-certified plastic surgeons or dermatologists (35.4%), followed by nonphysicians (27.7%). Only 2 of the 65 Websites were certified by the HONCode or the Information Standard.
Website Readability
Overall, sites had a mean FK grade level of 11.9 ± 2.6. The mean FRE score was 45.8 ± 10.3, which is within the range of having a readability level of “difficult.” Of all the categories, on average, medical spas and consumer guides had the highest FK grade levels (13.28 and 13.03, respectively) and professional societies had the lowest with a mean grade level of 10.6 (Fig. 2).
Fig. 2.
Average FK grade level by category.
LIDA Tool: Accessibility, Usability, and Reliability
The mean scores for the LIDA tool are reported in Table 2. Results of the statistical analyses for this tool are reported in Table 3.
Table 2.
Mean Scores and Results for JAMA Benchmarks and Accuracy
Table 3.
Results by Evaluation Tool
Physician private practice sites and medical spas were found to be less accessible, usable, and reliable when compared with academic centers (P < 0.05). Government agencies were found to be more accessible and reliable when compared with academic centers (P < 0.05).
Website Quality
Quality—JAMA Benchmarks
Out of a total of 4 points for the JAMA quality benchmarks, 61% of Websites scored between 1 and 2 points, 20% scored between 2 and 3 points, and 18.5% scored between 3 and 4 points.
When stratified by category, the majority of physician private practice Websites (79.2%), medical spas (83.3%), and consumer guides (69.2%) scored between 1 and 2 out of 4 on the JAMA quality benchmarks (Fig. 3). The majority of academic centers and professional societies scored between 2 and 3. Finally, all Websites categorized as an encyclopedia or government agency scored between 3 and 4 on the JAMA benchmarks (Fig. 3).
Fig. 3.
JAMA quality scores by category.
Quality—EQIP36 Tool
Out of a total of 36 points on the EQIP36 tool, the mean Website score was 17.0 ± 5.0. When compared with academic centers, media sources (P < 0.05), physician private practice (P < 0.05), and consumer guides (P < 0.05) had lower quality scores (Table 3).
When compared with authors who are board certified in plastic surgery or dermatology, sites authored by board-certified other (P < 0.001), nonphysicians (P < 0.05), and doctors of osteopathy (P < 0.001) had lower quality scores (Table 3).
Accuracy
Overall, based on the mean scores between the 3 reviewers, 12.3% of Websites scored <25% on accuracy, and 23.1% of Websites scored between 26% and 50% on accuracy. Finally, 32.3% of Websites scored between 51% and 75%, and 32.3% of Websites scored between 76% and 100% on the accuracy scale.
Based on category, compared with academic centers, media sources (P < 0.001), physician private practice sites (P < 0.05), medical spas (P < 0.05), and consumer guides (P < 0.05) had lower accuracy scores.
Compared with authors who are board certified in plastic surgery or dermatology, sites with an unknown author/provider (P < 0.001), doctors of osteopathy (<0.001), and authors under board-certified other (P < 0.05) had lower quality scores overall.
Top Sites Overall
Table 4 indicates the top 10 scoring Websites overall and their Google search rank. In cases where the scores were tied, the reviewers reassessed the entries, and agreement was determined by consensus.
Table 4.
Top 10 Scoring Websites Overall Across All Evaluation Tools and Their Google Search Rank
DISCUSSION
The Internet has become one of the most important sources of online patient information. Its accessibility, ease of use, and vast collection of information make it appealing to both patients and physicians. The emergence of the “self-educated” patient has changed the medical landscape arguably for the better as patients have gained increased autonomy over their healthcare. Many researchers in various specialties of medicine have attempted to analyze the online health content that their patients access.17,27,28 With the rise in popularity of soft-tissue fillers and the potential for devastating complications, it is vital to understand what type of information patients are using to inform their choices regarding this procedure. To our knowledge, this is the first study of its kind to thoroughly examine online health information for this procedure.
Our analysis indicates that the majority of information is authored by board-certified plastic surgeons or dermatologists (35.4%) that operate a private practice (36.9%). The inherent conflict of interest in this type of resource can often lead to incomplete disclosure of procedural risks and benefits. In fact, our study found that physician private practice sites did not report blindness at all and were only 12% as likely as the other categories to report skin necrosis as a potential complication. Incomplete disclosure of the risks and benefits of the procedure resulted in lower quality scores using the EQIP36 tool. Furthermore, our research indicated that physician private practice sites were found to have some of the lowest scores across all domains assessed. Previous studies have correlated poor quality scores with an increase in the amount of promotional advertising displayed.17 Although not statistically significant in our models, it is interesting to note that physician private practice sties, along with medical spas and consumer guides, were the only categories to offer numerous incentives, medical spa promotions, and television/magazine endorsements (Fig. 4).
Fig. 4.
Percent of marketing advertisements offered by category.
Although having good quality information is vital, it is equally, if not more, important to have a site that is easy to navigate so that the desired information can be found. Through our analysis, we found that the majority of Websites were not well organized, lacked visual appeal, or took multiple clicks and links before appropriate information could be found. Interestingly, using the online component of LIDA to assess accessibility, it was found that authors who were not certified or not certified with the American Board of Medical Specialties (ABMS) had sites that were more accessible. This demonstrates a need to improve accessibility for sites with higher quality information such as those run by professional societies and academic centers. The accessibility, usability, and reliability analysis using the LIDA tool returned low mean scores overall, particularly in the latter 2 domains (Table 2). Factors that lead to lower reliability scores included information that was not current or not referenced and inadequate reporting of conflicts of interest. Specifically, it was significant that physician private practice sites, medical spas consumer guides and pharmaceutical companies, had lower scores as compared with academic centers.
Readability is an important domain to consider in all sources of patient information. The US National Institute of Health recommends that patient information be written between a sixth- and seventh-grade level. Our study revealed that Websites across all categories had written content at a level that is too difficult to read and comprehend for many patients. This is similar to the results of recent research that analyzed readability of online patient material for breast reconstruction and found that all sites had exceeded the recommended age level.28 Although professional societies had the best scores in our results at a 10.62-grade level, this is still well above the recommending average. This indicates a need for improvement in the readability of online patient material for soft-tissue fillers.
As the Internet era is here to stay, it is important for physicians to continue to adapt and familiarize themselves with good quality Websites that they are comfortable recommending to patients. We found that the majority of Websites met less than 50% of the JAMA quality benchmarks. Although physician private practice sites did fare the worst, academic centers, encyclopedias, and professional societies had some of the highest scores overall. We found similar results when using the EQIP36 tool, which provided a very rigorous approach to assess quality. Providing complete details on the procedure, balanced information on the risks and benefits, and directing patients to other sources of reliable information contributed to higher quality scores.
Looking at all domains, we found that sites from academic centers, professional societies, and encyclopedias offered the most suitable information on soft-tissue fillers. However, the overall accessibility, usability, reliability, readability, quality, and accuracy of all Websites on this topic should be improved to provide patients with more appropriate information. To direct patients to the best current information on soft-tissue fillers, we recommend sites found in Table 4.
As previous researchers have indicated, the medical community should act as custodians of healthcare material. It is important, therefore, that plastic surgeons ensure online health information in the field of plastic surgery is of good quality, reliability, and accuracy. To accomplish this goal, regular audits of online medical content should be employed through similar studies. By this mechanism, we can remain up to date on what patients are accessing online. We can then confidently direct patients to trusted sources of information on plastic surgery procedures like soft-tissue fillers.
LIMITATIONS
A limitation of this study is not screening beyond the first 3 pages of search results for each search term. In future studies of this nature, additional screening can be undertaken.
Footnotes
Disclosure: The authors have no financial interest to declare in relation to the content of this article. The Article Processing Charge was paid for by the authors.
REFERENCES
- 1.Cohen RA, Adams PF. Use of the internet for health information: United States, 2009. NCHS Data Brief. 2011;66:1–8. Available at: http://www.ncbi.nlm.nih.gov/pubmed/22142942. [PubMed] [Google Scholar]
- 2.Frueh FS, Palma AF, Raptis DA, et al. Carpal tunnel syndrome: analysis of online patient information with the EQIP tool. Chir Main. 2015;34:113–121. doi: 10.1016/j.main.2015.04.003. [DOI] [PubMed] [Google Scholar]
- 3.Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor–Let the reader and viewer beware. JAMA. 1997;277:1244–1245. [PubMed] [Google Scholar]
- 4.Google Trends - Web Search interest: dermal fillers - Worldwide, 2004 - present. Available at: https://www.google.ca/trends/explore#q=dermalfillers. Accessed October 22, 2015.
- 5.American Society of Aesthetic Plastic Surgeons. Cosmetic Surgery National Data Bank Statistics. New York, NY: ASAPS; 2014. pp. 1–23. [Google Scholar]
- 6.American Society of Plastics Surgeons. Plastic Surgery Statistics Report 2014. 2014. pp. 1–23. Available at: http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Plastic+Surgery+Statistics+Report#1. [Google Scholar]
- 7.American Society of Plastics Surgeons. 2010 Plastic Surgery Procedural Statistics. 2010. [Google Scholar]
- 8.Wollina U, Goldman A. Dermal fillers: facts and controversies. Clin Dermatol. 2013;31:731–736. doi: 10.1016/j.clindermatol.2013.05.010. [DOI] [PubMed] [Google Scholar]
- 9.Administration USF and D. Soft tissue fillers. Dermatol Nurs. 2002;14(3):206. Available at: http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/CosmeticDevices/WrinkleFillers/ucm2007470.htm. Accessed September 14, 2015. [PubMed] [Google Scholar]
- 10.Daines SM, Williams EF. Complications associated with injectable soft-tissue fillers: a 5-year retrospective review. JAMA Facial Plast Surg. 2013;15:226–231. doi: 10.1001/jamafacial.2013.798. [DOI] [PubMed] [Google Scholar]
- 11.Carruthers JD, Fagien S, Rohrich RJ, et al. Blindness caused by cosmetic filler injection: a review of cause and therapy. Plast Reconstr Surg. 2014;134:1197–1201. doi: 10.1097/PRS.0000000000000754. [DOI] [PubMed] [Google Scholar]
- 12.Online Consumer Search Engine Trends | Experian Marketing Services. Available at: http://www.experian.com/marketing-services/online-trends-search-engine.html. Accessed September 14, 2015.
- 13.Küçükdurmaz F, Gomez MM, Secrist E, et al. Reliability, readability and quality of online information about femoracetabular impingement. Arch Bone Jt Surg. 2015;3:163–168. [PMC free article] [PubMed] [Google Scholar]
- 14.Peterson G, Aslani P, Williams KA. How do consumers search for and appraise information on medicines on the Internet? A qualitative study using focus groups. J Med Internet Res. 2003;5:e33. doi: 10.2196/jmir.5.4.e33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Boyer C, Baujard V, Geissbuhler A. Evolution of Health Web certification through the HONcode experience. Stud Health Technol Inform. 2011;169:53–57. [PubMed] [Google Scholar]
- 16.The Information Standard. http://www.england.nhs.uk/tis/. Accessed September 16, 2015.
- 17.Redmond CE, Nason GJ, Kelly ME, et al. Transrectal ultrasound guided biopsy of the prostate: is the information accessible, usable, Reliable and Readable? Curr Urol. 2015;8:32–37. doi: 10.1159/000365686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.How to Write Easy-to-Read Health Materials: MedlinePlus. Available at: https://www.nlm.nih.gov/medlineplus/etr.html. Accessed September 15, 2015.
- 19.LIDA tool: the Minervation validation instrument for healthcare websites. Available at: http://www.minervation.com/lida-tool/. Accessed September 15, 2015.
- 20.Kirthi V, Modi BN. Coronary angioplasty and the internet: what can patients searching online expect to find? J Interv Cardiol. 2012;25:476–481. doi: 10.1111/j.1540-8183.2012.00748.x. [DOI] [PubMed] [Google Scholar]
- 21.Grewal P, Alagaratnam S. The quality and readability of colorectal cancer information on the internet. Int J Surg. 2013;11:410–413. doi: 10.1016/j.ijsu.2013.03.006. [DOI] [PubMed] [Google Scholar]
- 22.Minnervation Ltd. The LIDA Instrument: Minervation Validation Instrument for Health Care Web Sites. Oxford: Minnervation Ltd; 2004. pp. 1–3. [Google Scholar]
- 23.Melloul E, Raptis DA, Oberkofler CE, et al. Donor information for living donor liver transplantation: where can comprehensive information be found? Liver Transpl. 2012;18:892–900. doi: 10.1002/lt.23442. [DOI] [PubMed] [Google Scholar]
- 24.Charvet-Berard a. I, Chopard P, Perneger T V. Measuring quality of patient information documents with an expanded EQIP scale. Patient Educ Couns. 2008;70(3):407–411. doi: 10.1016/j.pec.2007.11.018. [DOI] [PubMed] [Google Scholar]
- 25.Dy CJ, Taylor SA, Patel RM, et al. The effect of search term on the quality and accuracy of online information regarding distal radius fractures. J Hand Surg Am. 2012;37(9):1881–1887. doi: 10.1016/j.jhsa.2012.05.021. [DOI] [PubMed] [Google Scholar]
- 26.Morr S, Shanti N, Carrer A, et al. Quality of information concerning cervical disc herniation on the Internet. Spine J. 2010;10:350–354. doi: 10.1016/j.spinee.2010.02.009. [DOI] [PubMed] [Google Scholar]
- 27.Sobota A, Ozakinci G. The quality and readability of online consumer information about gynecologic cancer. Int J Gynecol Cancer. 2015;25:537–541. doi: 10.1097/IGC.0000000000000362. [DOI] [PubMed] [Google Scholar]
- 28.Vargas CR, Kantak N a., Chuang DJ, Koolen PG, Lee BT. Assessment of online patient materials for breast reconstruction. J Surg Res. 2015:3–9. doi: 10.1016/j.jss.2015.04.072. [DOI] [PubMed] [Google Scholar]