Abstract
Introduction:
Nowadays, hospitals have turned increasingly towards the Internet and develop their own web presence. Hospital Websites could be operating as effective web resources of information and interactive communication mediums to enhance hospital services to the public.
Aim:
Therefore, the aim of this study was to assess the quality of websites in Tehran’s public hospitals.
Material and methods:
This cross-sectional analysis involved all public hospitals in Iran’s capital city, Tehran, with a working website or subsites between April and June, 2014 (N=59). The websites were evaluated using three validated instruments: a localized checklist, Google page rank, and the Alexa traffic ranking. The mentioned checklist consisted of 112 items divided into five sections: technical characteristics, hospital information and facilities, medical services, interactive on-line services and external activities. Data were analyzed using descriptive and analytical statistics.
Results:
The mean website evaluation score was 45.7 out of 224 for selected public hospitals. All the studied websites were in the weak category based on the earned quality scores. There was no statistically significant association between the website evaluation score with Google page rank (P=0.092), Alexa global traffic rank and Alexa traffic rank in Iran (P>0.05). The hospital websites had a lower quality score in the interactive online services and external activities criteria in comparing to other criteria. Due to the low quality level of the studied websites and the importance of hospital portals in providing information and services on the Internet, the authorities should do precise planning for the appreciable improvement in the quality of hospital websites.
Keywords: Evaluation, Hospital Website, Quality, Content
1. INTRODUCTION
The Internet is a widespread, accessible means of communication, which has provided scientific space for exchanging the health information (1) and also the preferred choice of health information seekers to access the required information (2). Nowadays, the web has become one of the most crucial aspects of social life (3). In this context, the use of the Internet is increasingly grown for health purposes, and today it is a valuable source of health information for patients and their families (4).
In Iran, the number of Internet users, Persian health websites and the online health seekers continues to increase in recent years (4-6). Benefits of the Internet and its widespread use in modern societies have led hospitals to use this valuable technology to offer their services. They make great efforts to create and improve their websites to provide online information as well as the development of mutual and interpersonal communication (7). Therefore, hospital websites which have suitable design and high quality content, are used as a valuable medium in improving hospital services for public. Also, these sites are good sources of reference for general information about the hospital and can empower users to learn issues related to health, diseases and medication (2, 8).
Literature review in Iran showed that the majority of researchers investigated online health websites (9) and the websites of Iranian universities of medical sciences (10-13), and some studies has been done to evaluate hospital websites (7, 14, 15). According to studies in this field, some problems on the hospital websites were reported, such as, poor quality of medical services reservation and weaknesses in website management and application (4, 16-19). The active presence on the Internet has increasingly become a commitment to the health organizations. These organizations must develop websites with rich content and useful information, and continually monitor the performance of their website. Having a suitable and powerful portal was mentioned as one of the daily essential items in each hospital (14). In this regard, public hospitals websites are of particular importance as a means of providing electronic services (18, 19). Therefore, this study aimed to assess Tehran public hospitals’ websites.
2. MATERIALS AND METHODS
This study is a cross-sectional analytical study, which was conducted in 2014. The sample consisted of all active websites in Tehran public hospitals. To select a sample of websites, first of all, a list of 107 public hospitals of Tehran province was extracted from the hospital information and statistics website of Iran ministry of health and medical education (AVAB) (20).
At the time of study, 48 hospitals were excluded due to lack of active website or having a web-log, and finally 59 active hospital sites and sub-sites were studied. Then, Internet addresses of selected websites were determined by using the Avab website, entering the name of the hospital as a keyword in the Google search engine, or hospitals menu or health centers in the websites of Tehran province’s universities of medical science. The websites of the mentioned hospitals were studied by using a localized check list, Google PageRank (12) and Alexa traffic rank (25). The main structure of the checklist used in this study is based on a 89-items ad hoc Codebook developed by Maifredi et al (21). Its face and content validity, consistency between observers and internal coherence were approved for general use (4, 21). The ad hoc Codebook tool due to important features such as the integration of the items previously proposed by researchers studying hospital websites (19,21, 22, 23). The WHO forward/backward translation protocol was used to translate the English Codebook into Persian. New items were derived from circulars issued by Iran’ Ministry of Health and Medical Education Deputy of Curative Affairs (24) and also from the proposed variables of previous studies (7, 14). Finally, the 112 items localized checklist was designed by adding new items to the sections of technical content (12 items), hospital information and facilities (8 items), and external activities (3 items) in the Codebook to assess the qualitative assessment of the hospital websites. The validity of this tool for use in Iran was approved by a panel consisting of 8 experts in the field of library and medical information (N=2), health information management (N=2), Health service Management (N=2) and Medical Informatics (N=2). The reliability of this tool was approved by using the Kappa inter- rater agreement values of greater than 0.73. Therefore, the checklist used in this study consisted of 112 items divided into five sections, focusing on different contents: Technical contents: 31 items that included things like a site map and internal search engine, the certification of accessibility to people with disabilities provided by the Web Accessibility Initiative (WAI), W3C-css and W3Chtml technical informatics standards certifications, and the Health On the Net (HON) foundation’s logo as a proof of the validity of medical information websites.
Hospital information and facilities: 30 items concerning general Information, which included items such as the hospital age, location of hospitals and ways of reaching the hospital, and contact details of public relations office.
Medical Services and acceptance: 25 items related to hospital admission, discharge and everyday life during the hospitalization period and information about the doctors employed at the hospital. Interactive online services: using the Internet in hospitals was examined by 10 items, such as the availability of online reservations, being able to communicate with the hospital via Internet or email, and the presence of a health-related forum. External activities: this segment was consisted of 16 items, such as being able to obtain health information, job opportunities, and a list of conferences organized by the hospital (21). Using the Google Toolbars, the integer value (range 0 to 10) on the toolbar was recorded for all home pages of the selected websites on Oct 23, 2014. Also, the global and Iran Alexa traffic rank were extracted from the Alexa website on Oct 22, 2014. The results were analyzed by using descriptive and analytical approach. The evaluation score of each website and its main items was calculated on the basis of determining the presence or absence of each of the 112 items in the checklist (2=Yes, 1= Somewhat, 0 = No). It should be noted that, based on the consensus opinions of the experts’ panel, one point is considered for “somewhat” answer”. It can allow us to make a more accurate assessment of websites. The evaluated websites were classified with respect to the domain and total score percentage of qualitative assessment. As the percentage less than 50, between 50 to 75 and more than 75 were considered as weak, average and good.
3. RESULTS
Among the 107 public hospitals in Tehran, only 59 hospitals (55.1%) had an active website or sub-site. The characteristics of the studied hospitals were summarized in Table 1. Table 2, shows the evaluation results of the hospital websites. According to this table, all the referred websites were in weak quality (less than 50%) based on the average percentage of the total score (22.4%). However, the hospital websites achieved lower percent in criteria of “interactive online services” (4.7% of total score) and “external activities” (11% of total score) compared with other criteria. The results of the correlation study between the total quality score and each evaluated domain of the hospital websites showed that there was a significant positive relationship between the scores of three domains of website quality (hospital information and facilities, admission and medical service, and external activities) with total score (0.70< r <0.89, P< 0.001). Also, a significant relationship existed between these three sections of website evaluation and also the total score with activity and ownership type of the studied hospitals (p≤0.001). On the other hand, the academic teaching hospitals got higher scores than the others in these three domains and the total score (Table 3).
Table 1.
Hospital Characteristics
Table 2.
Website evaluation scores for each section in Tehran’s public hospitals
Table 3.
The evaluation score of the hospital websites based on activity and ownership type
Whereas, statistically significant difference was not seen between websites evaluation scores in general, specialized and super-specialized hospitals (P>0.05). As well as, there was no significant correlation between the hospitals age with the total quality score, Google PageRank and Alexa score in the world (P>0.05). There was no website with the Google PageRank more than 5. The majority of websites (56%) had a PageRank of 4/10. The teaching-treatment hospital sites had a better Google PageRank than the others. Although, any significant difference was not seen in the Google PageRank among the hospitals in terms of ownership (p=0.31), Care (p=0.28) and activity (p=0.55). The mean of Google PageRank for the websites was (2.8±1.7). There were only 33 (56%) websites with global Alexa ranking below 100,000 and 37 (62.7%) websites with Iran Alexa rank below 1000.
As Table 4 indicates, no significant correlation existed between the scores of five evaluated domains with Google PageRank and also world Alexa rank, while there was only a low significant inverse correlation between the global Alexa rank and the Google PageRank (r = -0.391, P = 0.002). Although, global Alexa traffic rank had a significant positive correlation with the Iranian ranking (r= 0.575, P < 0.001).
Table 4.
The correlation between the domains scores with Google page rank, Alexa rank in Iran & world. * Correlation is significant at the 0.05 level
In addition, the kind of ownership and activity had no significant correlation with the Iran Alexa score, but was related to global Alexa rank (P<0.001). As, the non-academic hospitals had the higher world Alexa rank. Also, there was a significant statistical difference about the Iran Alexa traffic score among the websites of hospitals providing general, specialty and super-specialty care (p=0.016). The super-specialized Hospitals had a higher Iran ranking score. However, level of care had not significant effect on the other scores (website quality scores, world Alexa ranking and Google PageRank)
Also, the type of hospital website (dedicated website/sub-site) had a significant relationship with the two domains of website evaluation [external activities (p=0.01) and technical content (P=0.02)] but was not associated with scores of the other areas. There was a statistical significant difference between the Google PageRank in the studied websites and sub-sites, (P < 0.001), i.e. the hospital dedicated websites had a higher Google PageRank.
4. DISCUSSION
The results showed that a large percentage of public hospitals in Tehran (44.9) had no active website or subsite. Similarly, in recent studies, there was no active website in a large number of hospitals (18, 14, 4). According to the results of this study, the content of hospital websites in all studied hospitals was in poor quality level, and they had poorer one in the areas of interactive online services and external activities. In like manner, previous Iranian researches have mentioned to the inappropriate condition of the Tehran private hospital ports (14) and also the very weak level of the content of Iranian educational hospital websites (7). Furthermore, studies in other countries indicate the poor quality of hospital websites (4, 17, 19, 26). All the assessed hospitals were lacking certifications concerning the technical content. Also, the general disclaimer, copyright notice and treatment of surfer personal data statement were not found in any of the websites. Also, in the study of Alipour Hafezi and colleagues, feature of web accessibility for people with disabilities was observed just in one port of the private hospitals (14).
The results of the current study revealed that the hospitals received a higher score in the area of technical content, hospital information and facilities. The portals of private hospitals in Tehran were in good or relatively favorable condition in terms of page design, and contact information (14). To improve quality of hospitals or medical web sites, these sites should consist items, such as the introducing hospital departments and facilities, the price list and the cost of using the facilities in various wards and complete information about physicians (27, 28). The academic teaching hospital websites had better quality than the nonacademic ones in these three areas (hospital information and facilities, admission and medical service, and external activities) and total score. This difference can be resulted from the hospital educational environment and the emphasis of Iran Ministry of Health to these hospitals upon having a qualified websites. In the Italian study, the public hospitals got the better website evaluation score than the private ones private (4).
The majority of studied websites were on the average level of Google PageRank (4/10), and also a few websites had a good Alexa ranking. As well as, there was no statistically significant association between the hospital website evaluation score with Google page and Alexa rank. In researches done in this area, similar results were obtained about these ranks in Persian health websites (9) and the websites of the Iranian Universities of Medical Sciences (29, 10). Moreover the correlation was not found between the popularity (Alexa rank) and importance (PageRank value) of the health websites with their quality scores (30). While based on the result of the present study, Google page and Alexa rank did not properly reflect the real quality of evaluated hospital website. This can be resulted from that the rank of hospital subsites was belonged to reference universities of medical science. The search icon (as a technical content question) was seen in the home pages of the some hospital websites, but it did not work or the search was done on the main reference website instead of subsite. While, a good search facility can save customer’s time and hence fetch popularity for the website, which will consequently improve the business. To investigate the ability to communicate with the hospital via the Internet or e-mail, one test email was sent to email address of all hospitals and e-mail accountability was checked for a three month period. But, only two hospitals had quick response and one hospital replied the email at an interval of two months. As well as, there was not a health-related forum and the possibility of interacting with any hospital through the “Contact Us” menu. The majority of the Greece public hospital websites were also failed to support necessary web-based transactions between the institutions and their stakeholders (18). Also, the provision of forum concerning health-related subjects were available in a very small percentage of Italian hospitals websites (4). The presence of website email address is considered as one of the important features for designing website (27).
Due to the low quality level of the studied websites, the several recommendations are offered to improve the quality of hospital websites: a) assigning organizations for periodic quality evaluation of hospital websites; b) labeling with defined standard logos for high-quality hospital websites; and c) establishing quality standards of hospital websites.
Acknowledgments
This work was supported by the grant number 1888 from Lorestan University of Medical Sciences (LUMS).
REFERENCES
- 1.Wen K-Y, Kreps G, Zhu F, Miller S. Consumers' perceptions about and use of the Internet for personal health records and health information exchange:analysis of the 2007 Health Information National Trends Survey. Journal of medical Internet research. 2010;12(4):73. doi: 10.2196/jmir.1668. doi:10.2196/jmir.1668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Raji SO, Mahmud M, Abubakr A. Evaluation of University Teaching Hospital Websites in Nigeria. Proceedia Technology. 2013;9:1058–64. doi:10.1016/j.protcy.2013.12.118. [Google Scholar]
- 3.Freire AP, Power C, Petrie H, Tanaka EH, Rocha HV, Fortes RP. Universal Access in Human-Computer Interaction Applications and Services. Springer; 2009. Web accessibility metrics:effects of different computational approaches; pp. 664–73. doi:10.1007/978-3-642-02713-0_70. [Google Scholar]
- 4.Maifredi G, Orizio G, Bressanelli M, Domenighini S, et al. Italian hospitals on the web:a cross-sectional analysis of official websites. BMC medical informatics and decision making. 2010;10(1):17. doi: 10.1186/1472-6947-10-17. doi:101186/1472-6947-10-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bahri N, SadeghMoghadam L, Khodadost L, Mohammadzade J, Banafsheh E. Internet addiction status and its relation with students' general health at Gonabad Medical University. Modern Care, Scientific Quarterly of Birjand Nursing and Midwifery Faculty. 2011;8(3):166–73. [In Persian] [Google Scholar]
- 6.Rezailashkajani M, Roshandel D, Ansari S, Zali MR. A web-based patient education system and self-help group in Persian language for inflammatory bowel disease patients. International Journal of Medical Informatics. 2008;77(2):122–8. doi: 10.1016/j.ijmedinf.2006.12.001. doi:10.1016/j.ijmedinf.2006.12.001. [DOI] [PubMed] [Google Scholar]
- 7.Moradi Gh, Ahmadi M, Zohoor A, Ebadifardazar F, Saberi M. Evaluation of Structure and Content of Websites of the Educational Hospitals in Iran-2007. Health Information Management. 2007;4(2):175–84. [In Persian] [Google Scholar]
- 8.Serati Shirazi M, Goltaji M. Top ranking hospitals' websites in worldwide:Webometric study. J National studies of information organization and Librarianship Quarterly. 2013;24(2):160–82. [In Persian] [Google Scholar]
- 9.Samadbeik M, Ahmadi M, Mohammadi A, Saravi BM. Health Information on Internet:Quality, Importance, and Popularity of Persian Health Websites. Iranian Red Crescent Medical Journal. 2014;16(4):e12866. doi: 10.5812/ircmj.12866. doi:10.5812/ircmj.12866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Aminpour F. How to Improve Webometric Ranks of Iranian Medical Universities. Journal of Isfahan Medical School. 2011;29(127):115–25. [In Persian] [Google Scholar]
- 11.Aminpour F, Kabiri P, Otroj Z, Keshtkar AA. Webometric analysis of Iranian universities of medical sciences. Scientometrics. 2009;80(1):253–64. doi:10.1007/s11192-008-2059-y. [Google Scholar]
- 12.Zahedi Sh. Evaluation of websites ' quality:instruments and criteria. J. Development and Evolution management. 2010;4:5–16. [In Persian] [Google Scholar]
- 13.Fathifar Z, Hosseyni F, Alibeyk M. Evaluation of Persian health and medicine websites based on silberg, discern and honcode criteria. Journal of Health Administraion. 2007;10(28):25–30. [In Persian] [Google Scholar]
- 14.Alipour-Hafezi M, Karimi M, Tofighi S. Conceptual Model to Improve Portal Services of Private Hospitals in Tehran. Health management journal. 2014;17(58):42–57. [In Persian] [Google Scholar]
- 15.Shadpour P, Teimourpour B, Asadi R. Webometrics-based Analysis and Ranking of Iranian Hospital Websites. International Journal of Hospital Research. 2013;2(2):77–84. [Google Scholar]
- 16.Noruzi A. Web Presence and Impact Factors for Middle-Eastern Countries. [[cited 2014 Oct 10]];Online. 2006 30(2):22–8. Available from: http://eprints.rclis.org/7264/1/Web_Presence-Middle-East.pdf . [Google Scholar]
- 17.Liu Ch, Arnett KP. Exploring the factors associated with Website success in the context of electronic commerce. Information & Management. 2000;38(1):23–33. doi:10.1016/S0378-7206(00)00049-5. [Google Scholar]
- 18.Patsioura F, Kitsiou S, Markos A. Evaluation of Greek public hospital websites. ICE-B 2009 - International Conference on E-business. [[cited 2015 Jan 10]]. pp. 223–229. Available from: http://www.amarkos.gr/en/research/pkm-ICEB2009.pdf . doi:10.5220/0002194402230229.
- 19.Mira JJ, Llinás G, Tomás O, Pérez-Jover V. Quality of websites in Spanish public hospitals. Informatics for Health and Social Care. 2006;31(1):23–44. doi: 10.1080/14639230500519940. [DOI] [PubMed] [Google Scholar]
- 20. http://avab.behdasht.gov.ir .
- 21.Llinás G, Rodríguez-Iñesta D, Mira JJ, Lorenzo S, Aibar C. A comparison of websites from Spanish, American and British hospitals. Methods Inf Med. 2008;47(2):124–30. doi:10.3414/ME0474. [PubMed] [Google Scholar]
- 22.Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web:a systematic review. JAMA. 2002;287(20):2691–2700. doi: 10.1001/jama.287.20.2691. [DOI] [PubMed] [Google Scholar]
- 23.Minerva working group 5:Quality Principles for Cultural Websites:a handbook. 2005. Available from: http://www.minervaeurope.org/publications/qualitycommentary/qualitycommentary050314final.pdf .
- 24.Deputy of Curative Affairs, Ministry of Health and Medical Education, Circular for hospital websites characteristics. [[Cited 2015 May 10]]. Available from http://medcare.behdasht.gov.ir .
- 25.Bageban H, Toodar R, Samdaliri Z, Naserimalvani A. The Evaluation of Islamic Azad University Website System and the Status of the Islamic Azad University Marvdasht Branch in this System. Journal of Information and Communication Technology in Educational Sciences. 2011;1(4):59–79. [In Persian] [Google Scholar]
- 26.Huerta TR, Hefner JL, Ford EW, McAlearney AS, Menachemi N. Hospital website rankings in the United States:expanding benchmarks and standards for effective consumer engagement. Journal of medical Internet research. 2014;16(2):e 64. doi: 10.2196/jmir.3054. doi:10.2196/jmir.3054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Rafe V, Monfaredzadeh M. A qualitative framework to assess hospital/medical websites. J Med Syst. 2012;36(5):2927–39. doi: 10.1007/s10916-011-9771-5. doi:10.1007/s10916-00. [DOI] [PubMed] [Google Scholar]
- 28.Hagigi N, Fakhrefatemi N. Assessing postgraduate students'view about affecting factors in quality of academic and informational websites in governmental universities in Tehran. Higher education in Iran Journal. 2010;2(4):29–53. [In Persian] [Google Scholar]
- 29.Erfanmanesh MA, Didgah F. Structural analysis, link analysis, and visit rate of the websites of the Iranian medical universities. Studies in library and information science. 2009;16(3):147–76. [In Persian] [Google Scholar]
- 30.Griffiths KM, Christensen H. Internet-based mental health programs:A powerful tool in the rural medical kit. Australian Journal of Rural Health. 2007;15(2):81–7. doi: 10.1111/j.1440-1584.2007.00859.x. doi:10.1111/j.1440-1584.2007.00859.x. [DOI] [PubMed] [Google Scholar]