Abstract
Introduction
Health care managers and personnel should be aware and literate of health information system in order to increase the efficiency and effectiveness in their organization. Since accurate, appropriate, precise, timely, valid information and interpretation of information is required and is the basis for policy planning and decision making in various levels of the organization. This study was conducted to assess the district health information system evolution in Iran according to WHO framework.
Methods
This research is an applied, descriptive cross sectional study, in which a total of twelve urban and eight rural facilities, and the district health center at Falavarjan region were surveyed by using a questionnaire with 334 items. Content and constructive validity and reliability of the questionnaire were confirmed with correlation coefficient of 0.99. Obtained data were analyzed with SPSS 16 software and descriptive statistics were used to examine measures of WHO compliance.
Results
The analysis of data revealed that the mean score of compliance of district health information system framework was 35.75 percent. The maximum score of compliance with district health information system belonged to the data collection process (70 percent). The minimum score of compliance with district health information system belonged to information based decision making process with a score of 10 percent.
Conclusions
District Health Information System Criteria in Isfahan province do not completely comply with WHO framework. Consequently, it seems that health system managers engaged with underlying policy and decision making processes at district health level should try to restructure and decentralize district health information system and develop training management programs for their managers.
Key words: Health care, management, HIS
1. INTRODUCTION
Organizational development and management in general, and healthcare organizations in particular is one of fundamental elements for organizational excellence and improvement of infirmity which otherwise, could have deleterious consequences. Dominant, effective and efficient management can ensure organizational development. So that, healthcare services managers and providers should have enough knowledge of health information management; Because existence and utilization of precise, valid, timely and credible data and information are the bases of decision making, policy and planning development (1, 2).
District Health Information System (DHIS) in itself is a mirror that shows the picture of DHIS to managers and policy makers, in district and national level. Whatever information transparency with assistance of proper and efficient information technology, and integrity in its components, shows a better picture of the organization, organizational units and services delivery methods (1). Therefore, reinforcement and improvement of DHID is one of the first steps in management and improvement of DHIS (3).
Unfortunately, in most of the developing countries, DHIS is inefficient in provision of management information. In these countries, there are some elements that limit utilization of DHIS and reduce effectiveness of healthcare services management. These elements include: lack of DHIS infrastructures, proper assessment of essential information needs, proper data gathering system, proper data processing and analyzing methods, appropriate methods of information presentation, proper interpretation of accumulated information, and lack of appropriate information based decision making and policy development (4, 5, 6, 7).
In one study by Humberto Muquingue on DHIS network in 14 African countries, the result showed that:
Data gathering has low priority.
Data is unreliable.
Data gathering is done continuously, although it has negative impact on proper information flow and analysis in their healthcare systems (8).
Also, one study in 2006 in Malawi on health information systems showed the challenges include:
In competency of data gathering in district level.
In competency of data analysis and distribution (9).
Regarding DHIS impacts in healthcare management, DHIS continuous assessment and improvement is necessary. In this study, DHIS assessment has been assessed in 8 following aspects:
DHIS infrastructures (training, hardware, software) (10).
Needs assessment of essential information (essential information needs for health facility patient management, health facility management, district healthcare center and facility performance assessment) (11).
Data gathering (data gathering tools for facility patient management, Warnerable diseases data gathering, administrative data gathering, organizational unit data gathering, and facility infrastructures data gathering) (12).
Data processing (data quality features, correcting of data errors, preventing of future data errors, horizontal and vertical data transmission) (13).
Data analysis (facility self-assessment, general indictors, human resources, transportation, laboratory, and financial indicators in DHIS) (14).
Information presentation (charting criteria of maternal health, vaccination, nutrition, epidemic diseases, therapeutic healthcare, venereal diseases, tuberculosis, and administrative activities) (2).
Information interpretation (comparing obtained information with national, provincial, and district goals, other health facilities, standards and criteria, and previous same period information) (2, 10).
Utilization of information for decision making (monthly written feedback from district health information unit to facility level, computerized reports for decision making, criteria for utilization in decision making) (15).
2. METHODS
This research is an applied, descriptive cross sectional study, in which a total of twelve urban and eight rural facilities, and the district health center at Falavarjan region were surveyed by using a questionnaire with 334 items. Content and constructive validity and reliability of the questionnaire were confirmed with correlation coefficient of 0.99. Obtained data were analyzed with SPSS 16 software and descriptive statistics were used to examine measures of WHO compliance.
3. RESULTS
Data in Table 1 appears DHIS aspects that in this study were surveyed and their results contain in Tables 2 and 3. Important findings include followings:
Table 1.
List of DHIS Aspects in This Study
| Infrastructures include 4G criteria. Essential information needs assessment include 32 criteria. Data gathering include 50 criteria. Data processing includes 41 criteria. Data analysis includes 65 criteria. Information presentation includes 49 criteria. Information interpretation includes 9 criteria. Utilization of information for decision making include 42 criteria. |
Table 2.
An Overview of DHIS Criteria Compliance with WHO framework
| Compliance Percentage | |||||
|---|---|---|---|---|---|
| Row | DHIS Aspects | District (Total) | District Health Center | Urban Health Facilities | Rural Health Facilities |
| 1 | Infrastructures | 21 | 45 | 20 | 20 |
| 2 | Essential information needs assessment | 60 | 80 | 60 | 60 |
| 3 | Data gathering | 70 | 80 | 72 | 72 |
| 4 | Data processing | 55 | 65 | 50 | 50 |
| 5 | Data analysis | 32 | 67 | 32 | 32 |
| 6 | Information presentation | 13 | 62 | 10 | 11 |
| 7 | Information interpretation | 18 | 45 | 17 | 17 |
| 8 | Utilization of information for decision making | 10 | 55 | 7.5 | 7.5 |
| Average of total aspects | 35.75 | 64.1 | 34.57 | ||
Table 3.
DHIS Criteria Compliance with WHO framework
| Compliance Percentage | |||||
|---|---|---|---|---|---|
| Aspect | Criteria Group | District [Total] | District Health Center | Urban Health Facilities | Rura Health Facilities |
| Infrastructures | Information Infrastructures | 30 | 40 | 30 | 30 |
| Training Infrastructures | 35.3 | 53.9 | 34.2 | 34.2 | |
| Computerized Reports Infrastructures | 1.9 | 37.5 | 0 | 0 | |
| Other DHIS Infrastructures | 50 | 50 | 50 | 50 | |
| Essential information needs assessment | Patient Management | 71.2 | 90 | 70 | 70 |
| Health Facility Management | 100 | 100 | 100 | 100 | |
| Health Facility Performance Assessment | 39.9 | 65.4 | 38.5 | 38.5 | |
| District Health Center | 87.5 | 87.5 | - | - | |
| Data gathering | Patient Management Data Gathering Tools | 66.7 | 66.7 | 66.7 | 66.7 |
| Warner able Diseases | 75.8 | 87.5 | 75 | 75 | |
| Health Facility Administrative Data | 100 | 100 | 100 | 100 | |
| Organizational Unit Data | 100 | 100 | 100 | 100 | |
| Health Facility Infrastructures Data | 81 | 100 | 80 | 80 | |
| Other Data Gathering Criteria | 66.2 | 70 | 58 | 58 | |
| Data processing | Data Quality | 75 | 75 | 75 | 75 |
| Data Error Correction | 100 | 100 | 100 | 100 | |
| Future Errors Prevention | 56.6 | 73.5 | 52.9 | 52.9 | |
| Vertical Data Transmission | 74 | 50 | 75 | 75 | |
| Horizontal Data Transmission | 26 | 41.7 | 25 | 25 | |
| Other Data Processing Criteria | 45.6 | 57.5 | 45 | 45 | |
| Data analysis | Health Facility Self Assessment | 50.3 | 56.3 | 50 | 50 |
| General Indicators | 31 | 72.5 | 28.8 | 28.8 | |
| Human Resources Key Indicators | 80.6 | 90 | 80 | 80 | |
| Transportation Indicators | 1 | 20 | 0 | 0 | |
| Clinical Laboratory key Indicators | 2.5 | 50 | 0 | 0 | |
| Financial Indicators | 52.5 | 100 | 50 | 50 | |
| Other Data Analysis Criteria | 16.7 | 16.7 | 16.7 | 16.7 | |
| Information presentation | Maternal Health Charting | 5 | 100 | 0 | 0 |
| Vaccination Charting | 2.5 | 50 | 0 | 0 | |
| Nutrition Charting | 5 | 100 | 0 | 0 | |
| Epidemic Diseases Charting | 1.5 | 25 | 0 | 0 | |
| Therapeutic Healthcare Charting | 2.5 | 50 | 0 | 0 | |
| Venereal Diseases Charting | 0 | 0 | 0 | 0 | |
| Tuberculosis Charting | 3 | 60 | 0 | 0 | |
| Facility Administration Charting | 16.6 | 57.1 | 14.3 | 14.3 | |
| Other Information Presentation Criteria | 34.7 | 53.3 | 26.7 | 26.7 | |
| Information interpretation | Comparing Information | 1 | 16.7 | 0 | 0 |
| Other Information Interpretation Criteria | 26.8 | 58.3 | 25 | 25 | |
| Utilization of information for decision making | Monthly Written Feedback | 0 | 0 | 0 | 0 |
| Computerized Standard Reports | 3.9 | 78.9 | 0 | 0 | |
| Other Criteria of Utilization of Information in Decision Making | 36.1 | 61.5 | 23.1 | 23.1 | |
| Average of total aspects | |||||
According to WHO criteria, the average score of DHIS assessment was 35.75 percent. The most score of DHIS criteria compliance with WHO, was related to data gathering criteria (70 percent) and the least score was related to utilization of information for decision making (10 percent).
In urban and rural facilities, the average score of compliance was 34.57 percent, the most and the least degree of compliance was related to data gathering criteria (72 percent) and utilization of information for decision making (7.5 percent), respectively.
In DHIS infrastructures, the average score of criteria compliance was 21 percent and the least of them was related to infrastructures of computerized reports (1.9 percent). Also, about DHIS essential information needs assessment, the average score of compliance was 60 percent that the most of compliance was related to needs assessment of health facility management information (100 percent) and the least degree of compliance was related to resource management information needs for facility performance assessment.
In DHIS data gathering criteria, the average score of compliance with WHO criteria was 70 percent. The most degree of them was related to administrative data gathering and facility organizational unit data criteria (100 percent).
In DHIS data processing criteria compliance with WHO, the averages score was 55 percent. The most and the least degree was related to data errors correction (100 percent) and horizontal data transmission (26 percent).
In DHIS data analysis criteria compliance with WHO, the averages score was 32 percent. The most of compliance was related to human resources key indicators and the least degree of compliance was related to transportation indicators (1 percent).
In DHIS information presentation criteria compliance with WHO, the averages score was 13 percent. Also, we should pay attention that venereal disease charting criteria has not been complied (0 percent).
In DHIS information interpretation criteria compliance with WHO, the averages score was 18 percent. The least degree of compliance was related to comparing obtained information with goals, standards, other health facilities information, and previous periods (1 percent).
In DHIS criteria about utilization information for decision making compliance with WHO, the averages score was 10 percent. The least of compliance was related to monthly written feedback from district health information unit to urban and rural health facilities (0 percent).
4. DISCUSSION
Health information system effective management is one of advanced and valuable outcomes of health system management and its important decisions infrastructure. Therefore, one of the greatest challenges for health system managers is commitment and deep belief of implementation and utilization of modern management in health information management area (16,17).
The results of this study showed that DHIS criteria compliance with WHO, was undesirable and has negative impacts on district health system management and healthcare delivery in this level. For true decision making in district level, health system management should restructure DHIS and hold further managerial and informational skills for district health system managers.
In general, DHIS focus on district health center and there is not enough attention to information system in urban and rural health facilities. In researchers opinions, DHIS infrastructures in urban and rural health facilities is undesirable and district health managers should equip health facilities with relevant software and hardware and train DHIS to facilities personnel. Also, these facilities managers have not enough knowledge of information system.
According to findings, utilization of information in decision making, information interpretation, and information presentation criteria compliance with WHO had the most undesirability. Authors believe that urban and rural health facilities managers should participate in proper training courses of information interpretation and utilization of information in decision making.
In this context, some study in Tanzania, Malawi, Ethiopia, Botswana, and Mongolia showed these challenges:
Native personnel who had DHIS skills did some other tasks; such as: service delivery to clients, data entry and reporting to regional level (18).
Because of a lot of shift in personnel positions and movement, there are not enough personnel for data gathering, analyzing, and distribution of information (9).
Inadequate access to DHIS skilled personnel (9).
Personnel resignation During DHIS activities, leads to improper DHIS implementation (19).
Not enough attention paid to district information technology protection and infrastructure to ensure DHIS soft ware maintenance (20).
No appropriate policy for hardware, software, and system development (21).
Health information personnel knowledge is low and it has negative effect on other personnel (19).
Not enough computers for data entry in health facilities and the existing ones are old and obsolete (9).
Not enough development Capacity for DHIS software (19).
The findings of the study also showed that main goal of information utilization in decision making and policy planning was not achieved and data were aggregated in district level. It is fair mentioning that DHIS suffers from inefficiency and close to 90 percent of district health system decisions and policies are made either experience based or subjective individual opinion, and not based on utilization of information. Health system managers have little managerial and informational skills and they should be trained in these areas.
In a study titled” Isfahan medical university hospital managers need assessment about comprehensive planning process in hospital information system, 2007” showed that the knowledge of hospital managers about hospital information system master planning process was average and further educational programs was needed (22).
Another study titled “Use of information technology in health information management” showed that application of information technology has been developed in various industries but not in the health sector. Therefore we should provide appropriate infrastructures for designing and implementing of these systems, taking in to consideration development and integration of existing information systems, information standards, health care personnel training, and employment of health information management experts (22).
The findings of our study also showed that in spite of infrastructure significance for proper implementation of DHIS, there is not enough attention paid to this subject in many other countries. For example, there is not primary infrastructure for team working, participation in information interpretation and decision making. Furthermore, DHIS training infrastructures is undesirable, first level data production and gathering is inefficient, personnel training has been neglected in primary health care, there is no computerized report in urban and rural health facilities, and namely, there is not enough information for decision making in this level.
According to WHO information criteria, user’s participation in all stages of DHIS designing is very important to comply with all users information needs (1, 23). This study showed that there is no user participation in DHIS designing stage.
In 2005, one study about DHIS in Kenya showed that information systems can not perform effectively unless users have enough knowledge of information system design, implementation and application and to be empowered about information culture, values and its impact on effective health services delivery (24).
The findings showed that health facilities performance is not assessed properly. Although the demographic, economic, and social profiles of target population are closely related to health program’s target population identification, There are no profile information available at these health facilities., The findings also showed that the needed socioeconomic data and information are neglected at district health center. These data should be identified properly and well utilized in health system decision making process. All in all the findings showed that DHIS essential information needs assessment is undesirable.
A study done in fourteen African countries on district health information systems discovered the followings:
Due to lack of necessary information for managerial processes, there is significant pressure on primary health care implementation approach (8).
One of the main challenges is lack of target population data in health care delivery system (18).
Lack of minimum data set definition in every level of district (21).
These findings are similar to our study results especially in the area of information for managerial decision making process and formation of minimum data sets.
Another area that should be paid attention to, is facility target population native disease status so that we can establish specialties clinics, provide necessary health resources, continuous personnel training program about diagnosis and treatment.
Our findings showed that administrative and organizational unit data gathering criteria were in compliance with WHO criteria in all district levels. This means that district health system management focuses more on facilities administrative activities, rather than clinical and patient management.
Data gathering criteria related to human resources and occupied positions, personnel skills and trainings did not comply with WHO criteria in urban and rural health facilities. Since human resources are one of the most important organizational strategic resource so, the human resources data gathering is very important and health system managers should pay adequate attention to this issue.
In total in the area of data gathering DHIS criteria were undesirable and in comparison with other aspects of DHIS criteria assessment was at the first rank. It is recommended that health information management should pay close attention to this aspect.
The most DHIS data processing criteria compliance with WHO criteria was data error correction criteria and the least of them was related to horizontal data transmission to relevant organizations and agencies such as transportation, agriculture, education and social agencies. It means that in our country, district health system is centralized and top level management authorization is necessary for information disclosure and they need to develop a proper program for prevention of future data errors. In relation to data access criterion, health system staffs do not have easy access to collected data. It is suggested that health system management should implement business process reengineering in order to simplify horizontal data transmission process and also stakeholders easy access to data. In general the finding showed that data processing criteria compliance with WHO criteria was undesirable.
The study of Mongolia health information systems showed incompetence and challenges in data processing, data access and utilization of information in decision making and the results conform to our study results (21).
The findings showed that DHIS data analysis criteria compliance with WHO criteria was undesirable. Human resources data key indicators comply more than other data analysis criteria and clinical indicators such as clinical library indicators were neglected. Also health system staffs lack calculating abilities in assessing data. Therefore staff self assessment and unit assessment is distorted in district health facilities. Quality of health care systems depends of a quality of mimimal data sets used by developing of DHIS. Measuring of structure, process and out-put variables of DHIS and its outcomes is also very important. Masic et all described their experinces in Bosnia and Herzegovina, esspecially in wartime conditions (25, 26, 27, 28).
According to findings, lack of human resources indicators led to false forecast of health services delivery staff needs. As a result, employment and distribution of human resources in health facilities do not have scientific support.
In urban and rural health facilities did not calculate clinical laboratory indicators. Therefore, there is no proper pattern for clinical library equipment and specialist distribution between district facilities. The lowest compliance of DHIS indicators related to staff annual budget cost per person.
Our research findings also showed that the least DHIS information presentation criteria compliance with WHO criteria was in urban health facilities. We probably could say that DHIS is centralized in Iranian district health centers and district health system management should decentralize the system, same as other successful countries.
For example there are not any vaccination, nutrition, therapeutic cares, venereal and epidemic diseases, tuberculosis, and target population charts and diagrams prepared and presented in urban and rural facilities. As well as there is no target population satisfaction, work load, staff vacation bar charts and maps of target population distribution in each facility.
Studies of DHIS done in other countries revealed that there are weak coordination between district health system and other sectors such as sewage, agriculture, housing and transportation and the health departments (8, 21).
Our study findings concerning DHIS information interpretation criteria compliance with WHO criteria was undesirable. For example there are not any comparison between obtained performance data and information with goals, standards, previous same periods and other same health centers. And therefore there is not any computerized report produced in urban and rural facilities.
Our study findings concerning DHIS monthly written feedback criteria compliance with WHO criteria was also undesirable due to lack of suitable surveillance that should be done on health facility performance.
In another study done in Malawi health information systems revealed the following challenges:
Lack of supportive surveillance data and information on district health center and its affiliated hospitals.
Lack of systematic utilization of data and information in decision making process (9).
It is worth mentioning that there has not been any study of DHIS done in Iran so far until now and although the majority of district health information systems researches done in other countries belongs to the last two decades, the Iranian health system has not paid enough attention to these studies and the major of Iranian health programs tend to concentrate on the primary health care only and not the related information systems.
5. CONCLUSION
DHIS assessment indicates that there are many areas to be improved. According to our findings the majority of DHIS criteria compliance with WHO criteria is undesirable and health system management should try to restructure the district health information system and develop managerial and informational skills in district managers. Also, we suggest that health system top managers try to plan the following programs:
DHIS development skills and its conceptual training.
Holding periodical courses and setup workshops in information interpretation and utilization in decision making process.
Software and hardware infrastructures development in all the district health system.
DHIS utilization policy and procedure.
Do researches on various aspects of DHIS.
Continuous assessment of DHIS throughout the country.
Assessment of DHIS results on health manager’s performance.
Assessment of other countries DHIS and develop appropriate pattern for Iran.
Assessment of DHIS role in national decision making and policy planning.
References
- 1.Lippeveld T, Sauerborn R, Bodart C.Design And Implementing of Health Information System. Translation: Farzadfar Farshad., et al. Tehran, Kelke Deerin Press, 1384: 1-2 [Persian]. [Google Scholar]
- 2.Heywood A., Rhode J.Using Information for Action. Translation: Zare Mohammad., Zahravi Farahnaz.Tehran, Simindokht Press; 1384: 1-4 [Persian]. [Google Scholar]
- 3.Tanner M, Lengeler C.From the efficacy of disease control tools to community effectiveness. Transactions of the Royal Society of Tropical Medicine and Hygiene, 1993; 87: 518-523 [DOI] [PubMed] [Google Scholar]
- 4.Tugwell P., et al. The measurement interactive loop: a framework for the critical appraisal of need, benefits, and costs of health interventions. Journal of chronic diseases, 1995; 38: 339-351 [DOI] [PubMed] [Google Scholar]
- 5.Omar MA., Charimari LS.The District Health Information System and its Potantial in the Management of District and Rural Hospitals. Nuffield Institute for Health, University of Leeds 71-75 Clarendon Road, Leeds LS2 9PL, United Kingdom Journal of world Hospitals., 30: 3. [PubMed] [Google Scholar]
- 6.World Health Organization Report of the Interregional Meeting on Strengthening District Health Systems. Based on Primary Health Care, Harare, Zimbabwe: 3 August1987. Geneva, World Health Organization: 1-42 (unpublished document WHO/SHS/DHS/87.13; available on request from Evidence and Information for Policy, World Health Organization, 1211 Geneva 27, Switzerland). [Google Scholar]
- 7.World Health Organization The challenge of implementation: district health systems for primary care. Geneva, World Health Organization, 1988. (unpublished document WHO/SHS/DHS/88.1; available on request from Evidence and Information for Policy, World Health Organization, 1211 Geneva 27, Switzerland:12). [Google Scholar]
- 8.Muquingue H.Overview of the HISP Network: District Health Information Systems-2007. available on web site: http://www.med.uio.no/iasam/inthel/english/hisp/DHIS%20elective%20-%20Top-ic%202%20-20Health%20district%20%20district%20health%20system%20-%20Oslo%202007ppt#256,2, District Health Information Systems [Google Scholar]
- 9.Muyepa A.HIMS in Malawi: Achievements, Challenges and the Way Forward. At the first International Workshop, Mpekweni South Africa, 9-12Oct 2006:14 [Google Scholar]
- 10.Barron P., et al. District Health Information System Guidelines. Europian Union, Department of Health. February1998:18 [Google Scholar]
- 11.Bulatao RA.Key indicators for family planning projects. Washington DC, World Bank; 1995. (World Bank Technical Paper, No.297): 45 [Google Scholar]
- 12.Woodall JP.Introduction: epidemiological approaches to health planning, management and evaluation. World health statistics quarterly. 1988; 41: 2-10 [PubMed] [Google Scholar]
- 13.Cleland JG.Data cleaning. Data processing. In: Pressat R, Wilson C. The dictionary of demography. Oxford, Basil Blackwell, 1985; 49-51 [Google Scholar]
- 14.World Health Organization Catalogue of health indicators. Geneva, World Health Organization; 1996. (unpublished document WHO/HST/96.8; available on request from Department of Organization of Health Services Delivery, World Health Organization, 1211 Geneva 27, Switzerland): 25-27 [Google Scholar]
- 15.Opit LJ.How should information on health care be generated and used? World, health forum, 1987; 8: 409-417 [Google Scholar]
- 16.Gilvary A.New Concepts in Information Sciences. Chapar Press; Tehran, 2008: 30-35 [Persian]. [Google Scholar]
- 17.Panahi A.Management Information Systems. Azarakhsh Press, Tehran, 2000: 22-24[Persian]. [Google Scholar]
- 18.Twaakyondo HM.DHIS Customization Tanzanian Experience, available on web site: http://www.hispkerala.org/conference/Presentations/Country%20pre-sentations%20Session%202/Ex-perience%20of%20DHIS%20customisation%5B2%5Dppt#256,l, DHIS customization Tanzanian experience [Google Scholar]
- 19.Hussein S.HISP Ethiopia: Current Status and Future Directions, available on website: http://www.hispker-ala.org/conference/Presentations/Country%20presentations°/o20Session%202/HISP%20Ethiopia.ppt
- 20.Chandna Omprakash.Evaluation of DHIS Pilot Project In Botswana. Ministry of health Botswana; 2006. available on web site: www.sim.hcuge.ch/helina/21.pdf [Google Scholar]
- 21.Govind S.National Health Management Information System in Mongolia: Issues, Challenges and Opportunities. Bi-Regional Consultation On Strengthening Health Information System in Asia and Pacific; Bangkok, 13-16December2004: 18-22 [Google Scholar]
- 22.Raeisi A.Isfahan Medical University Hospital Manager’s Needs Assessment About Comprehensive Planning Process in Hospital Information System. Research Paper. Isfahan University of Medical Sciences, 2006: 63-68 [Persian]. [Google Scholar]
- 23.Barron P., et al. District Health Information System Guidelines. Europian Union, Department of Health. February1998 [Google Scholar]
- 24.Odhiambo-Otieno GW., Odero Wilson WO.Evaluation Criteria for the District Health Management Information Systems: Lessons from the Ministry of Health, Kenya. African Health Sciences. 2005Mar; 5(1). [PMC free article] [PubMed] [Google Scholar]
- 25.Mask I.Use of Health Technology in Reconstruction of the Health Care System in Bosnia and Herzegovina. Med Arh. 1996; 50(1-2): 59-64 [PubMed] [Google Scholar]
- 26.Mask I., Toromanovic S, Masic I, Novo A, Kudumovic M, Zunic L.Criteria how to Chose Adequate Methodology and relevant Variables for Assessment of Quality of Primary Health Care. Med Arh. 2005; 59(1): 23-26 [PubMed] [Google Scholar]
- 27.Sabanovic Z, Masic I.Use of Spreadsheets as Decission Support Systems in Health Care. Med Arh. 1995; 49(1-2): 45-48 [PubMed] [Google Scholar]
- 28.Mask I, Niksic D.Quality and Quality Assurance in Health Care. Med Arh. 2003; 57(3): 189-196 [PubMed] [Google Scholar]