Feasibility of implementing recommendations to improve communicable diseases surveillance - a modified Delphi study

Abstract

Background

Effective control of communicable diseases requires an effective disease surveillance system (CDSS) which provides information for action on priority communicable diseases. It is the basis for public health decision-making worldwide. Implementation of any public health recommendations for improving a disease surveillance system depends on the acceptability of such recommendations by the relevant stakeholders and the feasibility of implementing the recommendations in practice.

Objectives

The aim of this study was to assess the acceptability and the feasibility of recommended improvements in CDSS among relevant stakeholders in Khartoum state.

Methods

A Delphi consensus process was used in the form of three analytical written rounds and individual face-to-face discussions among relevant stakeholders in Khartoum state to study the feasibility of the recommended improvements in CDSS in Khartoum state.

Resultss

The stakeholders in Khartoum state agreed with most of our statements to improve the CDSS core activities, supportive functions and quality in Khartoum State, and that the existing CDSS in Khartoum state needs to be strengthened with more effective coordination at different levels. Regarding the feasibility of implementing our recommendations, the results of our Delphi survey suggest neither expanding CDSS system, nor including vertical programmes; formulation of updated objectives; improving data management and feedback; strengthening epidemic management, as well as the quality of the system in terms of timeliness, completeness and acceptability.

Conclusion

This study added strength to our recommendations, based on two previous studies assessing the CDSS in Khartoum state. The Ministry of Health in Khartoum state can implement our consensus recommendations to improve the CDSS system in the future in order to achieve its targeted goals

Introduction

Public health surveillance is an ongoing, systematic collection, analysis, interpretation and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and mortality and to improve health¹. Effective communicable disease control needs an effective disease surveillance system, which provides information for action on priority communicable diseases. It is the basis for public health decision-making worldwide. The need to strengthen disease surveillance and response systems is recognized globally². In Sudan, communicable disease surveillance in Khartoum State is part of the National Surveillance System launched in 1994. The system depends on the passive surveillance for communicable diseases, which changes into active surveillance during epidemics or outbreaks⁴.

We have previously published two studies assessing the communicable diseases surveillance system (CDSS) in Khartoum State. In the first study, a descriptive, retrospective and cross-sectional study on the core activities and supportive functions of the CDSS for the period 2005 to 2007, the CDSS system was found to be centralized, not updated, poorly documented, and having a shortage of staff at lower levels⁵. The second, qualitative focus group study assessing the CDSS staff sought their views about the running of the system. The quality of CDSS was seen as poor because the system was not representative. It included neither the private nor the military sector nor the important teaching hospitals. It also lacked timeliness due to poor documentation in receiving and sending CDSS reports. It was inflexible since it did not rapidly respond to emerging and re-emerging diseases such as SARS and avian flu in its notification lists; and in addition, it did not use the data collected to apply interventions for control and prevention of communicable diseases on a routine basis⁶.

Implementation of public health recommendations depends on their acceptability by the relevant stakeholders and the feasibility of implementing the recommendations in practice. The Delphi technique, originally developed by the RAND Corporation, helps in structuring a group communication process that is particularly useful when there is little knowledge, or there is uncertainty surrounding a complex area being investigated³. Van Zoligen & Klaassen classify the Delphi technique into: the Classical Delphi - which establishes facts; the Policy Delphi - for generating ideas; the Decision Delphi for decisions making; and the Group Delphi for group discussion³. The Delphi technique has been successfully used to study the feasibility of policy options e.g. in creating international emerging infectious disease policies on SARS and similar threats⁷.

The aim of this study was to study the acceptability and feasibility of recommended improvements in CDSS among relevant stakeholders in Khartoum state.

Methods

A three-round analytic Delphi process was used, after which a face-to-face discussion meeting was arranged. A total of 50 experts in the field of communicable diseases surveillance in Sudan (doctors, health officers from central, locality and health area levels) with at least six months of experience were chosen randomly out of 175 experts, based on their direct link to CDSS in Khartoum state to participate in the study in the first written round and 47 replied (94%). In the second analytic round, 25 experts were chosen out of the 47 respondents of the first round based on the years of experience in CDSS in Sudan (minimum 2 years of experience), and the response rate was 100%. In the third analytic round, the top 10 experts (minimum 5 years of experience) were chosen, and the response rate was 100%. The 5 partcipants in the face-to-face meeting were high level experts and professionals (decision makers for the CDSS in Khartoum, with experience of more than 10 years in the system and with high qualifications either in epidemiology or diseases surveillance) from the participants in the written round. We involved different levels of experts in the Delphi rounds as in the Khartoum context we need to have acceptance of the highest level of experts to be able to implement the recommendations found in our 2 recent papers^5,6 to improve the CDSS system (Figure 1).

Figure 1.

Selection of professionals and experts for Delphi rounds

The pre-tested Delphi questionnaire was prepared based on the recommendations of our quantitative⁵ and qualitative studies assessing the CDSS in Khartoum⁶. The questionnaire was divided into 13 sections about recommended changes or improvements in core activities, supportive functions and quality of the CDSS in Khartoum state. In the first written round the questionnaire consisted of 47 statements; we repeated the questionnaire analysis without omitting any statement in the second and third rounds. A five-point Likert scale was used to measure the level of agreement. Consensus was defined as 80 % or a higher agreement rate on each statement.

Data were analyzed using Statistical Package for the Social Sciences (SPSS) version 16.0 to calculate frequencies, median and interquartile range.

The non-consensus statements from the written rounds were discussed with top experts (n = 5) via individual face-to-face meetings or phone conversations. In this round a further iteration of the non-consensus outcomes of the Delphi study resulted in the endorsement, modification, integration or rejection of individual statements.

Results

Written rounds

Out of the total of 47 statements in the three written rounds, 44 statements got consensus in the first round, 43 in the second round and 39 in the third round with percentages of 93.6%, 91.5% and 82.9% respectively (Table 1).

Table 1.

Consensus results from Delphi written rounds

Rounds	Consensus statements (> 80%)		Non consensus statements (<80%)
	No of consensus		No of non consensus
	statements	%	statements	%
First analytical round	44	93.6%	3	6.4%
Second analytical round	43	91.5%	4	8.5%
Third analytical round	39	82.9%	8	17%

The CDSS experts in Khartoum state agreed that the Khartoum CDSS needs important changes in its core activities, supportive functions and quality to achieve its targeted goals.

Type and structure of the CDSS

Experts participating in the three written Delphi rounds and individual face-to-face meeting agreed not to change the current sentinel site surveillance system to whole health facilities surveillance system. The first two Delphi rounds participants agreed that the Khartoum state must formulate a single surveillance system for all communicable diseases, meaning a system that includes all vertical communicable disease prevention and control programs, such as those for tuberculosis, leprosy, acute immno-deficiency syndrome, poliomyelitis and measles. However, the third round experts with a minimum of 5 years of experience were not in favour of a single surveillance system. They preferred the present system without including other vertical programmes.

Objectives of the CDSS

Experts agreed in the three rounds and individual face-to-face meetings that Khartoum state must formulate updated written clear objectives for the system and that the central level should not take the responsibility for the formulation of these objectives alone and then disseminate them to the lower levels of the system.

CDSS core activities

The experts throughout the Delphi process agreed that there should be the list of priority diseases for surveillance; standard specimen guidelines should be formulated and disseminated to all levels of CDSS; and an updated version of case definitions for communicable diseases should be formulated.

Experts recommended that a computer system is needed for data reporting at health facility level as well as the usage of a network for sending the CDSS reports from all levels. The experts also agreed that the system needs professional staff such as statisticians in order to fully implement data analysis at local levels.

The experts ruled out the feasibility of implementing monetary incentives for improving the data reporting at health facilities. The inclusion of the military hospitals in the Khartoum CDSS system was an important issue in the 3 rounds, but no agreement was achieved about the inclusion of private clinics and hospitals in the CDSS either in the 3 rounds or individual face-to-face interviews.

Khartoum CDSS needs to strengthen its zero reporting system for epidemic prone diseases at all levels of the system. Experts in the first round only agreed that data analysis should start at health facility level. All experts agreed that the system needs formulation of a standard format for the data analysis at all levels, as well as upgrading of data analysis from simple rates and ratios to higher levels. It also needs to use population statistics per area as the appropriate denominator for data analysis for all the diseases at health area, locality and central levels. All the experts in all rounds including the face-to face meeting agreed that using of geographical information system (GIS) for data analysis at central and local level would improve the system. The system must use the collected and analyzed data for performing real action to prevent and control communicable diseases in Khartoum State. The system should have a new informative standard feedback system for the surveillance data.

Experts in all rounds and individual face-to-face meetings recommended that the epidemic management system needs the following:

• Standard epidemic management plan at all levels must be updated;

• Standard specialized epidemic management committee is needed at the central level;

• formulation of rapid response team at central and locality level;

• Existence of ready emergency stocks of drugs, vaccines and supplies at central and locality level all the time;

• Availability of special budget every year for epidemic management at the central level at the time of suspected epidemics;

• Epidemic response must be done at the lower levels supported by central level;

• Establishment of standard epidemic reporting system;

• Updating of protocols for standard management of epidemic prone diseases; and

• A system for the evaluation of epidemic response after the end of each epidemic.

CDSS Supportive functions

Experts recommended that the CDSS supervision system should have a new standard check list and feedback systems, which would give supervision its vital role in monitoring and evaluating the system.

Both locality and health areas need more CDSS staff members and focal personnel are needed in the health facility level. Each hospital must have a public health office to manage notification and reporting of communicable diseases. All experts agreed that the CDSS must have a separate budget for all its activities.

CDSS Quality

Experts recommended that the CDSS should build a system for keeping the previous years' routine surveillance reports at all levels. Formulation of standard registry for sending and receiving times of the CDSS reports at all levels would provide a tool to measure the timeliness of the system. Experts agreed that the Khartoum CDSS is a flexible system in adopting changes. The system was seen to be simple and highly acceptable by stakeholders in the first and second round but neither in the third round nor in individual face-to-face meetings. The entire three rounds showed that CDSS was considered to be a highly useful system.

CDSS legislation

Highly experienced experts achieved no consensus about the formulation of local punishment regulatory system for delaying of the report at the recommended time. However, all agreed about formulation of local legislation that makes urgent notification of serious communicable diseases compulsory.

Discussion

Our study aimed to gather CDSS expert consensus about the acceptability, and the feasibility of implementing recommendations developed from empirical research to help the Ministry of Health in Khartoum state to implement the changes required to strengthen the CDSS. We were able to reach consensus on 39 out of 47 statements after the third Delphi round and individual face-to-face discussions of the highly experienced CDSS experts.

Experts pointed out that the current sentinel sites surveillance system for communicable diseases in Khartoum is ‘well functioning’. It provides an alternative to the population-based surveillance⁸ and can provide a simple, early detection of diseases⁹. This system is a suitable structure for Khartoum state whereas the idea of whole health facility surveillance is not likely to be applicable and would also be (more) costly in terms of monetary and human resources needed to run the system. The current system, if improved in its quality and representativeness, will be satisfactory and it might overcome the problem of the non-representativeness of the data for the entire population at risk¹⁰. Formulation of single surveillance systems for all communicable diseases, meaning including all vertical communicable disease systems such as tuberculosis, leprosy, acute immno-deficiency syndrome, poliomyelitis and measles, was not seen as a wise idea as it is very hard to combine these vertical, multi-component programmes in a single system. Moreover, by combining these programmes the quality of the single surveillance system would be affected and the system resources would be exhausted.

The CDSS clear written objectives at all levels of the system are of great value for the success of the system and these have helped the CDDS in Khartoum to function better than many other systems in the developed world¹¹.

Updating CDSS guidelines and manuals, especially their case definitions and specimen collection, is an important and necessary change facing the system. The system also needs to reset the communicable diseases notification list so that it can include new emerging diseases such as SARS and avian influenza⁵. This will improve the effectiveness of the system and make the system more flexible in adopting changes better than in other Sudan states¹² and other developing countries¹³.

Introduction of advanced technology for the CDSS data reporting in Khartoum state at the first level - health facility level, introduction of a computer system, usage of a network for sending the CDSS reports as well as provision of professional personnel or data reporting at local level are likely to lead to increased data accuracy, strengthening the CDSS system to the levels comparable to other countries^{14, 21}.

The experts agreed about the need of the CDSS in Khartoum state to improve its representativeness by including the military hospitals in the system to provide a sufficient volume of data to calculate statistically significant rates and ratios, which are important for assessing changes in the population health status¹⁰. However, the experts disagreed to include the private sector in the CDSS system. They justified this by claiming that the poor data registry in the private sector would negatively affect the quality of the system. This inclusion can be done later on if the data registry in the private sector improves. In the integrated disease surveillance strategy the data collected should be analyzed and used for action, especially at the health facility level¹⁵. It is important that the surveillance data analysis at the first point of its collection is used for action. However, Khartoum state health facilities are not yet well prepared to perform CDSS data analysis; thus the analysis should be done at higher levels of the system. This would make the Khartoum system superior compared to other systems in Sudan¹² and other African countries^16,17. Standardized, continuous, systematic and more detailed analysis of all data reported should be done by upgrading the data analysis and using appropriate dominators such as population per area to keep track of the disease situation in the area and to maximize and strengthen the CDSS effectiveness. The CDSS system is useful if it contributes to the prevention and control of adverse health-related events¹⁸. Functioning epidemic management systems are a major challenge for any CDSS system mainly in developing countries^12,13,18. The system in Khartoum needs urgent and major changes in the epidemic management system to provide the desired functions in controlling epidemics in a standard way in the state; theses changes include epidemic plan, epidemic committee, rapid response team financing and epidemic reporting system.

The CDSS system in Khartoum as well as elsewhere in Sudan has well trained and professional staff at the state level⁵. Both local and health area need more CDSS staff, which will increase the quality of the system as it will decrease work load, and there will be more time to make use of the collected data in performing the necessary actions. Highly experienced experts thought that no special incentives for staff are needed in the recent time as there are many CDSS priority areas needing money and because the surveillance is part of the job description of the staff.

The importance of reporting on timeliness has been documented^{14, 19}. Formulation of a standard registry for sending and receiving times of the CDSS reports at all levels would improve the Khartoum CDSS timeliness.

Experts considered the CDSS to be a highly useful system in monitoring communicable diseases in Khartoum state. This view is different from the opinions of the CDSS staff in our qualitative assessment of the system⁶. The highly experienced experts argued that the CDSS system, despite its limited resources was able to detect and manage all epidemics in the previous year on the acceptable level of performance and that was proven by the revision of the system records in that period⁵.

Surveillance systems should be as simple as possible in their structure and easy to operate while still meeting their objectives²⁰. However, the experts considered Khartoum CDSS to be too simple in its structure and operations at all levels as it used a simple data reporting and analysis system simulating some developed countries¹¹. The experts in this Delphi process considered Khartoum CDSS to be a flexible system as it responds to global changes which improve the quality of the system. However, this flexibility is not well combined in the existing system and it mostly produces separate systems for its data management outside of the original system. Interventions need to be performed to make all changes adopted within the original system otherwise the system will be fragmented, which negatively affect the quality of CDSS in Khartoum State.

However, due to a non-informative feedback system, the Khartoum CDSS was not highly acceptable among stakeholders. Proper communication methods, acknowleding staff members' contribution, dissemination of aggregate data back to reporting sources and interested parties, providing accurate, consistent, complete, and timely surveillance data, are key issues for the acceptability of the CDSS system.

Strengths and limitations of the study

The Delphi technique enabled CDSS experts in Khartoum State to freely give their opinions about our recommendations found in previous studies in Khartoum, to improve the CDSS system. The Delphi process supports our suggested policy recommendations but it also helps to clarify and justify many issues in our recommendations. Having different levels of experts based on the years of experience and professionalism in CDSS system gave more strength to our Delphi technique especially in the Sudanese context, where those in higher professional positions are the key persons for implementing the changes. As some of our experts were part of the running CDSS system in Khartoum, they brought into the Delphi their contextualized views, which are crucial for the implementation of changes in practice. The high response rate gives further strength to the study. Unfortunately, translation of the Delphi questions from English to Arabic and back to English might have affected the meaning of certain statements and experts' comments.

Conclusion

The results of the Delphi study added strength to the recommendations based on the two previous studies assessing the CDSS in Khartoum state^5,6. The Delphi panels agreed with most of our statements to improve the CDSS core activities, supportive functions and quality in Khartoum State. The Ministry of Health in Khartoum state can implement our consensus recommendations to improve the CDSS system in the future in order to achieve its targeted goals. Further research is needed to study the implementation of the recommendations made for the Khartoum CDSS system, especially their acceptability and cost-effectiveness.