Evaluation and costs of different haemoglobin methods for use in district hospitals in Malawi

Abstract

Aims: To evaluate the characteristics of manual haemoglobin methods in use in Malawi and provide evidence for the Ministry of Health in Malawi to enable them to choose a suitable method for district hospitals.

Methods: Criteria on accuracy, clinical usefulness, user friendliness, speed, training time, and economic costs were determined by local health professionals and used to compare six different manual haemoglobin methods. These were introduced sequentially into use in a district hospital in Malawi alongside the reference method.

Results: HemoCue was the optimal method based on most of the outcome measures but was also the most expensive (US$0.75/test). DHT meter and Jenway colorimeter were the second choice because they were cheaper (US$0.20–0.35/test), but they were not as accurate or user friendly as HemoCue.

Conclusions: The process for choosing appropriate laboratory methods is complex and very little guidance is available for health managers in poorer countries. This paper describes the development and testing of a practical model for gathering evidence about test efficiency that could be adapted for use in other resource poor settings.

As part of the Malawi government’s Poverty Reduction Strategy, the Ministry of Health and Population has developed an Essential Health Package.¹ This aims to provide better access to quality services for common conditions that disproportionately affect the poor. In Malawi, all health care is free so patients do not pay a fee for the service. To complement the Essential Health Package, the Ministry of Health and Population is developing a model package of essential laboratory services to improve the cost effectiveness of district hospital laboratories. The model is being developed and piloted in selected typical district hospitals in Malawi before being adapted for national implementation. Haemoglobin measurement is a key component of this service, so it is necessary to identify a method that is practical, accurate, and economically viable for use at district level.

“This is the first time that such a detailed effectiveness study involving all the processes required to determine the suitability of a test has been undertaken in a district hospital in a developing country”

Haemoglobin measurement is the most commonly performed laboratory test worldwide and is an essential component of any health system. In sub-Saharan Africa, haemoglobin estimations are used in district hospitals for individual patient management, to guide transfusion practice, and in the management of antiretroviral therapy. Surveys of haemoglobin concentrations are also used as tools to provide public health data, such as nutritional status, and to monitor malaria interventions. Despite the wide range of manual methods available for measuring haemoglobin in developing countries, no single technique has emerged as the most appropriate for this setting.

To support their Essential Health Package,¹ the Ministry of Health and Population in Malawi needed to identify a method for measuring haemoglobin that was simple, accurate, fast, and cheap and required minimal training and supervision that could be used in district hospital laboratories. Therefore, our study assessed the effectiveness, including technical, economic, and managerial aspects, of different manual techniques for measuring haemoglobin in routine practice in a typical district hospital in Malawi against criteria predetermined by local heath professionals. To our knowledge, this is the first time that such a detailed effectiveness study involving all the processes required to determine the suitability of a test has been undertaken in a district hospital in a developing country. The project received ethical approval from the Liverpool School of Tropical Medicine, UK and the Ministry of Health and Population in Malawi.

MATERIALS AND METHODS

Our study was carried out in Ntcheu District Hospital in the central region of Malawi. This district has a population of 500 000 and is served by a single hospital with 240 inpatient beds. The laboratory in the Ntcheu District Hospital performs approximately 6000 haemoglobin tests each year. It normally has a complement of two or three staff (laboratory assistants and qualified technicians), but two extra technicians were seconded to assist with our study.

The six haemoglobin methods to be evaluated were chosen because they were already in use in Malawi or at district level in other countries, and they could be supplied quickly for widespread use in Malawi.

The Lovibond comparator (ECHO International Health Services Limited, Coulsden, Surrey, UK)

This is the predominant method used in Malawian district hospitals and assessments were carried out using the equipment already in routine use. It is based on the oxyhaemoglobin method and involves matching the colour of the diluted blood sample with colours on a disc equivalent to known haemoglobin values in 20 g/litre increments.²

WPA colorimeter (WPA CO700D from WPA, Pocklington, York, UK) and Jenway colorimeter (Jenway 6051; Jenway, Dunmow, UK)

These methods are based on the conversion of haemoglobin to haemiglobincyanide. The Jenway colorimeter has a direct readout.

DHT meter (HMB.010; Developing Health Technology, Ipswich, UK)

This is based on the oxyhaemoglobin method and has a direct readout of the haemoglobin estimation.

HemoCue (HemoCue AB-haemoglobin photometer; Angelholm, Sweden)

Undiluted blood is placed directly into a microcuvette where the haemoglobin is converted to haemiglobinazide. The microcuvette is inserted into the photometer and the machine provides a direct readout of the haemoglobin result.

Haemoglobin colour scale (Copack GmbH; Oststeinbek, Germany)

A drop of undiluted blood is placed on specific chromatography paper and matched against a range of colours representing different haemoglobin values in 20 g/litre increments.

All methods were performed in accordance with the manufacturers’ instructions. The methods were evaluated sequentially on a total of 600 consecutive, unselected EDTA blood samples (that is, 100 samples for each method) sent to the laboratory for routine haemoglobin estimations during a two month period during the malaria season. To assess the accuracy of each method in routine use, the haemoglobin value of all samples was also measured by the haemiglobincyanide (HCN) method³ simultaneously with the test method, but by different technicians who were unaware of each other’s results.

Local laboratory technicians and clinicians devised the criteria by which each method was assessed. The criteria were designed to identify the method that was most appropriate for use in district hospitals in Malawi. The criteria comprised: (1) accuracy and clinical usefulness, (2) user friendliness, (3) time taken to learn and perform tests, and (4) economic costs.

Accuracy and clinical usefulness

The level of agreement between haemoglobin values generated by the individual methods compared with the reference HCN method was determined according to the statistical method of Bland and Altman.⁴ Statistical analyses were performed using SPSS version 10. The percentage of tests for each method that varied from the reference value by more than ±20 g/litre were regarded as wasted tests. This cutoff was chosen because local clinicians indicated that a result that deviated more than 20 g/litre from the true value would probably result in mismanagement of the patient and was therefore of no clinical use.

User friendliness

Local technicians themselves identified the five key attributes of an ideal method for haemoglobin measurement in a district hospital in Malawi as: technically undemanding, minimal training and supervision requirements, direct reading of results, simple quality control checks, and not reliant on electricity supply. The technicians independently rated each of these criteria on a scale of 1 (very poor) to 5 (excellent) for each test method and this was used to devise a score of user friendliness for each method (maximum score for each method, 25).

Time taken to learn and perform tests

As each new method was introduced, the time taken to learn the method so that laboratory technicians and attendants could perform it independently was recorded. The time taken to perform each test during routine use was documented, and the time spent on internal quality control and calibration processes for each batch of tests was included in the calculation and divided between the number of tests for each batch. The training costs for each method were calculated from the cost of practice tests and the salary of the trainee for the total training time. The Lovibond comparator was excluded from this analysis, because it was the method already in routine use in the laboratory, so it was familiar to the technicians.

Economic costs

To estimate the costs for each method, the actual value of all resources used to perform the 100 haemoglobin estimations (including control and calibration procedures) was calculated from: the cost of apportioned annualised capital equipment, supplies, and technician’s time taken to perform each test. An appropriate portion of the laboratory’s total electricity costs was included for all methods that used power. Building maintenance costs were assumed to be the same, irrespective of the method, and were therefore excluded from the cost analysis. These estimations were used to project the total annual costs of the haemoglobin workload for each method. Prices were obtained from the Malawi central medical store catalogue or directly from the suppliers, and included the standard 12% government discount rate. Costs were expressed in US$.

RESULTS

Accuracy and clinical usefulness

HemoCue was the most accurate method because it had mean and range values closest to those of the HCN reference method (table 1); 95% of HemoCue results differ by only 6% below and 16% above the reference results. None of the tests showed systematic bias and the degree of imprecision is reflected in the limits of agreement. The proportion of tests that were too inaccurate to be clinically useful varied from 1% for HemoCue to 59% for the Lovibond comparator (table 2).

Table 1.

 Accuracy of various haemoglobin methods compared with the reference haemiglobincyanide (HCN) method

Method	Mean (g/l)	Median (g/l)	SD	Range	95% limit below reference method	95% limit above reference method
HemoCue	94	104	46	1.6–16.5	6%	16%
HCN	94	106	46	1.9–16.9
Jenway colorimeter	65	59	30	1.9–16.1	14%	10%
HCN	67	60	30	2.1 –16.3
WBA colorimeter	58	52	25	1.9–14.5	31%	31%
HCN	61	56	28	1.9–16.0
Haemoglobin colour scale	81	80	35	4.0–14.0	48%	37%
HCN	78	72	35	2.3–17.0
DHT meter	81	77	30	2.8–16.8	19%	33%
HCN	75	68	30	1.9–15.4
Lovibond	105	110	36	4.0–14.0	83%	33%
HCN	81	78	40	1.5–16.5

One hundred different samples were used for each method.

Table 2.

 Clinical usefulness of 100 haemoglobin tests using different methods

Test method	Variation between haemoglobin results from test and reference method
	0 to 10 g/l	11 to 20 g/l	More than 20 g/l (“wasted” tests)
HemoCue	97	2	1
Jenway colorimeter	96	3	1
WBA colorimeter	76	21	3
DHT meter	67	30	3
Haemoglobin colour scale	57	27	16
Lovibond	14	27	59

User friendliness

HemoCue was rated as the most user friendly method by the technicians, achieving the maximum possible score of 25. The least user friendly methods were the reference HCN method and the WBA colorimeter because neither of these had a direct readout of the haemoglobin result (table 3).

Table 3.

 User friendliness of different haemoglobin methods

Characteristics (maximum score 5 each)	HemoCue	DHT meter	Haemoglobin colour scale	Jenway colorimeter	Lovibond	WBA colorimeter	HCN reference method
Easy to use	5	4	5	4	4	4	2
Minimal training/supervision required	5	3	5	2	5	2	1
Simple reading of results	5	5	2	5	2	3	3
Simple internal quality	5	2	0	3	0	3	3
Suitable for village use	5	5	5	1	4	1	1
Total (max 25)	25	19	17	15	15	13	10

HCN, haemiglobincyanide.

Time taken to learn and perform tests

Competency was achieved most quickly for the HemoCue method by both grades of laboratory staff (table 4).

Table 4.

 Comparison of training time and costs for different haemoglobin methods

Method	Training time* in minutes			Number of practice tests required				Total training costs† 2001 US$
	Lab attendant	LA/JT	Experienced technician	Lab attendant	LA/JT	Experienced technician	Total no of practice tests
DHT meter	200	90	60	50	40	25	115	18.37
Haemoglobin colour scale	160	60	30	40	20	15	75	24.52
HemoCue	110	50	20	35	20	10	65	45.97
WBA colorimeter	6000	4000	3000	200	150	100	450	228.62
Jenway colorimeter	6000	4000	3000	200	150	100	450	230.32
HCN	7000	5000	4000	200	150	100	450	245.79

*Training time excludes tutor’s time; †total training costs comprise trainees’ time, the cost of supplies for the number of practice tests required, and an apportioned element of the capital cost.

HCN, haemiglobincyanide; JT, junior technician; LA, laboratory assistant.

In routine use, the average time taken to perform a single test varied between three minutes (haemoglobin colour scale), five minutes (DHT meter, HemoCue, and Lovibond), and 15 minutes (colorimeters and HCN method). The training costs to achieve competency varied from US$18 to US$246, depending on the method. They were lowest for the DHT meter, the haemoglobin colour scale, and HemoCue. They were five to 10 times higher for the WBA and Jenway colorimeter methods and the HCN reference method.

Economic costs (table 5)

Table 5.

 Projected costs of resources for annual haemoglobin workload (6161 tests) in Ntcheu District Hospital using different methods (in 2001 US$)

	Haemoglobin colour scale (%)	Lovibond (%)	DHT meter (%)	WBA colorimeter (%)	Jenway colorimeter (%)	HCN (%)	HemoCue (%)
Staff*	202.8 (27.0)	338.0 (36.0)	338.0 (27.5)	1013.9 (46.7)	1013.9 (46.4)	1013.9 (28.8)	338.0 (7.3)
Supplies†	549.6 (73.0)	472.3 (50.3)	472.3 (38.4)	657.0 (30.3)	657.0 (30.1)	1587.1 (45.2)	3990.1 (86.0)
Equipment	0 (0)	129.1 (13.7)	183.3 (14.9)	264.7 (12.2)	278.6 (12.7)	677.3 (19.3)	74.1 (1.6)
Overheads	0 (0)	0 (0)	236.1 (19.2)	236.1 (10.9)	236.1 (10.8)	236.1 (6.7)	236.1 (5.1)
Total	752.4	939.4	1229.7	2171.7	2185.6	3514.4	4638.3
Cost/test	0.12	0.15	0.20	0.35	0.35	0.57	0.75

*Staff costs include the time taken to perform 6161 tests for each method; the cost of the time taken to calibrate machines was negligible; †the costs included in supplies are those associated with performing a test, the control test, and the material necessary for calibration.

HCN, haemiglobincyanide.

There was a fivefold variation in cost/test, from US$0.12 for the haemoglobin colour scale to US$0.75 for the HemoCue method.

The projected annual total costs of resources (staff, supplies, equipment, and overheads) that would be used for the 6161 haemoglobin estimations performed in Ntcheu laboratory in 2001 varied from 2001 US$752.4 to 2001 US$4638.3 for the haemoglobin colour scale and HemoCue, respectively.

DISCUSSION

To our knowledge, this is the first study to provide a detailed analysis of all the resources, including training inputs and recurrent costs, actually used to establish and maintain a variety of haemoglobin methods in an African district hospital. The criteria for the most appropriate method for their circumstances were determined by local users. On the basis of simplicity, speed, accuracy, and short training times, HemoCue would be the most appropriate method of all those assessed for district hospitals in Malawi. As a result of our study, the Ministry of Health and Population in Malawi has decided to introduce HemoCue as the standard haemoglobin technique at all their 32 district hospitals and urban health centres. Its major drawback is the cost of the disposable cuvettes which comprise 86% of the overall annual cost of this method, resulting in HemoCue having the highest annual costs at US$0.75/test. For all other methods except the haemoglobin colour scale, supplies make up less than 50% of the annual costs. Despite its advantages, the high recurrent costs will probably be a major disincentive for governments of poor countries to implement HemoCue at subdistrict levels where the burden of anaemia, and hence workload, is highest.⁵ It may be acceptable to compromise slightly on accuracy for haemoglobin methods that are used to screen large numbers at community level, as long as the method is fast and cheap,⁶ and a more accurate method is available at the referral hospital. Therefore, it has been suggested that the haemoglobin colour scale might be considered for use in antenatal and paediatric screening programmes. None of the other methods assessed in our study was ideal for district hospital use, where an accurate method is essential, because they involve a dilution step. This is a major source of inaccuracy in poorer countries where automatic pipettes are unavailable.

“One of the major roles that international organisations could play would be to provide detailed, independent evaluations of new laboratory methods”

The results of our study confirm findings from elsewhere that show that the Lovibond method currently in use in Malawi has an unacceptably low accuracy.⁷ It also indicates that methods that do not give a direct readout and which require the construction of calibration curves are technically too demanding for use in rural laboratories in poorer countries where the workload is high, many laboratory staff may have no technical qualifications, and there are no supervisory systems. The inputs needed to support the general laboratory system, such as training programmes, supervisory visits, and participation in external quality assurance systems, have not been included in our analysis. However, the simplest methods with the shortest training times will probably be those that require the least external supervision and monitoring.

Our study demonstrates the complexity of the decision making processes that are involved in an apparently simple task of deciding on an appropriate haemoglobin method for district hospitals in Malawi. In consultation with local users, we have developed and tested a model that can be used to assist in this process, and which could be adapted to suit other types of health technology and other settings. The appropriateness of tests will vary according to local clinical need and resources available, so the results of our study cannot be extrapolated directly to other studies. However, the evidence gathering, analysis, and decision making processes that we have described and that were developed in close collaboration with local users and suppliers of laboratory services, provide a model that could be adapted to other tests and settings.

Health managers in poorer countries do not have technical training in laboratory issues and have great difficulty in obtaining the evidence that will enable them to make rational decisions about purchasing technological services. They need to take account not only of costs, but also simplicity, accuracy, speed, available manpower, and technical skills of their laboratory workforce and the health needs of the population. Although the World Health Organisation does provide guidance on the selection of laboratory tests,^8,9 details of all the components needed to make an evidence based judgement, such as we have provided in our study, are lacking. One of the major roles that international organisations could play would be to provide detailed, independent evaluations of new laboratory methods, which include information about start up and recurrent costs, training duration and costs, complexity of the method, and the inputs needed to maintain calibrations and quality. This type of independent and comprehensive information would be invaluable to health purchasers in developing countries, who face conflicting pressures over equipment purchase from agencies such as bilateral donors, charities, and the commercial sector.

Take home messages.

• We assessed six haemoglobin tests currently in use in Malawi, taking into account accuracy and clinical usefulness, user friendliness, time taken to learn and perform tests, and economic costs

• On the basis of simplicity, speed, accuracy, and short training times, HemoCue was the most appropriate method and will be introduced as the standard haemoglobin technique in all Malawian district hospitals and urban health centres

• However, the high recurrent costs of this test mean that the haemoglobin colour scale might be considered for use at the community level, with the more accurate HemoCue method being available at the referral hospital

• We found that the process for choosing appropriate laboratory methods is complex and very little guidance is available for health managers in poorer countries

• We have developed and tested a practical model for gathering evidence about test efficiency that could be adapted for use in other resource poor settings

Abbreviations

• HCN, haemiglobincyanide