The good use of plasma. A critical analysis of five international guidelines

Abstract

Background

The clinical use of fresh-frozen plasma (FFP) is progressively increasing both nationally and internationally, despite the fact that many studies have shown the weaknesses of the indications for its use. Guidelines on the good use of plasma have, therefore, been adopted in various countries. The aim of the present study was to analyse some of the existing guidelines on the good use of plasma, applying a scientifically validated method, as a preliminary step in the implementation of Regional guidelines.

Methods

A bibliographic search (1990–2006) was conducted in databases, websites, and the archives of scientific societies. Relevant articles were recovered in full. The selected guidelines were evaluated using the AGREE instrument, which assesses the completeness and structural quality of the guidelines and, in some aspects, the contents of the recommendations. The project, co-ordinated by the Regional Centre for Co-ordination and Compensation (CRCC) and carried out by four Services of Immunohaematology and Transfusion (SIT) in Umbria, was funded by the Region of Umbria and approved by the four health care institutions involved.

Results

The bibliographic search yielded 3067 abstracts of which 239 were considered relevant. The analysis of these led to the recovery of 11 guidelines, among which five were selected: those from the British Committee for Standards in Haematology, the Agence Française de Securité Sanitaire de Produits de Sante, the Canadian Members of the Expert Working Group, the American Society of Anesthesiologists Task Force on Blood Component Therapy and the National Health and Medical Research Council (NHMRC)/Australasian Society of Blood Transfusion.

Conclusions

None of the guidelines analysed obtained a score higher than 50% in all the domains of the AGREE score. There was no evidence of a tendency to improvement over time in the guidelines analysed. Objective evaluation of the guidelines analysed could provide the starting point for the subsequent production of similar documents.

Introduction

The use of fresh-frozen plasma (FFP) has increased considerably in recent years: for example, the UK Transfusion Service distributed 365,547 units of FFP in 1999–2000, 374, 760 in 2000–2001 and 385, 236 in 2001–2002 ¹. This trend was confirmed in the USA where there was a 70% increase in the use of plasma in the 10 years between 1991 and 2001, up to 3.9 million units transfused in 2001². In Italy there was a 7% increase in the use of plasma between 2002 and 2005³. It should be noted that although the above data have sufficient internal consistency to demonstrate the general trend to increased consumption of plasma, they cannot be compared directly or extrapolated to other situations because they often refer generically to “units of plasma”, which are subject to notable geographical and temporal variability.

Nevertheless, such an increase in the use of FFP, combined with the fact that this product is not free of infective risks and that, when not used for clinical purposes, it is the basis of the production of plasma derivatives, explain the proliferation of studies evaluating the good use of FFP. In a series of 358 patients treated with FFP (with a total of 2372 units), Luk et al.⁴ showed that the requests for FFP were appropriate for 167 patients (47%), probably appropriate for 31 (9%) and inappropriate for the other 160 patients (45%). This percentage of inappropriate requests was within the range reported in previous studies (10%–73%)^4–8.

One of the instruments universally used to reduce inappropriate transfusions are guidelines, intended as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances”⁹. The function of guidelines is to “to make explicit recommendations with a definite intent to influence what clinicians do”⁹ and, when carefully drawn up, guidelines are indeed a valuable instrument for improving clinical practice^7,10,11, facilitating and rationalising the management of the public heath care system, with regards to both costs and care of patients¹², and stimulating clinical research¹³.

The aim of this study was to systematically analyse some of the existing guidelines on the good use of plasma, in preparation for the implementation of guidelines in the Region of Umbria.

Materials and methods

Participating centres and the working group

The four Services of Immunhaematology and Transfusions (SIT) in the Region of Umbria participated in this study; these SIT are located in the hospitals of Perugia, Terni, Città di Castello and Foligno. The working group was formed by staff members from the SIT involved and experts from the Internal and Vascular Medicine unit of the University of Perugia; all members of the working group are authors of this article.

Bibliographic search

The literature available on the use of plasma was identified by a search of the following databases, accessed on internet: MEDLINE, EMBASE, Cochrane Central, BioMED, SIGN, GIMBE, and NICE. Other sources searched were websites and archives of relevant scientific societies, the indexes of specialised journals and the proceedings of the main meetings in the related disciplines. The key words used for the search of the databases were: “plasma, fresh-frozen plasma, recommendations, guidelines, clinical use of plasma”. The relevance of the references retrieved was evaluated independently by two of the authors by reading the titles and the abstracts. The full papers of the articles considered relevant were then obtained. Finally, the working group selected the guidelines subsequently analysed in this study.

The AGREE instrument

The selected guidelines were appraised using the AGREE instrument^14–16 which was designed with the aim of providing a reference scheme for the evaluation of the quality of clinical guidelines. The AGREE instrument appraises the quality of both the contents explicitly reported in guidelines and some aspects of the recommendations. It provides a theoretical evaluation of the validity of a set of guidelines; in other words, the probability that the guidelines actually achieve their intended aims.

AGREE consists of 23 items divided into six domains, each of which deals with specific aspect of the quality of the guidelines (Table I). Each item is scored from 1 (complete disagreement) to 4 (complete agreement). After having read the guidelines, each member of the working group assigned a score to all the items.

Table I.

Domains and the items of the AGREE instrument for the appraisal of guidelines

DOMAIN (ITEMS RANGE)	ITEMS
	1) The overall objective(s) of the guideline is (are) specifically described.
Scope and purpose (1–3)	2) The clinical question(s) covered by the guideline is (are) specifically described.
	3) The patients to whom the guideline is meant to apply are specifically described.
	4) The guideline development group includes individuals from all the relevant professional groups.
Stakeholder involvement (4–7)	5) The patients’ views and preferences have been sought.
	6) The target users of the guideline are clearly defined.
	7) The guideline has been piloted among target users.
	8) Systematic methods were used to search for evidence.
	9) The criteria for selecting the evidence are clearly described.
	10) The methods used for formulating the recommendations are clearly described.
Rigour of development (8–14)	11) The health benefits, side effects and risks have been considered in formulating the recommendations.
	12) There is an explicit link between the recommendations and the supporting evidence.
	13) The guideline has been externally reviewed by experts prior to its publication.
	14) A procedure for updating the guideline is provided.
	15) The recommendations are specific and unambiguous.
Clarity and presentation (15–18)	16) The different options for management of the condition are clearly presented.
	17) Key recommendations are easily identifiable.
	18) The guideline is supported with tools for application.
	19) The potential organisational barriers in applying the recommendations have been discussed.
Applicability (19–21)	20) The potential cost implications of applying the recommendations have been considered.
	21) The guideline presents key review criteria for monitoring and/or audit purposes.
Editorial independence (22–23)	22) The guideline is editorially independent from the funding body.
	23) Conflicts of interest of guideline development members have been recorded.

The table summarises the 23 items of a guideline considered by the AGREE appraisal instrument; the items are grouped into domains as done in the original AGREE instrument.

Data analysis

All members of the working group appraised each of the selected guidelines using the AGREE instrument.

The scores assigned by the individual members of the working group were averaged using the method proposed in AGREE, reported here below:

$$\frac{obtained\hspace{0.17em}score-minimum\hspace{0.17em}possible\hspace{0.17em}score}{maximum\hspace{0.17em}possible\hspace{0.17em}score-minimum\hspace{0.17em}possible\hspace{0.17em}score}$$

Statistical analysis

In order to evaluate the agreement on individual items among the various raters using a different method, Kendall’s W statistic was applied to determine inter-rater agreement within each individual guideline and, on average, across all the guidelines.

Approval and funding of the project

The project was approved by the General Management of the four health care authorities involved and funded by the Region of Umbria as one of the research project aimed at producing the 2004 Document Evaluating Determinants of Health and of Strategies (DVSS) of the Regional Health Care Service.

Results

The bibliographic search, limited to the period from January 1990 to December 2006, yielded the following results: MEDLINE (1822 entries), EMBASE (1178 entries), Cochrane central (33 entries), BioMED (33 entries), SIGN (1 entry), GIMBE (no entries) and NICE (no entries). An initial screening of these references identified 239 potentially relevant articles, which were recovered in full. The analysis of these articles led to the identification of 11 guidelines on the good use of plasma^1,17–26.

The working group decided, for merely practical reasons, to limit the analysis to five of these guidelines^{1,17,19,23,25}, selecting those that were most complete and detailed. One of the guidelines was initially published in 1996²⁵ and subsequently updated in 2006²⁷; both publications were taken into account in the analysis. Two guidelines were excluded because of their language: one was in Dutch, the other in Japanese^18,24. A third one was excluded because the journal was not available²², two more because they were recommendations from experts rather than true guidelines^20,21 and one because the guideline dealt with plasma only marginally²⁶.

Table II reports a comparison of the indications for the use of FFP suggested in the five guidelines analysed in this study.

Table II.

Synoptic analysis of the major indications for the clinical use of FFP in the guidelines considered in the study

IDICATION	GUIDELINES
	British Committee for Standards in Haematology1	Agence Française de Securité Sanitaire de Produits de Sante20	Australasian Society of Blood Transfusion19	Canadian Members of the Expert Working Group17	American Society of Anesthesiologists18
	2004	2002	2001	1997	1996–2006
Dose	10–15 mL/kg	10–15 mL/kg	10–20 mL/kg	Not stated	10–15 mL/kg
Trigger	Fibrinogen <100mg/dL INR and aPTT ratio>1.5	Fibrinogen <100mg/dL INR and aPTT ratio>1.5–1.8 normal range	Coagulation values>1–1.5 normal range	Not stated	Fibrinogen <100mg/dL INR and aPTT ratio>2*
Inherited deficiency of single clotting factors	When specific, virus-safe or recombinant factors are not available. If the patient is bleeding.	When the specific factors are not available.	If the patient is bleeding or in preparation for surgery.	Only when DDAVP and specific factors are ineffective or not available and the patient is bleeding or there is reasonable expectation of bleeding.	When the specific factors are not available.
Disseminated Intravascular Coagulation (DIC)	If the patient is bleeding	Indicated for DIC in obstetric patients	If the patient is bleeding and there are coagulation abnormalities	If the patient is bleeding and the INR increase and Apt ≥ trigger unless the underlying cause can be treated effectively.	Not stated
Thrombotic Thrombocytopaenic Purpura (TTP)	During plasma exchange	During plasma exchange	During plasma exchange	During plasma exchange	Not stated
Haemolytic-uraemic Syndrome	Not stated	Yes	Not stated	Yes (except in children)	Not stated
Warfarin anticoagulation	If the patient has severe bleeding	If the patient is bleeding (However, prothrombin complex in association with vitamin K is preferable)	If the patient is bleeding, in association with vitamin K and factor IX concentrate	If the patient is bleeding. (However, prothrombin complex, in association with vitamin K, is preferable)	Yes (5–8 mL/Kg)
Vitamin K deficiency	No	Not stated	No	If the patient is bleeding	Not stated
Liver diseases	Routine use not appropriate	If the patient is bleeding and PT and apt ≥ trigger	If the patient is bleeding and PT and aPTT ge; trigger	If the patient is bleeding and PT and aPTTge;trigger	Not stated
Massive transfusion	Use guided by updated coagulation test carried out at the patient’s bedside	In cases of traumatic haemorrhagic shock that cannot be managed immediately by surgery	If PT and aPTT ge; trigger	If the patient has microvascular bleeding and PT and apt ≥ trigger In preparation for invasive procedures/sugerry	If the patient has microvascular bleeding and PT and aPTT ≥ trigger
Heart surgery	Prophylactic use not indicated	Prophylactic use not indicated	Not stated	Prophylactic and therapeutic use not indicated	Not stated
Liver biopsy	If PT ≥ 4 times higher than the normal value. Results unpredictable	Not stated	Not stated	If INR >2	Not stated
Not indicated	Hypovolaemia Plasma exchange except for TTP Test of clotting correction in the absence of haemorrhage	Hypovolaemia Plasma exchange except for TTP Test of clotting correction in the absence of haemorrhage	Not stated	Hypovolaemia Plasma exchange except for TTP Test of clotting correction in the absence of haemorrhage Hypoproteinaemia Hypogammaglobulinaemia	Hypovolaemia Albumin concentration

The table presents the results of the comparative analysis of the major indications for the use of FFP in the five guidelines considered in the study.

^*The PT and aPTT trigger value was > 1.5 in the 1996 edition (ref. ²⁵) and was changed to >2 in the 2006 version (ref. ²⁷).

Results of the AGREE appraisal

The standardised domain-specific score was calculated using the above-cited formula. Table III shows the results of the AGREE appraisal of the five guidelines. The domains in which the guidelines were weakest were applicability and statement of editorial independence. No evidence was found of a tendency to improvement in the guidelines over time in the period 1996–2004.

Table III.

Standardised domain-specific score assigned to each guideline.

DOMAIN	ITEM	GUIDELINES
		British Committee for Standards in Haematology	Agence Française de Securité Sanitaire de Produits de Sante20	Australasian Society of Blood Transfusion19	Canadian Members of the Expert Working Group17	American Society of Anesthesiologists18
		2004	2002	2001	1997	1996
Scope and purpose	1–3	87%	78%	61%	80%	78%
Stakeholder involvement	4–7	25%	36%	69%	58%	40%
Rigour of development	8–14	38%	56%	45%	75%	62%
Clarity and presentation	15–18	67%	74%	81%	75%	63%
Applicability	19–21	31%	33%	50%	36%	39%
Editorial independence	22–23	10%	7%	22%	47%	38%
Overall assessment		(2)	(2)	(2)	(2)	(3)

The standardised domain-specific score was obtained using the formula suggested in the AGREE instrument.

For each domain, the guideline that was assigned the highest score is shown in bold; scores above 70% are underlined.

The overall assessment score was assigned according the AGREE instrument in response to the question: “Would you recommend the use of this guideline in clinical practice?”: (1) Yes, I would strongly recommend it; (2) Yes, I would recommend it (with provisos and alterations); (3) No, I would not recommend it; (4) Not sure. The score was assigned by a procedure of formal consensus of the working group.

The concordance between the raters showed a general agreement of 0.816 (p<0.0001). The agreement concerning the evaluation of the individual guidelines were as follows: 0.986 (p<0.0001) for the guidelines by the British Committee for Standards in Haematology¹, 0.924 (p<0.0001) for the guidelines by the Agence Française de Securité Sanitaire de Produits de Sante¹⁷, 0.590 (p<0.0001) for the guidelines by the Canadian Members of the Expert Working Group¹⁹, 0.716 (p<0.0001) for the guidelines by the Australasian Society of Blood Transfusion²³ and 0.817 (p<0.0001) for the guidelines by the American Society of Anesthesiologists²⁵.

Discussion

A bibliographic search of databases, websites and archives of scientific societies in immunohaematology yielded 3067 studies including 11 guidelines, of which five were evaluated using standardised criteria. None of the guidelines was complete and well-structured in all their parts. The domains in which the guidelines could be improved were those related to the applicability of the guidelines, that is, the presence and appropriateness of those recommendations or considerations aimed at facilitating the practical implementation of the guidelines. It is not, therefore, surprising that, although the above-mentioned guidelines were written – as often explicitly stated – with the aim of improving the clinical use of plasma, the audits carried out and published in the literature have not shown a major impact on clinical practice^28–32.

On the other hand, the five guidelines analysed were documents issued at national level and thus inevitably not immediately applicable in a local context. This issue should be a strong stimulus to the production of local guidelines or the implementation of local adaptations of the national guidelines, with the aim of generating instruments suitable for use in clinical practice. The area concerning the rigour of the formal production of the guidelines also showed wide margins for improvement. Given that the average level of evidence available on the clinical use of FFP is low, it is often only possible to make weak recommendations, inevitably characterised by a high degree of subjectivity. However, there are two considerations that can be made: first, the methodology teaches us that it is precisely when the evidence is scarce and weak that the reasoning leading from the evidence to the recommendation must be most detailed, and secondly, even leaving aside a formal evaluation, the content of the guidelines is often poorly defined and contradictory. The analysis of the selected guidelines revealed wide discrepancies in the reasons considered valid for using plasma, even in highly specific clinical situations. Let us take the use of plasma in patients undergoing oral anticoagulation as an example: the guidelines by the American Society of Anesthesiologists²⁵ recommend elective use of FFP; those of the Canadian Members of the Expert Working Group¹⁹ emphasise the fact that there are published reports that prothrombin complex concentrate is more effective and acts faster than plasma in antagonising (even alone) warfarin-induced anticoagulation; on the basis of these considerations, the guidelines by the Canadian Members of the Expert Working Group advise using prothrombin complex concentrate in combination with intravenous vitamin K for the treatment of massive bleeding in patients with significant increases in PT, INR or aPTT and intravenous vitamin K alone in non-emergency conditions; the guidelines by the Australasian Society of Blood Transfusion²³ suggest the use of FFP, in addition to vitamin K and factor IX concentrate, in the presence of bleeding; according to the guidelines by the British Committee for Standards in Haematology¹ FFP should not be used to antagonise the effect of warfarin except in cases of severe haemorrhage; finally, the guidelines by the Agence Française de Securité Sanitaire de Produits de Sante¹⁷ advise using vitamin K in combination with FFP or prothrombin complex concentrate to correct the anticoagulant effect rapidly in cases of severe bleeding. Taking the correction of hypovolaemia as an indication for the use of plasma as another example, this is clearly advised against in the guidelines by the Australasian Society of Blood Transfusion²³, the British Committee for Standards in Haematology¹, the American Society of Anesthesiologists²⁵ and the Agence Française de Securité Sanitaire de Produits de Sante¹⁷; the guidelines by the Canadian Members of the Expert Working Group¹⁹ also advise against this practice although in a less direct manner, suggesting that there are other safer and more effective products available to correct some problems including hypovolaemia. Whether, however, the various recommendations depend mainly on reasons of safety, suitability or availability of the alternative treatments is unknown, given that no supportive evidence is reported.

The importance of our study lies in its uniqueness. To the best of our knowledge, this is the first study that has systematically reviewed and compared published guidelines on the good use of plasma using a recognised and standardised evaluation method, as is AGREE.

One of the main limitations of our evaluation is that the working group was composed exclusively of specialists in transfusion medicine or bleeding disorders. Furthermore, only a subset of the existing guidelines was analysed, because of linguistic barriers and limited availability of resources. The guidelines chosen for analysis were those which, mainly for reasons of cultural affinity, were considered most relevant for preparatory purposes prior to the production of guidelines for the Region of Umbria.

An intrinsic limitation of the working method is subjectivity in the appraisal. Despite the fact that an objective, well-described and validated method was used, the members of the working group appraised the guidelines independently from each other, expressing judgements with a substantial degree of subjectivity. The measure of the expression of this subjectivity in statistical terms as inter-rater agreement leads to some considerations. In fact, it must be asked whether it is the AGREE method, the competence and coherence of the working group or the quality of the guidelines that is being evaluated. In reality, a different study design would be needed to evaluate the efficiency of the AGREE method¹⁴, while to understand whether the variability depends on the composition and quality of the working group or the guidelines, it should be noted that the general agreement was 0.816, which can be considered more than satisfactory for an appraisal activity of this type, while values of agreement for the single guidelines vary between 0.986 and 0.590, indicating that the main source of variability is that intrinsic to the guidelines or, rather, that related to the difficulty in evaluating specific characteristics of the guidelines.

What are the practical implications of this study? The main one is the lack of both robust evidence and methodologically correct guidelines regarding indications for the use of FFP. This lack of proof of efficacy to support the effective and safe use of FFP, as well as to underpin and guide policy choices concerning the employment of FFP has been repeatedly highlighted by various bodies over the years. Nevertheless, the lack of evidence demonstrated by the systematic analysis of the specific guidelines is even more blatant and should stimulate researchers and administrators to concentrate their energy on resolving this important social and health care problem.

In addition, the identification of the weak areas in the existing guidelines could usefully be held in consideration when producing future guidelines. Indeed, we hope that the results of this study provide a solid basis for the production of local guidelines and the improvement of existing ones, as well as being a stimulus to undertake similar appraisals in other related areas.