Generation and Optimization of the Self-Administered Bleeding Assessment Tool (Self-BAT) and its Validation as a Screening Test for von Willebrand Disease

Summary

Introduction/Aim

Our aim was to generate, optimize and validate a self-administered bleeding assessment tool (BAT) for von Willebrand disease (VWD).

Methods

In Phase 1, medical terminology in the expert-administered ISTH-BAT was converted to a grade 4 reading level to produce the first version of the Self-BAT which was then optimized to ensure agreement with the ISTH-BAT. In Phase 2, the normal range of bleeding scores was determined and test-retest reliability analyzed. In Phase 3, the optimized Self-BAT was tested as a screening tool for first time referrals to the Hematology clinic.

Results

BS from the final optimized version of the Self-BAT showed an excellent intra-class correlation coefficient (ICC) of 0.87 with ISTH-BAT BS in Phase 1. In Phase 2, the normal range of bleeding scores for the optimized Self-BAT was determined to be 0 to +5 for females and 0 to +3 for males and excellent test-retest reliability was shown (ICC = 0.95). In Phase 3, we showed that a positive Self-BAT BS (≥ 6 for females, ≥ 4 for males) has a sensitivity of 78%, specificity of 23%, positive predictive value (PPV) of 0.15 and negative predictive value (NPV) of 0.86 for VWD; these figures improved when just the females were analyzed; sensitivity of 100%, specificity of 21%, PPV=0.17 and NPV=1.0.

Conclusion

We show an optimized Self-BAT can generate comparable BS to the expert-administered ISTH-BAT and is a reliable, effective screening tool to incorporate into the assessment of individuals, particularly women, referred for a possible bleeding disorder.

Introduction

Von Willebrand Disease (VWD) is characterized by increased or excessive mucocutaneous bleeding manifesting as epistaxis, bruising, prolonged bleeding from trivial wounds, oral cavity bleeding, bleeding after surgery, dental extraction or childbirth and menorrhagia. Additionally, the most severely affected individuals experience musculoskeletal bleeding including hemarthrosis and muscle hematomas.[1] VWD, especially Type 1 VWD, can be overlooked because of under-recognition of the importance of mucocutaneous bleeding symptoms. Additionally, unaffected individuals report hemorrhagic symptoms leading to an overlap with mildly affected individuals. VWF levels can vary as a result of environmental influences, therefore, patients with borderline low plasma levels of von Willebrand factor are often difficult to distinguish from those with normal levels.[2] In recent years, there has been increasing recognition of the value of standardized bleeding assessment tools in meeting these diagnostic challenges.

In 2005, Rodegheiro published a standardized bleeding assessment tool (BAT), which quantified hemorrhagic symptoms in VWD patients.[3] This questionnaire underwent subsequent revisions in order to improve its sensitivity, decrease administration time and make it applicable for children. Studies have shown that BATs can accurately distinguish normal from abnormal bleeding, are useful screening tools to identify patients with mild bleeding disorders, and can be used to describe bleeding severity.[4–7] In 2010, the International Society on Thrombosis and Haemostasis (ISTH) established a Working Party to develop and endorse a single BAT to standardize reporting of bleeding symptoms for use in adult and pediatric populations.[8] This, and all previously published BATs require expert administration, which can be problematic from a resource perspective, especially in a busy clinical setting. Additionally, the requirement for expert-administration is a barrier to more widespread use of such tools.

For this study, our objective was to modify the ISTH-BAT into a self-administered BAT that can be completed without assistance, optimize it to ensure the bleeding scores derived from its use were comparable to those from the ISTH-BAT and to validate its use as a screening tool in the Hematology clinic.

Methods

All research subjects were adults ≥ 18 years. Research Ethics Board approval was obtained from Queen’s University and written informed consent was obtained prior to participation.

Phase 1

To generate the first version of the Self-BAT, medical terminology in the ISTH-BAT was converted into lay language at a grade 4 reading level for example, the word epistaxis was changed to nosebleeds. Detail was added, where appropriate, to define terms. In the ISTH-BAT, the first question in the Epistaxis category is “Have you ever had spontaneous epistaxis” and during completion of the questionnaire, the expert helps define spontaneous for the patient. For the Self-BAT, we added potential causes of nosebleeds such as injury or dry air, to enable subjects to understand how to distinguish from spontaneous epistaxis themselves. Additionally, the order of questions was changed to make it flow more logically. For example, in the ISTH-BAT, detail about duration of bleeding is asked after whether medical attention was required, in the Self-BAT duration and frequency are captured prior to detail about medical attention. The “skip logic” of the ISTH-BAT was maintained, so if a subject reported no symptoms in a category, he/she was instructed to skip the questions about frequency and duration. Detail that does not directly affect the bleeding score was removed in the Self-BAT however; the integrity of the scoring system was maintained. Therefore, the scoring systems for both the ISTH-BAT and the Self-BAT are identical.

Both control and affected subjects were recruited for Phase 1. To recruit control subjects, an advertisement was placed in local newspaper. Healthy adults, who were not pregnant, did not have a known problem with bleeding or bruising and not on antiplatelet agents or anticoagulants were eligible to participate as controls. Affected subjects were recruited from the Inherited Bleeding Disorders Clinic of Southeastern Ontario and were required to have a confirmed diagnosis of Type 1 VWD and could not be pregnant at the time of the study. The study definition of Type 1 VWD was a VWF antigen (VWF:Ag) and/or VWF ristocetin cofactor activity (VWF:RCo) between 0.05 – 0.50 IU/mL on at least two occasions, RCo: Ag ratio > 0.6 and a normal VWF multimer profile for this and subsequent phases of the study. Whole blood was drawn by phlebotomy into 3.2% sodium citrate on each control and affected participant and VWF:Ag, VWF:RCo and FVIII were measured as previously described.[9] Type 1 VWD was chosen as the “affected” group for the optimization given that it is the most common mucocutaneous bleeding disorder, and the best studied from the perspective of BATs. It is also the most likely to present as a new referral to the adult Hematology clinic; Type 2 and 3 VWD will be more commonly diagnosed in childhood because of the increased severity of bleeding symptoms.

Control and affected subjects were administered the ISTH-BAT (by an expert) and Self-BAT (without assistance) in random order at least 2 weeks apart, followed by participation in a focus group. Both questionnaires were scored according to the ISTH-BAT 0 to 4 scoring key.[8] The ISTH-BAT bleeding score (BS) was compared to the Self-BAT BS using the intra-class correlation coefficient (ICC). Focus groups were held, each including ~5 – 7 individuals following a semi-structured interview format. Responses were recorded, collated and analyzed after each focus group. After consideration of both the ICC and the focus group feedback for the first group of subjects, revisions to the Self-BAT Version 1 were made, and new control and affected subjects, who had not previously participated, were recruited and the process repeated. Prior to the study, we had determined that Phase 1 would end when the ICC > 0.80 between the ISTH-BAT and Self-BAT BSs and no further feedback about questionnaire revision was received during the focus groups.

Phase 2

New control and affected subjects, who had not participated in Phase 1, were recruited for Phase 2, using the same criteria as for Phase 1. Each subject completed only the Self-BAT without assistance. Blood was collected and laboratory values analyzed as above. The normal range of bleeding scores for the final version of the Self-BAT was established from healthy control subjects recruited from laboratory and hospital staff. Healthy controls were not pregnant, had no known problem with bleeding or bruising and were not on antiplatelet agents or anticoagulants. Mean and standard deviation of the bleeding scores were calculated. Outliers were defined as +/− 3 SD from the mean and were excluded. Once outliers were removed, the middle 95 percentile was used to determine the normal range. A subset of affected and controls subjects completed the final optimized version of the Self-BAT a second time at least 2 weeks later in order to analyze test-retest reliability.

Phase 3

During Phase 3, adult patients referred for the first time to a Hematologist because of a problem with bleeding or bruising were recruited to test the diagnostic utility of the Self-BAT as a screening tool for VWD. Patients with a previous diagnosis of an inherited bleeding disorder, or those with an acquired cause of bleeding (low platelets, renal or liver disease) were excluded. All subjects were given the Self-BAT to complete without assistance, prior to their appointment with the Hematologist. After routine clinical assessment by the Hematologist, all were investigated with the usual laboratory work-up which included a CBC, INR/PT/PTT, thrombin time, fibrinogen, ferritin, ABO blood group and specific tests for VWD (VWF:Ag, VWF:RCo, FVIII and VWF multimers). Sensitivity, specificity, PPV and NPV were calculated by comparing BS (positive/abnormal or negative/normal) to the laboratory-defined diagnosis of VWD.

Results

Phase 1

There were a total of 38 control subjects (18 male, 20 female) and 20 affected subjects (2 male, 18 female) who participated in Phase 1. The mean age of the control subjects was 39 years while the mean age of the affected subjects was 36 years (see Table 1). Given our study definition of Type 1 VWD, (VWF:Ag and/or VWF:RCo < 0.50 IU/mL on at least two occasions) there are affected individuals with either VWF:Ag or VWF:RCo > 0.50 IU/mL, but not both. As expected, there is a predominance of blood group O in the affected subjects. Subjects in the affected group had a significantly higher mean Self-BAT BS and ISTH BAT BS when compared with controls. The ICC between scores obtained on the Self-BAT and on the ISTH BAT was only moderate for the first version, completed by 23 subjects (ICC=0.48), but steadily improved after each major revision (version 2 ICC=0.79 completed by 15 subjects, version 3 ICC=0.81 completed by 10 subjects). BS from the final version of the Self-BAT completed by 10 subjects (included as a Supplement) showed a high degree of correlation with ISTH-BAT BS (ICC=0.87).

Table 1.

Phase 1 Demographic Data, Laboratory Values and Bleeding Scores

	Control (n=38)	Affected (n=20)	p value
Male gender (%)	18 (47)	2 (10)	0.006
Mean age years (range)	39 (18 – 72)	36 (18 – 59)	0.294
Mean VWF:Ag IU/mL (range)	1.01 (0.65 – 1.46)*	0.59 (0.47 – 0.93)	<0.001
Mean VWF:RCo IU/mL (range)	0.69 (0.61 – 1.22)*	0.38 (0.21 – 0.58)	< 0.001
Mean FVIII IU/mL (range)	1.34 (0.71 – 1.49)*	0.71 (0.47 – 0.93)	< 0.001
Blood Group O (%)	15 (40)	16 (80)	0.001
Mean Self-BAT BS (range)	1.2 (0 – 4)	10 (4 – 17)	< 0.001
Mean ISTH-BAT BS (range)	1.5 (0 – 4)	11 (5 – 20)	< 0.001

VWF:Ag = VWF antigen, VWF:RCo = VWF ristocetin cofactor activity, FVIII = Factor VIII, Chi-square analysis was performed for categorical variables, and Mann-Whitney for continuous variables.

^*Data from 2 control subjects were excluded due to sample thaw prior to analysis.

The focus group feedback was liveliest during the early stages of Phase 1. In general, changes were made to the Self-BAT if more than one participant raised the same concern. In version 1 of the Self-BAT, category stems included the word problem, which created concern for all of the participants. For example, the section covering nosebleeds began with the question “Have you ever had a problem with nosebleeds?” All participants were uncomfortable with the word “problem”, and did not feel qualified to make that judgment. Therefore, this question was re-worded to “Have you ever had a nosebleed?” for version 2, which increased participant comfort with the questionnaire. Subsequent changes for version 3 included further defining medical terms, such as clarifying subdural hematoma as meaning bleeding into the brain. Little to no negative feedback was received following the focus groups for versions 3 and 4 and in combination with the high ICCs for these versions, led us to declare version 4 as optimized.

Phase 2

In total, 27 controls (10 male, 17 female) and 23 affected subjects (3 male, 20 female) completed Phase 2 of the study with a mean age of 44 years for the controls and 38 years for the affected subjects. The same pattern of VWF:Ag and VWF:RCo and blood group O was seen as in Phase 1. The bleeding scores obtained on the Self-BAT in the affected population were significantly higher than those obtained in the control population (see Table 2). The normal range of bleeding scores, determined from 57 individuals not known to have a problem with bleeding or bruising (34 females, 23 males) was 0 to 5 in females and 0 to 3 in males. Therefore, a positive or abnormal bleeding score is ≥ 6 for females and ≥ 4 for males. Ten of the affected subjects completed the final optimized version of the Self-BAT a second time 5–8 months after the first visit. The test re-test reliability was very high, with an ICC of 0.95.

Table 2.

Phase 2 Demographic Data, Laboratory Values and Bleeding Scores

	Control (n=27)	Affected (n=23)	p value
Male gender (%)	10 (37)	3 (13)	0.103
Mean age years (range)	44 (19 – 91)	38 (18 – 62)	0.342
Mean VWF:Ag IU/mL (range)	0.87 (0.60 – 1.40)*	0.49 (0.31 – 0.73)	<0.001
Mean VWF:RCo IU/mL (range)	0.66 (0.57 – 0.83)*	0.35 (0.13 – 0.55)	< 0.001
Mean FVIII IU/mL (range)	0.90 (0.60 – 1.48)*	0.71 (0.30 – 1.49)	< 0.001
Blood Group O (%)	12 (44)	18 (78)	0.001
Mean Self-BAT BS (range)	1.2 (0 – 4)	10 (4 – 17)	< 0.001
Mean ISTH-BAT BS (range)	1.5 (0 – 4)	9.7 (2 – 16)	< 0.001

VWF:Ag = VWF antigen, VWF:RCo = VWF ristocetin cofactor activity, FVIII = Factor VIII, Chi-square analysis was performed for categorical variables, and Mann-Whitney for continuous variables.

^*Data from 6 control subjects were excluded due to sample thaw prior to analysis.

Phase 3

For Phase 3, 64 subjects were enrolled (11 male, 53 female) with a mean age of 43 (range 18–73). Of the subjects enrolled, 47 were found to have a positive bleeding score (≥4 for males, ≥6 for females) while 17 were found to have a negative score (≤3 for males, ≤5 for females). Based on the laboratory investigations, nine were diagnosed with VWD and this included 2 males and 7 females. All of the females with VWD had positive bleeding scores but both of the males diagnosed with VWD had normal bleeding scores. (Figure 1). All of the subjects diagnosed with VWD during this study had mild Type 1 VWD and again, the same pattern of VWF:Ag and VWF:RCo was seen as in Phases 1 and 2. All newly diagnosed VWD patients were blood group O. These findings resulted in a sensitivity of 78%, specificity of 27%, with a positive predictive value of 0.15 and a negative predictive value of 0.88 for VWD. When the analysis was restricted to female subjects, the sensitivity was 100%, specificity was 21%, PPV was 0.17 and the NPV was 1.0. Additional investigations, including platelet function testing and platelet dense granule quantitation for those with no VWD is ongoing.

Figure 1. Phase 3.

Figure 1 shows the subjects recruited for Phase 3 including the results of the Self-BAT (Positive or Negative BS) and whether or not they were diagnosed with VWD.

Discussion

We report the development, optimization and validation of the Self-BAT in this manuscript. The development and optimization was undertaken carefully and shows that the bleeding scores derived from the optimized Self-BAT compare very well with those derived from the expert-administered ISTH-BAT. Of importance, we determined the normal range of BS to be 0 to +5 for females and 0 to +3 for males for the Self-BAT, which exactly match the normal range of BS for the ISTH-BAT. [10] This is an important strength of this research, as it means Self-BAT BS will be comparable to other published data. The results of Phase 3 support its use as a screening tool for VWD in the Hematology clinic, particularly for females.

A few critical issues about this study merit additional consideration. Gender skewing throughout the study is obvious. While gender distribution in the control group was almost even, in the affected group significantly more females participated in the study than males. This is reflective of our clinic population as there are many more females referred and diagnosed with VWD than males and therefore available for recruitment. This is certainly relevant for the male undiagnosed affected patient and, as presented above, the Self-BAT is not as effective for males as it is for females. Males, in general, are exposed to fewer hemostatic challenges than females and any BAT, whether expert or self-administered will suffer from this limitation. Interestingly, all of the new diagnoses of VWD made during this study were of Type 1 VWD. No new diagnoses of Type 2 or Type 3 VWD were made perhaps because of the more severe phenotype, and therefore, less likely referral as a new patient to an adult clinic. Lastly, this study is focused exclusively on adults. A similar study evaluating a self/parent-administered BAT in children and adolescents is ongoing.

In this study, we have shown the Self-BAT to be a reliable and effective screening tool, particularly for women being assessed for VWD. Ideally, a screening tool such as this would also be effective at identifying patients with other mild bleeding disorders (such as a platelet function disorder). Given the prevalence of VWD, this initial report is focused on that diagnosis, however an ongoing study is evaluating the Self-BAT’s performance as a screening tool for other disorders of hemostasis. We also have a future study planned to convert and validate the Self-BAT as a computer-based tool, with an ultimate vision of making it widely available outside of the Hematology clinic.