Use of electronic self-administered bleeding assessment tool in diagnosis of paediatric bleeding disorders

Abstract

Introduction:

In the era of electronic medical records, pen-and-paper-based physician-administered bleeding assessment tools (BAT) remain under-utilized in the clinical setting, as they are noted to be time-consuming.

Aim:

The current study reviews the use of an electronic self-administered bleeding assessment tool (eBAT) prospectively in a paediatric haematology clinic and in comparison with a physician administered BAT (pBAT).

Materials and Methods:

This was reviewed and approved in the current form because the aims statement includes the method regarding comparison of 2 groups. So no additional section required.

Results:

A total of 94 BAT response pairs were available for analysis. The median time required for patients or parents to complete the eBAT was 8 min, with less than a third of the patients requiring over 10 min. The median bleeding scores noted in this study were 4 for both the BATs, with strong positive correlation between the eBAT and the physician administered bleeding questionnaire. The eBAT had a sensitivity of 93.8% (95% CI 82.8%–98.7%), a specificity of 34.8% (95% CI 21.4%–50.3%), a positive predictive value (PV) of 60.0% (95% CI 54.5%–65.2%) and a negative PV of 84.2% (95% CI 62.5%–94.5%) for identifying a bleeding disorder.

Conclusions:

Findings indicate that eBAT is a valid and time-efficient screening tool for evaluating patients’ bleeding symptoms, which can improve clinical applicability of BATs by reducing time for bleeding history review.

1 |. INTRODUCTION

Identification of bleeding disorders relies heavily on the patient report of bleeding symptoms, which is essential in directing the appropriate diagnostic workup. Bleeding assessment tools (BATs) are helpful questionnaires that guide review of bleeding symptoms in a comprehensive manner.^1,2 Despite being identified as valuable screening tools for bleeding disorders in haematology research, the utilization of BATs in clinical practice remains low.³ For physicians unfamiliar with these tools, a BAT assessment can take up to 40 min to administer.³ Some experienced haematologists are familiar enough with these tools to administer them without the help of a paper instrument. Even for them, to mentally calculate a bleeding score while reviewing history, and the translation of information into their clinical documentation can be burdensome on busy clinic days. In the era of complete electronic medical charting, the practical utility of a paper-based-questionnaire that can take up a chunk of the typical patient appointment duration, is thus, limited.^2,4

There have been a few recent studies evaluating self-administered bleeding questionnaires, which make better direct use of patient information input.^5–8 If such patient-report tools are used ahead of the appointment – at home or in the clinic waiting room, a time gap which is often wasteful can be nicely utilized, while also allowing the patients to participate in their care. Additionally, this input at the onset of the visit lays some groundwork for the physician to be more focused on relevant bleeding symptoms during their history. Casey et al. evaluated use of such a self-administered Paediatric Bleeding Questionnaire (self PBQ), but the focus of the study was on validation of the tool for screening of von Willebrand disease, the tool was administered via pen and paper, and the results have not yet been replicated.⁷

Our goal was to prospectively evaluate the use of an electronic version of a self-administered bleeding assessment tool (eBAT) through a tablet/smart device, with patients and their parents prior to a haematology evaluation visit. The primary objective was to determine the accuracy of the self-administered eBAT by determining the level of agreement between patient/parent response and physician interviewbased response to the same BAT.

2 |. MATERIALS AND METHODS

2.1 |. Subject recruitment

This study was approved by the institutional review board at Nationwide Children’s Hospital for prospective data collection after obtaining informed consent, and assent where applicable. Identification of potential study subjects was achieved by reviewing haematology clinic schedules and referral forms at this quaternary-care paediatric facility. Patients referred to the clinic for bleeding diatheses evaluation were approached either during clinic triage in the waiting room or via telephone ahead of their appointment to introduce the study to them.

The consent form was embedded into the eBAT as the very first page, mandating completion of the consent form for all study participants before proceeding to the questionnaire. Assent, wherever applicable (based on the patient’s date of birth), was also similarly included as the second page.

2.2 |. Study population

The study included patients of age 0–21 years, who were evaluated at outpatient haematology clinic sites of Nationwide Children’s Hospital (Columbus, OH) between December 2017 and November 2018. Referral reasons could include a personal history of bleeding symptoms, a family history of a bleeding disorder, or abnormal coagulation labs. For this study, the questionnaires were exclusively presented in English, thus, only English literate patient families could be enrolled.

2.3 |. Study instruments

The questionnaire utilized in this study was based on the Self-Paediatric Bleeding Questionnaire (PBQ) developed by Casey et al., and validated as a screening tool in children being referred to the haematology clinic.⁷ The self-PBQ combines the questions from the International Society of Thrombosis and Haemostasis Bleeding Assessment Tool (ISTH-BAT, an electronic version of this is accessible at https://bleedingscore.certe.nl/) and the PBQ in paediatric-specific language.^1,7,9 This was validated for use for minimum grade 4 reading level. For this study, the self-PBQ was adapted into a REDCap® survey form, which could be accessed by the investigators and subjects with unique electronic links. REDCap® is a secure, web-based software platform designed to support electronic data capture for research studies.¹⁰

The eBAT had programming functionality to make the questionnaire user-friendly and automated. It improved ease of administration as it could skip sections that were answered in the negative (e.g., if no history of tooth extractions, survey would skip to the section for patient surgeries), making it more time-efficient. When the information for patient date of birth and gender was entered, the subsequent portions of the eBAT would become gender and age specific. The scoring algorithm was also built-in and for every completed questionnaire, the bleeding score would be automatedly calculated on a subsequent program page visible to the investigators, thus eliminating the time needed to manually tally the bleeding score. In the scoring algorithm, the score of 2 for medical evaluation beyond primary care, for referral to a specialist or for being offered detailed laboratory investigation, was only answered in the affirmative if the services offered beyond primary care were outside of a haematology clinic (e.g., surgery, otolaryngology, or obstetrics/gynaecology).¹ This was done to eliminate selection bias as all participants were referred to haematology as that was the location of the study. While building the scoring algorithm, two investigators (S.H.O and D.K.) reviewed scoring scenarios together to identify areas of scoring variations and disagreement and agreed upon standardized scoring practices. The questionnaire could be completed on the study tablet when in the waiting room, or on the family’s own devices prior to their haematology visit if a link were emailed to them.

2.4 |. Study procedures

Consenting families answered the bleeding questionnaires twice – once via a self-administered eBAT questionnaire on a tablet device via REDCap® and once through a physician-administered bleeding assessment tool (pBAT). The participants were randomly assigned to completing the eBAT first or the pBAT first. The pBAT was administered to all study participants by one investigator (D.K.) to standardize practice even when patients were being seen by different clinic providers. Patients older than 9 years of age could respond to the questionnaire themselves with help from their parents. For patients younger than 9 years, parents or guardians were asked to complete the questionnaires. The resulting bleeding score was recorded for all participants. When the investigators were the treating physicians, the score was documented in the electronic medical record, with details of history review being noted in the history and assessment/plan section. All treating physicians had access to the pBAT findings in terms of the bleeding scores as well as pertinent history findings communicated to them by the investigator prior to the appointment, verbally and as review of the eBAT summary page on REDCap.

The laboratory testing during bleeding diathesis evaluation was based on the discretion of the treating physician and was not standardized. The general testing practice for a possible bleeding disorder at our institution includes a complete blood count (CBC), prothrombin time (PT), partial thromboplastin time (PTT), platelet function assay (PFA), von Willebrand factor (VWF) antigen and activity (ristocetincofactor based) testing, factor VIII level, fibrinogen level and thrombin time. Further testing such as additional quantitative factor evaluations, VWF multimer evaluation and platelet electron microscopy or aggregation studies are pursued if there is a persistent concern for a primary haemostatic abnormality. If this work up did not reveal a diagnosis, a separate diagnostic category such as bleeding of unknown cause (BUC) was not utilized. If some patients had high bleeding score but no known bleeding diagnosis, they were simply noted as false positives.

2.5 |. Statistical analysis

Patient characteristics and bleeding scores were summarized using descriptive statistics. To assess the correlation between eBAT and pBAT, Spearman’s Rho correlation coefficient was used. Weighted Cohen’s kappa statistic was calculated to evaluate the agreement between eBAT and pBAT. Per the Landis and Koch criteria, kappa values were classified as: ≤.20 representing poor agreement, .21–.40 fair agreement, .41–.60 moderate agreement, .61–.80 substantial agreement, and values ≥.81 representing almost perfect agreement.¹¹

The bleeding scores were also classified into categorical variables with a score being either ‘normal’ or ‘abnormal’. A bleeding score of 2 or lower was noted as normal, while 3 and above was abnormal, based on past studies with BATs in a similar population.^{6,7,9,12–14} We additionally evaluated the eBAT as a screening tool for inherited bleeding disorders and calculated its sensitivity, specificity, and negative and positive predictive values with corresponding 95% confidence intervals (CI). All statistical analyses were performed using the base R statistical (R Foundation for Statistical Computing, Vienna, Austria).

3 |. RESULTS

After approaching 126 patients, a total of 107 patients agreed to participate in the study. Five patients were excluded due to incomplete eBAT data (Figure 1). Another eight patients were excluded either due to incomplete laboratory evaluations, loss to follow up, or indeterminate diagnostic results. Thus, 94 BAT response pairs were available for analysis.

FIGURE 1.

Flow diagram showing study enrolment, exclusions and inclusions (CONSORT)

The majority of the study population was female, n = 64 (68%). None of the study subjects self-identified as transgender. The median age of subjects was 13.5 years (range: .04–21). Over a third of our subjects (37%; n = 35) needed their parents to complete the questionnaire (five needing partial help); while 59 subjects could complete the eBAT themselves. In 46 instances of questionnaire administrations, the physician interview occurred first, whereas in 48 instances, eBAT was completed first by patient/parent. There was no evidence that eBAT scores were any different based on the order in which the two tools were administered (p= .48). The median time required for patients or parents to complete the eBAT was 8 min (range 2–28), with the majority of participants requiring < 10 min to complete the survey (72%, n = 68/94). The median time for pBAT completion was 10 min (range 3–19).

The median bleeding scores on both BATs was 4 (ranges: eBAT 0–15; pBAT 0–11). On the eBAT, 80% (n = 75) of patients had scores considered abnormal for children (≥3), whereas 74% (n = 70) of patients had abnormal scores with the pBAT. Overall, there was very strong positive correlation between eBAT and pBAT bleeding scores, (Spearman’s correlation coefficient .89, p < .0001; Figure 2). A weighted Kappa statistic for the comparison between self eBAT and pBAT was .60, indicating substantial agreement between the bleeding scores (p-value<.0001). There was a robust direct correlation between bleeding score and the time it took the responder to complete the eBAT (Spearman’s correlation coefficient .54, p< .0001).

FIGURE 2.

Scatter plot investigating the relationship between electronic bleeding assessment tool (eBAT) and physician administered BAT

Half of the patients in our study population (50%; n = 47) had a specific bleeding diathesis diagnosis. Of these 47 bleeding disorder patients, 19 (40%) were diagnosed with either von Willebrand factor (VWF) disease or low VWF levels, 15 (32%) with platelet storage pool disease or platelet function abnormalities, n = 15; 2 (.04%) with a collagen disorder/clinically diagnosed Ehlers Danlos syndrome, 10 (21%) with other single factor deficiencies, and 1 (.02%) was a genetically diagnosed haemophilia carrier. The median eBAT score for those with a bleeding disorder was 5 (mean = 5.9) whereas, the median eBAT score for those without a bleeding disorder was 4 (mean = 3.8, p= .0003). The eBAT had a sensitivity of 93.8% (95% CI 82.8%–98.7%), a specificity of 34.8% (95% CI 21.4%–50.3%), a positive predictive value (PV) of 60.0% (95% CI 54.5%–65.2%) and a negative PV of 84.2% (95% CI 62.5%–94.5%) for identifying a bleeding disorder (Table 1).

TABLE 1.

Characteristics of eBAT as a screening tool for identification of a bleeding disorder

	eBAT		pBAT
	Percent	95%CI	Percent	95%CI
Sensitivity	93.8%	82.8%–98.7%	89.6%	77.3%–96.5%
Specificity	34.8%	21.4%–50.3%	41.3%	27.0%–56.8%
Positive predictive value	60.0%	54.5%–65.2%	61.4%	55.1%–67.4%
Negative predictive value	84.2%	62.5%–94.5%	79.2%	60.8%–90.3%

Abbreviations: eBAT, electronic bleeding assessment tool; pBAT, physician administered bleeding assessment tool; CI, confidence interval.

4 |. DISCUSSION

While the value of BATs is well-recognized in identifying bleeding disorders, their use in clinical practice remains limited, due to their time-consuming application and the expertise required to administer one.¹⁵ The prospect of a smart questionnaire which is self-scoring and pulls in historical data into an electronic form, is thus, attractive. The current study compared the accuracy of a self-administered electronic paediatric bleeding questionnaire that addresses these concerns, by determining the level of agreement between patient/parent response and medical practitioner response to the same electronic questionnaire.

The eBAT proved to be an efficient tool, taking < 10 min for 72% of our study participants. With its short duration and automated simplified flow of the bleeding-review questions, using such a tool in the waiting room or pre-appointment would not be a burdensome ask of patients and families. Also, these durations of 8–10 min on the eBAT and 10 min on the pBAT, are still shorter than the typical reported time for BAT use.³ We did not assess the time saved for physician entry of bleeding history and score into the patient chart, as that was not planned as a comparative goal in this study owing to the wide amount of variation possible based on personal documentation style, typing speeds, use of shorthand etc. Even so, with the eBAT taking only about 10 min, this tool showed it can save valuable visit time – which could almost be a third of a short 30-min office visit. For busy physicians, these 10 min can be spent in explaining the likelihood of a bleeding disorder based on the bleeding score and management discussion, instead of learning the bleeding history. We also found that more time being spent answering the questionnaire correlated with higher bleeding scores (Figure 3).

FIGURE 3.

Scatter plot investigating the relationship between the electronic bleeding assessment tool (eBAT) score and the duration it took to complete the eBAT (in minutes)

The median bleeding scores noted in our study were 4 for both the eBAT and pBAT, which are comparable to past self-administered BATs.^7,16,17 The eBAT consistently recognized underlying bleeding disorders with high sensitivity at 94%. There was a strong positive correlation between the eBAT, and the physician administered bleeding questionnaire in this study. These findings suggest the eBAT to be a valid and reliable screening tool for evaluating patients’ bleeding symptoms.

The utility of the eBAT as a screening tool (Table 1) were similar to those previously reported for other BATs for mild to moderate bleeding issues.^7,13,18 The sensitivity and specificity of the eBAT noted in our study were fairly robust and compared well with the physician administered BAT. These were also similar to past reports of BAT characteristics, although the range for the latter is quite broad, possibly due to different scores being used as cut-offs.¹⁹ This can be explained with typical receiver operating curve characteristics of cut-off values for a screening test, in that a higher cut off score is likely to have lower sensitivity and higher specificity whereas, a lower cut-off score may identify more patient with bleeding abnormalities but has lower specificity with a higher false positive rate.

There have been a few recent studies reporting use of self-administered BATs, but these continue to be through pen and paper-based surveys, while most medical documentation today is in electronic medical records.^7,17 A recent study assessed an electronic version of a bleeding questionnaire through a social medium/website, but the aim was primarily to spread awareness and increase recognition for possible underlying bleeding disorders.⁶ This forum was not designed to validate the bleeding scores from the electronic questionnaire and information was not utilized in direct patient care. Two other studies have evaluated electronic bleeding questionnaires – one having been in Dutch language, as a one-page self-BAT administered electronically²⁰ and a second one, that evaluated a physician-administered electronic BAT.²¹ Both these studies reported good reliability of the instrument, which is replicated in our eBAT. The information collection and chart input would still require additional time from the patient visit being dedicated to the detailed history, which proves challenging for a busy clinical practice.² Our study shows potential reduction in this information collection piece, and further reduction in the information translation with the potential incorporation of the eBAT into electronic medical record systems. eBAT could translate information into the patient medical record in real-time, in the form of pertinent history data and bleeding scores, while allowing patients to participate in their care. An interesting finding in our study population was that the females made up about 70% of the sample, which is often the case in studies that include non-severe, non-haemophilia bleeding disorders.^5,16,22 This is attributable to a higher incidence of haemostatic challenges in women due to reproductive bleeding.²³

We are mindful of some limitations to our study. The laboratory evaluation for an abnormal bleeding score was not standardized and the consulting physicians were not the same for all patients. Although this likely led to some degree of testing variability, we find this to mirror true clinical practice. This being a questionnaire-based study, biases like recall, response bias, and interviewer bias were possible. To minimize response and interviewer bias – we standardized the administration of the pBAT to one investigator only. We also did not plan a defined washout period between the administration of the eBAT and pBAT for fear of detrimental effects on enrolment and retainment of study participants. Instead, we randomized the order in which the study participants completed the eBAT or the pBAT. Additionally, although the study only enrolled families/subjects that confirmed English literacy, we did not determine health literacy, which may have affected response accuracy and duration of time needed to complete the questionnaire.

There are many strengths to this study. Our study utilized eBAT in previously undiagnosed patients and is the first of its kind to prospectively evaluate an eBAT in a paediatric population and assess the validity of this tool. The eBAT was created on REDCap® and a second version created in the electronic medical record system EPIC® (Figure 4; this version in EPIC® could only be made available after the study was completed and hence could not be utilized for the study). Both these systems allow tool sharing across different centres through centrally maintained national libraries when appropriate permissions are obtained and when the tools are allowed upload into the central/national libraries.

FIGURE 4.

Screenshot of the bleeding assessment tool built into the electronic medical record system

5 |. CONCLUSIONS

In a paediatric haematology specialty clinic setting, the information gained from the eBAT was noted to be quite comparable to the physician-administered bleeding questionnaire, with 75% of participants being able to complete the eBAT in ≤10 min. The clinical application of eBAT has the potential to improve utilization of bleeding scores in the evaluation of patients with possible bleeding diatheses by addressing the limitations of a paper-pen based survey. When utilized in the waiting room or pre-appointment, it can provide valuable and detailed historical information in a time-efficient manner. Future work will focus on creating an electronic information bank from the eBAT, which houses symptomatology and bleeding scores as a repository of clinical information useful for future studies. As the eBAT was successful as a time-saving bleeding evaluation tool, its translations in additional languages should be validated to make it available to a larger population.

DATA AVAILABILITY STATEMENT

Our team is committed to the practice, policies, and goals of widespread data sharing among researchers to accelerate research on children and adolescents to the extent allowable in protecting human subject privacy. All data will be entered into a secure relational database. Once we have completed all analyses and publications related to the grant’s central aims, we will strip the data of all patient identifying information. Access to this data will be made available to other investigators upon request with the completion of a standard data use agreement, free of charge, within 2 years of the close of the project.