SPOT GRADE II: Clinical Validation of a New Method for Reproducibly Quantifying Surgical Wound Bleeding: Prospective, Multicenter, Multispecialty, Single-Arm Study

Abstract

The SPOT GRADE (SG), a Surface Bleeding Severity Scale, is a unique visual method for assessing bleeding severity based on quantitative determinations of blood flow. This study assessed the reliability of the SG scale in a clinical setting and collected initial data on the safety and efficacy of HEMOBLAST Bellows (HB), a hemostatic agent, in abdominal and orthopedic operations. Twenty-seven patients were enrolled across 3 centers and received the investigational device. Bleeding severity and hemostasis were independently assessed by 2 surgical investigators at baseline and at 3, 6, and 10 minutes after application of HB and compared for agreement. The mean paired κ statistic for assignment of SG scores was .7754. The mean paired κ statistics for determining eligibility for participation in the trial based on bleeding severity and the mean paired κ statistics determining the presence of hemostasis were .9301 and .9301, respectively. The proportion of patients achieving hemostasis within 3, 6, and 10 minutes of HB application were 50.0%, 79.2%, and 91.7%, respectively. There were no unanticipated adverse device effects and one possible serious adverse device effect, as determined by the Independent Data Monitoring Committee (IDMC). The reliability of the SG scale was validated in a clinical setting. Initial data on the safety and efficacy of HB in abdominal and orthopedic operations were collected, and there were no concerns raised by the investigators or the IDMC.

Background

Hemorrhage in surgical procedures has been shown to be related to 55% of perioperative complications and 27% of deaths.¹ The development of hemostatic agents aims to help reduce surgical blood loss and associated morbidity and mortality. Evaluation of hemostatic agents has been performed in prospective, controlled clinical trials.^2–28 However, these clinical trials have utilized subjective, nonclinically validated assessments for bleeding severity and hemostasis or no assessments at all. Use of a subjective assessment for determining bleeding severity in a specific treatment population (a subset of all bleeding severities) as well as for determining hemostasis (success) or the absence of such assessments can result in ambiguous and incomparable efficacy data. Therefore, these published multi-investigator, clinical studies may not able to reliably and consistently evaluate the comparative performance of hemostatic agents.

Previous work has been performed to develop objective bleeding scales; however, they have not been validated in controlled clinical settings.^29,30 The SPOT GRADE (SG) is a clinically validated Surface Bleeding Severity Scale that was developed for the quantitative assessment of target bleeding site (TBS) blood loss (Figure 1) and meets Food and Drug Administration (FDA) requirements for use in determining the degree of bleeding in a surgical wound.^31–33

Figure 1.

Surface Bleeding Severity Scale, the SPOT GRADE, including flow rates. ACS-ATLS indicates American College of Surgeons Advanced Trauma Life Support. Modified from Spotnitz WD, Zielske D, Centis V, et al. The SPOT GRADE: a new method for reproducibly quantifying surgical wound bleeding. Spine (Phila Pa 1976). 2018;43(11):E664-E671.

In addition to providing a consistent and reliable method for evaluating hemostatic agent performance, another potential benefit of the SG scale in surgery is to serve as an indicator of severity and volume of blood loss prior to any systemic blood pressure changes and reductions in hematocrit. Having a consistent, uniform scale for bleeding severity may allow further definition and delineation of appropriate actions to minimize intraoperative bleeding at each level of severity and help avoid the need for transfusion of blood products. Another major advantage of using the SG scale is that multiple bleeding sites can each be independently characterized and treated according to methods appropriate for the bleeding severity of each site. In addition, consistent calibration of assessments of bleeding severity among health professionals in a specific patient may also be facilitated, leading to more efficient communication and higher quality care. Thus, training of health professionals on such a scale may potentially enhance patient care.

The SG scale has been previously described based on an in vitro model of surface bleeding severity.³¹ The current study was executed to validate the use of the SG scale in a clinical setting to support employment of the scale in large multicenter randomized trials³³ as well as to provide an initial evaluation for the safety and efficacy of a local hemostat, HEMOBLAST Bellows (HB).

Methods

Study Design and Patient Population

A prospective, multicenter, multispecialty, single-arm clinical study was performed to evaluate the SG scale and the safety and efficacy of a hemostatic device, HB (Biom’Up France, SAS). The study was performed in abdominal and orthopedic operations under an FDA-approved Investigational Device Exemption and was registered on ClinicalTrials.gov (NCT02502019). Institutional review board approvals were obtained prior to any study-specific activities being performed, and the study was conducted in accordance with applicable regulations and the Declaration of Helsinki.

HEMOBLAST Bellows was supplied as a bellows applicator preloaded with 1.65 g of hemostatic powder consisting of a combination of porcine collagen, bovine chondroitin sulfate, and human-derived thrombin (1500 IU). The primary objective of the study was to assess the reliability of the SG scale in a clinical setting. Secondary objectives were to collect initial data on the safety and efficacy of HB.

Twenty-seven patients were enrolled across 3 institutions (University of North Carolina, Spectrum Medical, and University of Southern California). Patients older than 21 years undergoing nonemergent operations were evaluated for eligibility after giving written informed consent.

Patients were excluded if they met the following criteria: undergoing laparoscopic, thoracoscopic, robotic, spinal, neurologic, or emergency surgical procedures; pregnant, planning on becoming pregnant during the follow-up period, or actively breastfeeding; clinically significant coagulation disorder/disease (platelet count <100 000/µL and/or international normalized ratio >1.5) within 4 weeks of surgery; chronic corticosteroid use within 2 weeks before surgery; received intravenous heparin or oral warfarin within 24 hours of surgery; active or suspected infection at the surgical site; had or planned to receive organ transplantation; known sensitivity, allergy, or religious objections to any component(s) of HB; American Society of Anesthesiologists classification of >4; life expectancy of <3 months; known psychiatric disorder which would preclude the patient from completing the study; severe congenital or acquired immunodeficiency; HB would be used at the site of a porous-coated joint implant; participation in another investigational study within the past 30 days; and not appropriate for inclusion per the medical opinion of the investigator.

Eligibility was also assessed intraoperatively to confirm identification of a TBS with SG scale scores of 1 (minimal), 2 (mild), or 3 (moderate) bleeding³¹ for which conventional means for hemostasis were ineffective or impractical based on the indications for HB. Patients were evaluated preoperatively, intraoperatively, postoperatively, and at 6 ± 2 weeks.

The first participant for each investigational site was treated as a lead-in patient; lead-in patients were considered part of the safety population but not part of the efficacy analysis population.

Clinical investigators, 2 at each site consisting of the surgeon of record and an additional surgeon or a physician’s assistant, underwent proprietary training and were qualified by testing on the SG scale prior to the enrollment of any patients.³¹ The training consists of viewing a series of 36 videos depicting 6 levels of bleeding (Figure 1) with 2 videos at each SG scale severity level on 3 different sized plexiglass plates simulating wounds of 1, 10, and 50 cm². The investigators are then tested on a series of 42 videos to confirm their learning of the SG scale. After the surgeon of record identified a TBS with the required characteristics, baseline SG scores for the TBS of each patient were independently and concurrently assigned by the 2 investigators. These same 2 investigators also independently and concurrently assigned SG scores for the TBS at 3, 6, and 10 minutes after HB application. Hemostasis was defined as an SG score of 0.

Study End Points

The primary objective of the study was to quantify inter-rater reliability, as measured by Cohen κ, for the assignment of SG scores by investigators. The secondary end points for the trial were a priori specified as (1) the proportion of patients achieving hemostasis within 6 minutes of HB application, (2) the proportion of patients achieving hemostasis within 10 minutes of HB application, (3) the proportion of patients achieving hemostasis within 3 minutes of HB application, and (4) the incidence of adverse events (AEs) through final follow-up. The safety success criterion was no more than 1 unanticipated adverse device effect (UADE) or serious adverse device effect (SADE), as determined by the Independent Data Monitoring Committee (IDMC).

Gauze weights were also used to collect data on bleeding severity. Preweighed stacks of gauze were held against the TBS at baseline and each evaluation time point for 5 seconds and weighed afterward to calculate the mass of blood loss from the TBS.

Statistical Analysis

Patient characteristics were summarized using the sample mean and variance for continuous covariates and frequency percentage for discrete covariates. The primary end point, inter-rater reliability of SG scores, was summarized by Cohen κ statistic, a common summary measure for quantifying the percent agreement between investigators that is above what is expected by chance. Specifically, the κ statistic can range from −1 to +1, where 0 represents the amount of agreement that can be expected from random chance, and 1 represents perfect agreement between the raters. κ values are commonly interpreted as follows: values ≤0 as no agreement and .01 to .20 as none to slight, .21 to .40 as fair, .41 to .60 as moderate, .61 to .80 as substantial, and .81 to 1.00 as almost perfect agreement. Success for the clinical trial was a priori defined as a mean paired κ statistics for assignment of SG scores >.61 and mean paired κ statistics for determining eligibility and determining hemostasis using SG scores >.80.³⁴ Pairwise-weighted κ statistics were computed using all possible pairs of investigator scorings at all available time points. For an overall summary of investigator agreement, pairwise weighted κ statistics were averaged over all pairings.

Secondary end points were quantified by the sample proportion of patients achieving hemostasis at 6, 10, and 3 minutes. The incidence of AEs was also summarized as the proportion of patients experiencing an AE. Change in gauze weight by SG score was assessed via linear regression. All statistical analyses were completed using SAS System software, version 9.3 or above.

Results

A total of 27 patients were enrolled and received HB including 3 lead-in patients and were considered the safety analysis population (9 abdominal and 18 orthopedic). Twenty-four patients were included in the efficacy analysis population (8 abdominal and 16 orthopedic). All patients completed the study as planned.

The average age for all patients was 62.8 years, with 51.9% male and 48.1% female. Regarding ethnicity, 22.2% of the patients were Hispanic, 77.8% of the patients were Caucasian, and 22.2% were African American (Table 1).

Table 1.

Patient Demographic Data by Surgery Type (Safety Analysis Population).

Measure	All	Abdominal	Orthopedic
Age	62.8 ± 8.64 (27) 63.0 [55.0-68.0]	60.4 ± 10.70 (9) 55.0 [53.0-66.0]	64.0 ± 7.48 (18) 64.5 [62.0-68.0]
Gender
Male	14/27 (51.9%)	6/9 (66.7%)	8/18 (44.4%)
Female	13/27 (48.1%)	3/9 (33.3%)	10/18 (55.6%)
Ethnicity
Hispanic or Latino	6/27 (22.2%)	6/9 (66.7%)	0/18 (0.0%)
Not Hispanic or Latino	21/27 (77.8%)	3/9 (33.3%)	18/18 (100.0%)
Race
Caucasian	21/27 (77.8%)	9/9 (100.0%)	12/18 (66.7%)
African American	6/27 (22.2%)	0/9 (0.0%)	6/18 (33.3%)
American Indian or Alaska Native	0/27 (0.0%)	0/9 (0.0%)	0/18 (0.0%)
Asian	0/27 (0.0%)	0/9 (0.0%)	0/18 (0.0%)
Native Hawaiian or other Pacific Islander	0/27 (0.0%)	0/9 (0.0%)	0/18 (0.0%)
Other	0/27 (0.0%)	0/9 (0.0%)	0/18 (0.0%)

The surgical procedures consisted primarily of liver resections and total knee replacements, with the most common indications being metastatic cancers to the liver and osteoarthritis. The TBSs consisted of the soft tissue, muscle, parenchyma, and bone. The average dimensions of the TBS were 21.1 ± 75.9 cm², with larger dimensions in the abdominal arm (51.0 ± 131.0 cm²) and smaller dimensions in the orthopedic arm (6.1 ± 3.8 cm²).

For the primary end point, the mean paired κ statistic for the assignment of SG scores by 2 Investigators, the pairwise weighted κ statistics were .7441, .7640, and .8182 for each investigator pair; the mean paired κ statistic was .7754 (Table 2).

Table 2.

Mean Paired κ Statistic for Assignment of SPOT GRADE Scores.

Surgical investigator pair	Pairwise weighted κ
Investigator pair 1	.7441
Investigator pair 2	.7640
Investigator pair 3	.8182
Mean	.7754

The inter-rater agreement was also assessed for determining eligibility (SG score of 1, 2, or 3 [eligible] vs SG score of 0, 4, or 5 [ineligible]) and determining hemostasis (SG score of 0 vs SG score >0). The mean paired κ for determining eligibility and hemostasis were both .9301. The pairwise simple κ statistics for the investigator pairs along with the mean paired κ are provided (Table 3).

Table 3.

Mean Paired κ Statistic for Determining Eligibility and Hemostasis.

Surgical investigator pair	Eligibility: pairwise simple κ	Hemostasis: pairwise simple κ
Investigator pair 1	1.0000	1.0000
Investigator pair 2	0.8902	0.8902
Investigator pair 3	0.9000	0.9000
Mean	0.9301	0.9301

Change in gauze weight by SG score was examined. There was no statistically significant correlation between a change in gauze weight and SG scores.

Efficacy of HB was a secondary end point of this study. Rates of hemostasis across all study arms at 3, 6, and 10 minutes after HB application were 50.0%, 79.2%, and 91.7%, respectively (Table 4). Rates of hemostasis for the abdominal arm of the study at these time points were 25.0%, 50.0%, and 75.0% (Table 4). Rates of hemostasis for the orthopedic arm of the study at 3, 6, and 10 minutes after HB application were 62.5%, 93.8%, and 100.0%, respectively (Table 4).

Table 4.

Proportion of Patients Achieving Hemostasis at 3, 6, and 10 Minutes.^a,b,c

Time (minutes)	All	Abdominal	Orthopedic
3	12/24 50.0% (31.4%-68.6%)	2/8 25.0% (7.1%-59.1%)	10/16 62.5% (38.6%-81.5%)
6	19/24 79.2% (59.5%-90.8%)	4/8 50.0% (21.5%-78.5%)	15/16 93.8% (71.7%-98.9%)
10	22/24 91.7% (74.2%-97.7%)	6/8 75.0% (40.9%-92.9%)	16/16 100.0% (80.6%-100.0%)

^a Numbers are n/N percent (95% confidence interval).

^b Wilson confidence limits (score based) are used in the table.

^c Cumulative numbers of patients achieving hemostasis at each time are counted.

The median hospital stay was 4 days; no patients were noted to have any clinical signs or symptoms of postoperative bleeding and there were no reoperations for bleeding. A total of 10 (37.0%) patients experienced postoperative complications or AEs. Forty-one AEs were reported, with 8 deemed as severe adverse events (SAEs). The IDMC reviewed and adjudicated each SAE and identified one as a possible severe adverse device event (SADE)—the development of nonocclusive thrombi in the portal and superior mesenteric veins. This study patient underwent a liver resection for metastatic rectal cancer. Prior to initiation of the hepatic transection, the portal pedicle was clamped (Pringle maneuver) for 30 minutes without heparinization. HEMOBLAST Bellows was applied to segment 2 liver parenchyma. The patient presented with development of thrombi 6 days postoperatively, which resolved with heparin administration. Right hepatectomy, cancer, and longer duration of Pringle maneuver have been shown to be independent significant risk factors for portal vein thrombosis.^35–37 All other SAEs were determined by study investigators to be unrelated to HB. All predefined conditions of study success were met (Table 5).

Table 5.

Conditions of Study Success.

Parameter	Threshold for success	Study results	Meets threshold
Mean paired κ statistic	>0.61	0.7754	Yes
Mean paired κ statistic for determining eligibility	>0.80	0.9301	Yes
Mean paired κ statistic for determining hemostasis	>0.80	0.9301	Yes
No more than 1 UADE or SADE as determined by the IDMC	<1 UADE or SADE	1 possible SADE	Yes

Abbreviations: IDMC, Independent Data Monitoring Committee; SADE, serious adverse device effect; UADE, unanticipated adverse device effect.

In addition, the amount of HB available in each patient was determined to assure that the quantity of the powder supplied was adequate. The amount of HB available ranged from between 25% and 100% of the bellows, with the majority of cases having approximately 50% of the product available after TBS application (Table 6).

Table 6.

Product Available Following Application to Target Bleeding Site.

HEMOBLAST Bellows usea	Abdominal	Orthopedic
100% of product is available	0/8 (0.0%)	0/16 (0.0%)
75% of product is available	2/8 (25.0%)	2/16 (12.5%)
50% of the product is available	2/8 (25.0%)	10/16 (62.5%)
25% of product is available	2/8 (25.0%)	4/16 (25.0%)
0% of product is available	2/8 (25.0%)	0/16 (0.0%)

^a The amount of powder remaining in the Bellows applicator after application.

Discussion

SPOT GRADE is a visual scale based on quantitative determinations of blood flow and is used to assess bleeding severity and hemostasis. The scale has been shown to be reproducible and reliable.³¹ This clinical study was performed to validate utility of the SG scale in a clinical setting, as investigators were able to simultaneously visualize and independently score bleeding.

The mean paired κ statistic for independent investigator assignment of SG scores was .7754, indicating substantial agreement.³⁴ The mean paired κ for determining eligibility and hemostasis were both .9301, indicative of almost perfect agreement. These results confirm the validity of the SG scale when used clinically.

This study found that there was no correlation between the mass collected on the gauzes and SG score. The gauze weight method has multiple possible inherent flaws: (1) Other fluids in the vicinity of the TBS may be soaked up by the gauze (irrigation fluid, other bleeding sites, lymph, bile, etc); (2) variable amounts of pressure applied and imprecise timing of holding the gauze against the TBS; (3) requirement for maintenance of sterility when preweighing the gauze; (4) TBS assessment gauze may be mixed up with gauzes in the field; and (5) additional time and resources required to execute the measurements that are not practical during routine use of hemostats when determining eligibility for treatment of a TBS.

The SG scale is a visual assessment tool and requires no additional materials or time in the operating room to evaluate bleeding severity. The SG scale has been previously shown to be an effective method of training surgeons to assess intraoperative TBS bleeding severity.^31,33 The results of this study now validate the reproducibility and reliability of the scale when used clinically. Use of this scale may be the new standard for consistently and objectively assessing bleeding severity and hemostasis. This is especially important in randomized and controlled clinical investigations evaluating the performance of hemostatic agents.^32,33 Previous clinical investigations have often relied on subjective assessments for bleeding severity or no assessments at all; therefore, the treatment of the same levels of bleeding between different treatment arms or different investigations cannot be precisely determined and validated. In addition, the assessment of hemostasis—usually the efficacy end point of these clinical investigations—can also be considered subjective.^29,30

This clinical study demonstrates that surgeons who have undergone SG scale training and testing have substantial agreement on the assignment of scale scores and almost perfect agreement in the identification of eligible bleeding severities and hemostasis.

As noted previously, blood loss estimates in surgical procedures can be imprecise.³¹ The ability to more accurately assess blood loss during surgical procedures is valuable as it may help surgeons and anesthesiologists improve understanding of intraoperative hemorrhage, leading to improved patient blood management. Better assessment of TBS bleeding rates may allow for improved management in terms of surgical techniques as well as blood product use. Rapid and precise assessments may also enhance communications between health care professionals, leading to improved intraoperative, postoperative, and emergency care.

The safety and efficacy of HB initially demonstrated in this study have since been confirmed in a previous, large, prospective, randomized, multicenter study evaluating HB against a standard of care hemostat in cardiothoracic, abdominal, and orthopedic surgical procedures utilizing the SG scale.³³ In this current SG scale validation study, the scale was used to assess both eligibility of TBSs and successful hemostasis in both treatment arms. The SG scale may be usable as a standard means of measuring bleeding severity in future clinical trials designed in different specialties to evaluate the efficacy of a wide variety of hemostatic interventions.

The limitations of this study include the fact that it consisted of a single arm with no control agent. Without a comparator in this study, it was difficult to assess the function of HB in comparison to other available agents. In addition, the surgeons were not blinded to the agent being used and thus could have been biased in favor of the efficacy of HB. Neither of these limitations would have influenced the degree of agreement between the 2 investigators on bleeding severity at each TBS, which was the primary end point of the study.

Conclusion

This study supports the clinical validity of the SG scale as a reliable means of assessing bleeding severity. Thus, the SG scale is believed to be a useful tool as a now clinically validated, quantitative scale available to surgeons for assignment of bleeding severity during surgical operations. Given the promising clinical results of HB in both soft tissue and boney bleeding seen in this and other randomized clinical trials, further prospective investigations in other specialties evaluating the use of HB may be warranted.