Concerns with the methodology, analysis and discussion of the Buzzy^® and transillumination comparison article

In the article by Dr. Lima-Oliveira et al., “A new device to relieve venipuncture pain can affect haematology test results”¹, prolonged application of a Buzzy^®/ice pack unit with the elastic tourniquet included resulted in different laboratory values from those in free-flowing blood collected with transillumination. As the physician inventor and manufacturer of the Buzzy^® device, I appreciate the opportunity to respond to the findings, clarify appropriate use of our product, and illuminate a source of bias not considered in the discussion. As laboratory analysis and procedures are an integral part of the discussion, a statistician familiar with comparative sample methodology has reviewed the work as well.

Buzzy^® (MMJ Labs LLC, Atlanta, GA, Unites States of America) combines high frequency vibration with an optional ice pack; the healthcare version tested in this study came with an elastic black Velcro tourniquet to attach Buzzy^® to the arm. When placed immediately prior to venipuncture Buzzy^® has been shown to decrease pain significantly in adults without compromising the success of venous access². In children, Buzzy^® decreased pain by half compared to that present when a cold spray was used, and increased the likelihood of obtaining blood at the first attempt^3,4. In contrast to the statements made in the article, we have no data to support that the pain relief will “enhance patients’ compliance during venous blood collection”, although Buzzy^® has been used to enhance compliance with burning injections⁵. Our instructions state that Buzzy ^®should be applied “immediately before cleaning and inserting IV”. In contrast to recommending 15 to 60 seconds of direct application to relieve the pain of IM (intramuscular) injections, the package insert notes that “direct or prolonged application of ice could vasoconstrict or alter lab values”. The theoretical concerns of both vasoconstriction and triggering cold agglutinins do not support prolonged application of Buzzy^®, and we are concerned that the article by Lima-Oliveira et al. supports an incorrect usage of the device.

The article indicates that Buzzy^® was correctly placed 5 cm above the puncture site, but does not specify how Buzzy^® was held in place. An article published by the same authors 2 months earlier compares Buzzy^® to the same brand of transilluminator for blood chemistry evaluation, and uses the same sentences to describe methodology, including fasting, blood draw order, sequence, needle gauge, and 2 mL wastage. It is not stated in either paper whether Buzzy^® is applied with the black elastic tourniquet strap; however in the chemistry paper a picture shows the application with the black tourniquet elastic constricting a patient’s arm. It is not clear in either paper whether Buzzy^® was applied first for 1 minute, then blood was drawn from the opposite side using the transilluminator, then subsequently blood was drawn from the Buzzy^® side. Depending on the procedure, the total Buzzy^® application time would be between 90 and 180 seconds. The methods do specify that the transilluminator device was applied without a tourniquet. It is clear that there was no randomisation between whether blood was drawn first from the right or left arm. In all cases blood was drawn from the transilluminator side first, introducing order of draw as a potential confounder.

There were few significant differences between the laboratory results between the two methods, even with a highly powered paired design. The differences found echoed the differences Dr. Lima-Oliveira reported in a previous paper comparing a tourniquet vs the transilluminator device for the collection of blood. According to the 2011 tourniquet/transilluminator paper⁶, when a tourniquet was left in place for 90 seconds significant differences were found in red blood cell counts, haemoglobin concentration, haematocrit, as well as eosinophils and basophils. In the Buzzy/transilluminator paper, differences were found in red blood cell count, haemoglobin concentration and haematocrit. It seems logical that the major differences between Buzzy^® and the transilluminator may be largely due to the black elastic tourniquet, and not to the Buzzy.

The authors state that “there is a tangible risk that some physicians could make inappropriate clinical decisions, e.g. delay or avoid RBC transfusions” due to the differences in laboratory values. They base this potential clinical risk on a 2.5% difference in haemoglobin (141.4 g/L vs 137.9 g/L) or a haematocrit difference of 2.2% (41.5 vs 40.6). The Authors neglect to mention that in their own comparisons between the transilluminator device and application of a tourniquet for 90 seconds, the haemoglobin and haematocrit differences were greater than those in the Buzzy^®/transilluminator comparison (2.6% and 2.9%, respectively) (Table I). Following this logic, use of the transilluminator would be even more likely to contribute to the same dire clinical results when compared to the gold standard of tourniquet use, yet the authors praise the transilluminator in their previous articles and refer to it as the “gold standard” in their Buzzy^® comparison papers.

Table I.

Comparison between values in blood collected using a transilluminator (trans) and Buzzy^® left 90–180 seconds¹, and a transilluminator and tourniquet left 90 seconds⁶, with mean difference between paired results. Items in bold are statistically significantly different.

	Units	Trans	Buzzy®	% diff	Trans	Tourniquet	% diff
RBC	1012/L	4.80±0.55	4.90±0.55	2.0	4.68 (0.45	4.81 (0.46)	2.8
Hb	g/L	137.9±12.7	141.4±13.2	2.5	14.1 (1.4)	14.6 (1.4)	2.6
Hct	%	40.6±4.0	41.5±4.0	2.2	41.7 (4)	42.9 (4)	2.9
MCV	fL	84.4 (81.8–88.3)	84.6 (81.9–88.1)	0.2	88 (5)	89 (5)	1.1
RDW	%	12.7±0.5	12.7±0.6	0
WBC	109/L	7.35±1.94	7.10±1.89	−3.5	6.59 (1.87)	6.9 (2.02)	4.8
Neu	106/L	4.27±1.57	4.15±1.49	−2.9	3.72 (1.31)	3.87 (1.4)	4.2
Lymp	106/L	2.41±0.80	2.32±0.80	−3.9	2.23 (0.73)	2.29 (0.75)	2.6
Mono	106/L	0.29±0.08	0.28±0.05	−3.6	0.33 (0.11)	0.34 (0.13)	3.9
Eos	106/L	0.16±008	0.16±0.07	0	0.30 (0.34)	0.37 (0.36)	24.1
Baso	106/L	0.046±0.02	0.041±0.02	−12.2	0.026 (0.02)	0.021 (0.02)	23.8
Plt	106/L	274±66	272±66	−0.7	200 (46)	208(46)	3.6
MPV	fL	9.12±0.81	9.09±0.71	−0.3

RBC: red blood cell count; Hct: haematocrit; Hb: haemoglobin concentration, MCV: mean corpuscular volume; RDW: red blood cell distribution width; WBC: white blood cell count; Neu: neutrophil count; Lymp: lymphocyte count; Mono: monocyte count; Eos: eosinophil count; Baso: basophil count; Plt: platelet count; MPV: mean platelet volume.

As a clinician, the most important finding comes from comparing the two transilluminator studies. While the prolonged tourniquet application caused platelet, white blood cell and neutrophil counts to increase by 4.8%, 4.2% and 3.6%, respectively (P=NS), in the Buzzy^® study they fell by 2.9%, 3.9%, and 0.8%. The fact that these differences were not statistically significant in a paired evaluation may mean the differences were not clinically relevant, but this finding is worth reporting as many paediatric clinics may use Buzzy^® when determining whether a patient is neutropenic. These results would cause a clinician to err conservatively, but are worth knowing in a worst-case application scenario. Because the Buzzy^®/transilluminator paper does not mention the previous tourniquet/transilluminator paper, this result is not emphasised.

For their analysis, Dr. Lima-Oliveira et al. used as a standard the allowed bias for laboratory quality control, and appear to have extrapolated that these constitute clinically significant differences. In laboratory quality control, multiple analyses of the same sample are run and compared to a reference database for optimal minimum differences, which are not generally achieved in the clinical laboratory setting⁷. According to the authors of the reference database Lima-Oliveira cites, there is a measure appropriate for clinical differences. Specifically, “The numerical value that delineates medically significant changes between two results, classically named “critical difference”⁸ and today called Reference Change Value (RCV), comes from the formula:

$$\text{RCV}=k\times \sqrt{2}\times \sqrt{C{V_A}^2+C{V_I}^2}$$

with k=1.65 for a one tail test and a probability risk α of 95%, and CV_A and CV_I the analytical and the within-subject (or intra-individual) coefficients of variation, respectively⁹. It is unclear why this value was not used or at least reported, as it would provide the clinician with vital information that the “desired bias” values do not.

While both transilluminator papers refer to the device as the “gold standard”, traditional laboratory value textbooks presumably used tourniquet samples rather than this new transilluminator device. As such, the term “gold standard” is possibly premature.

Finally, the discussion of the effect of compression by Buzzy’s elastic tourniquet is absent. Dr. Lima-Oliveira has published five papers discussing the differences between constricted blood draws and free flowing blood draws using the transilluminator device, made in Brazil near his laboratory. For this reason, omission of the discussion of the contribution of prolonged tourniquet application is perplexing, as the papers were published prior to the current Buzzy^®/transilluminator article.

Given the use of Buzzy^® against package insert instructions, lack of full discussion of methods, the knowledge that tourniquet compression causes greater laboratory changes than those found in the Buzzy^® study, and the choice of analytic methods, the conclusion that “the novel Buzzy^® device should be used with caution,” seems excessive and inexplicably biased. The knowledge that few laboratory values were in any way clinically different despite the prolonged application is useful, and we appreciate the time of the author in conducting this study.