Jesse Bullowa, specific treatment for pneumonia, and the development of the controlled clinical trial

Introduction

Jesse Bullowa's 1928 report on the use of antipneumococcic serum in lobar pneumonia¹ was subtitled ‘Data necessary for a comparison between cases treated with serum and cases not so treated, and the importance of a significant control series of cases’. It stands out as one of the most sophisticated invocations of the controlled clinical trial in the 1920s, and as a self‐conscious attempt to justify and promote the methodology. Indeed, Bullowa's contribution to a symposium on the treatment of pneumonia at the New York Academy of Medicine in December 1927 was entitled simply ‘The control’.² In focusing explicit attention on methodological issues, Bullowa made clear that he wished to determine ‘whether there was conclusive proof that the serum is of value’.¹

Bullowa, nearly 50 years old at the time of his presentation at the New York Academy of Medicine, was a clinical professor at the New York University College of Medicine³ and co‐director, with Milton Rosenblüth, of Harlem Hospital's pneumonia service, then ‘the largest therapy unit of any hospital in the city’.⁴ During the previous decade, Rufus Cole and his colleagues at the Hospital of the Rockefeller Institute had introduced serotype‐specific antiserum for the treatment of pneumococcal pneumonia (as of 1928, only four such pneumococcal serotypes were known). When the Metropolitan Life Insurance Company lost 24 million dollars in excess death benefits during the influenza epidemic of 1918, it established an Influenza Commission, which would soon focus its attention upon pneumonia and its treatment with antipneumococcal antiserum.⁵ Intending to fund alternate control studies at multiple institutions,⁶ the company would ultimately fund published studies arising from Bellevue and Harlem Hospitals in New York, and from Boston City Hospital.^7–9 However, none of the pneumonia investigators – who included such eventual luminaries as Russell Cecil (at Bellevue Hospital in New York) and Maxwell Finland (at Boston City Hospital) – would so explicitly advocate the methodology of the controlled clinical trial as would Jesse Bullowa.

The challenge of variable prognosis

Bullowa had not used or advocated the use of a controlled series in his previous work on influenza and the role of the tonsils in systemic illness.^10,11 The novel specific treatment for pneumonia appears to have provided a turning point for Bullowa to focus upon the controlled clinical trial as a means of examining and justifying the utility of potential remedies. He started out by grounding his controlled study in the particularities of studying pneumonia. A theoretical rationale for such a study was based on the very variability of pneumococcal pneumonia's severity:

‘The evaluation of the effect of any therapeutic procedure in pneumonia is attended with certain inherent difficulties. Probably seven of every ten patients recover regardless of treatment, and therefore if one chances on a succession of favorable cases one is apt to attribute the benefit to the special treatment then in use. … [Conversely,] a short series of fatalities, unless carefully controlled and analyzed, may lead to a condemnation of what is really a very useful procedure.’¹

More practically, the study was grounded in the need, if antiserum's efficacy was confirmed, to convince clinicians – who had generally ‘not as yet adopted the serum treatment for pneumonia’ – of serum's utility.¹ To this end, an entirely separate pneumonia service had been established atHarlem Hospital in New York, with its own resident physician, four bacteriologists and two chemists who could all give ‘their full time to the investigations’ conducted upon the large numbers of patients admitted with the dreaded disease.⁸

Comparing like with like

Such a service was indeed impressive, and 365 cases would be investigated by December 1927, with 793 investigated by June 1928.⁸ Yet Bullowa's true innovation was embedded in the conduct and explanation of the investigation itself. The methodological first step of the controlled study was to ensure that ‘there must be no selection of patients’. Bullowa relied upon a strict ‘alternation’ of patients to serum treatment versus standard treatment groups, in which ‘only the order of arrival in the ward determines whether a patient is to receive serum’.¹

Bullowa, moreover, took special care to ensure and emphasize the fidelity of the implementation of such alternation. When planning the study, Bullowa established explicit criteria for the ‘rejection of patients’,¹ what would be considered ‘exclusion criteria’, in modern parlance. Only pneumococcal pneumonia patients were to be included; and if a ‘tuberculous pneumonia’ case was accidentally included, the case would be rejected, and the ‘patient's place [was] not filled, but the alternation continued as before in order to avoid selection’.¹ Yet for the pneumococcal pneumonia patients, reported Bullowa, even ‘if a patient dies of a surgical accident or complication, even though he has recovered from the pulmonary involvement, it is charged against the series’,¹ in anticipation of what today would be considered an ‘intention to treat analysis’. At the conclusion of the trial, Bullowa was able to point out that the integrity of such measures was confirmed by the equivalence in pneumococcal serotypes and in patients arriving early versus late in the serum‐treated versus serum‐untreated groups. Yet, for all of his stated commitment to an intention‐to‐treat analysis, Bullowa excluded four ‘deaths within 24 hours of admission’ (three Type I serum‐treated cases, and only one control) from his final analysis of the 1927 data.¹

Nevertheless, Bullowa's chief innovation was in the care he took in comparing the outcomes in the comparison groups. As a foundation, he constructed an elaborate ‘severity rating’ for patients, a 100-point scale ‘for subsequent comparison of cases of like degree’, based upon pulmonary, neurological, circulatory and gastrointestinal status, as well as upon such ‘complications and special factors’ as pregnancy and bacteremia.¹ Through such a rating system, Bullowa could sub‐stratify patients not only by pneumococcal serotype, but into three proposed classes of ‘poor’, ‘fair’ and ‘good’ baseline characteristics.

Taking account of the play of chance

Finally, Bullowa could turn to the statistical comparison of outcomes between the serum‐treated and control groups. Tellingly, he felt the need to begin this section with ‘a brief digression’ regarding the tests he was to apply ‘to see whether proof is adequate and how we can determine the number of cases necessary for a fair evaluation’, dependent upon ‘what difference in results is statistically significant’.¹ Reflecting Raymond Pearl's own explication of the subject in his Introduction to Biometry and Statistics,¹² Bullowa went on to concisely illustrate such contemporary biometric notions concerning how each group's outcomes could be visualized as a bell‐shaped curve, the width of which would be representative of the ‘standard error’ of the distribution of outcomes for that group. The more samples obtained for each group, the narrower the corresponding curve; and the less the curves for discrete groups overlapped, the more likely the differences between them were ‘significant’. Numerically, this could be represented as a ratio of outcome differences between the two groups divided by the square root of the sum of the squares of the two groups' standard errors (in which the larger the sample size for a particular group, the smaller its standard error). If such a ratio exceeded two, then the differences were to be considered “statistically significant”.¹ [Bullowa's calculation was a bit vague in the 1928 paper, and was presented simply as the ‘ratio of the difference to its standard error’.¹ He would define the calculation more precisely in his 1937 textbook on pneumonia,¹³ in which he would equate a ratio of 2.0 with what in modern parlance would be considered a ‘p value’ of <0.045.]

As Bullowa emphasized, such methodology held several implications. Logistically, as was the case by October of 1927 for Type I serum (where the ratio was 1.9), such outcome differences could be rendered ‘significant’ not only through obtaining more beneficial serum results, but simply through obtaining the same results over a larger group of patients, as would indeed be the case by June 1928.^1,8 Hence, as Bullowa concluded, and in anticipation of later considerations of statistical power: ‘The size of the series requisite is determined by a consideration of the standard error’¹ – or as he stated in his address at the New York Academy of Medicine, but did not record in his journal article: ‘Naturally, as physicians, we considered how long we were justified in continuing to deprive patients in the control series of what may be a valuable and available therapeutic aid’.²

Ultimately, and critically for Bullowa, such methodological considerations were to be tied to the moral conduct of therapeutics itself. As he declared: ‘When for type I pneumonia the ratio of the difference in percentage fatality between the serum‐treated and non‐serum‐treated cases, to its standard error, becomes more than 2 or 3, it will be our duty to administer serum in all type I cases and to urge its administration on others’.¹ And by the time Bullowa next addressed the New York Academy of Medicine on antipneumococcal antiserum, in November of 1928, he could report: ‘In fact, we have ceased, at Harlem Hospital, to alternate the use of serum in cases in which we find an invasion of the blood stream with Type I organisms, for it is felt that, for these cases, statistical proof has been given, and it is unjust to withhold an available life‐saving procedure; the ratio of the difference to the standard error, in this type, is 2.4 to 1’.¹⁴

Context and irony

William H Park, director of New York City's Bureau of Laboratories, served as a member of the Metropolitan Life Influenza Commission; and his own biographer states not only that he and Bullowa had first investigated the use of serum together at Harlem Hospital in the spring of 1926, but that ‘the entire project in New York City was under the general guidance and supervision of Park’.¹⁵ Nevertheless, despite Park's own prior interest in the use of alternate controls in adjudicating the serum treatment of both diphtheria and scarlet fever,^16–18 it remains unclear the degree to which Park ultimately influenced Bullowa in this respect. Park seemed somewhat uneasy regarding the ethics of such controls – i.e. withholding serum treatment from the control groups – and looked for the earliest opportunity to render such controls unnecessary.^16,17 Bullowa, however, seems to have been the one to tie such trial endpoints themselves to statistical notions of significance.

In this latter respect, Bullowa would also note that ‘Dr Louis I Dublin and his staff of the Statistical Bureau, Metropolitan Life Insurance Company, tabulated the data’ generated by the investigation,² and later he would add ‘and guided us in their interpretation’.¹⁴ And unfortunately, while an ethos favoring controlled studies seems to have pervaded the Metropolitan Life‐inspired studies generally, the extent to which Dublin and his staff influenced Bullowa and/or his paper remains unclear. ^a Each of the Metropolitan Life‐inspired studies of antipneumococcal antiserum embodied such an ethos of ‘control’. Russell Cecil and Wheelan Sutliff, for example, would compare characteristics of their serum‐treated and control groups, generated by alternation, to ensure that like was being compared with like.⁷ Again, however, Bullowa was exceptional in tying not only the methodology, but also the moral outcome of the studies, to contemporary notions of statistical significance. And among the lineage of alternate control studies in pneumonia, neither the subsequent study of digitalis at Bellevue Hospital (built upon the antiserum ‘machinery that could readily be adjusted to include an investigation of the effects of the use or nonuse of digitalis as routine therapy in this disease as well’),¹⁹ nor the Medical Research Council study of antiserum in Great Britain²⁰ would rely upon such mathematical ‘proof’ of significance. Paradigmatically, by 1937, the same year Austin Bradford Hill would publish the first edition of his Principles of Medical Statistics,²¹ Bullowa would begin the ‘Serum Therapy’ chapter of his Management of the Pneumonias with a 13-pageextension of his statistical rationale.¹³

By the late 1930s, the advent of the sulfa drugs (considered ‘chemotherapy’ at the time) threatened to displace the use of the more expensive and labor‐intensive antipneumococcal serotherapy.⁵ Sulfapyridine, the first truly anti‐pneumococcal sulfa drug, would not be available to investigators until the 1938–1939 winter pneumonia season, though this did not stop clinicians from attempting to treat patients with the first of the sulfa drugs, sulfanilamide. Tellingly, Bullowa wrote of sulfanilamide in his book: ‘If it shall be shown in an alternated series of patients of the same type and age that bacteremic incidence is reduced or that bacteremic patients are saved the value of the drug for human pneumonias will be demonstrated. This evidence has not been collected as yet.’¹³

And with the introduction of sulfapyridine, a cadre of researchers, including Bullowa, sought to evaluate the possibility of combination serochemotherapy, intending to compare this with monotherapy with sulfapyridine alone. Given that Bullowa had worked so long and hard to equate the physician's moral ‘duty’ to prescribe antiserum with its justification through the controlled clinical trial, it is ironic that, in defence of antiserum, he more than others in his generation drew attention to the limits of the applicability of the results of controlled trials.

By early 1939, Bullowa had ‘rotated in treatment’ 324 adults among serotherapy, sulfa monotherapy and combination therapy, and found that, on average, sulfa monotherapy was indeed superior to combination therapy. Yet in re‐interpreting his data after post‐hoc sub‐stratification, he claimed that cases treated within the first four days of illness seemed better after combination therapy, this difference nearly achieving statistical significance.²² Thus, despite his admission that ‘final conclusions regarding the effect of drug and antibody in the treatment of the pneumonias must await additional observations’, Bullowa felt free to posit, based upon immunological expectations, ‘that serum plus sulfapyridine is a more effective therapeutic agent than either acting alone in the early cases when autogenous antibody cannot be expected to be present’.²²

By mid-1941, however, Norman Plummer's group at Bellevue Hospital²³ – following Russell Cecil, and alternating 607 patients – had found not only that the sulfa monotherapy group fared better than the combination therapy group on average, but that such findings held true regardless of whether the patients were treated early or late in the disease. Indeed, Plummer continued his post‐hoc analysis by showing that stratification for age of the patient and bacteremia failed to identify any subgroup favoring combination therapy, thus seemingly putting an end to the ‘combination therapy’ question.

But Bullowa refused to give up on serum, arguing for ever‐finer sub‐stratification. Taken to its logical extreme – that ‘under certain conditions, one remedy may act more favorably than the other, so that there may be an appropriate remedy for each patient, as well as a better remedy for the majority’²⁴ – Bullowa's defense of serum threatened the very foundation of the controlled study generally, based upon the limitations of its applicability in individual patients. Yet more narrowly, it appears he simply looked forward to a more comprehensive machinery capable of more extensive and appropriate stratification, in which ‘all the factors which are known to influence prognosis, such as age, duration of illness, bacteremia, detection of capsular polysaccharide and concurrent disease, must be taken into consideration’.^25–27 Either way, by this time, Bullowa had outrun the machinery capable of rendering such an investigation. Indeed, he died two months after publishing these lines, and antipneumococcal antiserum would itself pass away from the therapeutic armamentarium within the two succeeding years. But Bullowa had left a profound legacy, in demonstrating to his contemporaries the potential utility – as well as the limitations – of the controlled clinical trial as a tool for adjudicating therapeutic efficacy in the emerging era of modern specific treatments.

Footnotes

DECLARATIONS —

Competing interests None declared

Funding None

Ethical approval Not applicable

Guarantor SHP

Contributorship SHP is the sole contributor