Book Review

In “Statistical Rules of Thumb,” Gerald van Belle covers a wide range of topics, everything, it seems, except a definition of the phrase “rule of thumb.” Wikipedia offers a useful definition (Wikipedia, 2009). “A rule of thumb is a principle with broad application that is not intended to be strictly accurate or reliable for every situation. It is an easily learned and easily applied procedure for approximately calculating or recalling some value, or for making some determination.” The key words in the definition that you need to keep in mind are “not intended to be strictly accurate or reliable for every situation.” No one ever got thrown in jail for ignoring a rule of thumb. So although this reviewer is sorely tempted to say “yes but” to about half of the rules in this book, he will resist the temptation. Dr. van Belle, himself, often acknowledges these “yes buts” in his book. This requires a careful balance. He avoids a rigid listing of statistical dogmatisms but doesn't timidly offer all possible perspectives as if they carried equal weight.

Dr. van Belle wrote these rules of thumb partly from his own experience but more from discussions with other statisticians and a review of the classic references in statistics. The bibliography is outstanding. The three quartiles of publication years in the bibliography are 1984, 1997, and 2001, but they reach as far back as 1925. Rules of thumb represent established practice developed over decades of experience, so the emphasis on older publications should be expected.

I had seen this book when it was just a single chapter on the Web prior to publication of the first edition. Some of the strongest material of the book is the rules of thumb involving sample size determination. I have used many of the sample size rules in this chapter, and I credit van Belle's work in this area with providing me the ability to rapidly recognize when a client is proposing a grossly inadequate sample size.

Several sample size rules in Chapter 2 are worth quoting. In a two-group comparison, 16 divided by the square of the effect size produces a reasonable sample size per group. Thus, a good power for a half standard deviation shift requires about 16/0.5² = 64 patients in each group. A rule for calculating power for a relative change in means given only a coefficient of variation (relative standard deviation) is also very useful. Finally, if your client is expecting to prove that an outcome is rare, it helps to set sample size targets based on the rule of 3 (if you observe 0 events out of n trials, 3/n is an approximate 95% upper confidence bound). To establish that an outcome occurs less than 1% of the time, sample at least 300 patients and hope that none of them experience the event.

Curiously, the best rule on the Website, the rule of 50, did not make it into the first or second edition of the book. If you are comparing a rare binary outcome in two groups, and you expect to see a 50% reduction in the probability of that event in the treatment group, you should design the study so that you would expect to see 50 events in the control group. You can derive this result trivially by using the classic binomial sample size formulas with power of 80%, a two-sided alpha level of 5%, and a simplifying approximation that sets the complementary probability (the probability that the event does not occur) equal to 1 in both groups. Thus, a control rate of 4% (treated rate of 2%) would require 50/.04 = 1250 control subjects and an equal number of treated subjects to achieve adequate power.

In the other chapters, the rules of thumb are more likely to be qualitative rather than quantitative. In the chapter on covariation, for example, Dr. van Belle admonishes you not to summarize a regression model with a correlation coefficient, and to be wary about correlating two rates or ratios that share the same denominator. Chapter 9 (Words, Tables, and Graphs), starts with the advice proposed originally by Edward Tufte, that you should use text for a few numbers, tables for many numbers, and graphs for complex relationships. It is inevitable that there will be a lot of overlap between chapters. The chapters on environmental studies and epidemiology, for example, address many of the same issues. Dr. van Belle is very good at cross-referencing throughout the book.

The second edition is much like the first, which received a positive review in Hayden (2004). If you own the first edition and use it regularly, the second edition may not have quite enough new material to justify its purchase. Most of the chapters from the first book have been reorganized, and many chapters have added a new rule or two. The total number of rules of thumb is 122 in the second edition compared to 99 in the first edition. You should consider the second edition if two new chapters on observational studies (7 rules) and evidence-based medicine (9 rules) appeal to you. I, for one, am thrilled that Dr. van Belle recognizes the major role that statisticians can contribute to the proper application of evidence-based medicine.

Experienced statisticians will see a few surprises among these rules of thumb, but most of the time, you'll just nod your head in agreement. The real value of the book to an experienced statistician is the careful way that Dr. van Belle supports all of these rules. A new statistician would find great benefit in this book because it offers pragmatic advice obtained through the collective experience of many statisticians. Learn these rules, and you will come across as wise beyond your years. Sections on The Basics (Chapter 1); Design, Conduct, and Analysis (Chapter 8); and especially Consulting (Chapter 10) are of particular interest to beginners.

I like this book a lot, and much of the appeal comes from the audacity of the premise. Can the practice of statistics be captured in a few pithy statements that are simple and memorable? Of course not, is the first reaction of those of us proud of the depth and intricacies of statistical practice. But when you read through this book, you see that a lot of what we do has been summarized well. Perhaps the best endorsement of this book that I can offer is that I use these rules of thumb all the time in my consulting practice.