Statistical reporting of clinical pharmacology research

Abstract

Research in clinical pharmacology covers a wide range of experiments, trials and investigations: clinical trials, systematic reviews and meta‐analyses of drug usage after market approval, the investigation of pharmacokinetic–pharmacodynamic relationships, the search for mechanisms of action or for potential signals for efficacy and safety using biomarkers. Often these investigations are exploratory in nature, which has implications for the way the data should be analysed and presented. Here we summarize some of the statistical issues that are of particular importance in clinical pharmacology research.

Descriptive vs. confirmatory investigations

The development of drugs is generally based on sequential phases, aiming first to learn and subsequently to confirm the mechanisms of efficacy and safety 1. Although early clinical pharmacology investigations often benefit from extensive preclinical results, there is still much to be learned during the first human trials when the first clinical data are obtained, described and interpreted. Learning can often be achieved by quantifying how measurements that we make are changed by the treatments we give. Such changes are best described using standard descriptive statistics of the data, such as the mean and standard deviation (as well as median, minimum, quartiles and maximum values).

Inferential statistical analyses are performed to draw conclusions about general (patient) populations. Those analyses might involve statistical models (e.g. analysis of covariance), and the results would be reported using estimates of population parameters (such as means or proportions) and their standard errors or confidence intervals, which quantify the variability and uncertainty of the reported estimates.

Confidence intervals vs. P‐values

When comparing treatment outcomes, confidence intervals for parameters of interest (such as the difference between treatment means) provide insight into the potential clinical relevance of a treatment effect in a clinical trial or observational study. Confidence intervals are on the same scale with the same units as the mean, which makes them a preferred choice for reporting results.

In contrast, P‐values can be used to demonstrate whether or not the difference between two treatments is statistically significant, but statements of statistical significance alone do not necessarily indicate the clinical relevance of an effect and hence might not adequately address the objectives of a clinical pharmacology trial. P‐values do not measure the size of an effect or the importance of a result, and do not provide a good measure of evidence regarding a hypothesis 2. These limitations hold in particular for comparisons that had not been adequately powered.

Consequently, P‐values that indicate the rejection of hypotheses should be reported only sparsely. Instead, use of confidence intervals is recommended for reporting trial results.

Type‐I‐error and absence of significance

We invite authors to use significance thresholds other than 0.05 (or equivalently 95% confidence intervals) when interpreting their results. In particular, in early stages of development, levels larger than the standard 0.05 might be acceptable, as the consequences for terminating a potentially useful intervention might be more harmful than the risk of not confirming efficacy in subsequent clinical trials. Each case might be different; hence authors should provide the rationale for choosing different thresholds.

Importantly, it is incorrect to interpret the absence of a statistically significant difference as evidence of the absence of any real effect. In particular, lack of statistical significance and imprecision in the effect estimates for adverse drug reactions should not be construed as demonstrating the intervention to be safe and well‐tolerated. Furthermore, if a study's objective is the demonstration of equivalence or noninferiority, authors should explicitly state this objective and detail how the margins for claiming equivalence were obtained. While regulatory guidance exists for pharmacokinetic bioequivalence studies 3, such margins would need to be developed and justified for most other endpoints.

The impact of multiple testing (e.g. several biomarkers simultaneously) and the need for adjustment of significance levels depends on the context. In exploratory trials the provision of unadjusted confidence intervals (and of unadjusted P‐values) is generally acceptable. In confirmatory investigations, more stringent approaches might be required, such as adjusting the α‐level for tests.

Statistical analysis model

Any statistical analysis of data will be based on assumptions, either implicitly or explicitly. Hence these assumptions should be reasonably justified a priori, as data driven decisions about which analyses to use might introduce bias to the results. Authors should review their analysis for the fulfilment of the assumptions to support the interpretation of the results. Deviations from prespecified analyses – which may be justified in exploratory pharmacology trials – should be indicated in the manuscript, as should all posthoc analyses. Whether these considerations would lead to a change of the statistical model is a case‐by‐case decision.

A common situation is that many pharmacokinetic measurements are log‐normally distributed, at least approximately, so that the logarithmic transformation is recommended 4, 5. It is generally not required to perform statistical tests of normality since results of such tests are not always reliable (particularly with small sample sizes). Even if deviation from normality (or log‐normality) might be statistically significant, this deviation might not be relevant for validity of the results 6, 7.

Adjusting for potential covariates is necessary when aiming to quantify treatment effects, particularly in nonrandomized investigations. Such an analysis would often be performed using an analysis of covariance (ANCOVA) or mixed effects model. For instance, if age or sex might be expected to impact the drug effect, then the main analysis should adjust for the potential differences of these demographic covariates, and the methods section should explain this adjustment.

It is generally not expected to drop preplanned covariates if they are not statistically significant, as nonbalanced covariates could cause bias. In exploratory trials, it would be acceptable to present posthoc adjustments, if this posthoc decision can be justified. In epidemiological and pharmacometric research, additional model building and model checking techniques are available for selection of covariates 8.

Determination of sample size

Some clinical pharmacology trials require a formal power calculation, if the objective of the trial is to investigate a specific hypothesis and demonstrate a statistically significant difference between treatments regarding specified study endpoints. In this case, full details of the assumptions (e.g. anticipated magnitude of effect and its variability) should be presented.

However, the majority of clinical pharmacology trials are exploratory, aiming to quantify mechanisms, rather than trying to prove them. Documentation of sample size considerations is still beneficial, for example to achieve a specific precision of an estimate. Exploratory investigations, too, are most useful if they are planned carefully, to increase the awareness for the potential of obtaining exaggerated results 9.

In exploratory trials, the sample size should consider how to present useful data for the key (primary) endpoints (e.g. standard errors of a particular size). A formal power calculation may not be necessary, but some motivation for discussion on the appropriateness of the sample size is recommended.

Illustrations

Figures can illustrate key features of a study and its results, and often provide (together with the abstract) the primary indication to readers whether the paper is relevant to them. A good graphical presentation of the data can require some time to produce, but this will often be time well spent.

The appropriate selection of ranges for the axes will help to present the important aspects of the data. Excellent examples are plots with inset figures for pharmacokinetic profiles. An inset could present the longer time frame with concentration on logarithmic scale to inform about the exponential disposition and the terminal half‐life, while the maximum concentration and its timing is presented on the original scale with a shorter time frame 10.

We also recommend including a graphical description of the trial design as this is often useful to understand the design more clearly.

Presenting individual data vs. summary measures

Clinical pharmacology trials are often performed in small samples and so that one has the option to present the individual subject data together with descriptive statistics, rather than descriptive statistics alone. The individual data can enable the readers to judge for themselves how to interpret the study results.

Similarly, many trials obtain not only a single endpoint but time profiles of key endpoints (repeated observations). The presentation of the means and confidence intervals or standard deviations of such time courses will give an indication on differences between short‐term and long‐term effects. However, summary characteristics (such as areas under the curve or weighted averages) can also be useful to summarize the information of a time profile. In some cases, it might be useful to present both population profiles as well as individual summary characteristics to offer different views on the data (an example is shown in 10).

Systematic reviews

Although the comments above primarily relate to reports on individual studies, they are applicable for systematic reviews as well. For example, prespecification of the analysis methods is also important for systematic reviews. The registration of the analysis protocol (e.g. at the PROSPERO register) will raise the credibility of the review as it lowers the potential for data‐driven bias. However, even in the absence of registration, authors should explain which posthoc choices regarding the statistical analysis strategy they made, for example with regard to the selection of subgroups, the combination of outcome measures or the statistical analysis model.

Furthermore, it is sometimes more important or meaningful to visualize the spread or variability in the effect measures across the trials, rather than to focus solely on the pooled effect.

Practical guidance

In summary, we recommend the following strategy for reporting study outcomes.

• For both confirmatory and exploratory research, the primary objective of the investigation should be stated explicitly, together with the key (primary) endpoint(s) to evaluate the trial objectives.

• The statistical methods section should clarify which analyses where exploratory, and how variability and uncertainty were addressed.

• For the key (primary) endpoints, detailed information on the derivation of the data as well as the statistical model for their analysis should be presented in detail in the Statistical methods section. For these endpoints, both individual data (where practical) should be presented as well as comparisons between treatments as means and their confidence intervals, either in Figure or Table format in the results section. P‐values might be added for key endpoints, but they would only be necessary if a specific hypothesis is stated in the trial protocol. Usually, confidence intervals will be more helpful.

• Further (secondary) endpoints that support the interpretation of the key endpoints or the study objectives should be presented in table format, with means and confidence intervals, either as tables or figures. P‐values associated with secondary endpoints are generally less useful.

• Additional endpoints should be reported as mean (standard deviation) – or other appropriate measures of location and variability such as medians and quartiles.

• We encourage the presentation of important but ancillary information as supporting information on the journal's website. This information could include detailed results for secondary or additional endpoints, or results of additional analyses (e.g. subgroup analyses).

The editorial board has adapted the Instructions to Authors to reflect these principles for clinical pharmacological research. Further discussions on statistical issues in pharmacology can be found in 11 and other articles of the Virtual Issue on Best Practice in Statistical Reporting. To view the other articles in this section visit http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1476‐5381/homepage/statistical_reporting.htm. Additional recommendations on statistical reporting in clinical pharmacology can be found in 5.

Competing Interests

There are no competing interests to declare. Arne Ring is statistical editor, and Simon Day is a former statistical editor of the BJCP.

Ring, A. , Schall, R. , Loke, Y. K. , and Day, S. (2017) Statistical reporting of clinical pharmacology research. Br J Clin Pharmacol, 83: 1159–1162. doi: 10.1111/bcp.13254.