Response to M Krkošek, C W Revie, B Finstad and C D Todd's comment on Jackson et al. ‘Impact of Lepeophtheirus salmonis infestations on migrating Atlantic salmon, Salmo salar L., smolts at eight locations in Ireland with an analysis of lice-induced marine mortality’

Jackson et al. (2013a) was published in Volume 36, Number 3 of the Journal of Fish Diseases. In the same issue of the journal there was an article describing a similar long-term study carried out in Norway. Both studies were large scale long-term field studies covering periods from 2001–2009 and 1997–2009 respectively. The Norwegian article, which had six authors (O T Skilbrei, B Finstad, K Urdal, G Bakke, F Kroglund & R Strand) from four research institutes in Norway, not only used the same statistical methodology to analyze its results it also reported similar findings. Skilbrei et al. (2013) reported that their “results are similar to those of Jackson et al. (2011a)” while Jackson et al. (2013a) reported that “the analysis carried out here on a much larger data set” support the conclusions of Jackson et al. (2011a). Jackson et al. (2013a) reported an absolute difference in risk of 1% and an odds ratio of 1.14:1 and Skilbrei et al. (2013) reported an odds ratio of 1.17.

The authors would like to point out that both Jackson et al. (2013a) and Skilbrei et al. (2013) were subject to full scientific peer review prior to publication. The claim by Krkosek et al. that we made methodological errors is not justified.

The primary analysis we presented, a logistic regression model to compare the difference in the (log) odds of a treated fish returning compared to a control, incorporating a random effect to model the degree of heterogeneity between river (e.g. release date environmental conditions) and adjusting for the significant year effect, is the correct approach when modelling a binary response variable arising from such a designed experiment (McCullagh & Nelder 1989; Pinheiro & Bates 2000; Zuur et al. 2009; Skilbrei, Finstad et al., 2013).

Krkosek et al., in their comment, present an analysis using weighted summary data at the river level and not at the individual fish level as in our analysis. Their use of the word ‘paired’ is misleading as the data do not represent a single sample with a pair of measurements per fish. Presumably, the use of the word ‘paired’ refers to within river comparisons. Their criticism that the ‘paired’ structure of the data was not utilized in our analyses is not correct. The Chi squared tests reported in our analysis (Jackson et al. 2013a, Table 1) are within river comparisons of the proportions surviving incorporating the ‘paired’ structure present in the design.

The analyses presented by Krkosek et al., a weighted within river comparison of the difference in the percentage returning between treated and untreated fish, highlight the pitfalls of analyzing data using paired differences (regardless of transformation) and as a meta-analysis in the presence of considerable heterogeneity. Krkosek et al. present results based on a meta-analysis using summary data from Table 1 of our article. Meta-analysis is useful and valid when study summaries, rather than the raw data, are available for analysis and studies are comparable (i.e. homogeneous). The results from a meta-analysis are highly unreliable, however, when the variability between studies (i.e. heterogeneity) is large.

Heterogeneity is evident in a meta-analysis by a high value of the I² statistic and a meaningful difference between the fixed and random effect estimates. In the analysis presented by Krkosek et al. the I² statistic, the percentage of variation due to heterogeneity, was extremely high at 93% (Fig. 1) and the test for heterogeneity was highly significant (P < 0.001). Using a combined estimate base on a meta-analysis with such heterogeneity is highly questionable.

Figure 1.

Meta –analysis used by Krkosek et al. when calculating the 1.39 odds ratio including weights (fixed and random effects) and the I² statistic (93%) not originally included in their letter to the Journal of Fish Diseases. Using a combined estimate base on a meta-analysis with such heterogeneity is highly questionable. The claim that the Odds Ratio is incorrect by a multiple of 30 is therefore based on flawed inference.

Krkosek et al. did not report heterogeneity (nor explore its source), which is a mandatory requirement for reporting meta-analyses as outlined by the PRISMA document (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, http://www.prisma-statement.org/), an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

The study weights used when calculating the 1.39 odds ratio for the random effect analysis (as quoted by Krkosek et al. for their justification of a ‘30 times …’ comment) are given in Fig. 1. The weight each river is given is driven by the heterogeneity as all rivers are weighted similarly and essentially by ‘noise’ and no longer weighted purely by sample size. The claim made by Krkosek et al. that the Odds Ratio is incorrect by a multiple of 30 is based on flawed inference.

One explanation for the source of heterogeneity is due to the significant change in percentage returning over time as reported in our article. Their statement about not adjusting for inter-annual fluctuations is indirectly stating that one should not look for sources of heterogeneity.

How best to report the results of a logistic regression, i.e. as an odds ratio estimating the multiplicative effect on the odds of a treated fish returning compared to a control or as a difference in percentage returns on an absolute scale is a valid question.

In our article, we reported both relative and absolute risk differences which is consistent with the CONSORT statement (Schulz, Altman & Moher 2010), which encompasses various initiatives to alleviate the problems arising from inadequate reporting of randomized controlled trials (Consolidated Standards Of Reporting Trials, http://www.consort-statement.org/).

In reporting the impact of lice infestation as a source of mortality in Atlantic salmon post smolts, it is important to recognize that over time the level of lice-induced mortality has remained relatively constant (Jackson et al. 2013a; Skilbrei et al. 2013) at approximately 1% or 10 fish in a thousand whereas the other mortality factors have increased substantially leading to a drop in the survival of Atlantic salmon over the study period from the region of 20% to <5% (Peyronnet et al. 2007; Jackson et al. 2011a,b; Jackson et al. 2013b).