|
‘the lack of actual behavioural outcomes or incentive compatible measures that represent how individuals would make choices in the broader context of purchasing are quite serious.’ (p. 31)
|
The complexity of smoking uptake decisions and the fact that, for young people, cigarette packs may be proffered by friends rather than purchased appears not to be recognised. Nor does the fact that the outcomes were based on previous research (including tobacco industry market research) and specifically explore the mechanisms through which packaging (and SP) is likely to impact on smoking behaviour. |
Insisting on methodological uniformity |
|
Behavioural task was used: respondents were asked which, if any, packs they would like to be sent upon conclusion of the study. |
Lack of rigour |
|
Studying actual cigarette pack purchasing behaviour or any close incentive compatible proxy among young non-smokers (as included in the Hammond study) is likely to be deemed unethical. |
Seeking methodological perfection |
|
‘The “Male brands” that are meant as controls have completely different brand names, pack dimensions and colours.’ (p. 34)
|
In line with evidence cited by Devinnney that intentions most closely relate to actual purchasing when they are for existing products (p. 12), brands used are real brands. The reality is that male brands are different from female brands and thus the two cannot be both real and identical. |
Lack of rigour |
|
The above is acknowledged in the paper and for this reason the male brands are excluded from some analyses. |
Lack of rigour |
|
‘The research uses 5-point scales … these 5-point scales are then arbitrarily aggregated so that they are dichotomous (i.e., “1” and “0”). Such arbitrary aggregation is completely unacceptable based on the norms and standards of market research.’ (p. 32)
|
This was not a market research study. |
Insisting on methodological uniformity |
|
Dichotomous scales were not arbitrarily selected but used to provide a more intuitive metric. Furthermore, all analyses were repeated using both the five-point and the dichotomous outcome variable with the same pattern of results. |
Lack of rigour |
|
‘Questions relating to “tar delivery” … and “health risks” are: (a) assuming that the individual is competent to understand the meaning of “tar delivery”; and (b) define “health risks” in a manner that is comparable between individuals…’ (p. 32). And later: ‘(a) it must be clear what is being rated – the “object” must be clear to those being asked to do the rating, (b) what makes up the construct is well articulated – that the “attributes” of the object are understood and valid, and (c) all of the raters are comparable – in other words, the raters are knowledgeable and relevant.’ (p. 33)
|
Research shows most smokers and non-smokers have knowledge about the health risks of smoking and understand that tar is a toxin in cigarette smoke [92], [93].Decisions on tar delivery/health risks are made routinely when purchasing cigarettes without additional information being available to the purchaser. Provision of additional information in the study would introduce an artificiality that Devinney would seek to avoid. |
Lack of rigour |
|
Tobacco companies have used similar measures in their own research. |
Lack of rigour |
|
While it is possible that respondents had different conceptualisations of tar and health risk, etc., the differences will be balanced across experimental conditions given participant random allocation. |
Lack of rigour |
|
‘The vast majority of the analyses are based upon pair-wise comparisons… where the scores are completely dependent upon the alternative against which they are being compared.’ (p. 35)
|
Packages were rated individually, one at a time, and so the scores were not dependant on the comparator as suggested. |
Lack of rigour |
|
‘The experimental conditions… do not allow for effective and efficient comparison of the package attributes, as the design is not efficient, orthogonal or balanced. For example, the brands appear different numbers of times … A properly designed study would control for brand effects, dimension effects, colour effects, price and other package and product attributes.’ (pp. 34–35)
|
This fails to acknowledge the purpose of the study: to test the effect, on young women, of pack design (descriptors, colour and imagery) and of removing these elements (as would occur with SP). Had the intention been to study pack size, price, brand family, etc., and to determine which particular combination was more appealing (as might occur in a market research study), then these should have been balanced as suggested, but it was not. |
Insisting on methodological uniformity |
|
Instead, and consistent with the study's objectives, the only differences between conditions were the elements being examined (pack design), while other elements (pack size and shape, brand family) were constant and the brands appeared an equal number of times across the three female experimental conditions. |
Lack of rigour |
|
Various other complaints about the statistical analysis.
e.g.: ‘Econometrically, the study has a number of flaws. Statistical efficiency would require that analysis of the preferences for specific brands be estimated using a pooled regression where the independent variables are conditional on the alternatives examined. In other words, because each participant will see a different mixture of eight packages we would want to know if the comparison set influences the choice. It is good practice to control for the choice set in which the evaluations are being made. Some bias might be mitigated by the ‘one at a time’ approach… but it is also likely that the mixture of items seen in the set of eight will have an influence. Normally good research practice is to control for items appearing in the evaluative set as a means of seeing whether any undue bias occurs.’ (p. 35)
|
Incorrect interpretation of the study design and objectives as outlined above; all participants in each condition saw the same set of eight packages. |
Lack of rigour |
|
While market research may have taken a different analytical approach, the critique fails to acknowledge that the between-group analysis in which participants are randomly allocated to groups is designed to reduce bias and that confounding was controlled for in the analysis. |
Insisting on methodological uniformity |
|
Silent on study strengths
|
Study strengths, including those consistent with the evaluative criteria set (Devinney p. 13–17) were: recent study; question design based on existing research; good sample size; subjects of relevant age (18–19 years); randomisation producing similar groups thus minimising bias; potential confounders controlled for; results statistically significant and consistent; included a behavioural task. |
Seeking methodological perfection |
