Quality of information accompanying on-line marketing of home diagnostic tests

Abstract

Objective To assess the quality of information provided to consumers by websites marketing medical home diagnostic tests.

Design A cross-sectional analysis of a database developed from searching targeted websites.

Setting Data sources were websites written in English which marketed medical home diagnostic tests.

Main outcome measures A meta-search engine was used to identify the first 20 citations for each type of home diagnostic medical test. Relevant websites limited to those written in English were reviewed independently and in triplicate, with disputes resolved by two further reviewers. Information on the quality of these sites was extracted using a pre-piloted performer.

Results 168 websites were suitable for inclusion in the review. The quality of these sites showed marked variation. Only 24 of 168 (14.2%) complied with at least three-quarters of the quality items and just over half (95 of 168, 56.5%) reported official approval or certification of the test. Information on accuracy of the test marketed was reported by 87 of 168 (51.7%) websites, with 15 of 168 (8.9%) providing a scientific reference. Instructions for use of the product were found in 97 of 168 (57.9%). However, the course of action to be taken after obtaining the test result was stated in only 63 of 168 (37.5%) for a positive result and 43 of 168 (25.5%) for a negative result.

Conclusions The quality of information posted on commercial websites marketing home tests online is unsatisfactory and potentially misleading for consumers.

INTRODUCTION

Home diagnostic medical tests are part of a rapidly growing market of health-related products available for purchase on the internet.^1,2 These tests allow consumers a combination of ease and privacy of use, avoiding the potential inconvenience and embarrassment associated with visiting a doctor or pharmacist. Despite their increasing popularity, there are concerns that inadequate, misleading or confusing advertisement can potentially result in inappropriate use of theses tests and lead to false reassurance or unnecessary anxiety, which may have serious health consequences. There is a lack of formal licensing for such products which can potentially lead to poor consumer protection.^1-7 Previous studies have shown that health-related websites are often of variable or poor quality,^8-11 but no study has yet assessed the quality of websites marketing home diagnostic medical tests. We examined the quality of information provided by such websites using a prospective protocol.

METHODS

Identification of websites

An initial scoping search was performed to develop a strategy for identification of home diagnostic tests available for purchase online. Each of the products identified was then searched for individually. Through an iterative process we developed a sensitive search strategy to develop a comprehensive database for analysis. The meta-search engine Copernic (www.copernic.com) was used for conducting the search, as it has the advantage of using a range of search engines simultaneously and excluding duplicate sites automatically. From the individual engines searched through Copernic, the first twenty citations for each of the tests were included in the study.¹¹ Websites were excluded if they were not marketing home diagnostic medical tests or if the option of translation to English was not available. Where different methods of a particular test were marketed in a website, only the first one was included for review. If different tests were marketed for the same purpose (for example, mid-stream urine, cassette or urine strip for pregnancy testing) and the instruction for use varied for each, the entire website was reviewed. The tests were divided into two categories. In the first, samples were taken at home but analysed in a laboratory; in the second, samples were taken at home and the tests provided an instant result. All sites were reviewed in triplicate by TK, TT and AKD, with discrepancies resolved by a fourth reviewer—either TS or KKS.

Assessment of information quality

There were no specific guidelines for the online marketing of home diagnostic tests at the time of our study. However, the International Federation of Pharmaceutical Manufactures Associations (IFPMA) has an ethical code for products marketed online.³ We used this IFPMA code, the advice on medical web publishing produced by the British Healthcare Internet Association (BHIA),⁸ and other published quality assessment tools for general health information on the web^12-18 to generate an 11-point quality checklist for evaluating online diagnostic product information.

We believe that when advertising and selling home diagnostic tests online, it is important to provide documentation of official approval of the test. The UK Medicine and Healthcare Products Regulatory Agency (MHRA) has recently taken steps to bring the marketing of home diagnostic tests under the purview of new Medical Device Regulations.¹ Accordingly, all commercially available diagnostic tools are now expected to be CE certified (CertifiGroup Engineer: the European Union's quality benchmark).¹⁹ Similar control measures exist in the USA through approval by the Food and Drug Administration (FDA).⁵ The Department of Health of the US government also regulates laboratories by certifying Clinical Laboratory Improvement Amendments (CLIA).²⁰

In addition to official approval, the purpose of the test, the target population, instructions on safe use and the provision of advice following a positive or negative result are all important to the use of kits at home. We did not appraise any research data for its validity but sought for a scientific reference or FDA/CE approval as a demonstration that the product had been researched prior to marketing. This information should be supported by a date of publication and appropriate references or links. A mechanism of two-way communication or contact between the sellers and customers was considered desirable, as this may substitute the need for communicating with a pharmacist or other health care provider. Finally, it was considered important that websites took responsibility for their marketed product. In this regard, a statement explicitly indicating responsibility or lack of it was considered acceptable, while not mentioning anything was unacceptable.

Data extraction and analysis

The websites' compliance with quality items was calculated for all tests together as well as separately. In addition, the overall quality of each website was given a score out of 11, with one point for each of the 11 quality items reported. Compliance with eight or more items (i.e. more than 75%) meant that the website was considered to be of good quality. We compared tests where samples were taken at home and analysed in the laboratory with those where the tests provided instant result at home. Our rationale was that information provision may differ between these two groups, as we expected the quality in relation to instructions on safe use of testing kits at home and advice following a positive or negative result to be better when results were provided by a laboratory.

RESULTS

From a total of 1011 websites identified by the search, 168 websites marketing home diagnostic tests were downloaded for assessment (Figure 1). There were six categories of home tests: allergy tests, hepatitis C screening, Human Immunodeficiency Virus (HIV) serology, thyroid tests (TSH), prostate cancer screening and the urine test for Chlamydia. Of these, 47 websites (27.6%) provided tests which were designed to be sampled at home but analysed in the company's laboratory.

Figure 1.

Website Selection

There was a wide variation in the quality of websites (Figure 2). Overall, the quality of information was poor, with 94 of the 168 websites (55.9%) complying with less than half of the quality items; only 24 websites (14.28%) complied with eight or more quality items. Only one website demonstrated compliance with all our quality items.²¹ Information on the purpose of the test and details of how to contact a helpline were almost always available. Only 95 websites (56.5%) mentioned that the tests being marketed were approved or certified. Information on the accuracy of the test marketed was reported on 87 websites (51.7%), with 15 (8.9%) providing a scientific reference. Instructions for use were found on 97 websites (57.9%), but the course of action to be taken after obtaining the test result was stated on only 63 and 43 (37.5% and 25.5%) for positive and negative results, respectively. Except for approval or certification (70.2% versus 51.2%, odds ratio 0.45, P= 0.03), there was no difference in the quality of information between the websites marketing tests done entirely at home and those where the test samples was analysed in the laboratory (Figure 2).

Figure 2.

The quality of websites included in a review of quality information accompanying online marketing of home diagnostic tests

DISCUSSION

This study clearly shows that the majority of websites marketing home diagnostic tests provide information of inadequate quality, and often fail to demonstrate any evidence of official approval or certification.

The validity of our findings and inferences depends on the rigour of our methods and the criteria we used in our assessment checklist. A question may be raised as to whether we captured a relevant sample of websites. Our website search was developed after extensive iteration with input from an information specialist. We took care to best mimic the searches likely to be conducted by those people who might search for information on home diagnostic tests. There remains the possibility that not all potentially relevant sites were captured, although we believe that our sample is likely to be representative of websites that may be found by consumers. There were some inconsistencies between reviewers initially, so we extracted all data at least in triplicate to avoid errors and biases in area where judgements were required. We chose not to prioritize items of quality as there is no empirical research on whether one item is more important than another. In the absence of any legally enforced quality guidelines for the online marketing of home tests, our selection of quality items was based on published authoritative guidance.^{3,5,8,9,13-18} We are therefore confident that our observations concerning the overall quality of websites marketing home tests merit consideration.

There have been no other studies that have examined the online marketing of such diagnostic tests; however, there has been research into the quality of health information provided online. In keeping with our study, it has generally been found that online health information is of variable and commonly poor quality.^10,11

There is increasing use of the internet by patients^1-3,5 and self-diagnosis can be pursued at home through tests purchased online. If the test is accurate and adequately applied, a negative result can provide reassurance and reduce the use of health-care resources. Similarly, an accurate positive result can direct appropriate use of health care—home pregnancy tests, for example, serve their purpose very well in this regard. On the other hand, erroneous results can give false reassurance or generate unnecessary anxiety while increasing the use of health-care resources. This study highlights the potential for misdiagnosis and undesirable consequences through the use of inadequate tests where sales have not been associated with provision of good quality information online. The need to improve the web advertising of such health products is obvious, given the implications for health care resulting from home testing. Online marketing needs to ensure that consumers are adequately informed, not only about how to conduct and interpret the test at home, but also about the consequences of testing (e.g. verification of a screening test result will require a diagnostic test).

Steps towards the regulation of home diagnostic tests have begun,^1-5,19,20 initially with enforcement of approval or certification, but from our findings it is clear that there is a long way to go. There is a need to establish good practice standards for the online marketing of home tests. This may need to be achieved through self-regulation, as control of websites' content is unlikely to be straightforward even with legalization. A standard template for delineating the features of tests being marketed on websites may be agreed and employed by sales and marketing agencies. Unregulated sales of home tests may otherwise result in potential misdiagnoses and inappropriate use of limited health care resources. We also recommend ongoing auditing and quality assurance of these sites in a similar manor to that undertaken for food and drugs.