Skip to main content
BMJ Open logoLink to BMJ Open
. 2015 Dec 22;5(12):e010017. doi: 10.1136/bmjopen-2015-010017

Awareness of the severity of liver disease re-examined using software-combined biomarkers of liver fibrosis and necroinflammatory activity

Thierry Poynard 1,2, Olivier Deckmyn 3, Mona Munteanu 3, Yen Ngo 3, Fabienne Drane 3, Jean Marie Castille 3, Chantal Housset 2, Vlad Ratziu 1,2, on behalf of the FIBROFRANCE Group
PMCID: PMC4691773  PMID: 26700292

Abstract

Background

Effective antiviral treatment (direct-acting antiviral agents (DAAs)), the requirement for a fibrosis score to support DDA reimbursement and a screening strategy, such as the USA baby boomer campaign, should lead to an increased awareness of liver disease severity.

Objective

To compare the awareness of liver disease severity between the USA and France, two countries with similar access to hepatitis C virus (HCV) and hepatitis B virus (HBV) treatments, similar rules for treatment reimbursement and similar availability of validated fibrosis tests, but with different policies, as France has no screening.

Method

The global database of the FibroTest–ActiTest, including 1 085 657 subjects between 2002 and 2014, was retrospectively analysed. Awareness was defined as the test prescription rate and was compared between the USA and France, according to year of birth, gender and dates of DAA availability and screening campaign (2013–2014).

Results

In the USA 252 688 subjects were investigated for HCV, with a dramatic increase (138%) in the test rate in 2013–2014 (119 271) compared with 2011–2012 (50 031). In France 470 762 subjects were investigated (subjects with HCV and other disease) and the rates were stable. In USA 82.4% of subjects and in France 84.6% were classified as either the highest or lowest priority. The most striking difference was the higher test rate in women born between 1935 and 1944 in France 30 384/200 672 (15.1%) compared with the USA 8035/97 079 (8.3%) (OR=1.98 (95% CI 1.93 to 2.03) p<0.0001). This resulted in twice as many cases of cirrhosis being detected, 2.6% (5191/200 672 women) and 1.3% (1303/97 079), respectively, despite the same prevalence of cirrhosis in this age group (17.1% vs 16.2%) and without any clear explanation as to why they had not been included in the USA screening.

Conclusions

This study highlighted in the USA the association between awareness of liver disease and both the HCV campaign and DAA availability. In comparison with France, there was a dramatically lower awareness of cirrhosis in the USA for women born between 1935 and 1944.


Strengths and limitations of this study.

  • Using fibrosis biomarkers it was possible to assess the relationships between fibrosis, activity, age and gender at a scale (more than 1 million subjects) unreachable by biopsy.

  • In comparison with France there was a dramatically lower awareness of cirrhosis in the USA for women born between 1935 and 1944. Women born before 1945 with a life expectancy of >80 years should be considered for non-invasive screening of liver fibrosis if cost-effective.

  • In the USA 82% and in France 85% of subjects were classified, as either the highest or lowest priority for treatment.

  • This was not a standard observational study and bias associated with the prescription of patented tests is possible.

Introduction

The global burden of cirrhosis and primary liver cancer, due mainly to chronic hepatitis C (CHC), is increasing and has reached 1.7 million related deaths a year worldwide.1 2 The burden of CHC could be greatly reduced, given the emergence of highly effective direct-acting antiviral agents (DAAs)3 and the availability of effective non-invasive biomarkers for identifying patients with severe liver disease,4–6 who should be given priority for the use of these expensive DAAs.7–9

The failure of risk-based screening with hepatitis C virus (HCV) serology testing to identify the 3 million cases of CHC and the over-representation of CHC in people born between 1945 and 1965 (‘baby boomer’ population) has led to birth cohort screening in the USA, regardless of whether risk factors are present or not.10 The effectiveness of such a strategy has not been proved.

Several barriers need to be overcome to completely realise the benefit of screening in reducing liver-related morbidity and mortality. Even for CHC, where effective treatments exist, fewer than 10% of cases are cured. Among these barriers, assessment of the extent of histological damage (fibrosis stage and necroinflammatory activity, which define severity) is one important component of patient evaluation. When a severe disease is identified, any barrier to care and starting treatment must be overcome.11

Liver disease severity has traditionally been assessed by liver biopsy. Since the standardisation of fibrosis and grading of activity in CHC,12 it has been possible to assess the main factors (age and gender) associated with the natural history of fibrosis and its progression to the last stage, cirrhosis.13 However, these studies, including ours,13 14 are rare, and limited by their sample size, with few studies including more than 2000 cases, which did not allow for follow-up of the awareness of fibrosis (see online supplementary data S1). Since 2002 the availability of biomarkers for identifying severe liver disease,15 and the centralisation of software-combined biomarkers, such as the FibroTest (FT),5 16 17 has enabled us to assess the global awareness of liver fibrosis and activity in 100 times more patients than was possible with biopsy (see online supplementary data S2). The FT has been extensively validated, using biopsy as a reference, in CHC,4–6 15–18 chronic hepatitis B (CHB),19 alcoholic liver disease (ALD)20 and non-alcoholic fatty liver disease (NAFLD);6 21 22 this has included modelling fibrosis progression or regression,22 and prognostic values.20 23–25 The reliability rate among 345 695 consecutive tests was >97%, even in tertiary centres.17 Several screening studies using FT have been performed in general populations,26–29 in patients with diabetes25 30–32 and in those with dyslipidaemia.33

Increased awareness of the severity of liver disease should result from effective antiviral treatment (DAAs), the requirement for a fibrosis score to support DDA reimbursement and a screening strategy, such as the USA baby boomer campaign. We aimed to compare the awareness between USA and France, two countries with similar access to HCV and hepatitis B virus (HBV) treatment, rules for treatment reimbursement, availability of validated fibrosis tests for more than 12 years, but different health strategies, as only the USA started a national screening campaign.4 34–37 Our first objective was to compare the incidence rate after 2013, when the USA campaign started. Second, we aimed to assess whether the 1945 birth-year threshold of the USA campaign was appropriate in view of the increased life expectancy of women.38 The third objective was to estimate, on a large scale, the proportion of patients with CHC and severe liver disease who might benefit from prioritisation to expensive DAAs.7–9

To achieve these objectives we analysed retrospectively a centralised database which combined fibrosis and necroinflammatory activity blood biomarkers (n=1 081 657 FibroTest–ActiTest (FT–AT)). For the first time we were able to describe the awareness of both liver fibrosis and activity, on a scale that was impossible using biopsy.

Methods

This non-interventional study was exempt from institutional review board (IRB) review (ethical committee of ‘Comité de Protection des Personnes of Paris, Ile-de-France,’ FIBROFRANCE project. CPP-IDF-VI, 10–1996-DR-964, DR-2012-222 and USA-NCT01927133). No patient consent was required, as all data were analysed anonymously. Clinical investigation was conducted according to the principles of the Declaration of Helsinki. All authors had access to the study data and reviewed and approved the final manuscript.

Test prescriptions and validation

FT–AT is a patented ‘In Vitro Diagnostic Multivariate Index Assay’ for the diagnosis of fibrosis stages (FT), including cirrhosis, and for activity grades (AT). FT includes serum α2-macroglobulin, apolipoprotein A1, haptoglobin, total bilirubin and γ-glutamyltranspeptidase, adjusted for age and gender. AT includes the same components plus alanine aminotransferase (ALT). FT–AT is exclusively available online, including security algorithms16 17 (see online supplementary data S2). Modelling of fibrosis progression or regression and prognostic performances were similar for FT and liver biopsy whatever the cause of the liver disease;4 6 18–24 both methods had the same limitations for discriminating between intermediate stages of fibrosis.39

The inclusion criterion was an FT–AT prescription for each subject, through an authorised laboratory connected to the BioPredictive website. Exclusion criteria were possible duplicates (same date of birth, country and gender) and non-reliable results identified using security algorithms.

Impact of the HCV screening campaign, on FT prescription rates, in the USA

Our hypothesis was that the FT prescription rates could be used as an index of fibrosis awareness, as assessing fibrosis stage is a major requirement for access to antiviral treatment and cost coverage by health systems. Therefore our aim was not to assess the prevalence of CHC or other chronic liver disease, but to estimate the awareness of advanced liver fibrosis during the past 12 years and the association of this awareness with age and gender—the two major risk factors,40–42 and screening policy. We focused, on the USA and France with highest number of FT–AT subjects—about two-thirds of the database.

Appropriateness of the 1945 birth-year threshold of the USA campaign in women

We acknowledge that screening recommendations must be shaped by data collected with a minimum of bias and that those recommendations must be based on cost-effectiveness analyses. Our database was the first to assess fibrosis severity in a large population of patients aged >65 years. Our previous hypothesis was that the 1945 birth-year threshold should have been earlier for women without associated severe disease. Since 1997,13 ageing has been shown to be the most significant independent risk factor of fibrosis severity among patients with CHC,26 25 31–34 40 and in 2010 the mean life expectancy of women in the USA reached 80.5 years (95% CI 80.5 to 80.6).38 Therefore, women born between 1935 and 1944 should be included in the HCV screening as the risk of cirrhosis would be expected to be very high in this subgroup.

HCV infection started to rise in the USA in subjects born between 1920 and 1930,10 with an estimated prevalence of HCV infection reaching 2.0% at the birth year of 1935 (shown in figure 6 of the landmark article of Armstrong et al43). For the period 1935–1944, no comparative data were found for women versus men, but even if infection in women were half that in men the greater life expectancy of women in this category would justify their inclusion in screening. We therefore planned to compare the prevalence of cirrhosis detected in women born between 1935 and 1944 in the USA, and a screening strategy starting at birth-year 1945, with France, where FT was prescribed without any age threshold (see online supplementary data S3). The same comparison was performed for men with the same years of birth to assess possible differences associated with shorter life expectancy.

Proportion of priority groups according to the context of use of DAA in CHC

The consensus in the guidelines was that for expensive DAA treatment the highest priority should be given to its use in patients with CHC with stages F3F4 and the lowest priority to patients without significant fibrosis or activity (F0F1–A0A1).3 6–9 11 The intermediate priority groups were patients with stage F2 and those with non-modifiable host factors associated with accelerated fibrosis progression, such as activity grade.8 9 11 We assessed the proportion of patients in these priority categories according to a combination of these factors in the USA results, where FT had a specific indication in CHC under the commercial brand name HCV-FibroSure (see online supplementary data S4).

Sensitivity analyses

We performed three sensitivity analyses on primary endpoints taking into account the main limitations of our study: no cause of liver disease in France, no biopsy and age and gender as confounding factors.

First, as our database did not include the cause of liver disease in France, we compared the endpoints in a cohort of patients with and without CHC from our Paris daycare (PDC) hospital for which causes of liver diseases were established, and used the USA CHC population as control (see online supplementary data S5). We estimated the representativeness of our samples using published prevalences of different liver diseases in France and the USA (see online supplementary data S6).

Second, FT is a surrogate and not a direct marker of fibrosis severity. Therefore we assessed the endpoints using biopsy. For the impact of the campaign in the USA, we searched Medline for the rate of biopsies performed in each of five countries over the past 10 years. For the appropriateness of the 1945 threshold, we obtained the same density plots for fibrosis estimated using biopsy and year of birth, in an integrated database of 6412 patients with naive CHC (see online supplementary data S7).

Third, FT includes age and gender in its algorithm, which might lead to overestimation of the impact of these factors on fibrosis prevalence. We performed analyses that neutralised the weight of age, gender and both (see online supplementary data S8). To identify possible bias among the CHC population, we also compared PDC characteristics with a tertiary centre population, an HIV tertiary centre and with outpatients reimbursed for HCV-FT testing (see online supplementary data S9).

Statistical methods

To compare the prevalence of cirrhosis awareness between USA and France, we performed a multivariate regression analysis that was adjusted for age and gender, and used the USA prevalence as a reference. Relative risk was expressed by OR and 95% CIs. To estimate the prevalence of cirrhosis severity, we used previously validated universal cut-off points for CHC and CHB,23 24 including F4.1—that is, ‘compensated’ cirrhosis without a high risk of portal hypertension (bleeding), F4.2 (high risk of bleeding) and F4.3. The F4.2 prevalence is representative of effective screening, as bleeding prevention can be started early to reduce mortality. Data were expressed using the median and 95% CI. The ORs were compared with Z tests. NCSS V.9.0 and R software V.3.1.2 was used.

Results

We constructed a centralised database of 1 081 657 FT–AT that were performed consecutively on fresh serum between 2002 and 2014. After exclusion of possible duplicates (n=65 100), 252 688 subjects in USA, 470 762 in France and 293 107 in other countries had an FT–AT performed. Details of the fibrosis stage, the proportion of baby boomer population, extent of fibrosis according to gender and year of birth and the prescription rates of biomarkers are given in figure 1. In other countries, in comparison with the USA and France, a higher proportion of the subjects investigated had cirrhosis, despite a younger age, the same association between fibrosis severity, age and male gender, with an exponential prescription rate was but without a dramatic increase in 2013–2014.

Figure 1.

Figure 1

Awareness of liver fibrosis according to the FibroTest in USA, France and in other countries. First column (main characteristics): number of subjects, median age, percentage of women and of subjects with cirrhosis. Second column (fibrosis stage spectrum): presumed METAVIR stage by the FibroTest—F0, no fibrosis; F1, minimal fibrosis; F2, few septa; F3, many septa; F4, cirrhosis. Cirrhosis F4.1 (red), compensated cirrhosis without risk of bleeding; F4.2 (violet), compensated cirrhosis with risk of bleeding; F4.3 (black), decompensated cirrhosis (cancer, bleeding, liver failure). Third column (baby boomer spectrum): number of subjects according to year of birth—before 1945 in orange, 1945–1965 in green, after 1965 in blue. Fourth column (fibrosis–age–gender): fibrosis density plots according to fibrosis presumed by FibroTest (y-axis) and year of birth (x-axis). All subjects are represented. The concentric black lines are density centiles, with the highest density being in the inner circle; subjects outside the outer line comprise <10% of the population. The two vertical black lines indicate the years of birth 1945 and 1965. The two crossing lines represented linear regression between FibroTest and date of birth, according to gender (men in blue and women in red). Fifth column (FibroTest (FT) prescription rate): number of subjects according to the test year between 2005–2006 (orange) and 2013–2014 (rose).

Impact of the ‘baby boomer’ HCV screening campaign, on FT prescription rates in the USA

The official USA recommendation, released in 2013, was associated with a dramatic 138% increase in FT–AT prescriptions from 50 031 in 2011–2012 to 119 271 in 2013–2014. This was significantly higher (p<0.0001) than the previous regular trend of 36.7% in 4 years, from 36 600 in 2007–2008 to 50 031 in 2011–2012. Multivariate analyses showed that the association between cirrhosis awareness and the start of the ‘baby boomer campaign,’ in the USA, persisted after taking into account age and gender (table 1). No such dramatic increase was seen in France. The prescription rate, which had increased after FT reimbursement in 2006,35–37 decreased from 98 833 tests in 2007–2008 to 81 661 tests in 2013–2014—that is, a 17% decrease.

Table 1.

Risk factors of cirrhosis awareness, presumed by FibroTest, in the US sample (n=252 688), assessed by multivariate regression analysis

Factor Odds ratio Low 95% CI High 95% CI Significance
Age 1.0595 1.0583 1.0606 <0.0001
Male gender 3.21 3.13 3.29 <0.0001
Tested after 2013* 1.11 1.08 1.13 <0.0001

*Hepatitis C virus (HCV) birth-year screening USA campaign and availability of direct acting antiviral agents for HCV started in 2013.

Appropriateness of the 1945 birth-year threshold in women of the US campaign

In France, slightly more women than men were investigated in comparison with USA (OR=1.19 (95% CI 1.18 to 1.20) p<0.0001), and twice as many women born between 1935 and 1944 were investigated in France than in the USA, 15.1% versus 8.3% of all women respectively (OR=1.98 (95% CI 1.93 to 2.03) p<0.0001) (table 2). These differences explain the marked difference in the number of cases of cirrhosis in women, identified in France during this period (n=5191), compared with in the USA (n=1303) (OR=2.15, 95% CI 2.02 to 2.29; p<0.0001). These differences were not related to the higher rate of cirrhosis in women from this birth category in France versus the USA, 17.1% and 16.2%, respectively (OR=1.06 (95% CI 1.00 to 1.14); p=0.06). The increase in the percentage of cirrhosis identified in men born between 1935 and 1944 in France compared with the USA was much less (OR=1.50 (95% CI 1.44 to 1.56)) than among women (table 3).

Table 2.

Cirrhosis awareness in women born between 1935 and 1944 in France versus USA samples, presumed by FibroTest

France USA OR France vs USA (95% CI)
All subjects, n 470 762 252 688
Women/all subjects 200 672/470 762 (42.6) 97 079/252 688 (38.4) 1.19 (1.18 to 1.20)
Women 1935–1944/all women 30 384/200 672 (15.1) 8035/97 079 (8.3) 1.98 (1.93 to 2.03)*
Cirrhosis/women 1935–1944 5191/30 384 (17.1) 1303/8035 (16.2) 1.06 (1.00 to 1.14)†*
Cirrhosis/women 1945–1965 8345/107 478 (7.8) 6567/63 806 (10.3) 0.73 (0.71 to 0.76)*
Cirrhosis/women >1965 884/48 790 (1.8) 342/22 851 (1.5) 1.21 (1.07 to 1.38)‡
Cirrhosis in women 1935–1944/all women 5191/200 672 (2.6) 1303/97 079 (1.3) 1.95 (1.84 to 2.08)
Cirrhosis in women 1935–1944/all subjects 5191/470 762 (1.1) 1303/252 688 (0. 5) 2.15 (2.02 to 2.29)

Results are shown as number (%) unless stated otherwise.

*There was no more cirrhosis among women born in 1935–1944 in France compared with the USA, but this age group was much more frequently investigated.

†Not significant (p=0.06).

‡p=0.002. All other ORs were highly significant (p<0.0001).

Table 3.

Cirrhosis awareness in men born between 1935 and 1944 in France versus USA samples, presumed by FibroTest

France USA OR France vs USA (95% CI)
All subjects, n (%) 470 762 252 688
Men/all subjects 270 090/470 762 (57.4) 155 609/252 688 (61.6) 0.84 (0.83 to 0.85)
Men 1935–1944/all men 23 277/270 090 (8.6) 8750/155 609 (5.6) 1.58 (1.54 to 1.62)*
Cirrhosis/men 1935–1944 8430/23 277 (36.2) 3034/8750 (34.7) 1.07 (1.02 to 1.13)†
Cirrhosis/men 1945–1965 32 371/149 932 (21.6) 31 223/114 038 (27.4) 0.73 (0.72 to 0.74)
Cirrhosis/men >1965 5896/88 021 (6.7) 2107/30 936 (6.8) 0.98 (0.93 to 1.03)‡*
Cirrhosis in men 1935–1944/all men 8430/270 090 (3.12) 3034/155 609 (1.95) 1.62 (1.55 to 1.69)
Cirrhosis in men 1935–1944/all subjects 8430/470 762 (1.79) 3034/252 688 (1.20) 1.50 (1.44 to 1.56)

Results are shown as number (%) unless stated otherwise.

*There was slightly more cirrhosis among men born in 1935–1944 in France compared with the USA, but this age group was much more frequently investigated. †p=0.01.

‡not significant (p=0.50). All other ORs were highly significant (p<0.0001).

Of the 139 baby boomer women with non-decompensated cirrhosis, viral cure (or sustained virological response (SVR)) was obtained in 13 women and no deaths or liver complications were seen at 7 years (100% survival). In the 126 patients without SVR, 24 liver-related complications (52.7% at 8 years without complications (27.8% to 77.6%) p=0.04 vs SVR) and 11 deaths occurred (survival 79.8% (62.5–97.2) NS vs SVR). Of the 47 older women (66–75 years of age at inclusion), similar survival rates were seen: SVR was obtained in six women, all without the occurrence of liver complications at 7 years (100% survival); one non-liver-related death occurred (survival 69.2% (44.9% to 93.6%)). In the 41 patients without SVR, there were three liver-related complications (84.1% without complications (66.6% to 100%) NS vs SVR) and seven deaths (survival 79.8% (62.5% to 97.2%) NS vs SVR) (see online supplementary data S3). Despite the small number of patients, these data suggest that the baby boomer age limit could be extended to women born between 1935 and 1944, in the absence of severe associated disease and if cost-effectiveness analyses studies confirm the efficiency of DAA treatment.

Proportion of patients with CHC in the USA who might benefit from prioritisation for DAA treatments, as indicated by biomarkers

Using FT and AT as markers of inflammation grade (see online supplementary data S4), we ranked the patients into five categories based on priority for treatment. For the first time necroinflammatory activity presumed by the ActiTest was assessed for each stage of fibrosis (figure 2 and online supplementary video 1). In the USA, 82.4% (208 174/252 688, (95% CI 82.2% to 82.5%)) of investigated patients could be classified according to consensual recommendations, either as the highest priority (n=83 058; 32.9% (32.7% to 33.1%)) or the lowest priority (n=125 116; 49.5% (49.3% to 49.7%)) (table 4).

Figure 2.

Figure 2

The extent of fibrosis according to activity grade as predicted by the ActiTest biomarker (see online supplementary data video 1 for the USA). First column: percentage of activity grade ranges presumed by the ActiTest, no activity (A0) and severe necroinflammatory activity (A3). Second column: among subjects without activity (A0), fibrosis density plots according to fibrosis presumed by FibroTest (y-axis) and year of birth (x-axis). Concentric black lines indicate centiles of density, with higher density in the inner circle; subjects outside the outer line comprise <10% of population. The two vertical black lines indicate the years of birth 1945 and 1965. Third column: fibrosis density plots among subjects with minimal activity (A1). Fourth column: fibrosis density plots among subjects with moderate activity (A2). Fifth column: fibrosis density plots among subjects with severe activity (A3). For all countries and all grades of activity there was a strong association between fibrosis severity, age and male gender. The 2 crossing lines represented linear regression between FibroTest and date of birth, according to gender (men in blue and women in red).

Table 4.

Priority ranking for direct-acting antiviral agents in five categories according to the METAVIR scoring system, obtained by the FibroTest and ActiTest, in the USA sample

Population Count % (95% CI)
Highest priority (F3F4) 83 058 32.9 (32.7 to 33.1)
Second priority F2 active (F2–A2A3) 7877 3.1 (3.0 to 3.2)
Third priority (F2–A0A1) 16 904 6.7 (6.6 to 6.8)
Low priority (F0F1–A2A3) 19 733 7.8 (7.7 to 7.9)
Lowest priority (F0F1–A0A1) 125 116 49.5 (49.3 to 49.7)
Total 252 688 100%

Sensitivity analyses

We found no estimate for biopsy prescriptions after the USA campaign started. Before 2013, similar trends were seen between the biopsy rate increase in Canada (41% (95% CI 23% to 61%))44 and the FT rate increase in our US population (37%) (see online supplementary data S6).

For the appropriateness of the 1945 threshold, similar results were seen using biopsy or FT. In women born between 1935 and 1944 we observed a high prevalence of cirrhosis (n=60) out of the 6412 (0.9%) patients biopsied (see online supplementary data S6).

The proportion of consensual priority groups estimated using biopsy in CHC was lower than in the USA cohort using FT–AT estimates in USA, both for the lowest priority category (A0A1F0F1: 26.0% vs 49.5%; p<0.0001) and highest priority (F3F4: 19.1% vs 32.9%; p<0.0001).

As expected allowing for the effect of age in FT dramatically modified the extent of liver fibrosis as well as the fibrosis spectrum (see online supplementary data S8). The strength of association between fibrosis severity and birth year, was still significant but with a smaller slope. The highest prevalence of cirrhosis identified in France, in women born between 1935 and 1944, persisted compared with the USA (OR=1.97 (1.83 to 2.23) ), or for men (OR=1.52 (1.43 to 1.61) ). Making an allowance for age reduced the proportion of the highest priority from 32.9% to 25.8% and increased the proportion of lowest priority category from 49.5% to 55.5%.

Sensitivity analyses among the French populations showed that patients with CHC in the PDC, in comparison with the overall French sample, were older (55.7 vs 50.7 years), with a similar proportion of women (43.4% vs 42.6%) and a similar proportion of cases of cirrhosis among the women born between 1935 and 1944 (1.2% vs 1.1%). The much lower prevalence of cirrhosis among women born between 1935 and 1944 in patients without CHC (0.2% in CHB, 0.3% in NAFLD and 0.5% in ALD) suggested that the higher prevalence of cirrhosis detected in France was indeed related to CHC in this age group (see online supplementary data S5). In patients of the Paris HIV centre, fibrosis was most severe with more cirrhosis despite younger age, with a significantly higher slope of fibrosis according to age, both for men and women (see online supplementary data S9).

Results of the analyses of samples in the USA, France and in other countries are shown in online supplementary files S10-A, S10-B and S10-C, respectively. Interestingly, despite the non-specific prescription of FT for CHC, there was a bimodal distribution of liver severity proportions with the same total of highest priority plus lowest priority in France 84%=27%+57% and in other countries 84%=33%+51%, similar to that seen in the USA for CHC only 83%=33%+50%. France had the highest proportion of low and lowest priority subjects (57%).

Discussion

In this study we measured the positive effect of the USA campaign on significant fibrosis awareness, even if this effect cannot be dissociated from the simultaneous awareness of the availability of DAAs. In France, a country without a screening campaign, no such increase was seen despite the same awareness of DAAs. Comparisons with France, where fibrosis biomarkers were widely used, suggest that in countries where women's life expectancy is >80 years, an extension of screening to women born between 1935 and 1944 could be discussed after a cost-effectiveness analysis. Biomarkers such as FT–AT enabled assessment of the proportion of patients at highest and lowest priority for DAA treatment for CHC.

The main advantage of this study was to analyse the burden of asymptomatic severe liver disease on a scale that is not possible with biopsy. The main limitation was the retrospective design and the dependence of test prescriptions mostly driven in the USA by health insurance policies requiring proof of advanced fibrosis for DAA treatment. Therefore our design could not guarantee that the samples were representative of the true fibrosis spectrum, or of the prevalence of different confounding factors. We did not record data on treatments and their effectiveness.

An unmet need for decreasing the burden of chronic liver diseases has been the absence of non-invasive biomarkers. The natural history of liver fibrosis in individuals has been primarily based on biopsy, which is the only direct estimate of fibrosis, and although accurate despite a large sampling error, is limited by its applicability, cost and risk.4 6 11 35–37 39 44–47 These limitations introduced bias into our previous awareness of fibrosis severity, as details of biopsies were mostly provided by the availability of treatments and integrated databases of phase 3 trials (see online supplementary data S1).14 47 In larger populations studies, the severity of liver disease has been based on transaminases such as alanine aminotransferase (ALT),48 a biomarker of limited use as it is mainly associated with activity, and not specific or sensitive for the diagnosis of fibrosis. Furthermore, ALT decreases with age and cirrhosis occurrence.49

FT is one of many methods used to determine fibrosis stage. It has limitations, but it has key features, not achieved by other methods, such as APRI (aspartate aminotransferase to platelet ratio index), FIB-4 (patient age, aspartate aminotransferase, ALT and platelets) (both including transaminases) or transient elastography. These include its applicability (97%), its performance in the diagnostic and prognosis of the most frequent types of fibrosis,4–6 15–37 its association with the ActiTest—a validated biomarker of activity,16 21 and its centralised assessment, using pre-analytical and analytical standardisations with reliability algorithms.17 We acknowledge that we have a conflict of interest and that FT is not a perfect blood test. Using biopsy, even a perfect test cannot have an area under the receiver operating characteristic curve >90%.50 FT is more expensive than APRI and FIB-4, but it has a higher diagnostic and prognostic performance, with more validations in CHC, CHB, NAFLD and ALD, including direct comparisons in an intention to diagnose.5 6 51 It also has higher applicability than transient elastography,46 51 and is implemented at the same cost with a biomarker of activity. The cost of such accurate biomarkers must continue to decrease in order to increase the global awareness of fibrosis. Only one independent cost-effectiveness study has been published so far, in favour of FT in CHC.52

The increase of FT prescriptions since 2013 is not sufficient to validate the screening strategy in the USA, as the start of the campaign was associated with the availability of DAAs together with health insurance policies requiring proof of advanced fibrosis for DAA coverage.8–11 However, here we observed a 138% increase in FT prescriptions in the USA in 2013–2014 in comparison with 2011–2012, which might be due to campaign effectiveness, as such changes were not seen in France, which had the same rules of reimbursement for DAAs. This highlights a difference between the two countries in who is screened for HCV and who will receive a fibrosis test. Our study also ignored the significant lag time between a recommendation for increased HCV screening and the implementation of increased screening, not to mention fibrosis marker testing. In the USA, the Centers for Disease Control and Prevention recommendation for baby boomer testing was released in 2012 but it was only when the US Preventive Services Task Force released its report in 2013 that baby boomer HCV antibody screening became universally reimbursable. A longer follow-up should clarify the independent weight of the screening campaign. In other countries the increase in FT prescription rate has been continuous and exponential for the past 10 years (figure 1).

One explanation for the absence of such an increase in France is the very large use of FT in France as 470 762 subjects were investigated, which is much greater than the estimate of 183 528 subjects aware of their HCV status. Therefore the vast majority of French patients with CHC was already aware of their fibrosis stages (estimated 77%) and did not need to be investigated in 2013–2014 when use of DAAs started (see online supplementary data S6c). In the USA the proportion of subjects investigated by FT among those aware of their HCV status was estimated to be <15% (see online supplementary data S6c), which leaves room for an increase in test prescriptions associated with DAA reimbursement and screening.

We suggested an improvement of the USA screening strategy by extending screening to women born between 1935 and 1944 (up to 80 years in 2015), if our results are confirmed by an independent study. These women had an HCV infection prevalence already >1%,10 43 and were screened two times less often in the USA than in France, owing to the 1945 threshold choice. In a small sample of French women in this category, no liver-related complications occurred at 5 years in women who had been virologically cured, in contrast to non-responders or women who had not been treated. A cost-effectiveness analysis is mandatory before such change in recommendations is made, but there is no clear rationale for refusing treatment and reimbursement in women who have cirrhosis and no associated severe disease, with life expectancy already >80 years in 2010. We are less convinced of the efficacy of extended screening in men, who in the USA have a mean life expectancy of <80 years.

We also assessed the proportion of the different priority categories for DAA treatment—a major economic problem. For the first time at such a large scale, using both staging by FT and grading by AT, we found that that the highest priority group F3F4 included 33% of patients and the lowest priority group, 50%. These estimates could be useful for determining health policies. For instance, extending the treatment indication to F2–A2A3 (the second priority category) in the USA, would increase the high priority population by only 2.8% (see online supplementary data S4). The same proportions were seen in other countries (see online supplementary data S10-C).

We compared the effect of screening patients with a specific cause of liver disease—HCV in the USA—with screening of fibrosis whatever the cause—in France. In the USA the sample investigated comprised 15.1% of HCV-aware subjects (see online supplementary data S6). Interestingly, the prevalence of cirrhosis was similar to that of a recent large US HCV cohort using liver biopsy—21.4% vs 18.5%, respectively47 (see online supplementary data S5). In France, fibrosis awareness was obviously not restricted to CHC and CHB. Indeed, the number of French subjects aware of their fibrosis stage was greater than the estimates of French subjects aware of their anti-HCV or HBsAg status. The remaining subjects were presumably investigated for other causes, mainly NAFLD and ALD (see online supplementary data S9). Assuming that CHC represented at least 30% of investigated subjects, our sample might have represented 77% of HCV-aware subjects (see online supplementary data S6). The non-identification of CHC among the overall French sample is a limitation. However, the sensitivity analysis in the PDC subpopulation showed that patients with CHC, in comparison with the overall French sample, had a similar proportion of cirrhosis among the women born between 1935 and 1944 (1.2% vs 1.1%), in contrast to patients without CHC: 0.2% in CHB, 0.3% in NAFLD and 0.5% in ALD. These latter proportions were much lower than in patients with CHC (1.2%), suggesting that the higher prevalence of cirrhosis detected in France was indeed related to CHC in this age class. Another limitation was the absence of information about the ethnicity of all the populations.

Despite these limitations, the main results were robust using sensitivity analyses. The same results and conclusion were found using biopsy or FT adjusted for age and gender. Although we consider these data promising, further validation of biomarkers such as FT is needed to prove their efficiency in screening for fibrosis. We recognise that the overall success of any screening programme should be measured by the outcome of long-term treatment. We cannot comment on treatment success rates or reductions in mortality, as these data were not prospectively collected. Several barriers need to be overcome to fully define the potential benefit of any screening in reducing liver-related morbidity and mortality.

In conclusion, this is a proof of concept of a new generation of observational made possible by the global centralisation of such tests, enabling collection of 1 081 657 consecutive tests. The design did not allow us to control for confounding variables and bias as with classic epidemiological studies. Owing to the simultaneous awareness of the HCV campaign and of the reimbursement rules for effective DAAs, our study describes only a marked increase in fibrosis awareness without identifying the respective weight of these two drivers. Comparisons with France according to age and gender suggested that women born between 1935 and 1944 are a  group at high risk for cirrhosis and premature death if not treated, who were excluded from baby boomer screening in the USA. This might explain why they have been less identified than in France. It is suggested that a cost-effectiveness analysis should be performed before decisions about whether or not to extend screening are made. Finally, it was reassuring that in patients investigated, >80% had criteria showing them to be highest priority or lowest priority for expensive antiviral treatments.

Footnotes

Contributors: TP: study concept and design; analysis and interpretation of data; statistical analysis; drafting of the paper; study supervision. OD: study concept and design; analysis and interpretation of data; statistical analysis; drafting of the paper. MM, YN: acquisition of data; analysis and interpretation of data; statistical analysis; drafting of the paper; critical revision of the manuscript. FD: acquisition of data; drafting of the paper. CH, VR: drafting of the paper; critical revision of the manuscript.

Funding: No special funding but OD, MM, YN, JMC and FD were employees of BioPredictive.

Competing interests: TP is the inventor of FibroTest/SteatoTest and the founder of BioPredictive, the company that markets these tests. Patents belong to the French Public Organization Assistance Publique-Hôpitaux de Paris (APHP). OD, MM, YN, JMC and FD are BioPredictive employees.

Ethics approval: FIBROFRANCE-project. CPP-IDF-VI, 10-1996-DR-964, DR-2012-222.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

References

  • 1.US Burden of Disease Collaborators. The state of US health, 1990–2010: burden of diseases, injuries and risk factors. JAMA 2013;310:591–608. 10.1001/jama.2013.13805 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Mohd Hanafiah K, Groeger J, Flaxman AD et al. . Global epidemiology of hepatitis C virus infection: new estimates of age-specific antibody to HCV seroprevalence. Hepatology 2013;57:1333–42. [DOI] [PubMed] [Google Scholar]
  • 3.[No authors listed]. The price of good health. Nat Med 2014;20:319 10.1038/nm.3538 [DOI] [PubMed] [Google Scholar]
  • 4.Poynard T. First-line assessment of patients with chronic liver disease with non-invasive techniques and without recourse to liver biopsy. J Hepatol 2011;54:586–7. 10.1016/j.jhep.2010.09.012 [DOI] [PubMed] [Google Scholar]
  • 5.Chou R, Wasson N. Blood tests to diagnose fibrosis or cirrhosis in patients with chronic hepatitis C virus infection: a systematic review. Ann Intern Med 2013;158:807–20. 10.7326/0003-4819-158-11-201306040-00005 [DOI] [PubMed] [Google Scholar]
  • 6.European Association for Study of Liver; Asociacion Latinoamericana para el Estudio del Higado. EASL-ALEH Clinical Practice Guidelines: Non-invasive tests for evaluation of liver disease severity and prognosis. J Hepatol 2015;63:237–64. Connected 23 April 2015 10.1016/j.jhep.2015.04.006 [DOI] [PubMed] [Google Scholar]
  • 7.Kuehn BM. Guideline: new HCV drugs should go to sickest patients. JAMA 2014;312:1084–5. 10.1001/jama.2014.11373 [DOI] [PubMed] [Google Scholar]
  • 8.AASLD guidelines: http://www.hcvguidelines.org/full-report/hcv-testing-and-linkage-care (accessed 19 Apr 2015).
  • 9.European Association for Study of Liver. EASL Clinical Practice Guidelines: management of hepatitis C virus infection. J Hepatol 2014;60:392–420. 10.1016/j.jhep.2013.11.003 [DOI] [PubMed] [Google Scholar]
  • 10.Smith BD, Morgan RL, Beckett GA et al. . Hepatitis C virus testing of persons born during 1945–1965: recommendations from the Centers for Disease Control and Prevention. Ann Intern Med 2012;157:817–22. 10.7326/0003-4819-157-9-201211060-00529 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Afdhal NH, Zeuzem S, Schooley RT et al. . The new paradigm of hepatitis C therapy: integration of oral therapies into best practices. J Viral Hepat 2013;20:745–60. 10.1111/jvh.12173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Bedossa P, Poynard T. An algorithm for the grading of activity in chronic hepatitis C. The METAVIR Cooperative Study Group. Hepatology 1996;24:289–93. 10.1002/hep.510240201 [DOI] [PubMed] [Google Scholar]
  • 13.Poynard T, Bedossa P, Opolon P. Natural history of liver fibrosis progression in patients with chronic hepatitis. Lancet 1997;349:825–32. 10.1016/S0140-6736(96)07642-8 [DOI] [PubMed] [Google Scholar]
  • 14.Poynard T, McHutchison J, Manns M et al. . Impact of pegylated interferon alfa-2b and ribavirin on liver fibrosis in patients with chronic hepatitis C. Gastroenterology 2002;122:1303–13. 10.1053/gast.2002.33023 [DOI] [PubMed] [Google Scholar]
  • 15.Imbert-Bismut F, Ratziu V, Pieroni L et al. . Biochemical markers of liver fibrosis in patients with hepatitis C virus infection: a prospective study. Lancet 2001;357:1069–75. 10.1016/S0140-6736(00)04258-6 [DOI] [PubMed] [Google Scholar]
  • 16.Poynard T, Munteanu M, Ngo Y et al. . ActiTest accuracy for the assessment of histological activity grades in patients with chronic hepatitis C, an overview using Obuchowski measure. Gastroenterol Clin Biol 2010;34:388–96. 10.1016/j.gcb.2010.05.001 [DOI] [PubMed] [Google Scholar]
  • 17.Poynard T, Munteanu M, Deckmyn O et al. . Applicability and precautions of use of liver injury biomarker FT. A reappraisal at 7 years of age. BMC Gastroenterol 2011;11:39 10.1186/1471-230X-11-39 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Poynard T, Ngo Y, Munteanu M et al. . Biomarkers of liver injury for hepatitis clinical trials: a meta-analysis of longitudinal studies. Antivir Ther 2010;15:617–31. 10.3851/IMP1570 [DOI] [PubMed] [Google Scholar]
  • 19.Xu XY, Kong H, Song RX et al. . The effectiveness of noninvasive biomarkers to predict hepatitis B-related significant fibrosis and cirrhosis: a systematic review and meta-analysis of diagnostic test accuracy. PLoS ONE 2014;9:e100182 10.1371/journal.pone.0100182 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Naveau S, Gaudé G, Asnacios A et al. . Diagnostic and prognostic values of noninvasive biomarkers of fibrosis in patients with alcoholic liver disease. Hepatology 2009;49:97–105. 10.1002/hep.22576 [DOI] [PubMed] [Google Scholar]
  • 21.Poynard T, Lassailly G, Diaz E et al. . Performance of biomarkers FT, ActiTest, SteatoTest and NashTest in patients with severe obesity: meta-analysis of individual patient data. PLoS ONE 2012;7:e30325 10.1371/journal.pone.0030325 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Poynard T, Munteanu M, Deckmyn O et al. . Validation of liver fibrosis biomarker (FT) for assessing liver fibrosis progression: proof of concept and first application in a large population. J Hepatol 2012;57:541–8. 10.1016/j.jhep.2012.04.025 [DOI] [PubMed] [Google Scholar]
  • 23.Poynard T, Vergniol J, Ngo Y et al. . Staging chronic hepatitis B into seven categories, defining inactive carriers and assessing treatment impact using a fibrosis biomarker (FT®) and elastography (FibroScan®). J Hepatol 2014;61:994–1003. 10.1016/j.jhep.2014.06.027 [DOI] [PubMed] [Google Scholar]
  • 24.Poynard T, Vergniol J, Ngo Y et al. . Staging chronic hepatitis C in seven categories using fibrosis biomarker (FT™) and transient elastography (FibroScan®). J Hepatol 2014;60:706–14. 10.1016/j.jhep.2013.11.016 [DOI] [PubMed] [Google Scholar]
  • 25.Perazzo H, Munteanu M, Ngo Y et al. . Prognostic value of liver fibrosis and steatosis biomarkers in type-2 diabetes and dyslipidaemia. Aliment Pharmacol Ther 2014;40:1081–93. 10.1111/apt.12946 [DOI] [PubMed] [Google Scholar]
  • 26.Poynard T, Lebray P, Ingiliz P et al. . Prevalence of liver fibrosis and risk factors in a general population using non-invasive biomarkers. BMC Gastroenterol 2010;10:40 10.1186/1471-230X-10-40 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Zelber-Sagi S, Ratziu V, Zvibel I et al. . The association between adipocytokines and biomarkers for nonalcoholic fatty liver disease-induced liver injury: a study in the general population. Eur J Gastroenterol Hepatol 2012;24:262–9. 10.1097/MEG.0b013e32834f15dd [DOI] [PubMed] [Google Scholar]
  • 28.Salles N, Dussarat P, Foucher J et al. . Non-invasive evaluation of liver fibrosis by transient elastography and biochemical markers in elderly inpatients. Gastroenterol Clin Biol 2009;33:126–32. 10.1016/j.gcb.2008.12.003 [DOI] [PubMed] [Google Scholar]
  • 29.Grattagliano I, Ubaldi E, Napoli L et al. . Utility of noninvasive methods for the characterization of nonalcoholic liver steatosis in the family practice. The “VARES” Italian multicenter study. Ann Hepatol 2013;12:70–7. [PubMed] [Google Scholar]
  • 30.Jacqueminet S, Lebray P, Morra R et al. . Screening for liver fibrosis by using a noninvasive biomarker in patients with diabetes. Clin Gastroenterol Hepatol 2008;6:828–31. 10.1016/j.cgh.2008.03.005 [DOI] [PubMed] [Google Scholar]
  • 31.de Lédinghen V, Vergniol J, Gonzalez C et al. . Screening for liver fibrosis by using FibroScan(®) and FibroTest in patients with diabetes. Dig Liver Dis 2012;44:413–18. 10.1016/j.dld.2011.12.005 [DOI] [PubMed] [Google Scholar]
  • 32.Petit JM, Guiu B, Masson D et al. . PNPLA3 polymorphism influences liver fibrosis in unselected patients with type 2 diabetes. Liver Int 2011;31:1332–6. 10.1111/j.1478-3231.2011.02566.x [DOI] [PubMed] [Google Scholar]
  • 33.Ratziu V, Giral P, Munteanu M et al. . Screening for liver disease using non-invasive biomarkers (FibroTest, SteatoTest and NashTest) in patients with hyperlipidaemia. Aliment Pharmacol Ther 2007; 25:207–18. 10.1111/j.1365-2036.2006.03182.x [DOI] [PubMed] [Google Scholar]
  • 34.Poynard T, Ratziu V, Benhamou Y et al. . Biomarkers as a first-line estimate of injury in chronic liver diseases: time for a moratorium on liver biopsy? Gastroenterology 2005;128:1146–8. 10.1053/j.gastro.2005.02.056 [DOI] [PubMed] [Google Scholar]
  • 35.Fontaine H, Petitprez K, Roudot-Thoraval F et al. . Guidelines for the diagnosis of uncomplicated cirrhosis. Gastroenterol Clin Biol 2007;31:504–9. 10.1016/S0399-8320(07)89420-6 [DOI] [PubMed] [Google Scholar]
  • 36.Castera L, Denis J, Babany G et al. . Evolving practices of non-invasive markers of liver fibrosis in patients with chronic hepatitis C in France: time for new guidelines? J Hepatol 2007;46:528–9. 10.1016/j.jhep.2006.12.002 [DOI] [PubMed] [Google Scholar]
  • 37.l'Association Française pour l'Etude du Foie. Hepatitis B: the French point of view on EASL guidelines. Gastroenterol Clin Biol 2009;33:535–8. 10.1016/j.gcb.2009.05.008 [DOI] [PubMed] [Google Scholar]
  • 38.Wang H, Dwyer-Lindgren L, Lofgren KT et al. . Age-specific and sex-specific mortality in 187 countries, 1970–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380:2071–94. 10.1016/S0140-6736(12)61719-X [DOI] [PubMed] [Google Scholar]
  • 39.Poynard T, Lenaour G, Vaillant JC et al. . Liver biopsy analysis has a low level of performance for diagnosis of intermediate stages of fibrosis. Clin Gastroenterol Hepatol 2012;10:657–63. 10.1016/j.cgh.2012.01.023 [DOI] [PubMed] [Google Scholar]
  • 40.Rüeger S, Bochud PY, Dufour JF et al. . Impact of common risk factors of fibrosis progression in chronic hepatitis C. Gut 2015;64:1605–15. 10.1136/gutjnl-2014-306997 [DOI] [PubMed] [Google Scholar]
  • 41.Eslam M, Hashem AM, Leung R et al. . Interferon-λ rs12979860 genotype and liver fibrosis in viral and non-viral chronic liver disease. Nat Commun 2015;6:6422 10.1038/ncomms7422 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Gower E, Estes C, Blach S et al. . Global epidemiology and genotype distribution of the hepatitis C virus infection. J Hepatol 2014;61:S45–57. 10.1016/j.jhep.2014.07.027 [DOI] [PubMed] [Google Scholar]
  • 43.Armstrong GL, Alter MJ, McQuillan GM et al. . The past incidence of hepatitis C virus infection: implications for the future burden of chronic liver disease in the United States. Hepatology 2000;31:777–82. 10.1002/hep.510310332 [DOI] [PubMed] [Google Scholar]
  • 44.Myers RP, Fong A, Shaheen AA. Utilization rates, complications and costs of percutaneous liver biopsy: a population-based study including 4275 biopsies. Liver Int 2008;28:705–12. 10.1111/j.1478-3231.2008.01691.x [DOI] [PubMed] [Google Scholar]
  • 45.Udompap P. Advanced fibrosis is common in individuals whose hepatitis C has not been diagnosed: results from the National Health and Nutrition Examination Survey 2001-2012. EASL. April 22–26 2015, Vienna, Austria. [Google Scholar]
  • 46.Castera L. Noninvasive methods to assess liver disease in patients with hepatitis B or C. Gastroenterology 2012;142:1293–302. 10.1053/j.gastro.2012.02.017 [DOI] [PubMed] [Google Scholar]
  • 47.Holmberg SD, Lu M, Rupp LB et al. , Chronic Hepatitis Cohort Study (CHeCS) Investigators. Noninvasive serum fibrosis markers for screening and staging chronic hepatitis C virus patients in a large US cohort. Clin Infect Dis 2013;57:240–6. 10.1093/cid/cit245 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Kim WR, Flamm SL, Di Bisceglie AM et al. . Serum activity of alanine aminotransferase (ALT) as an indicator of health and disease. Hepatology 2008;47:1363–70. 10.1002/hep.22109 [DOI] [PubMed] [Google Scholar]
  • 49.Chao DT, Lim JK, Ayoub WS et al. . Systematic review with meta-analysis: the proportion of chronic hepatitis B patients with normal alanine transaminase ≤40 IU/L and significant hepatic fibrosis. Aliment Pharmacol Ther 2014;39:349–58. 10.1111/apt.12590 [DOI] [PubMed] [Google Scholar]
  • 50.Mehta SH, Lau B, Afdhal NH et al. . Exceeding the limits of liver histology markers. J Hepatol 2009;50:36–41. 10.1016/j.jhep.2008.07.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Houot M, Ngo Y, Munteanu M et al. . Systematic review with meta-analysis: direct comparisons of biomarkers for the diagnosis of fibrosis in chronic hepatitis C and B. Aliment Pharmacol Ther Published Online First: 30 Oct 2015. doi:10.1111/apt.13446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Liu S, Schwarzinger M, Carrat F et al. . Cost effectiveness of fibrosis assessment prior to treatment for chronic hepatitis C patients. PLoS ONE 2011;6:e26783 10.1371/journal.pone.0026783 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES