Validation of new biomarkers in systemic autoimmune diseases

Abstract

Biomarkers have an important influence on the clinical decision-making processes involved in diagnosis, assessment of disease activity, allocation of treatment, and determining prognosis. The clinical usefulness of a biomarker is dependant on demonstration of its validity. Ideally, biomarkers should provide information not available from currently available tests and should be tested as they would be used in clinical practice; however, potential biomarkers could be affected by many different clinical or patient variables—such as disease activity, therapeutic intervention, or the presence of comorbidities—and validation studies might not include all the design features that are required to ensure that the biomarker is a true measure of the clinical process it is intended to reflect. In this Review, we appraise studies that have been conducted to validate six promising new biomarkers for diagnosis, disease activity assessment, or prognosis in patients with systemic autoimmune diseases. We discuss the validity of these six biomarkers with particular reference to the features of the studies that lend weight to or distract from their findings. The intent of this discussion is to draw attention to elements of validation study design that should be considered when evaluating the robustness of a biomarker, which differ according to the marker's intended use.

Introduction

Biomarker discovery is one of the major areas of emphasis in translational research. Biomarkers, defined as charac teristics that are objectively measured and evaluated as indicators of normal and pathogenic biological processes or pharmacologic responses,¹ can be used to help diagnose diseases, to assess disease activity and response to treatment, or to predict prognosis. The value of biomarkers in diagnosis, patient evaluation, and prognosis is dependant on demonstration of the validity of their association with a specific disease, or a particular manifesta tion of that disease.^2,3 Biomarkers reflect biological processes, which often vary naturally with age, gender or other patient characteristics; for example, therefore, in investi gating whether a biomarker can be used as a diagnostic test, matching patients with the disease with healthy controls on the basis of age and sex could be important. Medications might also influence biomarker expression, and tests of biomarker validity should investigate whether the differences observed between patients and controls are the result of therapeutic intervention rather than being directly related to the disease itself. Thus, validation studies should include treatment-naive patients or demonstrate a similar correlation between the biomarker and the disease process in subsets of patients receiving different treatments. Alternatively a literature reference should be provided in the validation study report regarding associations between the potential biomarker and any medication. Accurate diagnostic biomarkers should also provide the opportunity to identify patients with the disease regardless of the level of disease activity, and not miss patients with inactive disease.⁴ This requirement makes it uncommon for a single biomarker to show validity as both an accurate diagnostic tool and a reliable measure of disease activity. Indeed, a key aspect of diagnostic tests is stability over time and under different clinical conditions, whereas the most important feature of bio markers of disease activity is their ability to reflect changes in clinical status. Longitudinal studies that investigate the same variables in the same individuals over a long period of time are, therefore, critical for the evaluation of bio-markers of disease activity and also prognosis. Important pre requisites of biomarker validation are the reliability and accuracy of biomarker testing under conditions experi enced in the clinic, feasibility of the measurement pro cedure, and reproduci bility.² Variations among centers in assay performance and standardization might contribute to divergent evaluation of biomarker validity.

We previously found that fewer than half of the translational research studies investigating potential bio-markers had study design features that would allow valid interpretation of their clinical utility.⁵ In this Review, we evaluate several promising new biomarkers of diag nosis, disease activity, and prognosis that have progressed beyond initial testing in patients with systemic auto-immune disease, and examine the validity of the clinical associations reported to date for these biomarkers. We chose to examine selected biomarkers that hold pro mise for clinical application, and used established considera tions of confounding and information bias as the basis for our evaluations.^2–6

Biomarkers for diagnosis

Diagnostic tests are judged by their ability to accurately distinguish individuals with the disease in question from those who do not have the disease. Proper testing of bio-markers as diagnostic tools requires that factors that could confound the association between the biomarker and disease status, such as age, are similar in the group of patients with the disease and the control population without the disease to which they are being compared.⁵ Although initial evaluations of diagnostic tests usually use healthy individuals as the unaffected controls, these comparisons typically overestimate the test's sensitivity and specificity.⁶ A more appropriate and realistic assessment of the clinical utility of a diagnostic test results from comparisons in which individuals with conditions that mimic or that could be confused with the disease in question are enrolled as controls. Study designs should also ensure that medications do not influence assessments of the effective ness of the diagnostic biomarker, as we have described. Ideally, the test should perform similarly in different subsets of patients, including those with recent-onset disease and longstanding disease, and in indivi duals with active disease or inactive disease. Fulfilling this requirement helps ensure that the bio-marker is indicative of the presence of the disease itself, and is not instead a measure of the stage of disease, organ damage, or disease activity. In the following sections we discuss these elements of study design in the context of studies conducted to assess the validity of two promising diagnostic biomarkers: agonistic antibodies to platelet-derived growth factor receptor (PDGFR) in systemic sclerosis (SSc), and serum pro calcitonin as a marker of infection in patients with autoimmune disease.

Agonistic antibodies to PDGFR in SSc

In 2006, Baroni and colleagues⁷ reported the presence of autoantibodies directed against the PDGFR in each of 46 patients with SSc but not in any of 20 healthy controls or 54 patients with either primary Raynaud phe nomenon, systemic lupus erythematosus (SLE), rheumatoid arthritis (RA), or idiopathic pulmonary fibrosis included in the study. These results indicate both a sensitivity and a specificity of 1.0 for this test. Importantly, the auto-antibodies associated with SSc were a distinct subset of agonistic antibodies, directed against a conformational epitope of PDGFR, and detected by their ability to induce fibroblasts to generate reactive oxygen species (ROS) in an in vitro bioassay.⁷ This test differs from simple detection of the level of auto antibodies to PDGFR (as achieved using an ELISA, for example), which would probably measure multiple auto antibodies subsets including those that are not PDGFR agonists and that might have different associations with SSc. Interestingly, generation of ROS is enhanced in patients with SSc, and is known to stimulate collagen production and fibroblast proliferation.⁸ Therefore, these agonistic autoantibodies might be pathogenic, and their discovery provides an intriguing link between the autoimmune and fibrotic aspects of SSc. The study by Baroni et al.⁷ fulfilled all the criteria for initial testing of bio marker validity (Table 1). Importantly, however, the same researchers reported that agonistic antibodies to PDGFR were also universally present in patients with extensive chronic graft-versus-host disease, a disease with features similar to SSc.⁹ This finding affects the specificity of this autoantibody as a diagnostic test for SSc.

Table 1.

Summary of validation studies of diagnostic testing for agonistic antibodies to PDGFR in SSc and serum procalcitonin in infection

Study	Number of cases	Sensitivity	Specificity	Controlled for				Disease controls included?	Accepted classification criteria used?	Studied patients with both early and late disease?	Included patients with active and inactive disease?
				Age	Gender	Race	Treatment
Agonistic anti-PDGFR antibodies in SSc
Baroni et al. (2006)7	46	1.00	1.00	Yes	Yes	Yes	Yes, (Patients were all treatment naive)	Yes	Yes	Yes (more patients with early than late disease were included)	Yes
Classen et al. (2009)10	37	ND	ND	No	No	No	No	Yes	Yes	No	No
Loizos et al. (2009)11	49	ND	ND	No	No	No	No	No	Yes	No	No
Serum procalcitonin in infection
Tamaki et al. (2008)24	29	0.53	0.97	Yes	Yes	Yes	Yes	Yes	Yes	NA	Yes
Scirè et al. (2006)25	19	0.73	0.89	No	No	No	Yes	Yes	Yes	NA	Yes

Abbreviations: NA, not applicable; ND, not detected; PDGFR, platelet-derived growth factor receptor; SSc, systemic sclerosis.

Two subsequent studies have failed to confirm the presence of stimulatory anti-PDGFR antibodies in 37 patients¹⁰ and 49 patients¹¹ with SSc, respectively. Com menting on the validity of these studies is difficult in the absence of detectable antibodies of interest (Table 1). The inability of these studies to replicate the findings of Baroni and co-workers might be the result of technical differences in the assays, which highlights the importance of reliable measurement techniques as a pre requisite for testing biomarker validity.¹² Two further studies reported the presence of autoantibodies against PDGFR in patients with SSc and controls, but the researchers did not specifically test for the subset of agonistic antibodies and, therefore, these data are not directly relevant to discussion of this biomarker.^13,14

Serum procalcitonin in bacterial infection

Procalcitonin, a precursor of the hormone calcitonin, is present in the serum of healthy individuals at concentrations of several picograms per milliliter;¹⁵ how ever, this concentration is increased several thousand-fold in the presence of certain neuroendocrine tumors and also in the setting of trauma, surgery, pneumonitis, pan creatitis, or bacterial infections. In bac terial infec tions, pro calcitonin is thought to be released by non-endocrine cells in multiple tissues in response to lipopoly saccharide and other inflammatory mediators.¹⁵ Measurement of procalcitonin levels as an early and sensitive diagnostic test for bacterial infection has been investigated in several studies, with a meta-analysis reporting a sensitivity of 0.88 and a specificity of 0.81 for discrimination of patients with bacterial infection from those with noninfectious causes of inflammation on the basis of elevated serum procalcitonin levels.¹⁶ The increases in serum procalcitonin levels surpassed those of C-reactive protein in the same studies.¹⁶ A second meta-analysis reported lower accuracy for elevated serum procalcitonin levels in the diagnosis of sepsis in the intensive care setting, with both a sensitivity and a specificity of 0.71.¹⁷

Fever and other constitutional symptoms are common to infection, active SLE and systemic vasculitis. Patients with SLE or systemic vasculitis are often treated with immunosuppressive medications, making infection a common concern; therefore, having an accurate diagnostic test that could quickly distinguish infection from noninfectious inflammation in patients with these diseases would help expedite treatment. Several studies have examined the accuracy of serum procalcitonin for the diagnosis of bacterial infections in patients with systemic autoimmune diseases. Eberhard and colleagues¹⁸ retrospectively compared serum procalcitonin levels between periods of infection and no infection in 35 patients with anti-neutrophil cytoplasmic antibody-associated vasculitis, and found elevated levels consistently during infection, but rarely during periods without infection. Importantly, elevated serum procalcitonin levels were associated with infection even in a subset of patients with active vasculitis; in this group, biomarker validity might have been expected to be confounded by elevated levels of procalcitonin related to vasculitis-associated inflammation. Assessment of the validity of these comparisons is difficult because the study analyzed multiple, rather than one, samples per patient, and, therefore, diagnostic performance characteristics could not be calculated.

The study by Eberhard et al.¹⁸ also included patients with SLE, but because no infections were observed in these patients, the performance of serum procalcitonin testing in this group could not be assessed. Nevertheless, the authors did report that procalcitonin levels were not associated with SLE activity, suggesting that elevated procalcitonin levels might be a reliable marker of infection in patients with this disease. However, two small studies in patients with SLE provided conflicting results. Shin et al.¹⁹ reported a statistically significant elevation in serum procalcitonin levels in 9 patients with SLE who were hospitalized with bacterial or fungal infection, compared with 7 patients with fever relating to SLE flares, 3 patients with SLE and viral infection, and 11 patients with inactive SLE, suggesting high sensitivity and specificity. By contrast, the second study by Lanoix et al.²⁰ demonstrated that 5 patients with bac terial or fungal infections had serum procalcitonin levels within the normal range, and these levels were similar to those detected in patients with active SLE. The type of infection and timing of assessment could account for the dif ferent findings reported, but the small number of patients included in both studies precludes drawing any definite conclusions regarding the utility of serum pro calcitonin in diagnosing infections in patients with SLE. Levels of serum procalcitonin have been reported to be elevated in patients with active granulo matosis with polyangiitis (GPA, formerly Wegener's granulomatosis), suggesting some limitation in the specificity of this test for diagnosis of infections in these patients, although these studies did not compare infected and uninfected states—only patients without infection were enrolled.^21–23

Two larger studies that examined patients with diverse inflammatory rheumatic diseases have provided additional useful information regarding the validity of procalcitonin as a diagnostic biomarker for infection in systemic autoimmune disease. In a study that compared 29 patients with bacterial infection with 70 patients with inflammatory disease flares, Tamaki et al.²⁴ reported that elevated serum procalcitonin levels had a sensiti vity of 0.53 and a specificity of 0.97 for infection, using a cut-off point of 0.5 ng/ml as the indicator of an elevated procalcitonin level. Furthermore, this study included all criteria for assessment of validity (Table 1), which enhances confidence in the conclusions drawn. A sensitivity of 0.73 and a specificity of 0.89 were calculated in the second study,²⁵ with the caveat that patients with adult-onset Still's disease had elevated levels of procalcitonin in the absence of infection,^25,26 and were excluded in this calculation. Elevated procalcitonin levels might still be diagnostic of infection in adult-onset Still's disease, but it has been proposed that a higher cut-off point of 1.4 ng/ml be adopted in patients with this disease.²⁶ This evidence suggests that the performance of serum procalcitonin measurement in the diagnosis of infection in patients with systemic autoimmune diseases is probably similar to that demonstrated in individuals without auto-immune disease, but larger studies are needed to provide a more definitive confirmation of validity.

Biomarkers for disease activity assessment

Researchers are engaged in a constant search for improved biomarkers that can accurately assess disease activity or predict future flares, and consequently guide therapeutic intervention in patients with autoimmune diseases.^27,28

In order to demonstrate validated clinical associations for these biomarkers, studies should report whether the patients with active and inactive disease enrolled were matched for age, gender and race (or whether statistical adjustment for these factors was performed), whether associations between the potential biomarker and treatment were examined, and whether the biomarker was assessed longitudinally.⁵ Cross-sectional studies compare the expression of biomarkers between patients with active and inactive disease, whereas longitudinal studies provide important and unique information regarding the change in biomarker expression (or absence of change) in relation to variation in disease activity. Longitudinal studies are thus necessary for testing biomarkers of disease activity because detection of change is their primary function in clinical practice. They might, for example, be used to monitor response to treatment or to predict whether a patient's disease is progressing, which might result in development of a particular complication. In addition to these study design criteria, validated disease activity measures should be used as standards to which the performance of the potential biomarker should be compared, and patients representing a wide range of disease activity should be evaluated to ensure that associations are not overlooked. Next we examine recent studies investigating the interferon (IFN) gene signature or cell-bound C4d as biomarkers of disease activity in SLE, in terms of these study design features.

The IFN pathway and SLE disease activity

The IFN gene signature

The type I IFN pathway is dysregulated in SLE and has been identified as a source of potential biomarkers of the disease.²⁸ Several studies examining gene expression in peripheral blood mononuclear cells collected from patients with SLE have found increased expression of genes induced by type I IFN (termed the ‘IFN gene signature’) compared with the levels detected in individuals with other diseases or healthy controls.^29,30 These findings have been extended in translational studies that aimed to test the potential of the IFN gene signature as a biomarker of disease activity in SLE.

In a study by Baechler and co-workers,³¹ the IFN gene signature was correlated with severe SLE manifestations such as renal, central nervous system and hematological involvement (Table 2). Four subsequent studies showed that the IFN gene signature correlated significantly with disease activity in patients with SLE.^32–35 These studies all used validated measures of disease activity and included a wide range of disease activity scores, in all but one the effects of treatment were controlled for,^32,33,35 half reported adjustment for age,^33,35 and one for gender (although studies in patients with SLE are ‘automatically’ controlled for gender to some extent because the vast majority of individuals with the disease are women).³³ All studies had a cross-sectional design, but one study also evaluated the expression of the candidate biomarker longitudinally in one patient (Table 2).³³

Table 2.

Validation studies of IFN gene signature as a potential biomarker for disease activity assessment in SLE

Study	Number of cases	Findings	Matching or adjustments	Controls for treatment effect	Validated disease-activity measures used	Study design	Included patients with a wide range of DAS?
Baechler et al. (2003)31	48	Expression levels of IFN-induced genes correlated with severe SLE manifestations	None	None	None	Cross-sectional	No
Bennett et al. (2003)32	30	Expression levels of IFN-induced genes correlated with disease activity	None	Controlled for corticosteroid treatment	SLEDAI	Cross-sectional	Yes
Kirou et al. (2005)33	77	Expression levels of IFN-induced genes correlated with disease activity	Matched for age and gender	Controlled for hydroxychloroquine treatment	SLEDAI-2K	Cross-sectional Longitudinal follow-up in 1 patient	Yes
Feng et al. (2006)34	48	Expression levels of IFN-induced genes correlated with disease activity	None	None	SELENA-SLEDAI	Cross-sectional	Yes
Nikpour et al. (2007)35	269	Expression levels of IFN-induced genes correlated with disease activity	Adjusted for age	Controlled for prednisone treatment	SLEDAI-2K	Cross-sectional	Yes
Landolt-Marticorena et al. (2008)36	94	Expression levels of IFN-induced genes linked to disease flares Expression levels of IFN-induced genes correlation poorly with longitudinal changes in disease activity	None	Controlled for use immunosuppressive agents	SLEDAI-2K	Cross-sectional Longitudinal (1 year) for 27 patients	Yes

Abbreviations: DAS, disease activity scores; IFN, interferon; SELENA-SLEDAI, Safety of Estrogens in Lupus Erythematosus: National Assessment (SELENA)-Systemic Lupus Erythematosus (SLE) Disease Activity Index (SLEDAI); SLEDAI-2K, SLEDAI 2000.

Landolt-Marticorena et al.³⁶ described an association between the expression levels of IFN-inducible genes and SLE flares, but poor correlation with longitudinal changes in disease activity during 1 year of follow-up. Landolt-Marticorena and colleagues incorporated all but one (matching of demographic characteristics) of the design features important for the proper evaluation of potential biomarkers of disease activity into their study (Table 2). Longitudinal follow-up of the expression levels of IFN-inducible genes in larger studies is warranted to determine their utility as measures of disease activity in SLE.

IFN-regulated chemokines

In addition to the IFN gene signature, elevated serum levels of IFN-regulated chemokines have been associated with SLE disease activity in a number of studies. These chemokines are a group of soluble mediators that promote recruitment of leukocytes to target tissues and whose synthesis by neutrophils, macrophages and other immune cells is induced by IFN. Increased serum levels of IFN-regulated chemokines^37,38 and IFN-inducible chemokine gene expression scores³⁹ have been identified in patients with SLE compared with healthy and disease control groups and correlated with SLE activity.

Narumi et al.⁴⁰ reported that serum levels of 10 kDa IFNγ-inducible protein (IP-10, also known as CXCL10) correlated with anti-double stranded DNA (dsDNA) antibody and complement levels (Table 3). No validated disease activity indices were used in this cross-sectional study, however, patients with a wide range of anti-dsDNA antibody or complement levels were included, and patients with active SLE and inactive SLE were matched for age, gender, race and prednisone dosage.⁴⁰ In all but one of five subsequent studies (Table 3), serum levels of several IFN-regulated chemokines correlated with disease activity scores and clinical test results (erythrocyte sedimentation rate [ESR], hypocomplementemia, anti-dsDNA anti bodies or leukopenia).^{38,39,41–43} Bauer et al.⁴¹ also showed that a chemokine score—an integrated index of the serum levels of several chemokines—was more closely correlated with disease activity, as measured by SLE Disease Activity Index (SLEDAI), Systemic Lupus Activity Measure-Revised (SLAM-R), ESR, and anti-dsDNA antibodies levels, than the IFN gene expression score—calculated based on 82 IFN-inducible transcripts. Four of the five studies had a cross-sectional design,^38,39,41,42 four assessed patients with a wide range of disease activity,^38,39,41,43 three controlled for treatment effects,^39,42,43 and two studies included patients with active and inactive disease who were matched for age and gender.^38,43 To validate the potential utility of these chemokines as biomarkers of disease activity, Bauer and co-workers⁴³ followed the serum levels of three IFN-regulated chemokines in 267 patients with SLE longitudinally for 1 year. Serum chemokine levels correlated with the SLEDAI, and changes in the SLEDAI were accompanied by considerable changes in chemokine levels. Interestingly, IFN-regulated chemo kine levels also seemed to predict future disease flares. This study fulfilled all cri teria for testing the validity of biomarkers for disease activity assessment and, therefore, suggests that the IFN-regulated chemokine score is valid for this purpose.

Table 3.

Validation studies of IFN-regulated chemokines as potential biomarkers for disease activity assessment in SLE

Study	Number of cases	Findings	Matching or adjustments	Controls for treatment effect	Validated disease-activity measures used	Study design	Included patients with a wide range of DAS?
Narumi et al. (2000)40	28	Serum IP-10 levels correlated with anti-DNA antibody and complement levels	Matched for age and gender	Controlled for treatment	Anti-DNA antibody and C3 levels	Cross-sectional	Yes
Lit et al. (2006)38	80	IFN-regulated chemokine levels correlated with disease activity	Matched for age, gender and race	None	SLEDAI	Cross-sectional	Yes
Bauer et al. (2006)41	30	IFN-regulated chemokine levels correlated with disease activity measures	None	None	SLEDAI SLAM-R	Cross-sectional	Yes
Vilá et al. (2007)42	62	No association between CCL3, CCL4, and CCL5 levels and disease activity scores	None	Controlled for pharmacological treatments	SLAM	Cross-sectional	Not stated
Fu et al. (2008)39	67	Chemokine score* correlated with SLEDAI-2K and C3 levels	None	Controlled for hydroxychloroquine, prednisone and immunosupressive agents	SLEDAI-2K	Cross-sectional	Yes
Bauer et al. (2009)43	267	CCL2, CCL19 and IP-10 levels as well as a chemokine score* correlated with current disease activity and were predictive of future disease flares	Matched for age, gender and race	Controlled for prednisone and hydroxychloroquine treatment and use immunosupressive agents	SLEDAI	Longitudinal (1 year)	Yes

^*A chemokine score is an integrated index of the serum levels of several chemokines. Abbreviations: CCL, CC-chemokine ligand; IFN, interferon; IP-10, 10 kDa IFNγ-induced protein; SLE, systemic lupus erythematosus; SLEDAI, SLE Disease Activity Index; SLAM, Systemic Lupus Activity Measure; SLAM-R, SLAM-Revised; SLEDAI-2K, SLEDAI 2000.

Erythrocyte-bound or reticulocyte-bound C4d

Measurement of serum C3 and C4 complement proteins has been used for decades to monitor SLE disease activity. Early studies suggested that elevated levels of complement cleavage products, which are produced upon activation of the complement cascade, reflect dis ease activity more accurately than conventional measure ments of complement, and indicated that these complement fragments might be useful in the prediction of impending disease flares.⁴⁴ The findings of subsequent studies of soluble and cell-bound complement activation products hold promise for achieving this goal.⁴⁵

C4d a cleavage product of C4, and is stable and readily detectable covalently bound to various cells, mainly erythrocytes, but also reticulocytes and platelets. Erythrocyte-bound C4d has been proposed as a sensitive diagnostic marker for SLE that might also serve as a biomarker of disease activity.⁴⁶ Manzi et al.⁴⁷ found that patients with SLE had higher levels of erythrocyte-bound C4d than patients with other diseases or healthy controls. This group also demonstrated that erythrocyte-bound C4d levels examined on different days in the same patient varied considerably, suggesting that these changes might reflect fluctuations in disease activity.⁴⁷

In the time since erythrocyte-bound C4d was first recog nized as a biomarker in SLE, Singh and colleagues⁴⁸ found that erythrocyte C4d had a low but posi tive correlation with the Safety of Estrogens in Lupus Erythematosus: National Assessment (SELENA) ver sion of SLEDAI (SELENA-SLEDAI); however, the assay method used might have differed from that used in other studies (Table 4). Moreover, Yang et al.⁴⁹ showed that erythrocyte-bound C4d levels were correlated with the SELENA-SLEDAI in a subgroup of patients without hemolytic anemia but not in those with hemolytic anemia. These studies had cross-sectional designs and did not provide any information about matching for age and gender, or controlling for treatment (Table 4).

Table 4.

Studies of erythrocyte-bound or reticulocyte-bound C4d as biomarkers for disease activity assessment in SLE

Study	Number of cases	Findings	Matching or adjustments	Controls for treatment effect	Validated disease-activity measures used	Study design	Included patients with a wide range of DAS?
Singh et al. (2008)48	111	Erythrocyte-bound C4d levels associated with disease activity but with low correlation	None	None	SELENA-SLEDAI	Cross-sectional	Not stated
Yang et al. (2009)49	63	Erythrocyte-bound C4d levels correlated with disease activity	None	None	SLEDAI	Cross-sectional	No (SLEDAI score 6 ± 0.52)
Kao et al. (2010)50	157	Erythrocyte-bound C4d levels correlated with disease activity	None	None	SLAM SELENA-SLEDAI	Longitudinal (5 years)	Yes
Liu et al. (2005)51	156	Erythrocyte-bound C4d and reticulocyte-bound C4d levels correlated with disease activity	None	None	SLAM SELENA-SLEDAI	Cross-sectional Longitudinal	Yes

Abbreviations: SELENA-SLEDAI, Safety of Estrogens in Lupus Erythematosus: National Assessment (SELENA)-Systemic Lupus Erythematosus (SLE) Disease Activity Index (SLEDAI); SLAM, Systemic Lupus Activity Measure.

In a study published in 2010, erythrocyte-bound C4d levels were observed to be higher in patients with ‘more active’ and ‘most active’ SLE compared with those with less active disease.⁵⁰ In addition to cross-sectional analysis, this study demonstrated, using longitudinal linear mixed-effects model analysis of disease activity in 156 patients over 5 years, that erythrocyte-bound C4d measure ments were associated with the SLAM and SELENA-SLEDAI, even after adjusting for serum levels of C3, C4 and anti-dsDNA antibodies.⁵⁰

In another study that included both cross-sectional and longitudinal arms, erythrocyte-bound C4d was investigated as a potential biomarker for diagnosis of SLE, but with more focus placed on C4d bound to immature erythro cytes called reticulocytes (Table 4).⁵¹ Reticulocytes are a short-lived cell type that circulate in the blood stream for only 0–2 days, and, therefore, C4d bound to these cells might provide a better snapshot of complement activation than erythrocyte-bound C4d. Patients with reticulocyte-bound C4d levels in the highest quartile, compared with those in the lowest quartile, had notably higher SELENA-SLEDAI and SLAM scores in the cross-sectional comparison.⁵¹ When followed over time in the longitudinal arm of the study, reticulocyte-bound C4d levels seemed to change promptly with the clinical course in individual patients.⁵¹ In addition to the investigations involving erythrocyte-bound and reticulocyte-bound C4d, a cross-sectional study by Navratil et al.⁵² demonstrated that C4d can also be deposited on human platelets, and that this deposition is highly specific for SLE and correlated with the SLEDAI.

Although the biomarkers for assessing disease activity in SLE that we have described are promising, they still need to be tested further in long-term studies. Valid interpretation of their clinical associations also depends on control or adjustment for demographic characteristics or treatment effect, which has often been omitted from studies performed to date.

Biomarkers for predicting prognosis

In addition to controlling or adjusting for demographic and clinical characteristics that might confound the associ ation between the biomarker and disease outcome, studies of biomarkers for prognosis should use validated outcome measures and test the predictive ability of the biomarker in a longitudinal fashion.⁵ Prognostic bio-markers are most valuable if they are so tightly linked to the outcome that they can act as surrogate measures of it in tests of medication efficacy or treatment response. Here we examine studies of two biomarkers of bone and cartilage turnover, collagen C-telopeptides I and II, as predictors of structural damage in patients with RA.

Collagen C-terminal crosslinked telopeptides

C-terminal crosslinked telopeptide of type I collagen (CTX-I), a carboxy-terminal peptide fragment of collagen type I, is released during bone resorption and is detectable at quantifiable levels in serum or urine.⁵³ CTX-I is not a specific marker of bone degradation, but because bone represents the largest repository of type I collagen, CTX-I levels principally reflect bone catabolism. Bone erosions and osteoporosis both occur as consequences of RA; therefore, CTX-I levels have been examined as prognostic biomarkers in patients with RA. However, joint involvement in RA might be more directly reflected by cartilage damage; thus, C-terminal crosslinked telo peptide of type II collagen (CTX-II), which is a biomarker of breakdown of calcified articular cartilage,⁵⁴ could represent a more useful prognostic biomarker in patients with RA.

Most studies of the prognostic value of CTX-I for radio-graphic damage in RA were conducted longitudinally and used validated outcome measures (Table 5).^55–62 The weight of evidence suggests little associ ation between CTX-I levels and progression of radiographic damage, although differences might be evident after extended follow-up, or in a subset of patients with early RA and no baseline joint damage. By contrast, increased CTX-II levels have been consistently associated with more rapid progression of joint damage, particularly in studies that monitored changes over 2 years or more (Table 6).^{56,57,59,62–69} Trials in which Larsen scores were used seem less likely to demonstrate associ ations between CTX-II and joint damage than those using modified Sharp scores, which probably reflects better sensitivity to change of the modified Sharp score. Most studies of both CTX-I and CTX-II examined and adjusted for potential confounding factors, although adjustment for treatment effects was less comprehensive. In doing so, these studies appropriately emphasized that to be clinically useful, new prognostic biomarkers should supply information beyond that provided by risk factors already available and currently used in the prediction of disease outcomes.⁷⁰ Despite being conducted before the release of the latest expert panel recom mendations for assessment of biomarkers for damage endpoints,⁷¹ the studies of CTX-I and CTX-II listed in Table 5 and Table 6 largely met the recommendations subsequently made for patient enrollment, study design, consideration of treatment, measurement of RA dis ease activity, and duration.However, sample handling and assay fidelity were i dentified as areas needing further investigation.⁷⁰

Table 5.

Validation studies of CTX-I as a prognostic biomarker for structural damage in RA

Study	Number of cases	Findings	Adjustments	Controls for treatment effect	Validated outcome measure used	Study design
Garnero et al. (1999)55	318	CTX-I levels were higher in patients with destructive arthritis	Age, gender, menopausal status	Controlled for corticosteroid and methotrexate use	Larsen wrist score	Cross-sectional
Garnero et al. (2002)56	110	Higher levels of CTX-I predicted more rapid progression, but only in those without damage at baseline	Baseline modified Sharp score, ESR, DAS28, seropositivity	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (median 4 years)
Landewé et al. (2004)57	110	Baseline CTX-I levels and early changes in CTX-I levels were not associated with change in damage	Baseline modified Sharp score, ESR, DAS28, seropositivity	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (median 4 years)
Jansen et al. (2004)58	279	Higher levels of CTX-I were not associated with change in damage	Age, ESR, DAS28, seropositivity, physical function	None	Modified Sharp score	Longitudinal (2 years)
Forsblad d'Elia et al. (2004)59	88	Change in CTX-I levels over 24 months was not correlated with change in damage score	None	None	Larsen score	Longitudinal (2 years)
Syversen et al. (2009)60	238	Higher levels of CTX-I were weakly associated with change in damage score	Baseline modified Sharp score, CRP, seropositivity	None	Modified Sharp score	Longitudinal (10 years)
Wislowska et al. (2009)61	50	Levels of CTX-I were lower in patients with more joint damage	Age, gender, height, body mass index, joint counts, ESR, CRP	Controlled for corticosteroid use	Larsen score	Cross-sectional
Van Tuyl et al. (2010)62	67	Higher baseline levels of CTX-I were associated with more rapid progression	Baseline modified Sharp score, ESR, seropositivity, RANKL:osteoprotegerin ratio	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (11 years)

Abbreviations: CTX-I, C-terminal crosslinked telopeptide of type I collagen; CRP, C-reactive protein; DAS28, disease activity score in 28 joints; ESR, erythrocyte sedimentation rate; RA, rheumatoid arthritis; RANKL, receptor activator of nuclear factor κB ligand.

Table 6.

Validation studies of CTX-II as a prognostic biomarker for structural damage in RA

Study	Number of cases	Findings	Adjustments	Controls for treatment effect	Validated outcome measure used	Study design
Christgau et al. (2001)63	27	CTX-II levels were higher in patients with destructive arthritis	None	None	Not stated	Cross-sectional
Garnero et al. (2002)64	116	CTX-II was no better than CRP in predicting progression	Baseline modified Sharp score, CRP	Treatment group (etanercept or methotrexate)	Modified Sharp score	Longitudinal over 1 year
Garnero et al. (2002)56	110	Higher levels of CTX-II predicted more rapid progression, but only in those without damage at baseline	Baseline modified Sharp score, ESR, DAS28, seropositivity	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (median 4 years)
Landewé et al. (2004)57	110	Baseline CTX-II levels and early changes in CTX-II levels predicted change in damage	Baseline modified Sharp score, ESR, DAS28, seropositivity	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (median 4 years)
Forsblad d'Elia et al. (2004)59	88	Change in CTX-II levels over 24 months was not correlated with change in damage score	None	None	Larsen score	Longitudinal (2 years)
Young-Min et al. (2007)65	118	Higher baseline CTX-II and in-course CTX-II levels predicted more rapid progression	Joint count, physician global, DAS, ESR, CRP	Controlled for methotrexate use	Larsen score	Longitudinal (2 years)
Marotte et al. (2009)66	66	CTX-II levels were not associated with greater change in damage score	None	Controlled for Infliximab use and methotrexate use	Modified Sharp score	Longitudinal (1 year)
Hashimoto et al. (2009)67	145	Higher baseline CTX-II levels predicted more rapid progression	Baseline modified Sharp score, body mass index	None	Modified Sharp score	Longitudinal (1 year)
Christensen et al. (2009)68	45	CTX-II levels were not associated with more rapid progression	Age, gender, erosions at baseline, CRP, seropositivity	Controlled for methotrexate use	Larsen score	Longitudinal (1 year)
Christensen et al. (2010)69	133	Higher baseline CTX-II levels predicted more rapid progression	Age, gender, seropositivity, baseline modified Sharp score, DAS28	Controlled for methotrexate use, cyclosporin use, and corticosteroid use	Modified Sharp score	Longitudinal (4 years)
van Tuyl et al. (2010)62	73	Higher levels of CTX-II at 3 months (but not baseline) were associated with more rapid progression	Baseline modified Sharp score, ESR, seropositivity, RANKL:osteoprotegerin ratio	Adjusted for treatment group (prednisolone with methotrexate and sulfasalazine, or sulfasalazine only)	Modified Sharp score	Longitudinal (11 years)

Abbreviations: CTX-II, C-terminal crosslinked telopeptide of type II collagen; CRP, C-reactive protein; DAS, disease activity score; DAS28, DAS in 28 joints; ESR, erythrocyte sedimentation rate; RA, rheumatoid arthritis; RANKL, receptor activator of nuclear factor-κB ligand.

Conclusions

Biomarker validation is the last stage in a long process of discovery, testing, refinement, and evaluation, but validation is itself an ongoing process. Each assessment in which a biomarker performs as predicted represents an additional validation. In studies of the six promising biomarkers discussed in this Review, we found mixed results concerning the extent to which potential threats to the validity of clinical associations were attended to. For studies of biomarkers for diagnosis, greater attention to potential confounding factors would help ensure accuracy. When examining biomarkers of disease activity, more emphasis should be placed on evaluating these markers longitudinally. In investigations pertaining to biomarkers of prognosis, longitudinal design and the use of validated outcome measures provide results in which we can have confidence. At present, each of the examined bio markers remains a research tool. An important criteria for all categories of biomarkers is how their performance compares to that of tests currently used in clinical practice. The value of any new biomarker is best shown by how well it outperforms currently available tests.

Key points.

• ■ Biomarkers are important for making informed decisions in the clinic, including those concerning diagnosis and allocation of treatment, as well as for assessing disease activity and prognosis

• ■ The clinical usefulness of any biomarker depends on the demonstration of its validity for a particular purpose

• ■ In the validation of biomarkers intended to aid diagnosis, greater attention needs to be placed on controlling or adjusting for the demographic and clinical characteristics of the disease being studied

• ■ Longitudinal studies are essential in the assessment of validity for biomarkers relating to disease activity and prognosis

• ■ New biomarkers of the greatest value are those that provide information that cannot be gained from existing tests

Review criteria.

We searched for original full-text articles published in English from 1966 through June 1^st 2011 using PubMed. Study selection was based on our previously used inclusion and exclusion criteria.⁵ The search terms used were: “platelet-derived growth factor receptor antibody” AND “scleroderma [OR systemic sclerosis]”; “serum procalcitonin” AND “systemic lupus erythematosus [OR vasculitis]”; “interferon gene expression [OR interferon gene signature]” AND “systemic lupus erythematosus” AND “disease activity”; “interferon-related chemokines” AND “systemic lupus erythematosus” AND “disease activity”; “erythrocyte-bound C4d [OR reticulocyte- bound C4d]” AND “systemic lupus erythematosus” AND “disease activity”; and “collagen telopeptide [OR CTX-I OR CTX-II] AND “rheumatoid arthritis.” We also included relevant articles, selected based on the authors’ knowledge of the literature.