Skip to main content
Drug Design, Development and Therapy logoLink to Drug Design, Development and Therapy
. 2018 Sep 6;12:2757–2773. doi: 10.2147/DDDT.S173671

Are laboratory parameter (biomarker) values similar to the healthy volunteer reference range in all patient populations?

David A Brott 1,, Michael J Goodman 1, Richard P Hermann 1, Michael Merz 2, Roser Calvo 1, Nadereh Poorkhalkali 3, Alexandre Kiazand 1
PMCID: PMC6132491  PMID: 30233139

Abstract

Background

Liver biomarkers alanine aminotransferase (ALT) and bilirubin in patients with hepatitis are above the healthy volunteer reference range (HVRR) at baseline (prior to receiving the clinical trial medication). Discussions continue as how to best distinguish drug-induced liver injury in patients with abnormal baseline values participating in clinical trials. This study investigated if other baseline routine clinical safety biomarkers (lab parameters) are different from the HVRR.

Materials and methods

Clinical trial data (TransCelerate dataset) from placebo and standard of care treated patients were compared to the HVRR using a 10% threshold above or below the HVRR to classify a lab parameter in a patient population as potentially different from the HVRR at baseline. The TransCelerate dataset, batch 4, contained data from patients with Alzheimer’s, asthma, COPD, cardiovascular disease, diabetes, hidradenitis, hypercholesterolemia, rheumatoid arthritis, schizophrenia, stroke, and ulcerative colitis. A subset of the 200 biomarkers in Trans-Celerate were evaluated in this pilot: glucose, platelet count, neutrophil count, ALT, aspartate aminotransferase (AST), and bilirubin.

Results

Glucose was potentially higher than the HVRR in patients with diabetes, COPD, cardiovascular disease, hypercholesterolemia, and schizophrenia. At least one or more of the hematology and hepatic biomarkers were different from the HVRR in at least one patient population, except bilirubin. All the patient populations, except Alzheimer’s and asthma, had at least one biomarker that was higher than the HVRR.

Summary

The routine biomarkers evaluated in this pilot study demonstrated that not all lab parameters in patient populations are similar to the HVRR. Further efforts are needed to determine which biomarkers are different from the HVRR and how to evaluate the biomarkers in patient populations for detecting drug-induced altered lab values in clinical trials.

Keywords: patient populations, reference range, healthy volunteer, biomarkers

Introduction

Clinical safety signals (adverse drug reaction: ADR) in drug development are typically recorded using Medical Dictionary for Regulatory Activities terms and a severity score based on Common Terminology Criteria for Adverse Events (CTCAE) grade or a similar scoring method. The CTCAE grades for many routine lab parameters are based on the healthy volunteer reference range (HVRR) at the site doing the sample analysis. For example, severity scores for the liver biomarker alanine aminotransferase (ALT) are based on the upper limit of normal (ULN) of the reference range with grade one being >1–3 times ULN, grade two being >3–5 times ULN, grade three being >5–20 times ULN, and grade four being >20 times ULN. The HVRR is calculated so that 95% of all healthy volunteers are within the range and that only 2.5% of the subjects would have values higher than the ULN and 2.5% of the subjects would have values lower than the lower limits of normal (LLN). However, it is known that most patients with hepatitis C, liver metastases, or nonalcoholic steatohepatitis (NASH) have ALT levels above the ULN at baseline (prior to receiving the clinical trial medication).14 These findings have led to the current controversy of how to detect drug-induced liver injury (DILI) in clinical trials for patients with elevated ALT levels at baseline.5

Determining that baseline ALT values in patients with hepatitis C are different from the healthy volunteer reference range was relatively straightforward as less than 1% of the patients had ALT values less than or equal to the ULN while more than 33% of the patients had ALT levels greater than three times the ULN.4 Given that the biopharmaceutical industry develops drugs for a wide range of indications, it is vital to ensure that the data collected during clinical development are utilized and interpreted appropriately in order to correctly identify patients with ADRs and adequately accounting for patients with high values due to the disease or disease progression. A large dataset would be needed to determine if a patient population(s) has a different baseline reference range than the HVRR currently used in clinical trials. Real world datasets such as Truven MarketScan contain large quantities of demographic and health care utilization data but very little routine lab data and therefore are not applicable for this effort. TransCelerate BioPharma Inc is a non-profit organization with a mission to collaborate across the biopharmaceutical research and development community to identify, prioritize, design and facilitate the implementation of solutions for delivery of new medicines. One workstream initiated by this consortium focuses on the formation of a placebo and standard of care (PSoC) database containing routine safety lab data from clinical trials across the biopharmaceutical industry. Batch 4 of the dataset contains lab data from more than 20,000 subjects spread across multiple disease populations. In this pilot study, the dataset was used to determine if routine safety lab parameters were consistently similar to the HVRR or if, similar to ALT in hepatitis patients, there were lab parameters in a patient population that are different from the HVRR.

Materials and methods

Dataset

Data were selected from batch 4 of the TransCelerate BioPharma PSoC dataset, which is designed to act as a historical control to facilitate development of new drug products. A full description of PSoC is contained in a White Paper published by the collaborative.6 Briefly, contributing companies provide data from the PSoC control arms of completed studies in standard format. Member companies of the consortia have access to the dataset, in which all subjects are deidentified. Internal review board or ethics committee review was done by each individual company before initiation of their clinical trial. At present, each collaborator holds a copy of the data (it is not stored on the cloud nor freely accessible to the scientific community).

Therefore, the PSoC database incorporates a large volume of Good Clinical Practices (GCP)-compliant clinical trial data (comprised of both placebo and standard of care) to facilitate the development of innovative drug products.

Baseline date was defined as the minimum laboratory day (lbdy) in a study data tabulation model (SDTM). If an individual had more than one value for a specific test on the baseline date, one of the values was randomly selected by assigning a random number from a uniform distribution and choosing the minimum random number. Only a single baseline value from each subject was used in this study, even though many subjects had longitudinal samples.

There are more than 200 different lab parameters within the TransCelerate dataset but only six lab parameters were evaluated to determine if lab parameters in PSoC patient populations were similar to the HVRR. The lab parameters selected for analysis in this study are routinely used in most clinical trials across patient populations: glucose, platelets, neutrophils, ALT, aspartate aminotransferase (AST), and bilirubin.

Data analysis

Reference ranges may be different within and between studies as they are determined at the lab doing the sample analysis. To eliminate the influence of different reference ranges, all lab values were normalized to the ULN by dividing the value by the ULN and were normalized to the lower limit of normal (LLN) by dividing by the LLN, on an individual sample basis.

All data processing was done in SAS Enterprise Guide Version 7.15 on the AZ SAS Grid installation. Data for this paper comes from two SDTM tables: LB (laboratory) and DM (demographics).

Box plots were generated using SAS. The horizontal axis was the condition. The vertical axis was the lab value normalized to ULN. The box plots contain the mean, median, interquartile range, upper and lower 95% CI and outliers. A lab parameter in a patient population was deemed potentially different from the HVRR if greater than 10% of the lab values from the patient population were above the ULN or below the LLN. The 10% threshold is four times higher than the expected percentage (2.5%), based on the definition of HVRR and was selected to minimize the risk of false positives (classifying a lab parameter as different from the HVRR).

Results

Studies

There are 82 clinical trials in the TransCelerate database, batch 4. The vaccine patient population consisting of one clinical trial did not have any lab data and so this patient population could not be evaluated in this study. The remaining 81 clinical trials are divided amongst 11 different patient populations: Alzheimer’s, asthma, COPD, cardiovascular disease, diabetes, hidradenitis, hypercholesterolemia, rheumatoid arthritis, schizophrenia, stroke and ulcerative colitis (Table 1).

Table 1.

Studies in TransCelerate (batch 4)

Patient pop (TransCelerate) Study patient population Study ID Subject number Title
Alzheimer’s Alzheimer’s M11-793 146 A randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of ABT-126 in subjects with mild to moderate Alzheimer’s disease on stable doses of acetylcholinesterase inhibitors
Alzheimer’s Alzheimer’s M10-985 180 A randomized, double-blind, placebo and active-controlled study to evaluate the efficacy and safety of ABT-126 in subjects with mild to moderate Alzheimer’s disease
Alzheimer’s Alzheimer’s M10-984 136 Randomized, double-blind, placebo and active-controlled, parallel group study to evaluate the efficacy and safety of ABT-126 in subjects with mild to moderate Alzheimer’s disease
Alzheimer’s Alzheimer’s CN156013 42 Multicenter, randomized, double-blind, placebo-controlled study of the safety, tolerability, pharmacodynamic and pharmacokinetic effects of BMS-708163 in the treatment of patients with mild to moderate Alzheimer’s disease
Alzheimer’s Alzheimer’s H6LMCLFAN 501 Effect of gamma-secretase inhibition on the progression of Alzheimer’s disease: LY450139 vs placebo identity: interrupting Alzheimer’s dementia by evaluating treatment of amyloid pathology
Alzheimer’s Alzheimer’s H6LMCLFBC 555 Effect of LY450139 a g-secretase inhibitor, on the progression of Alzheimer’s disease as compared with placebo
Alzheimer’s Alzheimer’s M10822 123 A randomized, double-blind, active- and placebo-controlled study to evaluate the efficacy and safety of ABT-288 in subjects with mild-to-moderate Alzheimer’s disease
Alzheimer’s Alzheimer’s M12033 132 A randomized, double-blind, placebo- and active-controlled, parallel group study to evaluate the efficacy and safety of ABT- 384 in subjects with mild-to-moderate Alzheimer’s disease
Alzheimer’s Alzheimer’s MEM-MD-50 335 A randomized, double-blind, placebo-controlled evaluation of the safety and efficacy of memantine in patients with moderate-to-severe dementia of the Alzheimer’s type
Alzheimer’s Alzheimer’s MEM-MD-71 129 A randomized, double-blind, placebo-controlled evaluation of the efficacy of memantine on functional communication in patients with Alzheimer’s disease
Asthma Asthma FFA109685 207 A randomized study to evaluate the efficacy and safety of an investigational drug in adolescent and adult subjects with asthma uncontrolled on low-dose ICS therapy
Asthma Asthma HZA106829 194 Efficacy/safety study of fluticasone furoate/vilanterol (GW642444) in adult and adolescent asthmatics
Asthma Asthma HZA106839 98 Randomized, double-blind, double dummy, active comparator, parallel group, multicenter study to evaluate the safety of once-daily fluticasone furoate/GW642444 inhalation powder for 52 weeks in adolescent and adult subjects with asthma
Asthma Asthma HZA113714 152 Evaluating the efficacy and safety of fluticasone furoate/vilanterol trifenatate in the treatment of asthma in adolescent and adult subjects of Asian ancestry
Asthma Asthma 2050418 269 A phase III, randomized, double-blind, placebo-controlled, parallel-group trial to evaluate efficacy and safety of tiotropium inhalation solution delivered via Respimat inhaler (2.5 and 5 mcg once daily) compared with placebo and salmeterol HFA MDI (50 mg twice daily) over 24 weeks in moderate persistent asthma
Asthma Asthma 2050419 254 A phase III randomized, double-blind, placebo-controlled, parallel-group trial to evaluate efficacy and safety of tiotropium inhalation solution delivered via Respimat inhaler (2.5 and 5 mcg once daily) compared with placebo and salmeterol HFA MDI (50 mcg twice daily) over 24 weeks in moderate persistent asthma
Asthma Asthma HZA113091 401 A randomized, double-blind, double-dummy, parallel-group multicentre study to assess efficacy and safety of fluticasone furoate/GW642444 inhalation powder and fluticasone propionate/salmeterol inhalation powder in the treatment of persistent asthma in adults and adolescents
COPD COPD DB2113359 108 A 52-week, multicenter, randomized, double-blind, parallel-group, placebo-controlled study to evaluate the safety and tolerability of GSK573719/GW642444 and GSK573719 in subjects with chronic obstructive pulmonary disease (COPD) (COPD nDPI)
COPD COPD 2050235 3,006 A randomized, double-blind, placebo-controlled, parallel group trial assessing the rate of decline of lung function with tiotropium 18 mcg inhalation capsule once daily in patients with chronic obstructive pulmonary disease (COPD) UPLIFT (understanding potential long-term impacts on function with tiotropium) study
COPD COPD 12220011 210 Randomized, double-blind, placebo-controlled, parallel group study to assess the efficacy and safety of 48 weeks of once daily treatment of orally inhaled BI 1744 CL (five mcg [2 actuations of 2.5 mcg] and 10 mcg [2 actuations of 5 mcg]) delivered by the Respimat inhaler, in patients with chronic obstructive pulmonary disease (COPD)
COPD COPD 12220012 216 Randomized, double-blind, placebo-controlled, parallel group study to assess the efficacy and safety of 48 weeks of once daily treatment of orally inhaled BI 1744 CL (5 mcg [2 actuations of 2.5 mcg] and 10 mcg [2 actuations of 5 mcg]) delivered by the Respimat inhaler, in patients with chronic obstructive pulmonary disease (COPD)
COPD COPD 12220013 225 A randomized, double-blind, double-dummy, placebo-controlled, parallel group study to assess the efficacy and safety of 48 weeks of once daily treatment of orally inhaled BI 1744 CL (5 µg [2 actuations of 2.5 µg] and 10 µg [2 actuations of 5 µg]) delivered by the Respimat inhaler, and 48 weeks of twice daily foradil (12 µg) delivered by the aerolizer inhaler, in patients with chronic obstructive pulmonary disease (COPD)
COPD COPD 12220014 235 A randomized, double-blind, double-dummy, placebo-controlled, parallel group study to assess the efficacy and safety of 48 weeks of once daily treatment of orally inhaled BI 1744 CL (5 µg [2 actuations of 2.5 µg] and 10 µg [2 actuations of 5 µg]) delivered by the Respimat inhaler, and 48 weeks of twice daily foradil (12 µg) delivered by the aerolizer inhaler, in patients with chronic obstructive pulmonary disease (COPD)
COPD COPD 12220051 565 A randomized, double-blind, parallel group study to assess the efficacy and safety of 12 weeks of once daily, orally inhaled, co-administration of olodaterol 5 mcg (delivered by the Respimat inhaler) and tiotropium 18 mcg (delivered by the HandiHaler) compared to once daily, orally inhaled, co-administration of placebo (delivered by the Respimat inhaler) and tiotropium 18 mcg (delivered by the HandiHaler) in patients with chronic obstructive pulmonary disease (COPD) [ANHELTO TM 1]
COPD COPD 12220052 569 A randomized, double-blind, parallel-group study to assess the efficacy and safety of 12 weeks of once daily, orally inhaled, co-administration of olodaterol 5 mcg (delivered by the Respimat inhaler) and tiotropium 18 mcg (delivered by the HandiHaler) compared to once daily, orally inhaled, co-administration of placebo (delivered by the Respimat inhaler) and tiotropium 18 mcg (delivered by the HandiHaler) in patients with chronic obstructive pulmonary disease (COPD) [ANHELTO TM 2]
COPD COPD DB2113361 273 A 24-week, randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of GSK573719/GW642444 inhalation powder and the individual components delivered once-daily via a novel dry powder inhaler in subjects with COPD
COPD COPD DB2113373 279 A 24-week, randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of GSK573719/GW642444 inhalation powder and the individual components delivered once-daily via a novel dry powder inhaler in subjects with COPD
COPD COPD HZC112206 206 A 24-week study to evaluate the efficacy and safety of fluticasone furoate (FF)/GW642444 inhalation powder and the individual components delivered once daily (AM) via a novel dry powder inhaler compared with placebo in subjects with chronic obstructive pulmonary disease (COPD)
COPD COPD HZC112207 203 A 24-week study to evaluate the efficacy and safety of fluticasone furoate (FF)/GW642444 inhalation powder and the individual components delivered once daily (AM) via a novel dry powder inhaler compared with placebo in subjects with chronic obstructive pulmonary disease (COPD)
COPD COPD HZC113107 262 A 12-week study to evaluate the 24 hour pulmonary function of fluticasone furoate (FF)/vilanterol inhalation powder (FF/VI inhalation powder) once daily compared with salmeterol/fluticasone propionate (FP) inhalation powder twice daily in subjects with chronic obstructive pulmonary disease (COPD)
Diabetes Diabetes 12180015 130 A randomized, double-blind, placebo-controlled, parallel group 24 week study to assess the efficacy and safety of BI 1356 (5 mg) in combination with 30 mg pioglitazone (both administered orally once daily), compared to 30 mg pioglitazone plus placebo in drug naive or previously treated type 2 diabetic patients with insufficient glycaemic control
Diabetes Diabetes 12180016 167 A randomized, double-blind, placebo-controlled parallel group efficacy and safety study of BI 1356 (5 mg administered orally once daily) over 24 weeks, in drug naive or previously treated (6 weeks washout) type 2 diabetic patients with insufficient glycemic control
Diabetes Diabetes 12180017 177 A randomized, double-blind, placebo-controlled parallel group efficacy and safety study of BI 1356 (one dose, eg, 5 mg), administered orally once daily over 24 weeks, with an open label extension to 80 weeks (placebo patients switched to BI 1356), in type 2 diabetic patients with insufficient glycaemic control despite metformin therapy
Diabetes Diabetes 12180018 263 A randomized, double-blind, placebo-controlled parallel group efficacy and safety study of BI 1356 (5 mg) administered orally once daily over 24 weeks, with an open-label extension to one year (placebo patients switched to BI 1356), in type 2 diabetic patients with insufficient glycaemic control despite a therapy of metformin in combination with a sulphonylurea
Diabetes Diabetes D1680C00001 264 A 52-week international, multi-center, randomized, parallel-group, double-blind, active-controlled, phase III study with a 52-week extension period to evaluate the safety and efficacy of saxagliptin in combination with metformin compared with sulphonylurea in combination with metformin in adult patients with type 2 diabetes who have inadequate glycaemic control on metformin therapy alone
Diabetes Diabetes D1680C00005 258 A 24-week international, multi-center, randomized, parallel-group, double-blind, placebo-controlled, phase III study to evaluate the efficacy and safety of saxagliptin in adult patients with type 2 diabetes who have inadequate glycaemic control with diet and exercise
Diabetes Diabetes H9XMCGBDA 141 A randomized, placebo-controlled comparison of the effects of two doses of LY2189265 or exenatide on glycemic control in patients with type 2 diabetes on stable doses of metformin and pioglitazone (AWARD-1: assessment of weekly administration of LY2189265 in diabetes-1)
Cardiovascular Deep vein thrombosis CV185056 2,691 A safety and efficacy trial evaluating the use of apixaban in the treatment of symptomatic deep vein thrombosis and pulmonary embolism Apixaban for the initial Management of PuLmonary embolIsm and deep vein thrombosis as First-line therapY (AMPLIFY)
Cardiovascular Atherosclerosis/coronary artery disease D356ic00001 586 Study of coronary atheroma by intravascular ultrasound: effect of rosuvastatin vs atorvastatin (SATURN)
Cardiovascular Hypertension NEBMD20 148 Randomized, double-blind, placebo-controlled, parallel-group study to evaluate the effect of 5 mg or 20 mg nebivolol once daily on blood pressure in patients with systolic stage 2 hypertension
Cardiovascular Hypertension NEBMD25 95 A randomized, double-blind, parallel-group study to evaluate the effects of first-line treatment with a free combination of nebivolol and lisinopril compared with placebo and the monotherapy components on blood pressure in patients with stage 2 diastolic hypertension
Cardiovascular DVT CV185057 820 A safety and efficacy trial evaluating the use of apixaban for the extended treatment of deep vein thrombosis and pulmonary embolism Apixaban after the initial Management of PuLmonary embolIsm and deep vein thrombosis with First-line therapY-EXTended treatment. The AMPLIFY-EXT study
Hidradenitis Hidradenitis M11313 201 A phase 3 multicenter study of the safety and efficacy of adalimumab in subjects with moderate to severe hidradenitis suppurativa – PIONEER I
Hidradenitis Hidradenitis M11810 214 A phase 3 multicenter study of the safety and efficacy of adalimumab in subjects with moderate to severe hidradenitis suppurativa – PIONEER II
Hypercholesterolemia Hypercholesterolemia 4,522IL0065 7,542 A 6-week, open-label, dose-comparison study to evaluate the safety and efficacy of rosuvastatin vs atorvastatin, pravastatin, and simvastatin in subjects with hypercholesterolemia. Statin therapies for elevated lipid levels compared across doses to rosuvastatin (STELLAR)
Rheumatoid arthritis Rheumatoid arthritis 015KCLRA21 72 A phase 2b, randomized, double-blind, parallel-group, placebo-controlled, dose-finding, multi-center study to evaluate the safety and efficacy of ASP015K in moderate to severe rheumatoid arthritis subjects who have had an inadequate response to methotrexate
Rheumatoid arthritis Rheumatoid arthritis 015KCLRA22 51 A phase IIb, randomized, double-blind, parallel-group, placebo-controlled, dose-finding, monotherapy, multi-center study to evaluate the safety and efficacy of ASP015K in moderate to severe rheumatoid arthritis subjects
Rheumatoid arthritis Rheumatoid arthritis C87027 201 A phase III, multicentre, double-blind, placebo-controlled, parallel-group, 52-week study to assess the efficacy and safety of 2 dose regimens of lyophilised CDP-870 given subcutaneously as additional medication to methotrexate in the treatment of signs and symptoms and preventing structural damage in patients with active rheumatoid arthritis who have an incomplete response to methotrexate Rheumatoid Arthritis PreventIon of structural Damage 1 (RAPID 1)
Rheumatoid arthritis Rheumatoid arthritis C87050 130 A phase III multi-center, double-blind, placebo-controlled, parallel-group 24-week study to assess the efficacy and safety of two dose regimens of liquid certolizumab pegol as additional medication to methotrexate in the treatment of signs and symptoms of rheumatoid arthritis and in prevention of joint damage in patients with active rheumatoid arthritis who have an incomplete response to methotrexate Rheumatoid Arthritis PreventIon of structural Damage 2 (RAPID 2)
Rheumatoid arthritis Rheumatoid arthritis C87076 98 A phase IIIB, multi-center, double-blind, placebo-controlled, parallel group study to evaluate the safety and efficacy of certolizumab pegol, administered with DMARDs, in patients with low to moderate disease activity rheumatoid arthritis CERTAIN (CERTolizumab pegol in the treatment of RA: remission INduction and maintenance in pts with low DA)
Rheumatoid arthritis Rheumatoid arthritis C87094 212 A phase IIIb, multicenter study with a 12-week double-blind, placebo-controlled, randomized period followed by an open-label, extension phase to evaluate the safety and efficacy of certolizumab pegol administered to patients with active rheumatoid arthritis. RA Evaluation in Subjects Receiving TNF Inhibitor Certolizumab Pegol (REALISTIC)
Rheumatoid arthritis Rheumatoid arthritis H9BMCBCDM 349 A phase 3, multicenter, randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of LY2127399 in patients with moderate to severe rheumatoid arthritis (RA) who had an inadequate response to methotrexate therapy
Rheumatoid arthritis Rheumatoid arthritis H9BMCBCDO 251 A phase 3, multicenter, randomized, double-blind, placebo-controlled study to evaluate the safety and efficacy of LY2127399 in patients with rheumatoid arthritis (RA) with or without background disease-modifying anti-rheumatic drug (DMARD) therapy
Rheumatoid arthritis Rheumatoid arthritis H9BMCBCDV 155 A phase 3, multicenter, randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of LY2127399 in patients with moderate to severe rheumatoid arthritis (RA) who had an inadequate response to one or more TNF-alpha inhibitors
Rheumatoid arthritis Rheumatoid arthritis IM101029 133 A phase III, MultiA phase III, multi-center, randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of BMS-188667 in subjects with active rheumatoid arthritis on background disease modifying anti-rheumatic drugs (DMARDS) who have failed anti-tumor necrosis factor (TNF) therapy Abatacept Trial in Treatment of Anti-TNF Inadequate Responders (ATTAIN)
Rheumatoid arthritis Rheumatoid arthritis IM119015 61 A randomized, parallel group, double-blind, placebo-controlled study to evaluate the clinical efficacy and safety of BMS-582949 given orally to subjects with rheumatoid arthritis having an inadequate response to methotrexate
Rheumatoid arthritis Rheumatoid arthritis IM126004 41 A multicenter, randomized, double-blind, placebo-controlled study to assess the efficacy, safety, tolerability, and pharmacokinetics of BMS-817399 in adults with active, moderate to severe rheumatoid arthritis and inadequate response to methotrexate
Rheumatoid arthritis Rheumatoid arthritis M10261 57 A multi-center, randomized, double-blind, placebo-controlled study comparing 80 mg of adalimumab with placebo, and demonstrating the non-inferiority of monthly 80 mg adalimumab dosing compared with 40 mg adalimumab every other week dosing
Schizophrenia Schizophrenia M10854 68 A randomized, double-blind, placebo-controlled, parallel-group, phase 2 study of the safety and efficacy of ABT-126 in the treatment of cognitive deficits in schizophrenia (CDS)
Schizophrenia Schizophrenia M13608 51 A randomized, double-blind, placebo-controlled, parallel-group, phase II study of the safety and efficacy of ABT-126 in the treatment of cognitive deficits in schizophrenia (CDS) in smokers
Schizophrenia Schizophrenia M10503 72 A randomized, double-blind, placebo-controlled, parallel-group, phase 2 study of the safety and efficacy of ABT-288 in the treatment of cognitive deficits in schizophrenia (CDS)
Schizophrenia Schizophrenia M10855 144 A randomized, double-blind, placebo-controlled, dose-ranging, parallel-group, phase II study of the safety and efficacy of ABT-126 in the treatment of cognitive deficits in schizophrenia (CDS) in nonsmokers
Schizophrenia Schizophrenia R076477BIM3001 122 A randomized, double-blind, placebo-controlled, parallel-group, dose-response, multicenter study to evaluate the efficacy and safety of three fixed doses of extended-release paliperidone in the treatment of subjects with acute manic and mixed episodes associated with bipolar I disorder
Schizophrenia Schizophrenia R076477SCA3001 107 A randomized, double-blind, placebo-controlled, parallel-group study to evaluate the efficacy and safety of two dosages of paliperidone ER in the treatment of patients with schizoaffective disorder
Schizophrenia Schizophrenia R076477SCA3002 95 A randomized, double-blind, placebo-controlled, parallel-group study to evaluate the efficacy and safety of flexible-dose paliperidone ER in the treatment of patients with schizoaffective disorder
Schizophrenia Schizophrenia R076477SCH301 102 A randomized, double-blind, placebo-controlled, parallel-group study with an open-label extension evaluating extended release OROS paliperidone in the prevention of recurrence in subjects with schizophrenia
Schizophrenia Schizophrenia R076477SCH3015 267 A randomized, double-blind, placebo-controlled, parallel-group study to evaluate the efficacy and safety of paliperidone ER compared to quetiapine in subjects with an acute exacerbation of schizophrenia
Schizophrenia Schizophrenia R076477SCH303 127 Randomized, double-blind, placebo- and active-controlled parallel-group, dose-response study to evaluate the efficacy and safety of 3 fixed dosages of paliperidone extended release (6, 9, and 12 mg/day) and olanzapine (10 mg/day) with open-label extension in treatment of schizophrenia
Schizophrenia Schizophrenia R076477SCH304 110 A randomized, double-blind, placebo- and active-controlled, parallel-group, dose-response study to evaluate the efficacy and safety of 2 fixed dosages of paliperidone extended release tablets and olanzapine, with open-label extension, in the treatment of patients with schizophrenia
Schizophrenia Schizophrenia R076477SCH305 123 A randomized, double-blind, placebo- and active-controlled, parallel-group, dose-response study to evaluate the efficacy and safety of 3 fixed dosages of extended release OROS paliperidone (3, 9 and 15 mg/day) and olanzapine (10 mg/day), with open-label extension, in the treatment of subjects with schizophrenia
Schizophrenia Schizophrenia R092670PSY3001 204 A randomized, double-blind, placebo-controlled, parallel-group study evaluating paliperidone palmitate in the prevention of recurrence in patients with schizophrenia. Placebo consists of 20% intralipid (200 mg/mL) injectable emulsion
Schizophrenia Schizophrenia R092670PSY3003 96 A randomized, double-blind, placebo-controlled, parallel-group, dose response study to evaluate the efficacy and safety of 3 fixed doses (50 mg eq., 100 mg eq., and 150 mg eq.) of paliperidone palmitate in subjects with schizophrenia
Schizophrenia Schizophrenia R092670SCH201 84 A randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of 50 and 100 mg-eq of paliperidone palmitate in patients with schizophrenia
Stroke Stroke F7ICH1371 96 No biomarker data
Stroke Stroke F7ICH1389 11 No biomarker data
Stroke Stroke F7ICH1641 266 No biomarker data
Stroke Stroke F7ICH2073 9 No biomarker data
Ulcerative colitis Ulcerative colitis IM101108 140 A phase 3, multi-center, randomized, placebo-controlled study to evaluate the clinical efficacy and safety of induction and maintenance therapy with abatacept in subjects with active ulcerative colitis (UC) who have had an inadequate clinical response and/or intolerance to medical therapy
Ulcerative colitis Ulcerative colitis M06826 223 A multicenter, randomized, double-blind, placebo-controlled study of the human anti-TNF monoclonal antibody adalimumab for the induction of clinical remission in subjects with moderately to severely active ulcerative colitis
Ulcerative colitis Ulcerative colitis M06827 260 A multicenter, randomized, double-blind, placebo-controlled study of the human anti-TNF monoclonal antibody adalimumab for the induction and maintenance of clinical remission in subjects with moderately to severely active ulcerative colitis
Ulcerative colitis Ulcerative colitis M10447 96 A multi-center, randomized, double-blind, placebo-controlled study of adalimumab in Japanese subjects with moderately to severely active ulcerative colitis
Vaccine No biomarker data

Within each broad indication, the patient population characterization is of relevance, in terms of both the type of disease (eg, cardiovascular disease) and the stage of disease (eg, rheumatoid arthritis). For example, the cardiovascular population consisted of a variety of conditions: deep vein thrombosis (DVT), hypertension and atherosclerosis/coronary artery disease (CAD). The DVT indication (two clinical trials) was the highest contributor in terms of number of patients (3,511 out of 4,380) within the cardiovascular disease patient population. Although it is recognized that the different subpopulations could have very different values within a lab parameter, for purposes of this pilot study the three cardiovascular disease patient subpopulations were kept as one population. Similarly, among the 13 rheumatoid arthritis clinical trials, many consisted of patients with moderate to severe disease but at least one study enrolled patients with low to moderate rheumatoid arthritis. Although it is acknowledged that there could be lab value differences between subjects across different stages of the disease, for this pilot study all rheumatoid arthritis subpopulations were kept as one patient population.

The distribution of sample numbers across studies was compared within each condition to determine if a clinical trial could potentially have a disproportionate impact on determining if a lab parameter was different from the HVRR. In this regard, the only two patient populations of potential concern were cardiovascular and hypercholesterolemia. The cardiovascular patient population had more than 50% of the patients in 1 of the five studies while all of the hypercholesterolemia patient population were in one study. Although the impact of a majority of subjects being from one clinical trial has the potential to skew the results, for the purpose of this pilot study the cardiovascular disease and hypercholesterolemia patient populations were acceptable.

Study and patient numbers

To determine if the patient population reference range for a lab parameter is potentially different from the HVRR for the same lab parameter, it is important to have sufficient numbers of biomarker sampled patients. The stroke population had 369 subjects with glucose values, while more than half of the other lab parameters evaluated had fewer than 100 subjects (Table 2). Due to the low number of subjects with a lab value (<400) and a high variability in subject numbers for each lab parameter (0–369) the stroke patient population was excluded from further analysis. Any lab parameter with data from less than 400 subjects was not evaluated and interpreted.

Table 2.

Number of subjects evaluated by biomarker

Disease Glucose Platelet Neutrophil ALT AST Bilirubin
Alzheimer’s 2,150 2,146 767 2,150 2,150 2,149
Asthma 1,052 1,040 689 1,052 1,052 1,052
COPD 2,215 2,208 1,877 2,214 2,215 1,980
Cardiovascular 4,339 4,081 4,333 4,339 4,338 4,339
Diabetes 1,400 1,400 1,259 1,400 1,400 1,400
Hidradenitis 415 415 415 415 415 415
Hypercholesterolemia 7,430 7,408 0* 7,449 7,449 7,449
Rheumatoid arthritis 1,810 1,809 1,808 1,810 1,810 1,810
Schizophrenia 1,737 1,722 953 1,739 1,740 1,740
Stroke 369* 0* 102* 19* 19* 19*
Ulcerative colitis 719 718 623 719 719 719

Note:

*

Less than 400 subjects.

Abbreviations: ALT, alanine aminotransferase; AST, aspartate aminotransferase.

Some lab parameters were evaluated in only a subset of the patients. For example, neutrophils were evaluated in only a third of the patients with Alzheimer’s. These parameters were further evaluated in this study only if the lab parameter was measured in a sufficient overall number of subjects (400; Table 2).

Demographics

Demographics are different in the various patient populations, as would be expected (Table 3). Of the 10 patient populations reported in this study, the youngest patient populations with the mean age in the 30s–40s were those with hidradenitis, asthma, schizophrenia and ulcerative colitis, while the Alzheimer’s population was the oldest with a mean age of 74 years. The rheumatoid arthritis patient population had the highest percentage of females with 82% and the COPD population had the lowest percentage of females with 26%.

Table 3.

Demographics

Disease Age (years) (mean±SD) Percent female Asian Black Hispanic Mixed race Multiple Other White
Alzheimer’s 74±8 59 10 2 3 0 0 1 84
Asthma 43±16 61 37 7 0 2 0 0 53
COPD 62±9 26 20 2 0 1 0 0 78
Cardiovascular 56±16 39 7 5 0 0 0 4 85
Diabetes 55±10 45 43 1 0 0 0 0 54
Hidradenitis 37±12 66 3 14 0 0 1 2 79
Hypercholesterolemia nd 48 2 8 0 0 0 6 85
Rheumatoid arthritis 52±12 82 16 6 0 0 1 2 72
Schizophrenia 40±11 36 15 26 0 0 0 4 55
Stroke 64±13 38 20 7 0 0 0 3 71
Ulcerative colitis 41±13 39 17 3 0 0 0 2 78

Abbreviation: nd, not determined (values were not in the TransCelerate database).

Endocrine – glucose

Half (5 out of 10) of the patient populations had glucose values higher than the healthy volunteer reference range: COPD, cardiovascular disease, diabetes, hypercholesterolemia, and schizophrenia (Figure 1). As expected, the mean glucose value was highest in patients with diabetes at 1.41±0.35×ULN (mean±SD) and 63% of the samples being above the ULN. All of the patient populations except ulcerative colitis had subjects with greater than 2×ULN with some subjects being greater than 4×ULN.

Figure 1.

Figure 1

Endocrinology safety biomarker glucose in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval, with the percentages of people above the healthy volunteer reference range for each patient population (in red).

Abbreviation: ULN, upper limit of normal.

Hematology – platelet and neutrophil counts

None of the patient populations were considered different from the HVRR for decreased neutrophils or decreased platelets (Figures 2 and 3, percentages not shown). However, neutrophil and platelet counts were increased compared to the HVRR in patients with rheumatoid arthritis and ulcerative colitis and neutrophils alone were increased in patients with hidradenitis.

Figure 2.

Figure 2

Hematology safety biomarker neutrophil count in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval with the percentages of people above or below the healthy volunteer reference range for each patient population (in red).

Abbreviation: ULN, upper limit of normal.

Figure 3.

Figure 3

Hematology safety biomarker platelet count in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval with the percentages of people above or below the healthy volunteer reference range for each patient population (in red).

Abbreviation: ULN, upper limit of normal.

Hepatic – ALT, AST, and bilirubin

ALT was higher than the HVRR in four of the ten patient populations: diabetes, hypercholesterolemia, rheumatoid arthritis. and schizophrenia (Figure 4) with the ALT being greater than 10×ULN in some hypercholesterolemia patients. ALT levels were as high as 10×ULN in patients with hypercholesterolemia and most patient populations had subjects with levels greater than 3×ULN.

Figure 4.

Figure 4

Hepatic safety biomarker alanine aminotransferase (ALT) in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval with the percentages of people above the healthy volunteer reference range for each patient population.

Abbreviation: ULN, upper limit of normal.

AST was higher than the HVRR in only the hypercholesterolemia patients with the values being up to approximately 5×ULN (Figure 5). AST levels were as high as 10×ULN in patients with cardiovascular disease and several patient populations had subjects with levels greater than 3×ULN.

Figure 5.

Figure 5

Hepatic safety biomarker aspartate aminotransferase (AST) in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval with the percentages of people above the healthy volunteer reference range for each patient population (in red).

Abbreviation: ULN, upper limit of normal.

Bilirubin was similar to the HVRR in all ten of the patient populations (Figure 6). However, most patient populations had subjects with bilirubin greater than 2×ULN.

Figure 6.

Figure 6

Hepatic safety biomarker bilirubin in various patient populations.

Note: Forest plot for each patient population with the 95% confidence interval with the percentages of people above the healthy volunteer reference range for each patient population (in red).

Abbreviation: ULN, upper limit of normal.

Differences from healthy volunteer reference ranges

All patient populations except Alzheimer’s and asthma had at least one lab parameter deemed potentially different from the HVRR and all lab parameters except bilirubin were deemed potentially different in at least one patient population (Table 4). Both patients with hypercholesterolemia and rheumatoid arthritis had three of the lab parameters potentially different from the HVRR. Patients with hypercholesterolemia had increased glucose, ALT and AST levels, whereas patients with rheumatoid arthritis had increased platelet count, neutrophil count and ALT.

Table 4.

Percent of lab values above the UDDer limits of normal LN

Lab parameter (biomarker) Alzheimer’s Asthma COPD Cardiovascular Diabetes Hidradenitis Hypercholesterolemia Rheumatoid arthritis Schizophrenia Ulcerative colitis
Glucose 7 8 13 13 63 8 14 4 12 5
Neutrophil 4 5 6 6 3 24 nd 20 6 30
Platelets 4 2 2 1 1 8 1 17 4 29
Alanine aminotransferase 3 4 3 9 16 3 14 10 11 6
Aspartate aminotransferase 3 2 3 6 6 3 14 6 8 4
Bilirubin 3 1 2 3 3 1 4 0 1 3

Notes: Green highlight - lab parameter in that patient population is similar to the healthy volunteer reference range (HVRR). Black highlight - lab parameter is potentially different from the HVRR.

Abbreviation: nd, not determined (did not calculate because too few samples).

For the lab parameters evaluated in this pilot study using the cutoff of 10% of the subjects having values greater than the ULN, 25% (15 out of 60) of the lab parameters analyzed in the various patient populations were higher than the HVRR. Approximately 10% of the lab values would still be potentially higher than the HVRR if using a higher cutoff of 20% of the values greater the ULN.

Discussion

The objective of this pilot study was to determine if routine safety lab parameters were similar to the HVRR or if like ALT in NASH or oncology patient populations, other lab parameters are potentially different from the HVRR.

The TransCelerate dataset consisting of clinical trial data from across pharmaceutical companies contained lab data from enough subjects (>400) within the various patient populations for comparison with the HVRR. The demographics of the various patient populations were consistent with expectations. For example, the Alzheimer’s patient population in the dataset had a mean age of 74 years and a preponderance of females (59%). In a case control study of 2,618 patients with Alzheimer’s the mean age was 76.1 years with 59% being female.7 However, there were limitations in the TransCelerate dataset such as 1) disease definition and 2) disease stage.

Clinical trials typically have a very specific patient population definition and it is important to know if the HVRR can be used for the various lab parameters in that patient population. However, the TransCelerate dataset combined three different patient subpopulations into their definition of cardiovascular disease: deep vein thrombosis, hypertension and atherosclerosis/coronary artery disease. This broad definition of cardiovascular disease could have created false negative results. Similar to a broad disease definition, a patient population consisting of wide disease stages can impact the evaluation of lab parameters. For example, the rheumatoid arthritis population consisted of clinical trials of patient from mild all the way to severe patients. To overcome these limitations, future studies will need to determine if there are differences between the different clinical trials within a patient population.

Using the TransCelerate dataset, a lab parameter was classified similar to the HVRR if less than 10% of the lab parameter values in a patient population were above or below the HVRR (ULN and LLN, respectively). In contrast, a lab parameter was classified potentially different from the HVRR if ≥10% of the lab parameter values in a patient population were above or below the HVRR. The 10% threshold was chosen to decrease the chance of false positive classification of a lab parameter being different from the HVRR without doing statistical analysis within this pilot study. The HVRR reference range is a 95% CI, meaning 2.5% of the values will be above the ULN and 2.5% of values will be below the LLN. The 10% cutoff would be four times the expected values to be above or below the HVRR. This study with the 10% cutoff determined that 25% of the lab parameters in the various patient populations evaluated were classified as potentially different from the HVRR. Of those, five lab parameters in various patient populations had more than 20% of the values greater than the ULN of the HVRR: glucose in patients with diabetes; platelets in patients with ulcerative colitis; neutrophils in patients with hidradenitis, rheumatoid arthritis, or ulcerative colitis.

There are three important pre-analytical ways a lab parameter could be different from the HVRR: physiologically associated with the disease, due to standard of care of the patient population or due to a comorbidity of the patient population. For example, ALT in patients with diabetes,5,8,9 increased platelet and neutrophil counts in patients with ulcerative colitis10,11 would be considered physiologically associated with the disease of the patient population. Glucose being higher than the HVRR in patients with schizophrenia is potentially due to the comorbidity of diabetes, since the incidence of diabetes is 2–4 times greater in patients with schizophrenia than in healthy volunteers.12 Whereas glucose being higher in COPD patients is a combination of disease severity with elevated glucose level associated with COPD exacerbation, comorbidity of diabetes is higher in patients with COPD than in the general population, as well as glucocorticoid treatment of patients with COPD potentially inducing hyperglycemia.13 To determine the potential cause(s) of a lab parameter being different in a patient population, future studies will need to compare lab values with co-morbidities and co-medications, both of which are included in the TransCelerate dataset.

The data from this pilot along with literature and medical judgement suggests that it is not uncommon for lab values to be different from the HVRR in various patient populations. There are two options that can be used in clinical trials to evaluate patients with a lab value outside the HVRR after treatment with an investigational new drug: 1) continue using medical judgement alone to determine if the lab value is drug-induced or due to the patient population/disease, or 2) combine medical judgment with better evaluation criteria for detecting a potential drug-induced abnormal lab value.

Using medical judgement alone in clinical trials requires a significant amount of time and is subjective due to varying levels of medical expertise. For example, our 10% cutoff used in this study indicates that at least 10% of the subjects would yield a false positive elevated signal in a clinical trial before and after subjects are treated with the investigational drug. For example, 24% of the patients with hidradenitis had neutrophil counts above the ULN, which means that for a study consisting of 200 subjects, then 48 subjects at each time point of lab analysis would need medical judgment even though the lab value above the ULN would be due to other causes and not because of the drug under investigation (false positives). If there are 10 timepoints within the study then there are at least 480 false positive values that would undergo unnecessary medical judgements and if each medical judgement took 5 minutes then better data evaluation would save 40 hours of time, reduce the degree of misinterpretation of the data, and human error or inconsistency.

Better evaluation criteria will decrease the number of medical judgements within each clinical trial, but the criteria must be based on medical evidence and validation. For example, the criteria historically used for a potential Hy’s Law has been ALT >3×ULN and BILI >2×ULN. But over the years it has been determined that many patients with NASH and metastatic cancer have ALT greater than the ULN before dosing with an investigative new drug.14 There has been much debate on how to better evaluate ALT for potential Hy’s Law cases in these patient populations.5 There have been recommendations of using multiples of ULN and multiples of the baseline values to identify DILI in patients with elevated ALT and bilirubin at baseline.14

Conclusion and next steps

These data clearly suggest that not all lab parameters in various patient populations are similar to the HVRR, but rather some lab parameters in some patient populations are potentially meaningfully different from the HVRR. Further studies will be required to determine if the potential differences are consistent across studies within a patient population and with possibly better evaluation criteria to be used during drug development and clinical practice. Limitations of the pilot that needs to be addressed in the next study(ies) include: 1) creating a more specific definition of patient populations and disease stage; 2) evaluating lab parameters for each sub-patient population separately; 3) examining between-study reproducibility; 4) assessing if inter- and intra-subject variability is different between healthy volunteers and patient populations; 5) determining if standard of care treatment and/or comorbidities explains increased lab values and should result in these subjects being evaluated differently than the other patients in this population; 6) determining the impact of age, gender, and ethnicity; 7) determining which biomarkers are different from the HVRR based on statistical analysis and medical relevance; and 8) evaluating a larger confirmatory dataset.

Footnotes

Disclosure

Authors were employees or contractors of AstraZeneca Pharmaceuticals when the experiments were conducted. The authors report no other conflicts of interest in this work.

References

  • 1.Neuschwander-Tetri BA, Brunt EM, Wehmeier KR, Oliver D, Bacon BR. Improved nonalcoholic steatohepatitis after 48 weeks of treatment with the PPAR-gamma ligand rosiglitazone. Hepatology. 2003;38(4):1008–1017. doi: 10.1053/jhep.2003.50420. [DOI] [PubMed] [Google Scholar]
  • 2.O’Connor BJ, Kathamna B, Tavill AS. Nonalcoholic fatty liver (NASH syndrome) Gastroenterologist. 1997;5(4):316–329. [PubMed] [Google Scholar]
  • 3.Shantakumar S, Landis S, Lawton A, Hunt CM. Prevalence and incidence of liver enzyme elevations in a pooled oncology clinical trial cohort. Regul Toxicol Pharmacol. 2016;77:257–262. doi: 10.1016/j.yrtph.2016.03.019. [DOI] [PubMed] [Google Scholar]
  • 4.Torriani FJ, Rodriguez-Torres M, Rockstroh JK, et al. Peginterferon Alfa-2a plus ribavirin for chronic hepatitis C virus infection in HIV-infected patients. N Engl J Med Overseas Ed. 2004;351(5):438–450. doi: 10.1056/NEJMoa040842. [DOI] [PubMed] [Google Scholar]
  • 5.Kullak-Ublick GA, Merz M, Griffel L, Kaplowitz N, Watkins PB. Liver safety assessment in special populations (hepatitis B, C, and oncology trials) Drug Saf. 2014;37(Suppl 1):57–62. doi: 10.1007/s40264-014-0186-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Bhuyan P, Chen C, Desai J, et al. Development and Implementation of a pharma-collaborative large historical control database. 2017. [Accessed March 30, 2017]. Available from: http://www.transceleratebiopharmainc.com/wp-content/uploads/2015/04/TransCelerate-PSoC-Data-Sharing-White-Paper.pdf.
  • 7.Wang JH, Wu YJ, Tee BL, Lo RY, Yj W, Ry L. Medical comorbidity in Alzheimer’s disease: a nested case-control study. J Alzheimers Dis. 2018;63(2):773–781. doi: 10.3233/JAD-170786. [DOI] [PubMed] [Google Scholar]
  • 8.Giandalia A, Romeo EL, Ruffo MC, et al. Clinical correlates of persistently elevated liver enzymes in type 2 diabetic outpatients. Prim Care Diabetes. 2017;11(3):226–232. doi: 10.1016/j.pcd.2016.12.001. [DOI] [PubMed] [Google Scholar]
  • 9.Harris EH. Elevated liver function tests in type 2 diabetes. Clinical Diabetes. 2005;23(3):115–119. [Google Scholar]
  • 10.Cioffi M, Rosa AD, Serao R, Picone I, Vietri MT. Laboratory markers in ulcerative colitis: Current insights and future advances. World J Gastrointest Pathophysiol. 2015;6(1):13–22. doi: 10.4291/wjgp.v6.i1.13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Polińska B, Matowicka-Karna J, Kemona H. Assessment of the influence of the inflammatory process on the activation of blood platelets and morphological parameters in patients with ulcerative colitis (colitis ulcerosa) Folia Histochem Cytobiol. 2011;49(1):119–124. doi: 10.5603/fhc.2011.0017. [DOI] [PubMed] [Google Scholar]
  • 12.Al-Atram AA. A review of the bidirectional relationship between psychiatric disorders and diabetes mellitus. Neurosciences. 2018;23(2):91–96. doi: 10.17712/nsj.2018.2.20170132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Mirrakhimov AE. Chronic obstructive pulmonary disease and glucose metabolism: a bitter sweet symphony. Cardiovasc Diabetol. 2012;11:132. doi: 10.1186/1475-2840-11-132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Lin X, Parks D, Painter J, et al. Validation of multivariate outlier detection analyses used to identify potential drug-induced liver injury in clinical trial populations. Drug Saf. 2012;35(10):865–875. doi: 10.1007/BF03261982. [DOI] [PubMed] [Google Scholar]

Articles from Drug Design, Development and Therapy are provided here courtesy of Dove Press

RESOURCES