Modeling rare genetic disease with patient-derived induced pluripotent stem cells: reassessment of the minimum numbers of lines needed

Abstract

Induced pluripotent stem cells (iPSCs) are widely used to model human genetic diseases. The most common strategy involves collecting cells from relevant individuals and then reprogramming them into iPSCs. This strategy is very powerful, but finding enough individuals with a specific genetic disease can be challenging, especially since most are rare. In addition, making numerous iPSC lines is time-consuming and expensive. As a result, most studies have included relatively small numbers of iPSC lines, sometimes from the same individual. Considering the experimental variability obtained using different iPSC lines, there has been great interest in delineating the most efficient number of lines needed to achieve a robust and reproducible result. Several recommendations have been published, although most conclusions have been based on methods where experimental variance from individual cases is difficult to separate from technical issues related to the preparation of iPSCs. The current study used gene expression profiles determined by RNA sequencing (RNAseq) to empirically evaluate the impact of the number of unique individuals and the number of replicate iPSC lines from each individual for modeling Lesch-Nyhan disease (LND). This disease is caused by mutations in the HPRT1 gene, which encodes the enzyme hypoxanthine-guanine phosphoribosyltransferase. Results for detecting disease-relevant changes in gene expression depended on the analytical method employed, and whether or not statistical procedures were used to address multiple iPSC lines from the same individual. In keeping with prior studies, the best results were obtained with iPSC lines from 3-4 unique individuals per group. In contrast to prior studies, results were improved with 2 lines per individual, without statistical corrections for duplicate lines from the same individual. In the current study where all lines were produced in parallel using the same methods, most variance in gene expression came from technical factors unrelated to the individual from whom the iPSC lines were prepared.

Graphical Abstract

Disease modeling involved 4 unrelated Lesch-Nyhan disease (LND) cases and 4 unrelated controls (CON), with 3 independently derived iPSC lines made from fibroblasts for each individual. All samples were subject to RNAseq in one batch. The RNAseq results were used to assess pluripotency, and different methods were used to assess the disease effect on gene expression, with and without applying methods to address multiple lines from the same case. Sensitivity and specificity when applying different methods to different numbers of iPSC lines were evaluated using permutation-based analyses, and source of variance in gene expression were evaluated using variance partitioning.

Significance statement.

This study addresses a gap in knowing the optimal sample size required for stem cell-based investigations of rare diseases. By using induced stem cell-based disease modeling, we provide a robust framework to ensure the reproducibility and reliability of disease modeling studies. These findings are particularly relevant for rare disorders such as Lesch-Nyhan disease where studies are restricted by the small number of samples that can be obtained. This work establishes a methodological standard that can improve the design and interpretation of induced stem cell-based modeling.

Introduction

Induced pluripotent stem cells (iPSCs) are widely used to model human diseases.^1-3 These cells can be differentiated into numerous different cell types, enabling studies of organ-specific pathology. They have proven valuable for elucidating pathogenesis, as platforms for assessment of treatments, and as replacements for lost or damaged cells.^1-3 These cells have been particularly valuable for rare genetic diseases, where studies are limited by the numbers of affected individuals, the numbers of biosamples that can be obtained, and limitations of animal models that may not recapitulate the disease. The most common strategy for modeling rare genetic diseases has involved collecting cells from affected individuals, and then reprogramming them into iPSCs. One limitation of this strategy is finding enough cases. Another limitation is that the method is time-consuming and expensive. As a result, most prior studies have included only 1-3 unique cases, often with only 1-2 sublines per case.^4,5

There can be considerable experimental variability among different iPSC lines.^4-14 Results vary with iPSC lines produced from different individuals with the same disease, and sometimes even when using different sublines from the same individual. This variability has led to a need to delineate the minimum number of independent cases and sublines from each case optimal for reproducible results. Several articles provide guidance regarding the numbers of lines needed, with current recommendations suggesting the ideal strategy involves studying a single iPSC line from at least 4 unrelated individuals,^5,15-17 along with statistical methods to address relatedness among iPSC lines when more than one subline per individual is included.^4,5 However, most recommendations were derived following aggregation of results from iPSC lines produced across different labs, sometimes using different methods. As a result, the variance from individual cases cannot be readily disentangled from technical variables relating to the way the iPSCs were produced. Empirical validation of these recommendations under conditions where these technical differences are controlled would be valuable to guide further research in modeling rare diseases.

The current study empirically evaluated the impact of both the number of unique individuals and the number of replicate iPSC lines from each individual for disease modeling. All lines were prepared and processed using the same methods. The target was a monogenic disease associated with the HPRT1 gene, Lesch-Nyhan disease (LND). This disorder provides a valuable prototype for modeling monogenic disease for several reasons. First, there is a strong relationship between the genotype and phenotype, where HPRT1 mutations that cause null enzyme function produce a consistent phenotype, regardless of the molecular defect or genetic background.¹⁸ Second, the disease is X-linked and recessive, so only one allele needs to be addressed in males, eliminating confounds related to changes in X-chromosome activity during culture.^19,20 Third, the gene product encodes an enzyme (hypoxanthine-guanine phosphoribosyltransferase, HGprt) which has a well-delineated function, with predictable consequences when it is lost.^21,22 The HPRT1 gene also has relevance to the biology of stem cells, because it is increased in rapidly proliferating cells, presumably as a strategy to maintain high levels of purines needed for DNA replication.²³ The main readout for these studies was the gene expression profile obtained by RNA sequencing (RNAseq), because this method provides a wealth of information regarding the condition of the cells, it has been widely applied for many other diseases, and methods are standardized with good metrics for quality control. A schematic for overall study design is shown in the graphical abstract.

Methods

Generation of iPSC lines

All iPSCs were prepared as previously described²⁴ by reprogramming cultured fibroblasts using a microRNA/mRNA method (Stemgent) to mitigate against insertional mutagenesis or enduring effects of viral vectors. Three independent sublines from each of 4 unrelated cases were generated (LND lines; total N = 12). A total of 3 independent lines from each of 4 unrelated controls (CON lines; total N = 12) were prepared in parallel for comparisons (Table 1). All iPSC lines came from unrelated male subjects (no siblings or parents).

Table 1.

Characteristics of iPSC lines.

Group	Case ID	Sex	Age	Race	HPRT1 gene	HPRT1 consequence	HGprt enzyme	Subline	HPRT1 mRNA
LND	L1	Male	46	White	c.371insTT	frame shift, premature stop	ND	01	5.6
								03	4.0
								04	4.4
	L2	Male	10	White	c.508C > T	premature stop	ND	02	6.4
								09	6.3
								12	7.8
	L3	Male	46	Black	c.151C > T	premature stop	ND	27	12.5
								29	12.3
								31	12.0
	L4	Male	43	White	E4del	deletion	ND	01	48.1
								02	68.5
								03	56.9
CON	C1	Male	21	White	normal	normal	31.6	01	57.6
								02	50.8
								03	54.7
	C2	Male	43	Black	normal	normal	62.8	03	60.1
								04	64.6
								06	59.5
	C3	Male	22	White	normal	normal	30.0	05	48.8
								06	63.7
								08	61.2
	C4	Male	34	White	normal	normal	44.2	01	57.8
								02	56.5
								03	67.8

Age refers to subject age at skin biopsy. The HPRT1 gene was assessed by RT-PCR and interrogating final RNAseq data. Enzyme activity was measured as previously described (ND = not detectable).²⁴ The HPRT1 mRNA levels are relative values from qPCR. Karyotypes are provided in Supplementary Figure S1. Representative immunostains for pluripotency markers are shown in Supplementary Figure S2. Trilineage differentiation was conducted using the StemGent Trilineage differentiation kit as described in the methods. Some of these lines were described already, where karyotypes and immunostains are reproduced.²⁴.

All lines were cultured and validated as previously described.²⁴ For all lines, a karyotype with a minimum of 20 metaphase cells at a minimum of 400 band resolution (WiCell Research Institute Cytogenetics Lab, Madison WI) showed no relevant abnormalities (Supplementary Figure S1). Pluripotency was assessed by immunostains (SSEA3, SSEA4, TRA1-60, TRA1-81, NANOG), gene expression profiles of pluripotency genes with RNAseq, and differentiation of each line into each of the 3 major germ cell layers using the STEMdiff Trilineage Differentiation Kit (Stemcell Technologies, Vancouver BC). The immunostaining markers included those typical of ectoderm (PAX6, NESTIN), endoderm (SOX17, FOXA2), and mesoderm (brachyury, NCAM) as previously described.²⁴ PluriTest pluripotency scores²⁵ were also derived from the RNAseq data, but the novelty scores for somatic cells were not calculated because the publicly available tool was validated only with microarray data. The starting HPRT1 gene variant was confirmed by the reverse transcriptase and polymerase chain reaction (rtPCR), and it also was visualized in RNAseq read alignments using the Integrative Genomics Viewer (https://igv.org). The HPRT1 mutations for 3 of the individuals studied are associated with nonsense-mediated decay with low HPRT1 mRNA levels, while the 4th individual had normal HPRT1 mRNA levels (Table 1). The enzymatic activity of HGprt was confirmed to be positive for all CON lines and negative for all LND lines using a radiometric assay (Table 1).²¹

Gene expression profiling

To mitigate technical batch effects, all samples were grown and collected by the same individual using the same methods, and all samples were subject to bulk RNAseq in a single batch, as previously described.²⁴ For each sample, sequencing targeted 50 million paired-end, 100bp reads. Actual reads ranged from 50 to 71 million (average 58 million) with no consistent differences among subgroups. Original reads were verified by FastQC v0.11.4 for quality and then aligned to reference human genome (UCSC RefGene; hg38 build, 26,485 genes) with STAR (Spliced Transcripts Alignment to a Reference) aligner.²⁶ The STAR quantMode GeneCounts was used to count mapped reads. Between 93% and 94% of reads were uniquely mapped, and quantification was based on these uniquely mapped reads. After obtaining raw gene counts, further analyses were performed using R Statistical Software (v4.2.3; R Foundation for Statistical Computing, Vienna, Austria) in Rstudio (v2023.03.1 Build 446). Gene counts were normalized using the trimmed mean of M-values method in edgeR v3.40.2 R package²⁷ and then transformed to log2 counts per million (CPM).

Analytical strategy for differential gene expression

Principal component analysis (PCA) was performed to identify relatedness among the different iPSC lines. PCA was performed using the prcomp method from the stats v4.2.3 R package. The R package ggplot2 v3.4.2 was used for visualization. Sources of variance in gene expression were determined using variance partitioning on the matrix of normalized gene counts with fitExtractVarPartModel and plotVarPart functions from the R package variancePartition v1.32.5.²⁸

Two methods were used (edgeR and limma) to identify disease-related genes that were differentially expressed between LND and CON lines. These methods are based on distinct statistical models, normalization methods, handling of low-count genes, and especially variance estimation techniques. EdgeR employs a negative binomial distribution specifically for RNAseq data^4,27 while limma relies on linear models with empirical Bayes for microarray data, adapted for RNAseq with limma-voom. Implementation of edgeR involved the likelihood ratio test with the glmFit and glmLRT functions. Post hoc pairwise tests were performed using the exactTest function.²⁷ To assess the impact of multiple lines from the same individual, the edgeR method was applied before and after pooling RNAseq reads from multiple lines from the same individual. Results were also analyzed using the limma-voom method from limma v3.54.2 R package,²⁹ with and without correction for replicate sublines from the same individual using duplicateCorrelation (DupCor) as implemented in the voomWrapper function of the iPSCpoweR R package.⁴

For both edgeR and limma-voom methods, the adjusted P value was determined by the false discovery rate (FDR).³⁰ Differentially expressed genes with FDR < 0.05 were considered significant. Volcano plots were used to visualize results and were generated using GraphPad Prism v9.5.1. Potential biological pathways for differentially expressed genes were identified using clusterProfiler v4.6.2 R package³¹ according to the Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways.³²

Sensitivity and specificity analyses

To determine the sensitivity and specificity of detecting genes differentially expressed between LND and CON across analytical methods, all available samples were systematically shuffled under the null hypothesis that they were all from the same distribution. Fifty randomly selected genes were then given pre-defined changes (fold changes of 1.5, 2, 3, 5 and their inverses) for a total of 400 gene changes. To detect these changes, differential gene expression analyses were conducted with systematic permutations of all possible combinations of samples.⁴ The sensitivity and specificity for detecting these known changes were then compared using edgeR or limma-voom, with or without strategies to correct for duplicate samples from the same sources.⁴ The area under the curve (AUC) metric was used to assess the optimal balance between sensitivity and specificity.⁴

Results

Differential gene expression using edgeR

Three iPSC lines were prepared from each of 4 unrelated LND individuals and 4 unrelated healthy controls (CON). All lines studied had similar pluripotency characteristics when using immunostaining for pluripotency markers or tri-lineage differentiation (Supplementary Figure S2). All lines expressed similar levels of pluripotency genes and had similar PluriTest scores, with no systematic differences among LND and CON, or among individual sublines (Figure 1). To estimate relatedness of these lines, RNAseq read counts for all genes measured were normalized across all 24 lines and subjected to PCA. As shown in Figure 2A, there was no obvious clustering of LND vs CON lines (N = 12 each) or lines from the same individual (N = 3 each). These results imply the disease did not have an obvious global effect on gene expression and that iPSC sublines from the same individual had limited relatedness (clustering) of gene expression.

Figure 1.

Pluripotency gene expression among all iPSC lines. Panels A-E show expression levels in counts per million or CPM for typical pluripotency genes measured via RNAseq for each of the iPSC lines. Panel F shows the “pluripotency score” from the PluriTest method applied to RNAseq-based gene expression for each line.²⁵

Figure 2.

Gene expression analyses. Panel A shows the relatedness of gene expression patterns among all 24 lines by principal component analysis (PCA, LND = filled symbols and CON = open symbols, replicate lines from different cases shown as distinct symbols); Panels B-D show differential gene expression using edgeR when all 12 LND and all 12 CON iPSC lines were treated as independent samples; panel B shows a volcano plot for differentially expressed genes (LND vs CON, horizontal line shows FDR < 0.05); panel C shows the relatedness among lines for differentially expressed genes by PCA (N = 232); panel D shows biological pathways defined by these differentially expressed genes. Panel E shows the relatedness of lines from different individuals when results from all lines from the same individual were first pooled before further analyses. Panels F-H show differential gene expression using edgeR after pooling; panel F shows a volcano plot for differentially expressed genes (LND vs CON, horizontal line shows FDR < 0.05); panel G shows the relatedness among lines for differentially expressed genes by PCA (N = 2183); panel H shows the biological pathways defined by these differentially expressed genes. Results for differential gene expression with limma-voom with and without correction for multiple lines from the same case are shown in panels I-J. The overlap among the different methods when using a pre-defined FDR < 0.05 as criteria for differential gene expression is shown in panels K (edgeR, with and without pooling) and L (edgeR and limma-voom without correcting for duplicates). Similarities among the different methods when using correlations among P-values for all differentially expressed genes are shown in panel M.

Differences in gene expression between LND and CON groups were assessed in several ways. First, each line was considered an independent biological replicate, a strategy used in many prior studies and supported by the PCA showing a lack of relatedness of lines from the same individual (Figure 2A). The edgeR method revealed significant differences between LND and CON for 232 genes after correction for multiple comparisons at FDR < 0.05 (Figure 2B, Supplementary Figure S3A). The PCA for these 232 genes revealed some segregation of LND from CON, as expected when considering only genes that showed significant group differences. However, there was limited clustering of lines from the same case (Figure 2C). As expected from the role of HGprt in purine metabolism, the most significantly impacted biological pathways altered in the LND lines were driven by overlapping genes defining nucleotide, purine, and drug metabolism (Figure 2D). Among the other pathways identified, only p53 signaling is clearly related to purine metabolism.³³

Because prior reports suggested that inclusion of multiple lines from the same individual may increase false positive findings due to artificially low overall experimental variances from low within-subject differences,^4,5,10 the same data were re-evaluated with edgeR after addressing the inclusion of multiple sublines from the same case. The strategy involved pooling results for 3 lines from each individual and then comparing LND (N = 4) versus CON (N = 4). With this strategy, there was some separation of LND and CON in the PCA plot (Figure 2E) with significant differences for 183 genes using edgeR (Figure 2F, Supplementary Figure S3B). The PCA plot for these 183 genes (Figure 2G) showed a more obvious segregation of LND from CON. The main biological pathways affected showed some overlap with analyses when all lines were considered independently (Figure 2H). The HPRT1 gene was significantly altered when lines were analyzed as independent samples, but not after read counts were pooled (Figure 2F).

Differential gene expression using limma-voom

The limma-voom method was independently evaluated, because statistical algorithms have been created to address duplicate iPSC lines from the same individual (DupCor).⁴ The limma-voom method without DupCor produced significant differences for only 23 genes between the 12 LND and 12 CON iPSC lines when they were considered as independent samples (Figure 2I, Supplementary Figure S3C) and significant differences for only 2 genes (HPRT1 and POMZP3) when DupCor was applied (Figure 2J). The numbers of differentially expressed genes found with the limma-voom methods were insufficient to assess potential biological pathways. These results show the limma-voom method is statistically more conservative than edgeR, especially with the DupCor algorithm.

Comparison of differential gene expression across analytical methods

The differentially expressed genes between the LND and CON groups were also compared across the methods. A total of 89 genes overlapped (Figure 2K) when comparing the edgeR method applied to all samples as independent lines (232 genes) versus pooling lines from the same case (183 genes). Nearly all 23 genes found using the limma-voom method overlapped with 232 genes found with edgeR applied to all samples as independent lines (Figure 2L).

The partial overlap among differentially expressed genes may reflect the statistical criteria used to identify them, with some genes apparently not overlapping because they fell just short of the arbitrary cutoff at FDR < 0.05. To assess this possibility, Pearson correlations were evaluated for P-values for all genes found to be significantly altered with any method (Figure 2M). This analysis showed strong positive correlations (R = 0.64) between the edgeR and limma-voom methods without correcting for lines from the same individuals. There was also a near-perfect correlation (R = 0.99) between limma-voom with and without DupCor, because the main difference involves the calculation of the cutoff for P-values. Correlations found with edgeR after pooling 3 lines from the same individuals were lower and sometimes negative.

Overall, these correlations between results from edgeR and limma-voom methods when samples are considered independent suggest the methods identify similar subsets of genes, even though the number of genes that reach statistical criteria for significance varies, with the limma-voom method being more conservative. Results obtained after pooling data for lines from the same case before edgeR analyses reduced statistical power, most likely because the apparent number of samples is reduced from 12 per group to only 4 per group. More importantly, pooling produced results that had poor correlations with other methods.

Sensitivity and specificity for detecting differential gene expression

One limitation of the analyses above is that the ground truth for changes in gene expression is unknown for LND, except for the HPRT1 gene, because most of the mutations chosen result in nonsense-mediated decay with reduced HPRT1 mRNA levels (Table 1). For changes in other genes, it is impossible to know which analytical method produces the “best” result. Indeed, many of the differentially expressed genes could reflect “noise” (false positives). This question was assessed by permutation-based analyses.⁴ With this method, specific changes were introduced into the dataset, and the ability to detect these changes was assessed, with varying combinations of individuals and lines per individual.

For the edgeR method applied to all samples as independent iPSC lines, increasing the number of unique cases per group increased sensitivity for the detection of true positives (Figure 3A). Increasing the number of lines per individual improved the detection of true positives. False positive findings decreased with increasing numbers of unique cases, but they increased with multiple lines per case. However, the magnitude of this effect of multiple lines from the same individual on false positives was small (Figure 3A). The AUC metric confirmed that larger numbers of individuals improved overall results (Figure 3B). Two lines per individual further improved results, but there was not much added value from 3 lines per individual (Figure 3B).

Figure 3.

Assessment of sensitivity and specificity with edgeR. Panel A shows results of all possible permutations of different numbers of individual cases and different numbers of iPSC lines for each case, with each line considered an independent sample. Panel B shows the median for the corresponding AUC (area under the curve) metric to assess overall sensitivity and specificity. Panel C shows the impact of the fold-change as the median value of all permutations. Panels D-F show parallel analyses following pooling iPSC lines from the same case.

The ability to detect changes in gene expression depends on the magnitude of the change, and the majority of true positives could be detected at FC = 3-5X, even with only two cases per group and a single line for each (Figure 3C). At the more commonly used FC cutoff of 1.5X, the median number of true positives detected was only 78 of 400 total (20%) across all permutations involving 1 line from 4 cases per group. Sensitivity increased to a median of 264 of 400 genes (75%) with 3 lines from each of 4 cases per group (Figure 3C). False positives are determined mostly by the random groupings of samples during permutation analyses, not by the FC introduced, so false positives did not change substantially with different FC (Figure 3C). The practical consequences of the effect of FC on overall results are substantial, because evaluation of the empirically derived data showed that a large proportion of differentially expressed genes had FC between 1.5-2.0 (Supplementary Figure S4A,B).

Sensitivity and specificity were also assessed using edgeR after pooling all 3 related lines for each case prior to permutation analyses. Sensitivity for detecting true positives was not improved following pooling (Figure 3D). The AUC metric for pooling produced trends that fell between using 1 line vs 2-3 lines individually (Figure 3E-F).

The limma-voom method produced similar trends as edgeR when multiple lines from the same case were treated as independent samples, but it was statistically more conservative. The number of false positives detected with limma-voom was lower than with edgeR, at the cost of increased false negatives (Figure 4A-C). Indeed, best outcomes with limma-voom detected only about half the true positives. The application of DupCor was even more statistically conservative, such that the majority true positives could not be detected without 3 lines per case (Figure 4D-F). Overall, permutation analyses suggested that correction for multiple iPSC lines from the same individual (either by pooling samples for edgeR or using DupCor in limma-voom) did not provide a statistical advantage over analyses without these corrections.

Figure 4.

Assessment of sensitivity and specificity with limma-voom. Panel A shows results of all possible permutations of different numbers of individual cases and different numbers of iPSC lines for each case, with each line considered an independent sample. Panel B shows the corresponding AUC (area under the curve) metric to assess overall sensitivity and specificity. Panel C shows the impact of the fold-change as the median value of all permutations. Panels D-F show parallel analyses following application of statistical methods to address iPSC lines from the same case (DupCor).

Reproducibility of differential gene expression results

The current results for gene expression involving 12 LND and 12 CON iPSC lines were also compared with a previously generated RNAseq dataset involving only 6 LND and 6 CON lines. The methods used across these studies were similar including derivation and growth of iPSC lines and RNAseq. With edgeR applied to results from each iPSC line as an independent sample (12 LND and 12 CON), there were 232 differentially expressed genes in the current dataset (12 LND and 12 CON) and 407 genes in the prior dataset (6 LND and 6 CON). Among all these genes, only 47 overlapped (Figure 5A, Supplementary Figure S5A,B).

Figure 5.

Robustness of results across two independent datasets. Panel A shows differentially expressed genes for the current dataset (232 genes) and a previous dataset (407 genes), both collected and analyzed in the same way. Although only 47 genes overlapped between these studies, there was a high correlation when considering the fold-change for either significantly differentially expressed genes (B) or all genes (C). Variance portioning was used to disclose the main source of variance. Panel D shows source of variance when considering expression of all genes in the current dataset, and panel E shows sources of variance when considering only differentially expressed genes. Similar analyses are also provided for a prior dataset for all genes (F) versus only differentially expressed genes (G). after combining both datasets into a single analysis, the effect of batch can also be seen (H).

This limited overlap reflects in part the arbitrary statistical cutoff at FDR < 0.05, with some genes just missing criteria. There was a high correlation between LND and CON for the fold changes for differentially expressed disease genes (R = 0.75, Figure 5B) as well as all genes across both datasets (R = 0.68, Figure 5C). These results imply that overall changes in gene expression were similar across these independent datasets, even when the same genes are not identified by the statistical criteria applied.

Sources of variance in gene expression

Prior studies have suggested that background genetic variation among individual cases are the main contributors to variance in gene expression among iPSC lines. However, these conclusions were often derived following analysis of results from different iPSC lines generated using different methods, sometimes in different laboratories. For the current dataset where these technical variables were controlled, variance partitioning was used to determine the main sources of variance across all 24 iPSC lines. When considering all genes, the disease (LND vs CON) had only a small effect on gene expression (Figure 5D). Differences between individuals contributed a greater effect to overall variance. However, the greatest effect was in residual variables. These results indicate that unmeasured technical factors have the greatest contribution to overall variance in gene expression, not differences between individual cases. When considering only the 232 genes that were differentially expressed between the LND and CON groups, the disease had a greater effect on variance, as expected (Figure 5E). However, residual technical variables still contributed the greatest effect to variance.

When considering sources of variance for all genes in the prior dataset, individuals contributed more variance than the disease, although the greatest variance again came from residuals. When considering only differentially expressed genes, the disease contributed more variance than individuals or residuals. When the current and prior datasets were analyzed together, the effect of the batch as a technical variable appeared to be greater than the disease or the individual.

Discussion

For modeling rare genetic diseases, the preparation of iPSCs is both time-consuming and expensive. Further, making multiple lines from the same individual is easier than making independent lines from multiple individuals. As a result, there has been great interest in delineating how many unrelated lines are needed, how lines from the same individual impact findings, and how to define a biological from a technical replicate. As highlighted in recent reviews, the majority of previously published studies of rare diseases include only 1-3 unique individuals per group, with only 1-2 sublines per case.^4,5 Current recommendations call for a minimum of 4 unique individuals per experimental group (4 diseases and 4 controls), and the application of statistical corrections when the study includes multiple sublines per individual.^4,5,15-17 The current results focusing on LND caused by mutations in the HPRT1 gene are only partly in line with these recommendations. For studies of differential gene expression, the current results support the recommendation for including iPSC lines from 4 unique individuals per group. The current studies do not support recommendations that caution against the use of multiple sublines from the same individual without statistical correction for relatedness of these lines. While methods to correct for duplicate lines from the same individual reduce false positive findings, the cost of this strategy is that the majority of true positive findings are missed.

Minimum number of iPSC lines for study

In keeping with existing recommendations, the empirical results described here suggest the need for iPSC lines from 3 to 4 unique individuals to achieve optimal sensitivity and specificity in differential gene expression studies. Including 2 sublines from each individual improves results, but adding a 3rd line had little added benefit. However, the number of lines required also depends on the magnitude of differences in gene expression. Very large changes in gene expression (3-5X) may be detectable with a single pair (one disease and one control) of iPSC lines. The magnitude of the disease's effect on gene expression also applies to other analytical endpoints. For example, mutations in HPRT1 have a robust effect on enzyme function, which can be detected with a single pair of iPSC lines (Table 1). The LND iPSCs also all waste large amounts of hypoxanthine, because it is a substrate that is not metabolized when the enzyme is missing.²⁴ Loss of enzyme activity and its immediate metabolic consequence are both robust effects of the disease which can be detected with a single pair of iPSCs (one disease and one control). The varying numbers of iPSC lines needed has also been reported for other experimental endpoints.⁵

The impact of HPRT1 mutations on the broader transcriptome appears to be weaker (Figure 5A) and therefore results with different analytical strategies vary more (Figure 2-4). Three of the four HPRT1 mutations studied are subject to nonsense-mediated decay with low mRNA levels (Table 1). As a result, reduced expression of HPRT1 mRNA could be detected using most analytical methods. The ability to detect changes in expression for other genes depended on the numbers of unique individuals studied, the number of sublines per source, the magnitude of any change, and the analytical methods applied (Figure 3-4).

It is important to consider the possibility that the differences in results obtained using different analytical methods mostly reflect “noise” (false positives). This explanation seems unlikely, because permutation analyses with known changes introduced into the dataset produced a relatively low rate of false positives when sample size was large (Figure 3-4). In addition, the results found in the current study were similar to those reported for a prior independent dataset involving a smaller number of samples.²⁴ For LND, the most consistent findings involved 2 genes (HPRT1 and POMZP3), because they were found using the most stringent statistical methods in the current dataset (Figure 2B,I,J) and in a prior independent dataset (Supplementary Figure S5B). Changes in HPRT1 might be expected. Changes in POMZP3 are not expected. This gene is therefore probably disease-relevant and merits attention in future studies.

Another insight from the current studies relates to the value of the AUC metric for identifying optimal sensitivity and specificity. An AUC > 0.8 is often considered adequate for most needs, but careful scrutiny of the data reveals that this value may be too generous. For example, an AUC > 0.9 is achieved even with a single line for only 2 cases per group; yet only half of the true positives are detected (Figure 3-4). These results argue against over-reliance on the AUC metric to guide optimal study designs. If needed, the empirical results can be subject to permutation analyses to estimate statistical power for any experiment.

Statistical correction for multiple iPSC sublines from the same individual

Numerous prior studies have cautioned against the use of multiple sublines from the same individual, unless statistical procedures for addressing the relatedness of these lines are employed.^4,5,15-17 The current results question the value of this recommendation. The PCA plots did not show obvious clustering for the 3 related lines from each of 4 CON or 4 LND cases, suggesting their gene expression patterns are sufficiently unrelated to be viewed as independent samples. Furthermore, strategies involving pooling results from related lines before analyses produced poor correlations with other methods, and using limma-voom with DupCor appeared to be excessively conservative. Indeed, the permutation analyses confirm that methods to correct for multiple lines from the same case reduce false positives, at the cost of missing the majority of true positives.

There are two main reasons why the current results do not match prior recommendations regarding the need to correct for multiple iPSC lines from the same individual. One reason is that some of these prior studies were based on aggregating results from iPSC lines prepared in different ways. For example, one early study evaluated more than 100 lines from 62 unique individuals produced by the HipSci Consortium along with more than 100 lines from 37 unique individuals from the NHLBI NextGen Consortium.⁴ HipSci used the Sendai virus method for reprogramming fibroblasts, while NHLBI used an mRNA method for reprogramming erythroblasts. For both collections, iPSCs were generated in separate batches across different labs. When aggregating results in this way, it is difficult to discriminate the proportion of variance that comes from the unique genetic background of individuals from other technical factors involved in the preparation of each line. Indeed, our results point to technical issues and batch effects as more important contributors to variance than individual cases (Figure 5).

These technical factors may include donor characteristics (eg, sex, age at sampling, race), starting type of donor cell (eg, fibroblast, erythroblast, leukocyte), method of derivation (eg, retroviral, Sendai virus, episome, mRNA), passage number (which may vary from 10 to more than 100), and unmeasured genetic or epigenetic contributions. Another potential technical source of variation relates to the pluripotency state of iPSCs. For disease modeling, pluripotency is usually defined using immunostains for specific pluripotency markers, gene expression studies documenting expression of relevant genes, differentiation of the cells into all three germ layers, and sometimes additional bioinformatics tools such as PluriTest²⁵ or ScoreCard.³⁴ All of these methods were applied here. However, the pluripotency state is not black or white, but encompasses numerous different states, which change dynamically in culture.^7,8 The assessment of differences in these states is not usually conducted in studies of disease modeling, because it requires single-cell RNAseq. Although it is not feasible to control for all potential variables for each experiment, the current study controlled for many including type of source cell (fibroblasts), sex (all males), roughly equivalent age groups (for cases and controls), same reprogramming method (mRNA), and similar passage level (~20 passages). Furthermore, all samples were collected by the same individual under the same conditions and studied in a single RNAseq run. This approach minimizes technical batch effects in ways not clearly excluded in prior studies.

Another explanation for why the current results do not match prior recommendations relates to how iPSCs are often derived. The most common strategy is to conduct reprogramming of somatic cells in one batch, for example a single culture plate.^{4,8,10,11,13,35} Potential iPSC colonies are then isolated from this batch by manual scraping of visible colonies or the sample is replated for isolation of individual colonies. This strategy carries the risk that multiple apparently independent isolates are not truly independent but derive instead from the same source clone. In this situation, the iPSC lines would be considered technical replicates, not independent lines. Numerous prior studies highlighting the relatedness of sublines from the same individual were conducted in this way, and in some cases the application of batch processing is not mentioned. In the current studies, special precautions were taken to avoid this risk for all lines by selecting only a single iPSC line from independently reprogrammed batches.

Conclusions

For disease modeling with iPSC lines, the best statistical approach for the number of unique starting lines and the number of lines from each source depends on the robustness of the disease effect. For strong disease effects, a single pair of case-derived lines may be sufficient. For weaker effects, a larger sample is needed. In this situation, comparisons should involve 3-4 unique individuals, with 1-2 lines for each. Available methods for addressing multiple lines from the same source appear to be very conservative, and they are probably not appropriate for exploratory studies. A PCA analysis assessing relatedness of iPSC lines may be useful to determine when correction for multiple lines from the same source may be warranted. Permutation analyses can also be conducted with any empirically derived dataset to estimate sensitivity and specificity.

When the effect of the disease is unknown, the best experimental design may also depend on whether the experimental goal is exploratory or hypothesis-testing. If the main aim is to confirm a preconceived hypothesis, then rigorous statistics to avoid incorrect conclusions from false positive findings is essential, and independent confirmation with other methods such as use of gene-edited iPSCs may be valuable. On the other hand, if the goal is exploratory and hypothesis-generating because the disease effects are not known, then it may be more important to avoid false negatives.

Another strategy for disease modeling involves gene editing of iPSC lines to introduce a disease-causing mutation into a normal iPSC line. It is sometimes claimed that gene-edited iPSCs may be preferable to case-derived lines. However, derivation of gene-edited lines is technically more demanding, and empirical comparisons of these two strategies for disease modeling are lacking. The gene editing method also produces considerable variance, although the sources of variance are somewhat different from those affecting case-derived lines. Whether the gene-editing strategy is superior to the case-derived strategy for disease modeling remains unclear because of the lack of rigorous empirical comparisons of these two methods.

Author contributions

ARD contributed to data curation, formal analysis, visualization, and writing the original manuscript draft. DJS contributed to methodology, investigation, data curation, validation, and review and editing of the manuscript. LG, AT and EO contributed to investigation, validation, and review and editing of the manuscript. EJH, JEV and SK contributed to study design and review and editing of the manuscript. HAJ contributed to conceptualization, analysis, supervision, project administration, funding acquisition, and writing the original manuscript draft

Conflicts of interest

The authors declare no competing interests.

Ethical statement

Cells from human subjects were obtained using appropriate ethical standards and approved by the Emory University Institutional Review Board (IRB protocol IRB00013934). For healthy control subjects, informed written consent was obtained. For LND subjects who have cognitive impairment and cannot hold a pen to write, informed written consent was obtained legal guardians, and verbal assent was obtained from the subjects.

Data availability

Data are available on request from the corresponding author.