InMeRF: prediction of pathogenicity of missense variants by individual modeling for each amino acid substitution

Abstract

In predicting the pathogenicity of a nonsynonymous single-nucleotide variant (nsSNV), a radical change in amino acid properties is prone to be classified as being pathogenic. However, not all such nsSNVs are associated with human diseases. We generated random forest (RF) models individually for each amino acid substitution to differentiate pathogenic nsSNVs in the Human Gene Mutation Database and common nsSNVs in dbSNP. We named a set of our models ‘Individual Meta RF’ (InMeRF). Ten-fold cross-validation of InMeRF showed that the areas under the curves (AUCs) of receiver operating characteristic (ROC) and precision–recall curves were on average 0.941 and 0.957, respectively. To compare InMeRF with seven other tools, the eight tools were generated using the same training dataset, and were compared using the same three testing datasets. ROC-AUCs of InMeRF were ranked first in the eight tools. We applied InMeRF to 155 pathogenic and 125 common nsSNVs in seven major genes causing congenital myasthenic syndromes, as well as in VANGL1 causing spina bifida, and found that the sensitivity and specificity of InMeRF were 0.942 and 0.848, respectively. We made the InMeRF web service, and also made genome-wide InMeRF scores available online (https://www.med.nagoya-u.ac.jp/neurogenetics/InMeRF/).

INTRODUCTION

Development of the high-throughput sequencing (HTS) technologies has enabled identification of a large number of single-nucleotide variants (SNVs) in human diseases in a reasonable time at a reasonable cost. Nonsynonymous SNVs (nsSNVs) constitute most of the identified pathogenic SNVs, although the numbers of pathogenic intronic splicing SNVs and pathogenic copy number variations are also increasing (1). The Human Gene Mutation Database (HGMD) (2) and ClinVar (3) are major databases of pathogenic SNVs. Along with advancement of the HTS technologies, in silico tools to predict pathogenicity of an identified nsSNV have been developed using various machine learning techniques, including (i) support vector machine (SVM) [CADD (4,5), MetaSVM (6) and MutPred (7,8)], (ii) random forest (RF) [DEOGEN2 (9), MutPred (7,8), REVEL (10) and VEST (11)], (iii) gradient boosting decision tree [M-CAP (12)], (iv) logistic regression [MetaLR (6)], (v) hidden Markov model [FATHMM (13)] and (vi) deep learning [DANN (14), PrimateAI (15) and MVP (16)].

We previously used SVM modeling to make IntSplice for predicting the splicing effect of SNVs at intronic positions −50 to −3 (17). In this communication, we made RF models to differentiate pathogenic nsSNVs in HGMD and common nsSNVs in dbSNP. Compared to SVM modeling, RF modeling is suitable for a large dataset, and requires less computer resources and less efforts in tuning hyperparameters (18,19). We named a set of our models ‘Individual Meta RF’ (InMeRF). We called it ‘individual’, because we generated an RF model for each of 150 amino acid (AA) substitutions. Among 20 × 19 = 380 possible AA substitutions, nsSNVs are able to cause 150 AA substitutions. Individual modeling alleviated a possible bias that makes radical AA changes being predicted to be pathogenic. We made an InMeRF web service. We also analyzed all possible nsSNVs in the human genome, and made the genome-wide InMeRF scores available online.

MATERIALS AND METHODS

Generation of InMeRF models

We used the HGMD Pro release 2015.2 (2) to obtain pathogenic nsSNVs. Among them, we filtered 72 556 pathogenic nsSNVs with both ‘CLASS = DM’, which stands for disease-causing mutation, and ‘MUT = ALT’, which indicates that an alternative nucleotide is a mutation. Even when an nsSNV was reported in multiple patients, the nsSNV was counted as a single nsSNV in the training dataset. We next extracted 166 161 common nsSNVs from dbSNP build 151 (20) included in dbNSFP v4.0a (21), for which a minor allelic frequency (MAF) was >0.001 in 1000Gp3_AF, UK10K_AF, ExAC_AF, gnomAD_exomes_AF or gnomAD_genomes_AF. We then filtered 162 918 common nsSNVs by removing nsSNVs either included in HGMD Pro release 2015.2 (2) or labeled as ‘clinvar_clnsig = Pathogenic or Likely_pathogenic’ in dbNSFP v4.0a.

Feature values to differentiate pathogenic and common nsSNVs were obtained from 37 tools in dbNSFP v4.0a. In dbNSFP v4.0a, the scores are normalized so that a pathogenic nsSNV gives rise to a rank score of 1 and a normal nsSNV returns a rank score of 0. We calculated percent coverages of the 37 tools for the pathogenic and common nsSNVs (Table 1). The coverages of M-CAP (12), MutPred (7,8) and LINSIGHT (22) for the common nsSNVs were as low as 37.24%, 6.21% and 3.52%, respectively (Table 1). We thus excluded these three tools, and rank scores of the remaining 34 tools in dbNSFP v4.0a were used as feature values [SIFT (23–26), SIFT4G (27), Polyphen2_HDIV (28), Polyphen2_HVAR (28), LRT (29), MutationTaster (30), MutationAssessor (31), FATHMM (13), PROVEAN (32,33), VEST4 (11), MetaSVM (6), MetaLR (6), REVEL (10), MVP (16), MPC (34), PrimateAI (15), DEOGEN2 (9), CADD (4,5), DANN (14), fathmm-MKL (35), fathmm-XF (36), Eigen (37), Eigen-PC (37), GenoCanyon (38), integrated_fitCons (39–41), GERP++ (42), phyloP100way_vertebrate (43,44), phyloP30way_mammalian (43,44), phyloP17way_primate (43,44), phastCons100way_vertebrate (43,44), phastCons30way_mammalian (43,44), phastCons17way_primate (43,44), SiPhy (45,46), and bStatistic (4,5)]. To make RF models, nsSNVs that lacked one or more of 34 rank scores in dbNSFP v4.0a were also excluded.

Table 1.

Thirty-seven tools in dbNSFP v4.0a (21) and their percent coverage for the sum of 72 556 pathogenic nsSNVs and 162 918 common nsSNVs, which were used to generate InMeRF models

Tool	Type	Pathogenic nsSNVs (%)	Common nsSNVs (%)	References
SIFT	Prediction	97.31	89.62	(23–26)
SIFT4G	Prediction	98.12	93.46	(27)
Polyphen2_HDIV	Prediction	92.14	80.88	(28)
Polyphen2_HVAR	Prediction	92.14	80.88	(28)
LRT	Prediction	93.93	72.45	(29)
MutationTaster	Prediction	99.94	95.72	(30)
MutationAssessor	Prediction	89.32	76.07	(31)
FATHMM	Prediction	98.27	87.35	(13)
PROVEAN	Prediction	98.29	90.39	(32,33)
VEST4	Prediction	99.35	95.72	(11)
MetaSVM	Prediction	99.40	94.08	(6)
MetaLR	Prediction	99.40	94.08	(6)
M-CAPa	Prediction	97.39	37.24	(12)
REVEL	Prediction	99.40	94.08	(10)
MutPreda	Prediction	81.09	6.21	(7,8)
MVP	Prediction	99.12	73.85	(16)
MPC	Prediction	91.76	75.79	(34)
PrimateAI	Prediction	96.72	85.13	(15)
DEOGEN2	Prediction	94.52	86.73	(9)
CADD	Prediction	100.00	100.00	(4,5)
DANN	Prediction	100.00	100.00	(14)
fathmm-MKL	Prediction	100.00	100.00	(35)
fathmm-XF	Prediction	86.76	92.20	(36)
Eigen	Prediction	87.62	92.02	(37)
Eigen-PC	Prediction	87.62	92.02	(37)
GenoCanyon	Prediction	100.00	100.00	(38)
integrated_fitCons	Prediction	87.68	97.44	(39–41)
LINSIGHTa	Prediction	0.07	3.52	(22)
GERP++	Conservation	99.98	98.51	(42)
phyloP100way_vertebrate	Conservation	100.00	99.97	(43,44)
phyloP30way_mammalian	Conservation	100.00	99.94	(43,44)
phyloP17way_primate	Conservation	100.00	99.90	(43,44)
phastCons100way_vertebrate	Conservation	100.00	99.97	(43,44)
phastCons30way_mammalian	Conservation	100.00	99.94	(43,44)
phastCons17way_primate	Conservation	100.00	99.90	(43,44)
SiPhy	Conservation	99.88	97.09	(45,46)
bStatistic	Conservation	98.93	98.02	(4,5)

^aExcluded from features to make our models and from comparison with InMeRF because of low nsSNV coverages.

The pathogenic and common nsSNVs were classified into 150 different nonsynonymous AA substitutions, which possibly occur by a single-nucleotide substitution. In order to select nsSNVs with high MAFs, the common nsSNVs were sorted in descending order of MAF of 1000Gp3_AF. When MAF was absent in 1000Gp3_AF, MAFs of TWINSUK_AF, ALSPAC_AF, ExAC_AF, gnomAD_exomes_AF and gnomAD_genomes_AF were used in this order. NsSNVs without any MAF were ranked lower than MAF-bearing nsSNVs. In order to make unbiased RF models, the same numbers of pathogenic and common nsSNVs were used for each AA substitution (Supplementary Table S1A). We confirmed that modeling with all pathogenic nsSNVs and all common nsSNVs made the RF models less dependable compared to the balanced training datasets (Supplementary Table S1B). We also confirmed that modeling with the same numbers of pathogenic nsSNVs and ‘rare’ common nsSNVs similarly made less dependable RF models (Supplementary Table S1C). In this ‘rare’ common modeling, nsSNVs were sorted in ascending order of MAF of 1000Gp3_AF to select nsSNVs with low MAFs.

The 34 rank scores in dbNSFP v4.0a were used as feature values to make RF models to differentiate pathogenic and common nsSNVs. RF modeling was performed using a machine learning library, scikit-learn (47), on Python version 3.7. In this modeling, hyperparameters were set to (i) n_estimators (range(10, 110, 10)), (ii) criterion (‘gini’, ‘entropy’) and (iii) max_depth (i for i in range(1, 6, 1)). These hyperparameters represent that (i) the number of trees in the forest is selected from 10 to 100 by 10 steps, (ii) the function to measure the quality of a split is selected as Gini impurity or information gain and (iii) the maximum depth of the tree is selected from 1 to 5 by one step. Optimal hyperparameters were automatically determined by 10-fold cross-validation grid search. RF models were generated independently for each of 150 different AA substitutions. An overview of InMeRF modeling is shown in Supplementary Figure S1A.

Comparison of InMeRF with 25 other prediction tools in dbNSFP v4.0a

To compare InMeRF with 25 other prediction tools (Table 1), we used three testing datasets of VariBenchSelected, predictSNPSelected and SwissVarSelected (48) derived from VariBench (49,50), predictSNP (51) and SwissVar (52), respectively (Supplementary Table S2). Among the 28 prediction tools in dbNSFP v4.0a, M-CAP, MutPred and LINSIGHT were excluded from the comparisons because of low nsSNV coverages, as stated earlier. Some tools might have used subsets of the three testing datasets to generate the models, and are expected to show better predictive performance than the others.

Comparison of InMeRF with 10 other tools in the lack of overlapping nsSNVs between training and testing datasets

To compare InMeRF with 10 previously reported tools that were generated by an identical CADD training dataset (48), we generated InMeRF-CADD using the CADD training dataset (Supplementary Figure S1B). When a rank score was missing for a specific nsSNV in the testing datasets, an average of all rank scores for an AA substitution of interest was used as a rank score of the missing tool.

Application of InMeRF to nsSNVs associated with congenital myasthenic syndromes and spina bifida

We predicted the pathogenicity of 155 pathogenic nsSNVs and 125 common nsSNVs in seven major genes (DOK7, MUSK, AGRN, LRP4, CHRNE, COLQ and GFPT1) causing congenital myasthenic syndrome (CMS) (53,54), as well as in VANGL1 causing spina bifida.

RESULTS

A reason why we developed InMeRF

We identified p.R335C in VANGL1 in a patient with spina bifida. An arginine-to-cysteine (R-to-C) substitution causes a radical change in AA properties based on the BLOSUM62 score (55). Indeed, more pathogenic nsSNVs have been reported in radical AA substitutions compared to common nsSNVs (Figure 1). We thus pursued the pathogenicity of p.R335C, but found that p.R335C had no effect on subcellular localization of Vangl1, interaction of Vangl1 with SCRIB and embryonic development of zebrafish (see the Supplementary Information). This failure prompted us to make a prediction model for each AA substitution. Among 20 × 19 = 380 possible AA substitutions, 150 AA substitutions can occur by an SNV, whereas the other 230 AA substitutions require two or three nucleotide changes.

Figure 1.

The ratio of pathogenic nsSNVs to common nsSNVs of each AA substitution is plotted on each category of the BLOSUM62 score (55). The BLOSUM62 score of -3 like p.R335C in VANGL1 indicates a radical AA substitution, whereas the score of 3 indicates a slightly deleterious AA substitution. The ratios and BLOSUM62 scores of 150 AA substitutions are indicated in Supplementary Table S1B. P = 0.001 by the Jonckheere–Terpstra trend test, which is for testing ordered differences in medians, on R version 3.5.

Generation of InMeRF models

To circumvent a possible bias that makes radical AA changes being predicted to be pathogenic, we made an RF model to discriminate pathogenic and common nsSNVs for each of the 150 AA substitutions. InMeRF is thus comprised of 150 RF models representing all possible AA substitutions caused by an SNV. We used 72 556 pathogenic nsSNVs and 162 918 common nsSNVs in dbNSFP v4.0a to make our training dataset. To unbiasedly predict pathogenic and common nsSNVs, we used the same numbers of pathogenic and common nsSNVs for each RF modeling. The number of pathogenic or common nsSNVs used to make each RF model was on average 256.8 ranging from 12 to 1494 (Supplementary Table S1A). Feature values to differentiate pathogenic and common nsSNVs were obtained from rank scores of 34 tools in dbNSFP v4.0a. Each rank score represents a normalized score of a previously reported prediction tool with a rank score of 1 being pathogenic and a rank score of 0 being normal. In our RF models, pathogenic nsSNVs were predicted to be 1, and normal nsSNVs were predicted to be 0. A threshold to differentiate pathogenic and normal nsSNVs was set to 0.5.

Ten-fold cross-validation of InMeRF models generated by our training dataset

We evaluated the performance of our RF models by making receiver operating characteristic (ROC) and precision–recall (PR) curves by 10-fold cross-validation. We plotted an area under the curve (AUC) of an ROC curve of each AA substitution against the number of nsSNVs used to make each RF model (Figure 2A and Supplementary Table S1A). We similarly plotted an AUC of a PR curve of each AA substitution (Figure 2B and Supplementary Table S1A). We observed that ROC-AUCs and PR-AUCs were mostly independent of the number of nsSNVs in the training dataset, and even <500 nsSNVs mostly gave rise to ROC-AUCs and PR-AUCs of >0.9.

Figure 2.

Scatter plots of the AUC of an ROC curve (A) and a PR curve (B) against the number of nsSNVs in the training dataset. Values of 150 RF models are individually plotted.

We next calculated seven statistical measures [accuracy, precision/positive prediction value (PPV), recall/sensitivity, specificity, F1 score, negative predictive value (NPV) and Matthews correlation coefficient (MCC)] according to the Human Mutation guidelines (48,56–57) using 10-fold cross-validation. Statistical measures of 150 RF models are indicated in Supplementary Table S1A, and their means and standard deviations (SDs) are indicated in Table 2. Among the seven statistical measures, MCC had the lowest mean statistical measure of 0.882, whereas specificity had the highest mean statistical measure of 0.949 (Table 2).

Table 2.

Seven parameters of InMeRF comprised of 150 RF models by 10-fold cross-validation to evaluate prediction performance indicated in the Human Mutation guidelines (48,56–57)

Comparison of InMeRF models with 25 other prediction tools in dbNSFP v4.0a

We compared InMeRF with 25 prediction tools including 4 recently reported tools (MVP, MPC, PrimateAI and DEOGEN2), which were newly added in dbNSFP v4.0a, using three testing datasets (VariBenchSelected, predictSNPSelected and SwissVarSelected) (Supplementary Table S2) (48). Compared to the four recently reported prediction tools, ROC-AUCs of InMeRF were ranked first in all three testing datasets (Figure 3). Similarly, compared to the 25 prediction tools, ROC-AUCs of InMeRF were ranked fourth in VariBenchSelected and first in predictSNPSelected and SwissVarSelected (Supplementary Figure S2 and Supplementary Table S3). On average, both ROC-AUC and PR-AUC of InMeRF were ranked first compared to the 25 prediction tools (Supplementary Table S3D). This analysis, however, can be possibly affected by type 1 circularity, which represents that the same nsSNVs are included in both training and testing datasets (48). Scores given by InMeRF and by the 25 prediction tools for each nsSNV in the three testing datasets are indicated in Supplementary Table S4.

Figure 3.

ROC and PR curves of InMeRF and four newly added prediction tools in dbNSFP v4.0a using testing datasets of VariBenchSelected (A, B), predictSNPSelected (C, D) and SwissVarSelected (E, F). ROC and PR curves of InMeRF and 25 other tools including the 4 tools in this figure are indicated in Supplementary Figure S2.

Comparison of InMeRF models generated by the CADD training dataset with seven other tools generated by the CADD training dataset

Grimm et al. (48) generated 10 tools using an identical training dataset derived from CADD (4). They also made three independent testing datasets (VariBenchSelected, predictSNPSelected and SwissVarSelected) that had no shared nsSNVs with each other or with the CADD training dataset (complete lack of type 1 circularity). One of the 10 tools, MutationTaster (30), however, had the type 1 circularity stated earlier. Another tool, FATHMM’s weighted (FATHMM-W) (13,48), had type 2 circularity, which represents that the model is tuned to differentiate genes carrying only common nsSNVs and genes carrying only pathogenic nsSNVs, but not to differentiate nsSNVs within a single gene (48). Excluding these two tools, we compared InMeRF with the eight remaining tools. To this end, we generated InMeRF-CADD using the CADD training dataset (Supplementary Figure S1B). Scores and labels given by InMeRF-CADD and by the eight tools for each nsSNV in the three testing datasets are indicated in Supplementary Table S5. Statistical measures of InMeRF-CADD and the eight tools are shown in Supplementary Table S6. The mean ROC-AUCs were ranked in the order of InMeRF-CADD, PolyPhen2, MutationAssessor, SIFT, CADD, FATHMM-U, LRT, PhyloP and GERP++. The ROC and PR curves of the four top-ranked tools (InMeRF-CADD, PolyPhen2, MutationAssessor and SIFT) are shown in Figure 4. Similarly, the ROC and PR curves of InMeRF-CADD and the eight tools are shown in Supplementary Figure S3. Although the ROC-AUC of InMeRF-CADD was ranked first for each of the three datasets, PR-AUC of InMeRF-CADD was ranked second, third and first for VariBenchSelected, predictSNPSelected and SwissVarSelected, respectively (Supplementary Table S7).

Figure 4.

ROC and PR curves of InMeRF-CADD and three top-ranked tools using testing datasets of VariBenchSelected (A, B), predictSNPSelected (C, D) and SwissVarSelected (E, F). InMeRF-CADD and the three tools (48) were generated by an identical CADD training dataset. ROC and PR curves of InMeRF-CADD and eight other tools including the three tools in this figure are indicated in Supplementary Figure S3.

Application of InMeRF to congenital myasthenic syndromes and spina bifida

DOK7, MUSK, AGRN, LRP4, CHRNE, COLQ and GFPT1 are frequently mutated genes in CMS (53,54). In addition, our motivation to develop InMeRF arose from our failure in proving the pathogenicity of p.R335C in VANGL1 in spina bifida. We thus applied InMeRF to 155 pathogenic nsSNVs and 125 common nsSNVs in these eight genes. InMeRF predicted that 146 out of 155 truly pathogenic nsSNVs are pathogenic (sensitivity = 0.942) and that 106 of 125 truly common nsSNVs are normal (specificity = 0.848) (Supplementary Table S8). The sensitivity of 0.942 with these eight genes was comparable to the sensitivity (0.930 ± 0.040, mean and SD of 150 RF models) obtained by 10-fold cross-validation of the training dataset (Table 2). In contrast, the specificity of 0.848 was slightly inferior to the specificity (0.949 ± 0.034, mean and SD of 150 RF models) obtained by 10-fold cross-validation (Table 2). Predictions of pathogenic and common nsSNVs in MUSK and VANGL1 are schematically shown in Figure 5.

Figure 5.

Examples of application of InMeRF to nsSNVs in MUSK (A) and VANGL1 (B). Domains and the positions of nsSNVs are drawn to scale. Pathogenic nsSNVs and common nsSNVs with MAF >0.001 are indicated in red and blue letters, respectively. The number of nsSNVs in each category is indicated in parentheses. Pathogenic and common nsSNVs that were correctly predicted by InMeRF are indicated by open circles, whereas erroneously predicted SNVs are indicated by closed circles. (A) Ig, immunoglobulin domains; CRD_FZ, frizzled-like cysteine-rich domain; TMD, transmembrane domain; and Pkinase_Tyr, protein tyrosine kinase. (B) TMD, transmembrane domain. Arrow indicates the position of p.R335C (see the Supplementary Information).

Web service of InMeRF

We made a web service for InMeRF at https://www.med.nagoya-u.ac.jp/neurogenetics/InMeRF/. The InMeRF web service accepts (i) a genomic coordinate according to GRCh37/hg19, (ii) a genomic coordinate according to GRCh38/hg38, (iii) HUGO Gene Nomenclature Committee gene symbol or (iv) Ensembl Transcript ID. An nsSNV is predicted to be pathogenic when the probability of pathogenicity is ≥0.5, and the predicted result is returned to the same web page. When a genomic coordinate is entered, prediction scores of all possible nsSNVs for all transcripts annotated in Ensembl Release 94 are returned. Figure 6 shows the prediction result of chr11:46897102 associated with CMS (58). We also analyzed all nsSNVs registered in dbNSFP v4.0a according to the Ensembl Release 94 annotations on GRCh37/hg19 and GRCh38/hg38. dbNSFP includes all possible nsSNVs in the human genome by in silico saturation mutagenesis, and we followed the strategy of dbNSFP. We made the genome-wide pre-analyzed InMeRF scores available under the download tab of the InMeRF web service.

Figure 6.

A screenshot of InMeRF web service. We previously reported that a heteroallelic C→T substitution in LRP4 at position 46 897 102 (GRCh37/hg19) on chromosome 11 predicting p.R1277H (red circle) in a patient with CMS compromises binding of LRP4 to MuSK and agrin, and reduces clustering of acetylcholine receptor (58). When the genomic coordinate ‘chr11:46897102’ is entered in the InMeRF web service, the result shows up on the same window.

DISCUSSION

Failure to identify pathogenicity of p.R335C in VANGL1 prompted us to develop InMeRF to predict the pathogenicity of genome-wide nsSNVs. We hypothesized that most of the currently available tools tend to predict that minor AA changes are benign, whereas major AA changes are damaging. We thus made individual RF models for 150 AA substitutions. One possible disadvantage of our approach was that the numbers of nsSNVs used to generate an RF model became low. However, we were able to make RF models with ROC-AUC ≥ 0.845 and PR-AUC ≥ 0.882 even with small numbers of training variants (Figure 2 and Supplementary Table S1A). As is commonly recognized in machine learning, we observed that RF models generated with equal numbers of pathogenic and common nsSNVs made more dependable models (Supplementary Table S1A) than RF models generated with all pathogenic and common nsSNVs (Supplementary Table S1B). We suppose that individual RF modeling for 150 AA substitutions enhanced the advantage of balancing pathogenic and common nsSNVs. InMeRF was superior to or at least not inferior to the other tools by comparing InMeRF with 25 other prediction tools in dbNSFP v4.0a (Figure 3, Supplementary Figure S2 and Supplementary Table S3) and by comparing InMeRF-CADD with 8 other tools (Figure 4, Supplementary Figure S3 and Supplementary Table S7) that were generated by the CADD training dataset.

Type 1 circularity indicates that the same variants are included in both training and testing datasets (48). With type 1 circularity, a tool can efficiently predict pathogenicity of nsSNVs in the training dataset, but the efficiency is reduced for novel nsSNVs. Prediction tools of nsSNVs are easily subject to type 1 circularity, which spuriously increase prediction accuracies (48). Although the comparison of InMeRF with 25 other tools could be affected by type 1 circularity, the comparison of InMeRF-CADD with the 8 CADD-based tools should be free from type 1 circularity.

Type 2 circularity indicates that a model differentiates genes exclusively with common nsSNVs and genes exclusively with pathogenic nsSNVs, but not both in a single gene (48). The SwissVarSelected testing dataset contains many genes carrying both pathogenic and common nsSNVs in a single gene (48). We found that both ROC-AUC and PR-AUC of InMeRF-CADD were ranked first with the SwissVarSelected testing dataset (Supplementary Table S7). Lack of type 2 circularity in InMeRF is also suggested by representative analysis of pathogenic and common nsSNVs in seven frequently mutated genes (DOK7, MUSK, AGRN, LRP4, CHRNE, COLQ and GFPT1) causing CMS, and the VANGL1 gene causing spina bifida (Figure 5 and Supplementary Table S8).

In RF modeling of InMeRF, we used rank scores of 34 previously reported tools as feature values (Supplementary Figure S1). Analysis of feature importance of each tool for 150 RF models disclosed that REVEL (10), MetaLR (6), VEST4 (11), MVP (16), DEOGEN2 (9) and MetaSVM (6) provided the best six feature values in InMeRF (Supplementary Table S9). These six tools have been reported in or after year 2016. Comparison of ROC-AUC of InMeRF with those of 25 other tools indeed showed that REVEL (10), MetaLR (6), VEST4 (11), MVP (16), DEOGEN2 (9) and MetaSVM (6) were on average ranked second to seventh after InMeRF (Supplementary Table S3D). We also reduced the number of feature values one by one in descending order of the feature importance (Supplementary Table S9), and generated 150 RF models. ROC-AUC remained high even when we eliminated less dependable feature values (Supplementary Figure S4). InMeRF is thus largely dependent on the performance of these recently reported tools. Superiority of InMeRF over these recently reported tools may lie in the ensemble of AA substitution-specific RF models.

Development of InMeRF disclosed that the degree of a change in AA properties is likely to have a substantial effect on the prediction of pathogenicity. Lastly, our first motivation to develop InMeRF was satisfied by finding that p.R335C in VANGL1 was predicted to be benign with an InMeRF score of 0.342.

DATA AVAILABILITY

The InMeRF web service and the genome-wide InMeRF scores are openly available at https://www.med.nagoya-u.ac.jp/neurogenetics/InMeRF/. The ClinVar, dbNSFP and CADD were obtained from openly available databases at https://www.ncbi.nlm.nih.gov/clinvar/, https://sites.google.com/site/jpopgen/dbNSFP/ and https://cadd.gs.washington.edu/, respectively. HGMD Pro is a commercial database, and is available upon purchase from QIAGEN.

SUPPLEMENTARY DATA

Supplementary Data are available at NARGAB Online.

FUNDING

Japan Society for the Promotion of Science [18K14684, 16H04657 and 19H03329]; Ministry of Health, Labour, and Welfare, Japan [H24-Shinkei-Kin-Ippan-005 and H29-Nanchi-Ippan-030]; Japan Agency for Medical Research and Development [19ek0109230, 20gm1010002, 20ek0109281 and 20bm0804005]; National Center of Neurology and Psychiatry (2-5); Hori Sciences and Arts Foundation.

Conflict of interest statement. None declared.