Abstract
Aim: The purpose of this review was to evaluate the expression patterns of miRNAs in periodontal and peri-implant diseases, while identifying potential miRNAs with the greatest diagnostic ability as an oral fluid biomarker. Materials and methods: Human and animal studies were included when evaluating expression of miRNAs between health and different forms/stages of diseases, in which microarray and/or real-time polymerase chain reaction (RT-PCR) was carried out to detect fold changes in gene expression. After full-text analysis, 43 articles were considered for a qualitative assessment, and 16 miRNAs were selected to perform meta-analysis. Results: Based on human studies, results showed an overall upregulation of most of the evaluated miRNAs in periodontitis, with miRNA-142-3p and miRNA-146a being the most conclusive on both microarray and RT-PCR values and potentially serving as diagnostic biomarkers for disease activity. Conversely, miR-155 was the only miRNA revealing a statistically significant difference (SSD) (p < 0.05*) in experimental periodontitis models from RT-PCR values. Scarce scientific evidence is available from peri-implant diseases, however, most explored miRNAs in peri-implantitis were downregulated except for miR-145. Conclusions: Although our results revealed that a distinct differential expression of specific miRNAs can be noted between the state of health and disease, future research remains necessary to explore the functional role of specific miRNAs and their potential as therapeutic targets in periodontal and peri-implant diseases. MeSH Terms: periodontitis, peri-implantitis, epigenomics, microarray analysis, real-time polymerase chain reaction, microRNAs. Clinical relevance: Scientific background: Although most research identified different expression levels of miRNAs in periodontal and peri-implant diseases compared to their counterparts, their actual role in the pathogenesis of these conditions remains unclear. Therefore, we aimed to present a systematic review and meta-analysis on the expression patterns of miRNAs in periodontitis and peri-implantitis, while identifying potential miRNAs with the greatest diagnostic ability as an oral fluid biomarker. Principal findings: In periodontitis-related studies, miRNA-142-3p and miRNA-146a were the most conclusive on both microarray and RT-PCR values. Scarce scientific evidence is available from peri-implant diseases. Practical implications: Both miRNA-142-3p and miRNA-146a might serve as future diagnostic biomarkers for disease activity in periodontitis. Yet, future research remains necessary to explore the functional role of specific miRNAs and their potential as therapeutic targets in periodontal and peri-implant diseases.
Keywords: periodontitis, peri-implantitis, epigenomics, microarray analysis, real-time polymerase chain reaction, microRNAs
1. Introduction
MicroRNAs (miRNAs) are a group of small noncoding RNAs of about 22 bp in length that regulate gene expression through post-transcriptional modifications by binding to the 3′-untranslated region of a target messenger RNA (mRNA) [1], which leads to suppression of gene expression either by degradation of a target mRNA or by prevention of its translation [2]. Interestingly, one miRNA can control the expression of several genes, whereas the expression of a certain gene can be controlled by several miRNAs [3].
MicroRNAs are considered as an epigenetic mechanism that modulate cellular processes, such as cell growth, apoptosis, and differentiation, and play fundamental roles in inflammatory responses and the development of diseases, e.g., cancer and rheumatoid arthritis [4].
Recently, expression of miRNAs in tissues affected by periodontitis and peri-implantitis has been investigated. Periodontitis and peri-implantitis are biofilm-induced conditions affecting teeth- and implant-supporting tissues [5], which mainly consists of Gram-negative, anaerobic, and microaerophilic bacteria that can colonize the subgingival tissues [6]. The bacterial biofilm elicits an inflammatory host response, which is influenced by environmental, genetic, and epigenetic factors [7], with the latter referring to alterations in the gene expression that are not encoded in the DNA sequence [8].
In this context, the interaction between miRNAs and Porphyromonas gingivalis, a key periodontal pathogen, has been extensively investigated. It was reported that miRNAs might mediate endotoxin tolerance through the modulation of mitogen-activated protein kinase (MAPK) signaling pathway, increase the sensitivity of toll-like receptors (TLRs) when exposed to bacterial lipopolysaccharide (LPS), or target the nuclear factor-kappa B (NF-κB) signaling pathway in response to bacterial stimuli [9]. Yet, the relationship between periodontal pathogens and miRNAs is still vague and requires further exploration and investigation, especially that epigenetic mechanisms have therapeutic potential for improving individualized drug therapy.
MicroRNAs have recently emerged as key regulators in bone hemostasis; they affect osteoclastogenesis either by directly regulating osteoclast activity, signaling intermediates, or through negative-feedback loops, while they control osteogenic lineage commitment of various stem cells through positive-feedback loops [10]. As such, they might mediate alveolar bone resorption, which characterizes both peri-implantitis and periodontitis [9].
Although most research identified different expression levels of miRNAs in periodontal and peri-implant diseases compared to their counterparts, their actual role in the pathogenesis of these conditions remains unclear. Conversely, miRNAs have been proposed as valuable future diagnostics tools in periodontal and peri-implant diseases as high-quality miRNAs are reportedly detectable in oral fluids [11,12,13].
Therefore, we aimed to present a systematic review and meta-analysis on the expression patterns of miRNAs in periodontitis and peri-implantitis, while identifying potential miRNAs with the greatest diagnostic ability as an oral fluid biomarker.
2. Results
2.1. Study Selection
Initial screening of electronic databases yielded a total of 1242 articles, and manual search identified 2 publications. After elimination of duplicated studies, 1240 titles and abstract were further evaluated. The screening of titles and abstracts resulted in 138 potentially relevant articles to be selected. The full text of the relevant studies was obtained and thoroughly evaluated. Next, 95 investigations were excluded due to the lack of inclusion of miRNA expression, no data on fold changes, no control group, or appropriate model as indicated in Table S3. Overall, 43 (32 human and 11 animal) studies fulfilled the inclusion and exclusion criteria and were assessed in this systematic review and meta-analysis (Figure 1).
2.2. Characteristics of Included Investigations
2.2.1. Human Studies
The characteristics of the included human studies are presented in Table S4. It must be noted that these studies were confined to periodontal disease, as no human studies are available on miRNA and peri-implantitis. Thirty-two articles [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42] evaluated miRNA expression in periodontitis among humans and were published from 2011 to 2020. The included studies were case–control, cross-sectional, and prospective studies, with sample sizes ranging from 3 to 150 and with an age range between 13 and 76 years.
Among these studies, 25 studies compared miRNA expression in periodontitis among healthy individuals, 3 studies were with healthy and obese, 4 studies used healthy individuals with systemic diseases, 2 studies included patients with acute coronary syndrome/coronary heart disease, while 2 studies investigated patients affected with diabetes.
The expression of miRNA was analyzed from different samples; the most harvested sample type was gingival biopsies (16 studies), followed by gingival crevicular fluid (GCF) (5 studies), periodontal ligament stem cells (PDLSC) from extracted teeth (5 studies), serum (4 studies), saliva (2 studies), and lastly, supragingival biofilm (1 study). It must be noted that in two studies, dual samples were harvested; 1 study analyzed expression from gingival biopsies and GCF, while the other study performed the analysis from serum and GCF. In all the included studies, broad and specific miRNA expression was analyzed with different techniques including RT-PCR, microarray, and next-generation sequencing.
2.2.2. In Vivo (Animal) Studies
The characteristics of the included animal (in vivo) studies are presented in Table S5. A total of 11 studies were included: 9 studies [43,44,45,46,47,48,49,50,51] on the expression of miRNAs in periodontitis and 2 studies [52,53] in peri-implantitis. Regarding the animal models used, 6 studies utilized rats, 2 utilized mice, and 1 utilized monkeys for experimental periodontitis. Ligature-induced model was utilized in one study in mice and four studies in rats. Polymicrobial inoculation was utilized in one study in mice and one study in rats. One of these studies considered a dual model of ligature and polymicrobial inoculation [51]. Lastly, one experiment selected monkeys with naturally occurring periodontitis [48]. Samples were harvested from serum, gingival biopsies, and block samples (histology). The applied techniques to analyze miRNA expression were RT-PCR and microarray.
As for peri-implantitis, both studies utilized ligature-induced model in dogs. Samples were strictly harvested from gingival tissues. The techniques applied to analyze the miRNA expression were microarray and RT-PCR.
2.2.3. Quality Assessment of Selected Human and Animal Studies
The quality assessment of all reviewed articles followed the Cochrane Handbook for Systematic Reviews of Interventions [54] (Tables S6–S8). Figure 2 shows the risk of bias for human nonrandomized cohorts/case–control (2a), human cross-sectional (2b), and in vivo (animal) (2c) studies. Although, the majority of human studies had clear inclusion criteria and well-accepted definitions for periodontal disease, no marginal bone levels were reported by any study and probing depths were, somehow, inconsistent with active breakdown. Therefore, the overall risk of bias was unclear for human studies. On the other hand, in vivo studies lack clinical information to determine the severity of experimental periodontitis/peri-implantitis. Nonetheless, all studies followed and complied with ethical guidelines associated with animal research and provided valid and reliable methods to assess miRNA expression. Consequently, most in vivo studies were given an unclear and low risk of bias.
2.3. Meta-Analysis for miRNA Expression
Among all 43 included articles, a total of 563 miRNAs were identified reporting fold changes in miRNA expression between health and periodontal/peri-implant disease (Tables S9 and S10). Due to extensiveness and limitations in the reported data, only 16 miRNAs were selected to perform a meta-analysis and were chosen if fold changes values of the same miRNA were reported in both animal and human studies or individually, in more than two and four articles from animal and human studies, respectively [13,15,17,18,20,21,22,23,26,28,29,31,32,33,35,37,38,41,42,44,45,46,47,48,49,50,51].
A summary of the results from the present meta-analysis is depicted in Table 1 and forest plots of the most relevant miRNAs from periodontitis and peri-implantitis studies are shown in Figure 3, Figure 4 and Figure 5. Based on human studies, there is an overall upregulation of most of the evaluated miRNAs in periodontitis, with miR-142-3p and miR-146a being the most conclusive on both microarray and RT-PCR values and potentially serving as diagnostic biomarkers for disease activity. Conversely, miR-155 was the only miRNA revealing a statistically significant difference (SSD) (p < 0.05 *) in periodontitis from RT-PCR values of animal studies. Interestingly, most explored miRNAs in peri-implant disease were noted to be downregulated with an exception of miR-145 [52,53].
Table 1.
Periodontal Health vs. Periodontitis (Humans) | ||||||
---|---|---|---|---|---|---|
miRNA | Lab Test | Fold.Dif | SE | CI 95% | p-Value | I2 |
miR 24-3p | Microarray | 0.58 | 0.29 | 0.00 1.16 | 0.049* | 62.0% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 27b-3p | Microarray | 0.05 | 0.22 | −0.37 0.48 | 0.803 | 93.1% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 30e | Microarray | 2.34 | 1 | 0.37 4.31 | 0.020* | 60.5% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 130a | Microarray | 2.84 | 0.82 | 1.23 4.44 | <0.001*** | 0.0% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 132 | Microarray | 0.75 | 1.22 | −1.65 3.15 | 0.538 | 85.1% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 142-3p | Microarray | 2.71 | 0.39 | 1.94 3.47 | <0.001*** | 0.0% |
RT-PCR | 2.54 | 0.93 | 0.72 4.53 | 0.006** | 54.8% | |
miR 144 | Microarray | 1.97 | 0.37 | 1.24 2.71 | <0.001*** | 0.0% |
RT-PCR | N/A | N/A | N/A | N/A | N/A | |
miR 146a | Microarray | 1.56 | 0.54 | 0.49 2.63 | 0.004** | 0.0% |
RT-PCR | 3.48 | 1.35 | 0.84 6.12 | 0.010* | 86.9% | |
miR 155 | Microarray | −0.01 | 1.7 | −3.34 3.32 | 0.995 | 88.1% |
RT-PCR | 0.72 | 2.7 | −4.57 6.01 | 0.79 | 91.4% | |
miR 210 | Microarray | 0.72 | 0.24 | 0.25 1.18 | 0.002** | 9.8% |
RT-PCR | 0.33 | 0.8 | −1.23 1.89 | 0.681 | 87.9% | |
miR 223 | Microarray | 2.51 | 0.46 | 1.60 3.42 | <0.001*** | 0.0% |
RT-PCR | 4.73 | 3.09 | −1.34 10.8 | 0.127 | 90.0% | |
Periodontal health vs. Periodontitis (Animals) | ||||||
miRNA | Lab test | Fold.Dif | SE | CI 95% | p-value | I2 |
miR 132 | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | 0.17 | 0.47 | −0.76 1.09 | 0.726 | 47.70% | |
miR 146a | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | 1.2 | 0.75 | −0.27 2.66 | 0.109 | 88.40% | |
miR 155 | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | 1.54 | 0.71 | 0.14 2.94 | 0.031* | 74.20% | |
Peri-implant health vs. Peri-implantitis (Animals) | ||||||
miRNA | Lab test | Fold.Dif | SE | CI 95% | p-value | I2 |
let-7g | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | −1 | 0.28 | −2 | <0.001*** | 0.00% | |
miR 27a | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | −0.4 | 0.12 | −0.8 | <0.001*** | 0.00% | |
miR 29a | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | −0.23 | 0.09 | −0.45 | 0.001** | 0.00% | |
miR 142 | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | −0.36 | 0.16 | −0.72 | 0.022* | 0.00% | |
miR 145 | Microarray | N/A | N/A | N/A | N/A | N/A |
RT-PCR | 1.13 | 0.32 | 0.50 1.76 | <0.001*** | 0.00% |
Fold.Dif: fold difference, SE: standard error, CI: confidence interval, N/A: not available.
A statistical analysis was not possible to evaluate differences between methods of sample collection as of tissue biopsies was the procedure of choice in the majority of studies followed by serum, PDLSC from extracted teeth, block samples, saliva, GCF, and subgingival biofilm. On the other hand, a weighted mean difference (WMD) was used to compare miRNA expression differences between microarray and RT-PCR revealing that RT-PCR values are 0.71 higher than microarray and not to be SSD (p = 0.107, CI 95% (−0.15 to 1.58), WMD = 0.71), indicating that microarray is more sensitive to miRNA expression analysis than RT-PCR.
3. Discussion
MicroRNAs regulate gene expression through post-transcriptional modifications [1]. They modulate cellular functions, such as apoptosis and differentiation, and play key roles in inflammatory responses and in the development of diseases such as cancer and rheumatoid arthritis [4]. As such, expression of miRNAs has been increasingly a subject of interest in periodontitis and peri-implantitis, to better understand their role in the development of both diseases and periodontal/peri-implant tissue hemostasis.
In the present review, results from meta-analysis of human studies reflected an overall upregulation in expression of the most evaluated miRNAs in periodontitis in comparison to healthy periodontium. Interestingly, miRNA-146a was the most conclusive on both the microarray (p = 0.004) and RT-PCR (p = 0.010) values. miRNA-146a plays a critical role in the negative regulation of the innate immune response, and its dysregulation has been associated with several inflammatory disorders [26,31,38]. Regardless if it is upregulated or downregulated in periodontal disease, miRNA-146a seems to play a functional key role in periodontitis. In fact, Motedayyen et al. reported that an overexpression of miRNA-146a in patients with rapidly progressive aggressive periodontitis was accompanied by a reduction in the levels of TNF-α, IL-1β, and IL-6 [26], suggesting that the elevation of the miRNA-146a level regulates pro-inflammatory cytokines through a negative-feedback loop. Moreover, the unexpected finding of a decreased expression of these major pro-inflammatory cytokines points to the contribution of other inflammatory mediators and/or a nonimmunologic-dependent mechanism in progression of the disease.
Furthermore, miRNA-223 is significantly overexpressed in periodontitis (p < 0.001) and is a key regulator of osteoclastogenesis [55]. Irwandi and Vacharaksa suggested that miRNA-223 might be an important player in periodontitis-induced alveolar bone loss, having shown a consistent pattern of expression profile in gingival tissues isolated from individuals with periodontitis [56].
Other miRNAs significantly upregulated in periodontitis were miRNA-30e, miRNA-130a, miRNA-142-3p, and miRNA-210. Interestingly, these four miRNAs were reportedly overexpressed in the presence of periodontitis and obesity [31]. These specific miRNAs expression in obesity could target and post-transcriptionally modulate cytokine mRNA. In addition, they might provide new insights of how certain risk factors influence periodontal inflammation, and they are considered as novel therapeutic targets. miRNA-24-3p and miRNA-144 were also upregulated in periodontal disease, however, their role in periodontitis remains unclear, which warrants further research on these miRNAs.
Based on experimental periodontitis models, miRNA-155 expression was significantly upregulated in periodontitis (p = 0.031). Interestingly, conflicting findings on the expression of miRNA-155 was noted when compared to humans [33,35,37]. Such discrepancy could be as a result of differences in sample size and method applied in miRNA profiling. Since epigenetic modifications might be influenced by genetic ancestry and environmental exposure, these factors might add up to the discrepancy in results. Conversely, the impact of genetic ancestry is clearly mentioned only in one study [29], and at present, it is merely a speculation [9].
Regarding results of miRNA expression in peri-implantitis, they are confined to findings from two animal studies [52,53]. Interestingly, most miRNAs in peri-implant disease (let-7g, miRNA-27a, miRNA-29a and miRNA-142) were significantly downregulated with an exception of miRNA-145, which was significantly upregulated. In brief, Wu and colleagues demonstrated that let-7g, miRNA-27a, and miRNA-142 influenced the onset, progression, and treatment of peri-implantitis in a canine ligature-induced peri-implantitis model, highlighting the potential biological effects of the differentially expressed miRNAs and the specific enrichment of target genes involved in the MAPK signaling pathway [52,53].
It is of paramount importance to mention that several miRNAs displayed a dual regulation in advanced stages of periodontal disease. A possible explanation for this phenomenon could be due to the fact that one specific miRNA can target and regulate multiple genes [13]. Therefore, it is important to further elucidate on how the dual regulation of a specific miRNA might play a role in the progression and regression of periodontal disease.
Understanding the functional roles of miRNAs in the pathogenesis of periodontitis and peri-implantitis is very important due to their strong potential as therapeutic targets in alveolar bone regeneration. In this context, miRNAs have been shown to play an important role in the differentiation of periodontal ligament stem cells into an osteogenic lineage [25,57,58]. This could be of a great value, since stem cell therapy has reached a central role in the regenerative medicine paradigm [59]. Taking all the previous findings into consideration, it seems that miRNA therapeutics hold great promise for the future of periodontal/peri-implant therapy based on their ability to modulate the immune response to infection, either when applied in conjunction with synthetic antagomirs or by utilizing a more straightforward delivery strategy [60].
4. Materials and Methods
4.1. Information Sources
Electronic and manual literature searches were conducted by two independent reviewers (CGP and FA) in several databases, without any language restrictions, including PubMed, MEDLINE, EMBASE, and Cochrane Central Register of Controlled Trials and Cochrane Oral Health Group Trials Register databases for articles up to February 2020 published in English. The present protocol was registered in an international database of systematic reviews (PROSPERO).
4.2. Review (PECO) Question
The focus question for the present systematic review was developed using the PECO criteria. Do human/animal subjects (P) with periodontal/peri-implant diseases (E) compared to healthy subjects (C) have different patterns of miRNA expression (O)?
4.3. Search Strategy
This review was conducted according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement. The following search strategy was used (((“periodontal diseases” [MeSH Terms]) or periodontitis [Title/Abstract]) or peri-implantitis [Title/Abstract]) and (((“microRNA” [MeSH Terms]) or miRNA [Title/Abstract]) or mRNA [Title/Abstract]).
A second broader screening was conducted owing to the unspecific articles found indexed with the preliminary screening strategy: (((periodontal diseases [All fields]) or periodontitis [All fields]) or peri-implantitis [All fields]) and (((microRNA [All fields]) or miRNA [All fields]) or mRNA [All fields]).
Additionally, a manual search of periodontics- and implant-related journals, including Journal of Dental Research, Journal of Clinical Periodontology, Journal of Periodontology, and The International Journal of Periodontics & Restorative Dentistry, up to January 2020, was also performed to ensure a thorough screening process (CGP and FA). References of the included articles were also screened to check all the available articles. For the search of the gray literature, Google Scholar was used to identify any articles not included in the databases above.
4.4. Eligible Criteria
Articles were included if they evaluated the expression of miRNAs between health and different forms of periodontal (i.e., chronic or aggressive periodontitis) and peri-implant diseases (i.e., peri-implantitis) in human and animal studies. Investigations were selected if microarray and/or real-time polymerase chain reaction (RT-PCR) was carried out to detect fold changes in gene expression, of at least, one miRNA. Articles under following criteria were excluded: (I) if no fold changes were reported, (II) if no disease/experimental group was considered, (III) studies reporting on the miRNA expression from in vitro studies using cell lines not related to the oral cavity, (IV) published materials that were review articles, letters, personal opinions, book chapters, conference abstracts, and (V) articles published in a language other than English.
4.5. Data Extraction and Analyses
The same two authors (CGP and FA) independently reviewed all full-text articles. Discussions between the two authors initially resolved any disagreements. If the two authors did not reach an agreement, a third author (LL) made the final decision. The primary outcome considered was the fold changes of miRNA expression in periodontal/peri-implant diseases. Due to the observational nature and reported outcomes of the included investigations, only a qualitative descriptive analysis was performed upon demographic data and systematically reviewed using tables. Despite the heterogeneity among some of study subjects and subgroups, a quantitative synthesis of the included studies was possible to perform a meta-analysis. This report adhered to the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analyses) statement.
4.6. Risk of Bias and Quality Assessment of Selected Studies
Assessment of study quality and bias was performed for the included studies using the Cochrane review guidelines. Human nonrandomized cohorts and case–control studies were assessed using the Newcastle–Ottawa Scale (NOS) [61]. The criteria for the quality assessment of cross-sectional studies were established from the Joanna Briggs Institute (JBI) critical appraisal checklist for studies reporting prevalence data [62]. Animal research was assessed according to ARRIVE guidelines for in vivo experiments and assigned to predefined grades [63,64] (Tables S1 and S2). Two authors (CGP and FA) independently evaluated the included studies and disagreement were resolved by discussion to produce final scores. Summary assessment of risk of bias for outcomes within and across studies is represented in risk of bias graphics following Cochrane review guidelines. In summary, a low risk of bias was estimated when plausible bias is unlikely to seriously alter the results or the risk of bias was low in all key domains. The unclear risk of bias was estimated when plausible bias that raises some doubt about the results of risk of bias was in one or more key domains, whereas the high risk of bias was estimated when plausible bias that seriously weakens confidence in the results or risk of bias was in one or more key domains [54].
4.7. Statistical Analysis
Using a random effect model, a meta-analysis was considered to establish a statistically significant difference (p-values = <0.05*, <0.01**, and <0.001***) among studies reporting fold changes in miRNA expression. The DerSimonian–Liard approach was used to estimate studies variances with confidence interval (IC) of 95% and estimations were depicted using forest plots. Study heterogeneity was calculated with I2 and Cochran’s Q statistics, whereas publication bias was analyzed with Egger’s test and funnel plots using R 3.5.1 [65].
5. Study Limitations
The present systematic review was not able to establish a difference of the miRNA expression between methods for sample collection due to a high heterogeneity (i.e., biopsies, GCF, serum, and saliva), which warrants further caution when interpreting the results. On the other hand, most studies barely elucidated the functional roles of upregulated/downregulated miRNAs in the pathogenesis of periodontal and peri-implant diseases. Furthermore, due to the preliminary nature of microarray studies, further confirmation of miRNA expression in periodontitis/peri-implantitis is required.
6. Conclusions
We concluded that a distinct differential expression of specific miRNAs can be noted in periodontal, peri-implant disease, and healthy specimens. Moreover, miRNA expression might be affected by systemic conditions including cardiovascular diseases, obesity, and diabetes. It was noted that microarray is more sensitive method than RT-PCR for miRNA profiling, with miRNA-142-3p and miRNA-146a being the most promising miRNA biomarkers for diagnostic purposes. Scarce research was available in regard to miRNA expression and peri-implant disease. Therefore, future research is still necessary to explore the functional role of specific miRNAs and their potential role as therapeutic targets.
Acknowledgments
The authors would like to thank Juan Luis Gomez (stHalley Statistics) for his support with the statistical analysis. FA is supported by the Osteology Research Scholarship (Osteology Foundation, Lucerne, Switzerland). For this study, FA also received additional funding from the Royal Society of Arts and Sciences in Gothenburg (KVVS, Sweden).
Supplementary Materials
Supplementary materials can be found at https://www.mdpi.com/1422-0067/21/11/4147/s1. Table S1. Criteria for quality assessment of animal studies using a predefined grading system (Schwarz et al. 2012) based on ARRIVE guidelines (Kilkenny et al. 2010). Table S2. Criteria for quality assessment of human cross-sectional studies using a predefined grading system based on The Joanna Briggs Institute (JBI) critical appraisal checklist. Table S3. Excluded articles. Table S4. Characteristics of human studies reporting miRNA expression in periodontal disease. Table S5. Characteristics of in vivo (animal) studies reporting miRNA expression in periodontal and peri-implant disease. Table S6. Quality assessment of animal research based on ARRIVE guidelines and predefined grading scores (Schwarz et al. 2012). Table S7. Quality assessment of nonrandomized studies based on the Newcastle–Ottawa Scale (NOS). Table S8. Quality assessment of human case reports, case series, and cross-sectional studies based on The Joanna Briggs Institute (JBI) critical appraisal checklist. Table S9. Overall miRNA fold expression changes from human studies based on microarray and RT-PCR reported values. Table S10. Overall miRNA fold expression changes from in vivo (animal) studies based on microarray and RT-PCR reported values.
Author Contributions
F.A., contributed to conception, design, data acquisition, analysis, and interpretation and drafted and critically revised the manuscript; C.G.-P., contributed to design, data acquisition, analysis, and interpretation and drafted and critically revised the manuscript; C.D., contributed to data interpretation and critically revised the manuscript; and L.L., contributed to data interpretation and critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by Royal Society of Arts and Sciences in Gothenburg (Kungliga Vetenskaps- och Vitterhets-Samhället i Göteborg “KVVS”): NA; Osteology Foundation: NA.
Conflicts of Interest
The authors report no conflicts of interest related with respect to this article, its publication, and authorship.
References
- 1.Larsson L., Castilho R.M., Giannobile W.V. Epigenetics and its role in periodontal diseases: A state-of-the-art review. J. Periodontol. 2015;86:556–568. doi: 10.1902/jop.2014.140559. [DOI] [PubMed] [Google Scholar]
- 2.Filipowicz W., Bhattacharyya S.N., Sonenberg N. Mechanisms of post-transcriptional regulation by micrornas: Are the answers in sight? Nat. Rev. Genet. 2008;9:102–114. doi: 10.1038/nrg2290. [DOI] [PubMed] [Google Scholar]
- 3.Selbach M., Schwanhausser B., Thierfelder N., Fang Z., Khanin R., Rajewsky N. Widespread changes in protein synthesis induced by micrornas. Nature. 2008;455:58–63. doi: 10.1038/nature07228. [DOI] [PubMed] [Google Scholar]
- 4.Sonkoly E., Pivarcsi A. Advances in micrornas: Implications for immunity and inflammatory diseases. J. Cell. Mol. Med. 2009;13:24–38. doi: 10.1111/j.1582-4934.2008.00534.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Berglundh T., Armitage G., Araujo M.G., Avila-Ortiz G., Blanco J., Camargo P.M., Chen S., Cochran D., Derks J., Figuero E., et al. Peri-implant diseases and conditions: Consensus report of workgroup 4 of the 2017 world workshop on the classification of periodontal and peri-implant diseases and conditions. J. Clin. Periodontol. 2018;45(Suppl. 20):S286–S291. doi: 10.1111/jcpe.12957. [DOI] [PubMed] [Google Scholar]
- 6.Page R.C., Kornman K.S. The pathogenesis of human periodontitis: An introduction. Periodontol 2000. 1997;14:9–11. doi: 10.1111/j.1600-0757.1997.tb00189.x. [DOI] [PubMed] [Google Scholar]
- 7.Kornman K.S. Mapping the pathogenesis of periodontitis: A new look. J. Periodontol. 2008;79:1560–1568. doi: 10.1902/jop.2008.080213. [DOI] [PubMed] [Google Scholar]
- 8.Adcock I.M., Tsaprouni L., Bhavsar P., Ito K. Epigenetic regulation of airway inflammation. Curr. Opin. Immunol. 2007;19:694–700. doi: 10.1016/j.coi.2007.07.016. [DOI] [PubMed] [Google Scholar]
- 9.Asa’ad F., Monje A., Larsson L. Role of epigenetics in alveolar bone resorption and regeneration around periodontal and peri-implant tissues. Eur. J. Oral Sci. 2019;127:477–493. doi: 10.1111/eos.12657. [DOI] [PubMed] [Google Scholar]
- 10.Luan X., Zhou X., Trombetta-eSilva J., Francis M., Gaharwar A.K., Atsawasuwan P., Diekwisch T.G.H. MicroRNAs and Periodontal Homeostasis. J. Dent. Res. 2017;96:491–500. doi: 10.1177/0022034516685711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mico-Martinez P., Garcia-Gimenez J.L., Seco-Cervera M., Lopez-Roldan A., Alminana-Pastor P.J., Alpiste-Illueca F., Pallardo F.V. Mir-1226 detection in gcf as potential biomarker of chronic periodontitis: A pilot study. Med. Oral Patol. Oral Cir. Bucal. 2018;23:e308–e314. doi: 10.4317/medoral.22329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Fujimori K., Yoneda T., Tomofuji T., Ekuni D., Azuma T., Maruyama T., Mizuno H., Sugiura Y., Morita M. Detection of salivary mirnas reflecting chronic periodontitis: A pilot study. Molecules. 2019;24:1034. doi: 10.3390/molecules24061034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Nisha K.J., Janam P., Harshakumar K. Identification of a novel salivary biomarker mir-143-3p for periodontal diagnosis: A proof of concept study. J. Periodontol. 2019;90:1149–1159. doi: 10.1002/JPER.18-0729. [DOI] [PubMed] [Google Scholar]
- 14.Amaral S.A., Pereira T.S.F., Brito J.A.R., Cortelli S.C., Cortelli J.R., Gomez R.S., Costa F.O., Miranda Cota L.O. Comparison of mirna expression profiles in individuals with chronic or aggressive periodontitis. Oral Dis. 2019;25:561–568. doi: 10.1111/odi.12994. [DOI] [PubMed] [Google Scholar]
- 15.Bagavad Gita J., George A.V., Pavithra N., Chandrasekaran S.C., Latchumanadhas K., Gnanamani A. Dysregulation of mir-146a by periodontal pathogens: A risk for acute coronary syndrome. J. Periodontol. 2019;90:756–765. doi: 10.1002/JPER.18-0466. [DOI] [PubMed] [Google Scholar]
- 16.Bao L., Zhang X., Xu Y., Wang M., Song Y., Gu Y., Zheng Y., Xiao J., Wang Y., Zhou Q., et al. Dysfunction of mir-148a-nrp1 functional axis suppresses osteogenic differentiation of periodontal ligament stem cells under inflammatory microenvironment. Cell Reprogram. 2019;21:314–322. doi: 10.1089/cell.2019.0026. [DOI] [PubMed] [Google Scholar]
- 17.Chen H., Lan Z., Li Q., Li Y. Abnormal expression of long noncoding rna fgd5-as1 affects the development of periodontitis through regulating mir-142-3p/socs6/nf-kappab pathway. Artif. Cells Nanomed. Biotechnol. 2019;47:2098–2106. doi: 10.1080/21691401.2019.1620256. [DOI] [PubMed] [Google Scholar]
- 18.Ghotloo S., Motedayyen H., Amani D., Saffari M., Sattari M. Assessment of microrna-146a in generalized aggressive periodontitis and its association with disease severity. J. Periodontal Res. 2019;54:27–32. doi: 10.1111/jre.12538. [DOI] [PubMed] [Google Scholar]
- 19.He F., Zhou Y., Wang X., Li L., Geng Y., Wang Z., Wang Y., Xu Y. Functional polymorphisms of ctla4 associated with aggressive periodontitis in the chinese han population. Cell Physiol. Biochem. 2018;50:1178–1185. doi: 10.1159/000494544. [DOI] [PubMed] [Google Scholar]
- 20.Jia S., Yang X., Yang X., Zhang F. Microrna-210 protects against periodontitis through targeting hif-3alpha and inhibiting p38mapk/nf-kappab pathway. Artif. Cells Nanomed. Biotechnol. 2020;48:129–136. doi: 10.1080/21691401.2019.1699818. [DOI] [PubMed] [Google Scholar]
- 21.Kalea A.Z., Hoteit R., Suvan J., Lovering R.C., Palmen J., Cooper J.A., Khodiyar V.K., Harrington Z., Humphries S.E., D’Aiuto F. Upregulation of gingival tissue mir-200b in obese periodontitis subjects. J. Dent. Res. 2015;94:59S–69S. doi: 10.1177/0022034514568197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lee Y.H., Na H.S., Jeong S.Y., Jeong S.H., Park H.R., Chung J. Comparison of inflammatory microrna expression in healthy and periodontitis tissues. Biocell. 2011;35:43–49. [PubMed] [Google Scholar]
- 23.Li L., Liu W., Wang H., Yang Q., Zhang L., Jin F., Jin Y. Mutual inhibition between hdac9 and mir-17 regulates osteogenesis of human periodontal ligament stem cells in inflammatory conditions. Cell Death Dis. 2018;9:480. doi: 10.1038/s41419-018-0480-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Liu Y., Liu C., Zhang A., Yin S., Wang T., Wang Y., Wang M., Liu Y., Ying Q., Sun J., et al. Down-regulation of long non-coding rna meg3 suppresses osteogenic differentiation of periodontal ligament stem cells (pdlscs) through mir-27a-3p/igf1 axis in periodontitis. Aging (Albany NY) 2019;11:5334–5350. doi: 10.18632/aging.102105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Liu Y., Liu W., Hu C., Xue Z., Wang G., Ding B., Luo H., Tang L., Kong X., Chen X., et al. Mir-17 modulates osteogenic differentiation through a coherent feed-forward loop in mesenchymal stem cells isolated from periodontal ligaments of patients with periodontitis. Stem Cells. 2011;29:1804–1816. doi: 10.1002/stem.728. [DOI] [PubMed] [Google Scholar]
- 26.Motedayyen H., Ghotloo S., Saffari M., Sattari M., Amid R. Evaluation of microrna-146a and its targets in gingival tissues of patients with chronic periodontitis. J. Periodontol. 2015;86:1380–1385. doi: 10.1902/jop.2015.150319. [DOI] [PubMed] [Google Scholar]
- 27.Na H.S., Park M.H., Song Y.R., Kim S., Kim H.J., Lee J.Y., Choi J.I., Chung J. Elevated microrna-128 in periodontitis mitigates tumor necrosis factor-alpha response via p38 signaling pathway in macrophages. J. Periodontol. 2016;87:e173–e182. doi: 10.1902/jop.2016.160033. [DOI] [PubMed] [Google Scholar]
- 28.Naqvi A.R., Brambila M.F., Martinez G., Chapa G., Nares S. Dysregulation of human mirnas and increased prevalence of hhv mirnas in obese periodontitis subjects. J. Clin. Periodontol. 2019;46:51–61. doi: 10.1111/jcpe.13040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ogata Y., Matsui S., Kato A., Zhou L., Nakayama Y., Takai H. Microrna expression in inflamed and noninflamed gingival tissues from japanese patients. J. Oral Sci. 2014;56:253–260. doi: 10.2334/josnusd.56.253. [DOI] [PubMed] [Google Scholar]
- 30.Ou L., Sun T., Cheng Y., Huang L., Zhan X., Zhang P., Yang J., Zhang Y., Zhou Z. Microrna-214 contributes to regulation of necroptosis via targeting atf4 in diabetes-associated periodontitis. J. Cell Biochem. 2019;120:14791–14803. doi: 10.1002/jcb.28740. [DOI] [PubMed] [Google Scholar]
- 31.Perri R., Nares S., Zhang S., Barros S.P., Offenbacher S. Microrna modulation in obesity and periodontitis. J. Dent. Res. 2012;91:33–38. doi: 10.1177/0022034511425045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Pettiette M.T., Zhang S., Moretti A.J., Kim S.J., Naqvi A.R., Nares S. Microrna expression profiles in external cervical resorption. J. Endod. 2019;45:1106–1113.e2. doi: 10.1016/j.joen.2019.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Radovic N., Nikolic Jakoba N., Petrovic N., Milosavljevic A., Brkovic B., Roganovic J. Microrna-146a and microrna-155 as novel crevicular fluid biomarkers for periodontitis in non-diabetic and type 2 diabetic patients. J. Clin. Periodontol. 2018;45:663–671. doi: 10.1111/jcpe.12888. [DOI] [PubMed] [Google Scholar]
- 34.Saito A., Horie M., Ejiri K., Aoki A., Katagiri S., Maekawa S., Suzuki S., Kong S., Yamauchi T., Yamaguchi Y., et al. Microrna profiling in gingival crevicular fluid of periodontitis-a pilot study. FEBS Open Bio. 2017;7:981–994. doi: 10.1002/2211-5463.12238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Stoecklin-Wasmer C., Guarnieri P., Celenti R., Demmer R.T., Kebschull M., Papapanou P.N. Micrornas and their target genes in gingival tissues. J. Dent. Res. 2012;91:934–940. doi: 10.1177/0022034512456551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Venugopal P., Koshy T., Lavu V., Ranga Rao S., Ramasamy S., Hariharan S., Venkatesan V. Differential expression of micrornas let-7a, mir-125b, mir-100, and mir-21 and interaction with nf-kb pathway genes in periodontitis pathogenesis. J. Cell. Physiol. 2018;233:5877–5884. doi: 10.1002/jcp.26391. [DOI] [PubMed] [Google Scholar]
- 37.Xie Y.F., Shu R., Jiang S.Y., Liu D.L., Zhang X.L. Comparison of microrna profiles of human periodontal diseased and healthy gingival tissues. Int. J. Oral Sci. 2011;3:125–134. doi: 10.4248/IJOS11046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Yagnik K., Mahendra J., Kurian V.M. The periodontal-cardiovascular alliance: Evaluation of mirna-146a in subgingival plaque samples of chronic periodontitis patients with and without coronary heart disease. J. Investig. Clin. Dent. 2019;10:e12442. doi: 10.1111/jicd.12442. [DOI] [PubMed] [Google Scholar]
- 39.Yoneda T., Tomofuji T., Ekuni D., Azuma T., Maruyama T., Fujimori K., Sugiura Y., Morita M. Serum micrornas and chronic periodontitis: A case-control study. Arch. Oral Biol. 2019;101:57–63. doi: 10.1016/j.archoralbio.2019.03.009. [DOI] [PubMed] [Google Scholar]
- 40.Zhang Y., Li S., Yuan S., Zhang H., Liu J. Microrna-23a inhibits osteogenesis of periodontal mesenchymal stem cells by targeting bone morphogenetic protein signaling. Arch. Oral Biol. 2019;102:93–100. doi: 10.1016/j.archoralbio.2019.04.001. [DOI] [PubMed] [Google Scholar]
- 41.Zhao S., Cheng Y., Kim J.G. Microrna-146a downregulates il-17 and il-35 and inhibits proliferation of human periodontal ligament stem cells. J. Cell Biochem. 2019;120:13861–13866. doi: 10.1002/jcb.28659. [DOI] [PubMed] [Google Scholar]
- 42.Zhou W., Su L., Duan X., Chen X., Hays A., Upadhyayula S., Shivde J., Wang H., Li Y., Huang D., et al. Microrna-21 down-regulates inflammation and inhibits periodontitis. Mol. Immunol. 2018;101:608–614. doi: 10.1016/j.molimm.2018.05.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Guo J., Zeng X., Miao J., Liu C., Wei F., Liu D., Zheng Z., Ting K., Wang C., Liu Y. Mirna-218 regulates osteoclast differentiation and inflammation response in periodontitis rats through mmp9. Cell. Microbiol. 2019;21:e12979. doi: 10.1111/cmi.12979. [DOI] [PubMed] [Google Scholar]
- 44.Lian J., Wu X., Liu Y., Qiu W., Zhu X., Wang X., Meng S., Valverde P., Steffensen B., Tu Q., et al. Potential roles of mir-335-5p on pathogenesis of experimental periodontitis. J. Periodontal Res. 2020;55:191–198. doi: 10.1111/jre.12701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Nahid M.A., Rivera M., Lucas A., Chan E.K., Kesavalu L. Polymicrobial infection with periodontal pathogens specifically enhances microrna mir-146a in apoe-/- mice during experimental periodontal disease. Infect. Immun. 2011;79:1597–1605. doi: 10.1128/IAI.01062-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Nayar G., Gauna A., Chukkapalli S., Velsko I., Kesavalu L., Cha S. Polymicrobial infection alter inflammatory microrna in rat salivary glands during periodontal disease. Anaerobe. 2016;38:70–75. doi: 10.1016/j.anaerobe.2015.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Sugiura Y., Yoneda T., Fujimori K., Maruyama T., Miyai H., Kobayashi T., Ekuni D., Tomofuji T., Morita M. Detection of serum mirnas affecting liver apoptosis in a periodontitis rat model. In Vivo. 2020;34:117–123. doi: 10.21873/invivo.11752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Sun H.T., Zhang J., Hou N., Zhang X., Wang J., Bai Y. Spontaneous periodontitis is associated with metabolic syndrome in rhesus monkeys. Arch. Oral Biol. 2014;59:386–392. doi: 10.1016/j.archoralbio.2014.01.004. [DOI] [PubMed] [Google Scholar]
- 49.Tomofuji T., Yoneda T., Machida T., Ekuni D., Azuma T., Kataoka K., Maruyama T., Morita M. Micrornas as serum biomarkers for periodontitis. J. Clin. Periodontol. 2016;43:418–425. doi: 10.1111/jcpe.12536. [DOI] [PubMed] [Google Scholar]
- 50.Xu R., Zeng G., Wang S., Tao H., Ren L., Zhang Z., Zhang Q., Zhao J., Gao J., Li D. Periodontitis promotes the diabetic development of obese rat via mir-147 induced classical macrophage activation. Biomed. Pharmacother. 2016;83:892–897. doi: 10.1016/j.biopha.2016.07.030. [DOI] [PubMed] [Google Scholar]
- 51.Zhou X., Luan X., Chen Z., Francis M., Gopinathan G., Li W., Lu X., Li S., Wu C., Diekwisch T.G. Microrna-138 inhibits periodontal progenitor differentiation under inflammatory conditions. J. Dent. Res. 2016;95:230–237. doi: 10.1177/0022034515613043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Wu X., Chen X., Mi W., Wu T., Gu Q., Huang H. Microrna sequence analysis identifies micrornas associated with peri-implantitis in dogs. Biosci. Rep. 2017;37:BSR20170768. doi: 10.1042/BSR20170768. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Wu X., Gu Q., Chen X., Mi W., Wu T., Huang H. Mir-27a targets dkk2 and sfrp1 to promote reosseointegration in the regenerative treatment of peri-implantitis. J. Bone Miner. Res. 2019;34:123–134. doi: 10.1002/jbmr.3575. [DOI] [PubMed] [Google Scholar]
- 54.Higgins J.P.T., Green S. Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration; London, UK: 2011. Version 5.1.0. [Google Scholar]
- 55.M’Baya-Moutoula E., Louvet L., Metzinger-Le Meuth V., Massy Z.A., Metzinger L. High inorganic phosphate concentration inhibits osteoclastogenesis by modulating mir-223. Biochim. Biophys. Acta. 2015;1852:2202–2212. doi: 10.1016/j.bbadis.2015.08.003. [DOI] [PubMed] [Google Scholar]
- 56.Irwandi R.A., Vacharaksa A. The role of microrna in periodontal tissue: A review of the literature. Arch. Oral Biol. 2016;72:66–74. doi: 10.1016/j.archoralbio.2016.08.014. [DOI] [PubMed] [Google Scholar]
- 57.Hung P.S., Chen F.C., Kuang S.H., Kao S.Y., Lin S.C., Chang K.W. MiR—146a induces differentiation of periodontal ligament cells. J. Dent. Res. 2010;89:252–257. doi: 10.1177/0022034509357411. [DOI] [PubMed] [Google Scholar]
- 58.Sun F., Ma Y., Cai Z., Yang Z. MiR-218 promotes osteogenic differentiation of periodontal ligament stem cell through activation of Wnt signaling by targeting SFRP2. Int. J. Clin. Exp. Pathol. 2016;9:10188–10196. [Google Scholar]
- 59.Tatullo M., Codispoti B., Pacifici A., Palmieri F., Marrelli M., Pacifici L., Paduano F. Potential Use of Human Periapical Cyst-Mesenchymal Stem Cells (hPCy-MSCs) as a Novel Stem Cell Source for Regenerative Medicine Applications. Front. Cell Dev. Biol. 2017;5:103. doi: 10.3389/fcell.2017.00103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Luan X., Zhou X., Naqvi A., Francis M., Foyle D., Nares S., Diekwisch T.G.H. MicroRNAs and immunity in periodontal health and disease. Int. J. Oral Sci. 2018;10:24. doi: 10.1038/s41368-018-0025-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.The Newcastle-Ottawa Scale (nos) for Assessing the Quality of Non-Randomised Studies in Meta-Analysis. [(accessed on 3 March 2020)];2011 Available online: http://www.ohri.ca/pro-grams/clinical_epidemiology/oxford.asp.
- 62.The Joanna Briggs Institute Critical Appraisal Tools for Use in Jbi Systematic Reviews. [(accessed on 3 March 2020)];2017 Available online: http://joannabriggs.org/research/critical-appraisal-tools.html.
- 63.Kilkenny C., Browne W., Cuthill I.C., Emerson M., Altman D.G., NC3Rs Reporting Guidelines Working Group Animal research: Reporting in vivo experiments: The arrive guidelines. Br. J. Pharmacol. 2010;160:1577–1579. doi: 10.1111/j.1476-5381.2010.00872.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Schwarz F., Iglhaut G., Becker J. Quality assessment of reporting of animal studies on pathogenesis and treatment of peri-implant mucositis and peri-implantitis. A systematic review using the arrive guidelines. J. Clin. Periodontol. 2012;39(Suppl. 12):63–72. doi: 10.1111/j.1600-051X.2011.01838.x. [DOI] [PubMed] [Google Scholar]
- 65.R: A Language and Environment for Statistical Computing. [(accessed on 9 March 2020)];2018 Available online: http://www.R-project.org/
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.