Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis

Abstract

Background

The accuracy of ultrasound in detecting invasive placentation, such as placenta accreta spectrum (PAS), remains a topic of debate. Accurate prenatal diagnosis is crucial to improve maternal outcomes, especially in women with high-risk factors..

Objective

This systematic review and meta-analysis aimed to evaluate the diagnostic accuracy of ultrasonography in predicting invasive placentation among at-risk pregnant women.

Study Design

A comprehensive search of multiple databases (MEDLINE, EMBASE, Cochrane Library, etc.) was conducted to identify studies assessing the diagnostic accuracy of ultrasound in detecting PAS. A total of 24 studies, including 1,509 high-risk pregnancies, met the inclusion criteria. Data on sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR-), and diagnostic odds ratio (DOR) were extracted and analyzed.

Results

The meta-analysis demonstrated that ultrasound has a high diagnostic performance, with a pooled sensitivity of 91.73% (95% CI: 88.3–94.7%) and specificity of 97.95% (95% CI: 97.4–98.6%). The DOR was 99.6 (95% CI: 49.9–200.1). Among ultrasound modalities, color Doppler showed the highest predictive accuracy with a sensitivity of 91.75% (95% CI: 86.3–95.6%) and specificity of 87.69% (95% CI: 85.7–91.5%).

Conclusion

Ultrasound, especially with the use of color Doppler, is highly effective in the prenatal diagnosis of PAS disorders in high-risk pregnancies. Early detection through ultrasound allows for better clinical management, reducing maternal morbidity by enabling planned interventions.

AJOG Global Reports at a Glance.

Why was this study conducted?

This study was conducted in order to evaluate the ultrasound's diagnostic accuracy, specifically color Doppler, in the prediction of placenta accreta spectrum (PAS) in high-risk pregnancies. Accurate diagnosis is critical for the improvement of maternal outcomes.

Key findings

Key findings indicate that ultrasound has a pooled sensitivity of 91.73% and a specificity of 97.95%, with color Doppler demonstrating the highest predictive accuracy. Early detection through ultrasound enables better clinical management, which can reduce maternal morbidity.

What does this study add to what is known?

The study adds robust evidence to the existing literature by confirming the high diagnostic performance of ultrasound in the detection of PAS. It emphasizes the benefit of color Doppler in improving diagnostic precision, offering valuable guidance for clinicians in managing high-risk pregnancies. This supports the standardized usage of ultrasound for prenatal PAS diagnosis in appropriate care settings.

Introduction

The pathological adhesion of the placenta due to localized or widespread aberrant trophoblast invasion into the myometrium is known as placenta accreta spectrum (PAS), or morbidly adherent placenta (MAP).¹ Over time, the rate of PAS is rising. In the 1970s and 1980s, it was estimated to be between 1 in 2510 and 1 in 4017, and between 1982 and 2002, it reached 1 in 533.²

The term “placenta accreta” describes a placenta that is difficult to separate from the uterine wall. It is still uncertain how exactly placental attachment abnormalities begin, although it is most likely more complex than just incorrect decidualization of a scarred region or absence of decidua in the lower uterine segment close to the cervix.³ PAS is characterized by an aberrant invasion of placental tissue (trophoblast) through the uterine serosa (referred to as accreta, increta, or percreta, respectively) into the inner or outer myometrium. Seven to 10 percent of maternal deaths are caused by this potentially fatal disease. The placenta becomes morbidly adherent when the decidua basalis and Nitabuch layer are absent, either completely or partially. Three forms of PAS are distinguished based on the extent of chorionic villi penetration: accrete, increta, and percreta.⁴

Significantly higher rates of cesarean sections are the cause of the considerable increase in placenta accreta occurrence. A previous cesarean birth, an accompanying placenta previa, and most likely a combination of the 2 are the 2 most significant risk factors. Additional risk factors include multiparity, advanced maternal age, prior uterine surgery such as curettage and dilatation, prior myomectomy (resulting in intrauterine adhesion, etc.), and more.⁵ A greater level of clinical suspicion is needed to diagnose PAS, and imaging can be a useful diagnostic technique. Ultrasonography is mostly used to identify placenta accreta, although it can occasionally fail to forecast the condition's prognosis, which can result in severe bleeding or even maternal mortality from insufficient preoperative assessment.⁶ It is commonly known that the best technique for precisely locating the placenta is transvaginal sonography (TVS). Even in cases of vaginal bleeding, TVS is safe for women with placenta previa. Clinical care can be guided and the diagnosis established with the use of appropriate antenatal ultrasonography. Grayscale sonography obtained in the second and third trimesters characteristics include the loss of a hypoechoic zone (myometrial zone) between the placenta and the myometrium, many vascular lacunae (irregular vascular spaces) within the placenta that give the appearance of Swiss cheese, the placental/myometrial site bulging into the bladder, and increased vascularity visible on color Doppler sonography.⁷

To maximize maternal outcomes, antenatal diagnosis of PAS is essential, and delivery should be scheduled at a level III or IV maternal care facility.⁸ The majority of women are diagnosed with PAS in the second or third trimester, despite the fact that ultrasonographic indications of the condition may be observed as early as the first trimester.⁹ Several placental vascular lacunae inside the retroplacental clear zone, myometrial thinning (less than 1 mm), disruption of the serosa–bladder interface, and placental extension into the myometrium, serosa, or bladder are among the gray scale criteria.¹⁰ Variations exist in ultrasound accuracy throughout several research. While some produced results with 100% accuracy, some reported substantially lower levels.¹¹

This systematic review and meta-analysis study aims to evaluate the diagnostic accuracy of ultrasonography in the prediction of invasive placentation among at-risk pregnant women.

Methods

Study Aim: The purpose of this meta-analysis and systematic review was to evaluate the accuracy of ultrasonography in predicting aberrant placenta adhesion throughout the second trimester. Our study was registered in PROSPERO by the ID: CRD42024609696.

Design of Operations: This research is a meta-analysis.

Gathering and analyzing data: A thorough search for papers published in English was conducted across several databases. Using keywords associated with obstetric ultrasonography, placenta accrete spectrum, and morbidly adherent placenta, a computerized study was conducted in the databases of PubMed, Medline, Elsevier, Scopus, Google Scholar, and the Cochrane Library.

Study Selection: In this meta-analysis, we included studies which fulfilled the following criteria:

• a.Randomized controlled trials (RCTs)
• b.Singleton pregnancy.
• c.Previous one or more CSs.
• d.Gestational age from 14 week (last menstrual period and/ or Ultrasound confirmation of gestational age).
• e.Patient diagnosed as having anterior placenta previa.
• f.Studies reporting pregnancy outcomes

Exclusion Criteria:

• a.Multiple gestational pregnancies.
• b.Fundal placenta.
• c.Primigravida.
• d.Gestational age less than 14 week.
• e.Posterior placenta previa.
• f.Multipara with previous vaginal delivary.
• g.Non-English publications
• h.Studies with data with inadequate randomization.

The prediction of placenta accreta is dependent upon two or more of the following sonographic findings:

• 1.Any portion of the echolucent region between the uterus and placenta that is obliterated is referred to as the "obliteration of the clear space."
• 2.The placenta's many linear, irregular vascular gaps, known as placental lacunae, are seen.
• 3.An abnormal serosa occurs when the typical continuous echolucent line is interrupted by a series of dashes at the contact between the uterus and the posterior bladder wall.
• 4.Focused exophytic tumors that invade the bladder are present.
• 5.Less than 1 mm of myometrium.

Color Doppler findings of placenta accreta

The presence of two or more of these signs is necessary for the diagnosis of placenta accreta:

• 1.Blood flowing erratically from the placenta into the environment.
• 2.Diffuse or focal flow of placental lacunae inside the parenchyma.
• 3.Hypervascularity in the vesico-uterine serosal interphase.
• 4.The retroplacental venous complex is prominent.
• 5.Doppler vascular signal loss in the retroplacenta.

Procedure for gathering data

The three researchers reviewed the abstracts and titles after eliminating duplicates. Any conflict was solved by the fourth author. Next, we examined the entire text using the study's inclusion and exclusion criteria.

Data Collection Process: After duplicates removed, the three researchers screened the titles and abstracts. Any conflict was solved by the fourth author. Then, we investigated the full text based on inclusion and exclusion criteria for this study.

Risk of bias assessment

The bias is a systematic error, or a deviation from the truth, in results or conclusions. The different biases may lead to underestimation or overestimation of the actual intervention effect. Quality assessment of the included studies was done following the Newcastle–Ottawa Scale (NOS) [11]; the researchers were in charge to assess the methodological quality for each included study.

Literature search

The search strategy yielded 156 relevant articles, 87 articles were excluded based on their title and abstract. Fifty-two articles were retrieved (Duplicated). Ten of these articles also excluded after full text review based on inclusion, exclusion criteria and eligibility. Process is shown in detail (Figure 1) flow chart according to PRISMA guidelines. At the end, a total of 24 studies (1509 participants) ultimately met the inclusion criteria.

Figure 1.

Flow chart of the search and screening process

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Statistical Analysis: Data entered was organized in Microsoft excel 2010 then exported to comprehensive meta-analysis software version 3.

Pooled for analysis of multiple studies, and found adjusted accumulative outcome.

Z score method: to test difference in mean.

Test for heterogeneity: Cochran's Q test and I²: Under null, it is approximately distributed as a chi-square with k-1 degrees of freedom for test heterogeneity and homogeneity of studies results and finding.

Results

A total of 24 studies, including 1509 women who met the inclusion criteria, were analyzed. Placenta accreta spectrum (PAS) was confirmed in 398 of these cases (Table 1).

Table 1.

Characteristics of studies

Reference	Diagnostic tools	Inclusion criteria	Trimester scan	Reference standard	Women scan	Invasion placenta
E. PILLONI et al.12	Transvaginal and transabdominal color Doppler	PP and/or previous CS or uterine surgery	Second and Third trimester	Surgical finding	314	37
Bhawna et al.13	Ultrasound color Doppler and MRI	PP and/or previous CS or uterine surgery	Third trimester	Pathology And Surgical Finding	30	8
J.Vojtech et al.14	Ultrasound color Doppler and MRI	PP with previous CS	Third trimester	Pathology	18	10
A.Elgebaly et al.15	Ultrasound color Doppler and MRI	PP	Third trimester	Surgical finding	100	34
Cali et al.16	2- and 3-dimension ultrasound color doppler	PP and previous uterine surgery	2 and 3 trimesters	Pathology	187	41
G. A. Kassem et al.17	2-dimension ultrasound	PP e previous CS	3trimester	Pathology and surgical finding	100	25
Fishman et al.18	2-dimension ultrasound	Placenta previa	3trimester	Pathology	154	23
Esakoff et al.19	2-dimension ultrasound	Placenta previa	3trimester	Pathology	108	19
Hamada et al.20	2-dimension ultrasound	PP and/or previous CS	3trimester	Pathology	70	5
Mansour et al.21	Ultrasound and MRI	PP, previous uterine surgery	3trimester	Surgical finding	35	15
Lim et al.22	Ultrasound and MRI	Previous CS and/or third-trimester bleeding	2 and 3 trimesters	Pathology	13	9
ElBehery et al.23	Cell free M RNA in maternal plasma e US color Doppler	PP, previous CS or uterine surgery	3trimester	Pathology and surgical finding	35	7
Shih et al.24	2- and 3-dimension ultrasound color doppler	PP and/or previous CS or uterine surgery	3trimester	Pathology	170	39
Chou et al.25	2- and 3-dimension US color Doppler	PP, previous CS	2 and 3 trimesters	Pathology	44	6
Wong et al.26	2-dimension ultrasound	PP and/or previous CS or uterine surgery	2 and 3 trimesters	Pathology and surgical finding	66	9
Dwyer et al.27	Ultrasound and MRI	PP and/or previous CS or uterine surgery	3trimester	Pathology and clinical finding	32	15
Minura et al.28	Cell free M RNA in maternal plasma e US	PP and/or previous CS	3trimester	Pathology	12	4
Japarj et al.29	Transabdominal and trans vaginal colour Doppler	PP, previous CS	3trimester	Pathology and surgical finding	21	7
Warshak et al.30	Ultrasound and MRI	PP and/or previous CS or uterine surgery	2 and3 trimester	Pathology	453	39
Yang et al.31	trans vaginal ultrasound	PP, previous CS	2 and 3 trimesters	Pathology	51	23
Comstock et al.32	Ultrasound colour Doppler	Low anterior placenta, previous CS	2 and 3trimester	Pathology	2002	4
Moodly et al.33	Ultrasound and MRI	Placenta previa	3trimester	Pathology and surgical finding	30	3
Chou et al.34	Ultrasound colour Doppler	PP and/or previous CS or uterine surgery	2 and 3trimester	Pathology	80	17
Twickler et al.35	Ultrasound colour Doppler	PP, previous CS	3trimester	Pathology and surgical finding	20	10

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Timing of Ultrasound Diagnosis

In 15 studies, all ultrasound diagnoses were performed during the third trimester.13, 14, 15, 16, 17, 18, 19, 20, 21^,23,24,27, 28, 29^,33,35 In contrast, 9 studies included diagnoses performed during both the second and third trimesters.^{12,16,22,25,26,30,31,32,34}

Diagnostic Performance of Specific Ultrasound Signs

The sensitivity and specificity of individual ultrasound characteristics for PAS were evaluated across different studies:

• 1.Placental Lacunae:This characteristic was assessed in 11 studies.^{16,20,21,23,24,26,28,29,31,32,35} The pooled sensitivity was 91.73% (95% CI: 88.3–94.7), and the specificity was 97.95% (95% CI: 97.4–98.6).
• 2.Lack of Clear Zone:

  The absence of a clear zone between the placenta and uterine wall was evaluated in 9 studies.^{16,20,21,23,24,26,28,29,30} This sign demonstrated moderate diagnostic accuracy, with detailed sensitivity and specificity shown in Table 3.

• 3.Uterus–Bladder Interface:

  Distortion or interruption of the uterus–bladder interface was examined in 7 studies.^16,23, 24, 25, 26^,29,35 The findings are summarized in Table 4.

• 4.Color Doppler Findings:

  Abnormal vascular patterns on color Doppler imaging, including turbulent flow and increased vascularity, were evaluated in 12 studies.23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33^,36 Among all ultrasound modalities, color Doppler showed the highest predictive accuracy, with a pooled sensitivity of 91.75% (95% CI: 86.3–95.6) and specificity of 87.69% (95% CI: 85.7–91.5).

Table 3.

The sensitivity and specificity of lack of clear zone in the studies

Author names	Sensitivity of lack of clear zone	Specificity of lack of clear zone
Cali et al.16	90%	81%
Mansour et al.21	87%	81%
Hamada et al.20	60%	98%
El Behery et al.23	100%	89%
Shih et al.24	44%	95%
Miura et al.28	100%	25%
Wong et al.26	100%	35%
Japaraj et al.29	43%	100%
Comstock et al.32	60%	99%

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Table 4.

The sensitivity and specificity of the uterus–bladder interface in the studies

Author names	Sensitivity of uterus – bladder interface	Specificity of uterus – bladder interface
Cali et al.16	71%	99%
El Behery et al.23	43%	96%
Shih et al.24	18%	100%
Chou et al.25	100%	100%
Wong et al.26	11%	100%
Japaraj et al.29	57%	100%
Twickler et al.35	90%	70%

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Overall diagnostic performance of ultrasound

Ultrasound as a diagnostic tool for invasive placentation showed high overall accuracy:

• •Sensitivity: 91.73% (95% CI: 88.3–94.7)
• •Specificity: 97.95% (95% CI: 97.4–98.6)
• •Positive Likelihood Ratio (LR+): 12.02 (95% CI: 6.2–21.1)
• •Negative Likelihood Ratio (LR–): 0.17 (95% CI: 0.12–0.24)
• •Diagnostic Odds Ratio (DOR): 99.6 (95% CI: 49.9–200.0)

Among the evaluated ultrasound characteristics, color Doppler demonstrated the best predictive accuracy, with a sensitivity of 91.75% and a specificity of 87.69%. Other ultrasound signs, such as placental lacunae, lack of a clear zone, and abnormalities in the uterus–bladder interface, also contributed to diagnosis but with varying levels of sensitivity and specificity (Table 2, Table 3, Table 4, Table 5, Table 6, Table 7).

Table 2.

The sensitivity and specificity of placental lacunae in the studies

Author names	Sensitivity of placental lacunae	Specificity of placenta lacunae
Cali et al.16	73%	86%
Mansour et al.21	87%	81%
Hamada et al.20	60%	71%
El Behery et al.23	86%	93%
Shih et al.24	54%	85%
Miura et al.28	100%	25%
Wong et al.26	78%	39%
Japaraj et al.29	86%	100%
Yang et al.31	87%	79%
Twickler et al.35	100%	70%
Comstock et al.32	80%	100%

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Table 5.

The sensitivity and specificity of ultrasound in the studies

Author names	Sensitivity of US	Specificity of US
E. PILLONI et al.12	73.13%	95.5%
Bhawna et al.13	53.3%	88%
J.Vojtech et al.14	87.5%	86.4%
A.Elgebaly et al.15	53.5%	82%
Cali et al.16	100%	37.5%
G. A. Kassem et al.17	95%	95%
Fishman et al.18	91%	100%
Esakoff et al.19	89%	91%
Hamada et al.20	60%	98%
Mansour et al.21	87%	80%
Lim et al.22	67%	50%
El Behery et al.23	100%	79%
Shih et al.24	100%	85%
Chou et al.25	100%	100%
Wong et al.26	89%	98%
Dwyer et al.27	93%	71%
Minura et al.28	100%	63%
Japarj et al.29	100%	100%
Warshak et al.30	77%	96%
Yang et al.31	87%	79%
Comstock et al.32	100%	99%
Moodly et al.33	67%	93%
Chou et al.34	82%	97%
Twickler et al.35	100%	60%

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Table 6.

The sensitivity and specificity of colour Doppler in the studies

Author names	Sensitivity of colour dopper	Specificity of colour Doppler
El Behery et al.23	76.7%	90.4%
Shih et al.24	90%	100%
Chou et al.25	76%	90%
Wong et al.26	86%	96%
Dwyer et al.27	100%	100%
Minura et al.28	100%	85%
Japarj et al.29	100%	50%
Warshak et al.30	88%	41%
Yang et al.31	100%	100%
Comstock et al.32	67%	85%
Moodly et al.33	100%	60%

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Table 7.

Pooled values for sensitivity, specificity, positive and negative likelihood ratios (LR and LR–), and diagnostic odds ratio (DOR) for ultrasound overall, and the different ultrasound signs in the identification of invasive placentation

Diagnostic method	Sensitivity (95% CI) (%)	Specificity (95% CI) (%)	LR+ (95% CI)	LR- (95% CI)	DOR (95% CI)
Ultrasound (overall)	91.73 (88.3– 94.7)	97.95 (97.4– 98.6)	12.02 (7.2– 21.1)	0.17 (0.12– 0.24)	99.6 (49.9– 200.1)
Placental lacunae	78.43 (71.2– 84.2)	96.03 (95.2– 96.9)	5.53 (3.6– 9.2)	0.30 (0.21– 0.44)	25.33 (10.14– 65.9)
Loss of hypoechoic space	67.25 (60.4– 74.7)	96.77 (95.2– 97.6)	6.65 (3.4– 15.2)	0.39 (0.21– 0.7)	22.99 (7.9– 71.7)
Abnormalities of uterus– bladder interface	50.67 (42.5– 59.0)	99.77 (99.6– 99.9)	31.57 (9.2– 116.6)	0.52 (0.35– 0.78)	94.71 (36.6– 248.6)
Color Doppler abnormalities	91.75 (86.3– 95.8)	88.69 (85.7– 91.5)	8.78 (4.4– 19.5)	0.18 (0.11– 0.3)	70.03 (23.9– 209.2)

Mohamed. Accuracy of ultrasound in prediction of abnormal placental adherence: a systematic review and meta-analysis. AJOG Glob Rep 2024.

Discussion

Principal findings

This systematic review and meta-analysis demonstrates that ultrasonography, especially with the use of color Doppler, is a highly accurate diagnostic tool for the detection of placenta accreta spectrum (PAS) in high-risk pregnancies. The pooled specificity and sensitivity of ultrasound were 97.95% and 91.73%, respectively, confirming its robustness in prenatal detection. In comparison, Color Doppler outperformed other modalities with a sensitivity of 91.75% and specificity of 88.69%. These findings are based on data from 24 studies that involved 1,509 pregnancies and are reflective of the strong diagnostic value of ultrasonography for PAS.

Results in the context of what is known

The findings of our study align with prior research emphasizing the diagnostic utility of ultrasonography in invasive placentation. Previous studies have reported specificity and sensitivity rates which are comparable to those found in this analysis, highlighting the consistency of ultrasound's performance. For instance, studies have emphasized the effectiveness of color Doppler and grayscale ultrasonography in identifying features such as loss of the hypoechoic zone, abnormal vascular patterns, and placental lacunae. While some studies suggested MRI as a potential adjunct, the current evidence supports ultrasonography as the primary tool due to its accessibility, reliability, and non-invasive nature.

Clinical implications

This study's findings underscore the clinical value of ultrasonography in the early detection of PAS, allowing for strategic and timely clinical decision-making. Early identification facilitates multidisciplinary management plans like scheduled cesarean delivery, advanced surgical preparation, and interventional radiology in order to mitigate risks such as massive hemorrhage and maternal morbidity. These results support the integration of detailed ultrasonographic evaluation, including color Doppler, into routine care for high-risk pregnancies. However, further research is needed for standardizing diagnostic protocols and confirming the applicability of these findings across diverse clinical settings and populations.

Research implications

Despite the strong diagnostic accuracy of ultrasonography for PAS, several questions remain unanswered. Future research should explore its utility in fundal and posterior placentas, which were excluded in most studies. Additionally, prospective, blinded studies are needed to minimize bias and improve the generalizability of findings. Research is also warranted for comparing ultrasonography with alternative modalities like MRI in specific high-risk subgroups and to refine criteria for diagnosis, particularly in cases with ambiguous ultrasound findings.

Strengths and limitations

The study's strengths include a rigorous systematic approach, comprehensive data analysis, and inclusion of diverse ultrasound modalities, enhancing the reliability of the findings. However, limitations exist. Variability in study design and ultrasound protocols across included studies may have introduced heterogeneity. Many studies were retrospective, which could bias the results, and blinding was not consistently reported. Moreover, the findings are limited to anterior placentas in high-risk women, excluding those with fundal or posterior placentas.

Conclusions

Ultrasonography, particularly when combined with color Doppler, is highly accurate in diagnosing PAS in high-risk pregnancies. This diagnostic tool allows for early detection and better clinical management, ultimately reducing maternal morbidity. However, standardization of diagnostic criteria and further research addressing current limitations are necessary to optimize its application in clinical practice.

CRediT authorship contribution statement

Ekramy A. Mohamed: Writing – review & editing, Visualization, Validation, Supervision, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation. Ruqayyah Ali Ahmed: Writing – review & editing, Writing – original draft, Validation, Software, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation. Nada Yasser Metwali: Writing – original draft, Validation, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation. Jumana Hussain Timraz: Writing – original draft, Validation, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation. Ahmed Mohamed: Writing – original draft, Data curation, Investigation, Writing – review & editing, Methodology, Software. Hossam Abdelfatah Mansour: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.