Abstract
Background
Postpartum stress urinary incontinence significantly impacts the quality of life and the physical and mental health of women. A reliable predictive model for postpartum stress urinary incontinence can serve as a preventive tool. Currently, there have been numerous studies developing predictive models to assess the risk of postpartum stress urinary incontinence, but the quality and clinical applicability of these models remain unknown.
Objective
To systematically review and evaluate existing models for predicting stressful postpartum risks.
Methods
PubMed, EBSCO, The Cochrane Library, Embase, Web of Science, China National Knowledge Infrastructure, WanFang Data, SinoMed, and VIP Data databases were systematically searched from the time of database construction to October 2023. Two researchers used Critical appraisal and data extraction for systematic reviews of prediction modeling studies: the CHARMS checklist for data extraction. Three researchers used The Prediction Model Risk of Bias Assessment Tool (PROBAST) checklist for bias and applicability assessment.
Results
Eight papers including ten postpartum stress urinary incontinence prediction models were finalized. The most common predictors in the prediction models were urinary incontinence (UI) during pregnancy, followed by mode of delivery, Maternal age, parity, and UI before pregnancy. Nine of the prediction models reported discrimination with an area under the ROC curve (AUC) or C-index between 0.680 and 0.850. All included studies were at high risk of bias, mainly due to mishandling of continuous predictors, unreported or mishandled missing data, and inadequate assessment of predictive model performance.
Conclusions
Postpartum stress urinary incontinence risk prediction models are in the initial development stage, and existing prediction models have a high risk of bias and poor modeling methodological quality, which may hinder their clinical application. In the future, healthcare practitioners should follow the norms of predictive model development and reporting to develop risk prediction models with superior predictive performance, low risk of bias, and easy clinical application.
Keywords: Stress urinary incontinence, Postpartum, Risk prediction model, Systematic review
1. Background
Postpartum stress urinary incontinence is the first occurrence of involuntary loss of urine during exertion or physical labor (including physical activity), sneezing, or coughing during the postpartum period and up to 12 months postpartum [1]. It is a common pelvic floor dysfunction disorder, along with postpartum fecal incontinence and pelvic organ prolapse [1]. Postpartum stress urinary incontinence occurs in close association with pregnancy and childbirth [2]. It may be related to the damage to the pelvic floor structure caused by pregnancy and childbirth and the hormonal changes resulting in decreased pelvic floor muscle strength, ligament laxity, and abnormal urethral contractile function, among other factors [[3], [4], [5]]. Studies report varying prevalence rates of postpartum stress urinary incontinence, ranging from 16.41 % to 31 %, with an increasing incidence observed with age [[6], [7], [8]]. In addition, studies have shown that among women who reported symptoms of stress urinary incontinence (SUI) at 3 months postpartum, SUI symptoms persisted up to 12 years postpartum, and the prevalence of persistent SUI at 6 and 12 years postpartum was 24 % and 37.9 %, respectively [9,10]. Although postpartum stress urinary incontinence is a relatively common condition, it is not a fatal or life-threatening disease. However, it is a growing global public health problem, as it often hurts the quality of life and affects several aspects of physical activity and emotional relationships, and treatment is expensive and burdensome for both families and society [[11], [12], [13]]. Despite these effects, lack of awareness of the disease, shame, and finances result in only a small percentage of people with postpartum stress urinary incontinence seeking medical help or discussing their problems with their partners and relatives [[12], [13], [14]].
The prevention and treatment of postpartum stress urinary incontinence should emphasize early screening, early diagnosis, and early treatment integrated comprehensive management. There are numerous of risk factors for postpartum stress urinary incontinence, including vaginal delivery, advanced age, high body mass index (BMI), excessive weight gain during pregnancy, episiotomy, forceps delivery, and UI during pregnancy [7,15]. Early screening and identification of pregnant women who have not yet developed postpartum stress urinary incontinence, coupled with the effective management of controllable risk factors (including high BMI, excessive weight gain during pregnancy, etc.) to reduce the incidence of postpartum stress urinary incontinence [16]. Currently, prediction models that combine multiple variables to estimate individual risk can help healthcare workers screen people at high risk of postpartum stress urinary incontinence, but most of the existing research focuses on the development or validation of prediction models, and the quality and applicability of the prediction models are still unknown, making it difficult for healthcare workers to choose the appropriate prediction model to accurately identify people at high risk of postpartum stress urinary incontinence and tailor interventions accordingly.
The objective of this systematic review is to comprehensively gather studies on postpartum stress urinary incontinence risk prediction models. Our aim is to systematically summarize, compare, and analyze these models with regard to their fundamental features, construction methods, predictive factors, model performance, and methodological quality. This effort is geared towards providing accurate guidance for clinical practitioners in selecting appropriate postpartum stress urinary incontinence prediction models. Furthermore, the review seeks to contribute to the enhancement and optimization of these prediction models.
2. Methods
The protocol of our systematic review was registered in the International Prospective Registry of Systematic Reviews (PROSPERO) database under the number CRD42023470416. We reported this review according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) [17].
2.1. Data sources and search strategy
We systematically searched nine databases, PubMed, EBSCO, The Cochrane Library, Embase, Web of Science, China National Knowledge Infrastructure（CNKI）, WanFang Data, SinoMed and Technology Journal Database (VIP), with a time frame of database construction to October 2023. The keywords used in the search included medical subject headings (MESH) and the following entry terms combined with synonyms: “Postpartum”, “Puerperium”, “Postnatal”, “Stress urinary incontinence”, “Score∗”, “Predict∗”, “Model?”, “validity”, “nomogram?”. Our search was limited to titles and abstracts, and we screened the reference lists of all original literature to identify any potentially relevant studies. Detailed strategies for retrieval use can be found in the supplementary material.
The PICOTS system recommended by the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist was used in our systematic review [18]. It helped us to define the purpose of the review, develop a search strategy for the studies, and inclusion and exclusion criteria [19]. The key items of this systematic review are described below:
P (Population): Adult women (age ≥18 years) presenting with stress urinary incontinence after childbirth.
I (Intervention model): Developed and published a predictive model for the risk of postpartum stress urinary incontinence (predictor ≥2).
C (Comparator): None.
O(Outcome): Outcome was defined as post-partum symptoms of stress incontinence.
T (Timing): Outcomes were predicted based on general maternal demographics, information on the gestation period, and delivery information.
S (Setting): The use of risk prediction models is to individualize the prediction of postpartum stress urinary incontinence and to implement timely interventions for pregnant women.
2.2. Inclusion and exclusion criteria
Studies were included if they met the following criteria: (1) target population: adult women (age ≥18 years) who were postpartum, including vaginal or cesarean deliveries; (2) study content: development of the postpartum stress urinary incontinence prediction model (with or without external validation), external validation of the postpartum stress urinary incontinence prediction model (with or without updating); and (3) study design: prospective cohort, retrospective cohort, nested case-control or case-cohort studies, cross-sectional designs, randomized trials. (4) Predicted outcome: postpartum stress incontinence, regardless of degree.
Studies were excluded if they were (1) studies of postpartum stress urinary incontinence predictors and risk factors only, without constructing a predictive model; (2) studies that validated the original predictive model only; (3) studies for which the full text was not available; (4) studies reported in the form of headlines, abstracts, letters, conference announcements, or draft interventions; (5) duplicated publications; and (6) non-English-language literature.
2.3. Study selection
2 researchers (WLY and ZMH) independently screened studies in the database based on inclusion and exclusion criteria. Duplicate studies were first removed. Secondly, irrelevant studies were excluded by reading the titles and abstracts. We then performed full-text reading to determine eligibility for inclusion in the systematic evaluation, and we also checked the reference study lists of eligible studies for any potentially relevant studies. In the event of disagreement, the decision to select studies was made after discussing the disagreement with a third researcher (QYQ) on our review panel.
2.4. Data extraction
After identifying studies for inclusion, 2 researchers (WLY and DQQ) independently performed data extraction and cross-checking according to the Critical appraisal and data extraction for systematic reviews of prediction modeling studies: the CHARMS checklist [18]. The extraction included: first author, country, year of publication, type of study, target population, candidate variables, modeling sample size, missing data and treatment, modeling method, model performance, presentation format, and predictors.
2.5. Assessment of risk of bias
Predictive model risk of bias was independently assessed by 3 researchers (WLY, ZMH, and QYQ) using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [20]. The PROBAST risk of bias assessment encompassed 4 domains: participant, predictor, outcome, and analysis, with a total of 20 signaling questions. The applicability assessment includes 3 domains: participant, predictor, and outcome. The answer to each question is "yes," "probably," "no information," "no," or "may or may not". "possible", and the risk of bias and applicability for each domain was judged to be high, low, or unclear. When all domains in a study were rated as low risk, the overall risk of bias was low; when one or more domains were rated as high, the overall risk of bias was high. A study was considered to have an unclear risk of bias when one or more domains were considered unclear, while other domains were assessed as low risk. When there was disagreement, the disagreement was resolved through discussion with the panel.
2.6. Data synthesis
We conducted a descriptive analysis of the included studies using a tabular format to synthesize key features of the predictive model. The table summarizes the data sources of the included studies, including characteristics such as country of study, participants, study design, and sample size. Additionally, it outlines the candidate predictors for the predictive model, methods for processing continuous predictors, techniques for predictor selection, approaches to handling missing data, methods for constructing the model, presentation of the model, and strategies for model validation. Finally, we summarized that the predictive performance of the postpartum stress urinary incontinence prediction model is assessed from two main perspectives: discrimination and calibration. We use the area under the curve or C-statistic to assess the discrimination of the prediction model, when the area under the curve or C-statistic >0.7 indicates that the postpartum stress urinary incontinence prediction model has good discrimination. The calibration of the postpartum stress urinary incontinence prediction model was assessed using calibration plots or the Hosmer-Lemeshow test, and when the Hosmer-Lemeshow test P > 0.05 or the calibration slope was close to 1 indicated a good model fit. Due to the heterogeneity of the included studies (participants, predictors, clinical background), we decided not to perform a meta-analysis first.
3. Results
3.1. Search results
We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 flowchart to describe the retrieval process and results of this systematic review (see Fig. 1). We initially retrieved 3063 records and firstly removed 1099 duplicates and then remained 1964 studies. We further screened researches by title and abstract, and after removing 1929 irrelevant or ineligible records, the remaining 35 documents were further evaluated by full-text reading. In the subsequent evaluation we excluded 28 studies that did not meet the inclusion criteria and finally included and analyzed 8 studies with a total of 10 predictive models for analysis [[21], [22], [23], [24], [25], [26], [27]].
Fig. 1.
Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flowchart of literature search and selection.
3.2. Characteristics of model development for inclusion in the study
3.2.1. Summary of design and participant characteristics of included studies
Table 1 summarizes the design and participant characteristics of a total of 10 prediction models from the 8 studies we included. They were published in 2012–2013, of which 5 studies (n = 62.5 %) [22,23,[25], [26], [27]] were developed based on Chinese populations, 1 study (n = 12.5 %) [24]was developed based on US populations, and 2 study (n = 14.2 %) [21,28] was developed in Brazil. Most of the predictive models were developed based on cohort studies, of which 4 studies were (n = 50 %) [22,[24], [25], [26]] prospective cohort studies, 2 studies (n = 25 %) [23,27] were retrospective cohort studies, 1 study (n = 12.5 %) [28] was a case-control study, and 1 study (n = 12.5 %) [21]was a cross-sectional study. 3 studies (n = 37.5 %) [[22], [23], [24]]were multicenter studies, and 5 studies (n = 62.5 %) [21,[25], [26], [27], [28]]were single-center studies. 3 studies (n = 37.5 %) [21,23,24] targeted only primigravid women, 4 studies (n = 50 %) [[25], [26], [27], [28]] included primigravid women together with menstruating women, and in addition, 1 study (n = 12.5 %) [22] had separate prediction models for primigravid women and menstruating women. Their prediction times ranged from 42 days to 7 months postpartum.
Table 1.
Overview of basic data of the included studies.
| Author (year) | Country | Study design | Participants | Data source | Timing of Prediction | Main outcome | Outcome Measurement Methods | Number of participants modeling/number of events (EPVs) | Number of candidate predictors/predictors included in the final model | Predictors |
|---|---|---|---|---|---|---|---|---|---|---|
| Jelovsek (2013) | America | Prospective cohort study | Primiparous women who delivered a child at 37 weeks or more of pregnancy | the publicly available data from the Pelvic Floor Disorders Network Childbirth and Pelvic Symptoms Study (CAPS) | 6 months postpartum | PSUI | Medical, Epidemiological, and Social Aspects of Aging Questionnaire (MESA) | 921/237 | antepartum models：-/7 antepartum plus L&D models：-/13 |
antepartum models：race、UI before pregnancy、UI during pregnancy、Pre-pregnancy BMI、Predelivery BMI、Maternal age、Planned Mode of Delivery antepartum plus L&D models：Duration of second stage of labor、Urinary incontinence before pregnancy、Urinary incontinence during pregnancy、Mode of delivery 、Episiotomy、Perineal laceration degree、infant weight、Fetal head circumference、Fetal head position at crowning、Race、Pre-pregnancy BMI、Predelivery BMI、Maternal age |
| Leroy (2016) | Brazil | Case-control study | women with up to 90 days postpartum |
the obstetrics outpatient facility of a public tertiary teaching hospital in the state of São Paulo, Brazil | 90 days postpartum | PSUI | International Consultation on Incontinence Questionnaire – Short Form (ICIQ-SF) | 344/77（EPV＞20） | 14/4 | UI during pregnancy、multiparity、gestational age at birth greater or equal to 37 weeks、constipation |
| Xu (2023) | China | Retrospective study | women with singleton, full-term, and vaginal deliveries | a tertiary specialized hospital in Shanghai, China | 42–100 days postpartum | PSUI | International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form (ICIQ-UI-SF) | 2441/431（EPV＞20） | 17/5 | Maternal age, parity, second stage of labor, infant weight, and forceps delivery |
| Chen (2020) | China | Prospective cohort study | singleton pregnant women without incontinence before pregnancy who were 18 years or older | 2 hospitals in Shenzhen, China | 6 weeks postpartum | PSUI | Question the mother about whether she experiences involuntary urine loss during physical activity or when coughing or sneezing | primiparous women:393/49(EPV＞20） multiparous women:334/67（EPV＞20） |
Model of primiparous women:13/4 Model of multiparous women:13/4 |
Model of primiparous women: Maternal age, abortion/miscarriage history, SUI during pregnancy, mode of delivery Model of multiparous women: pre-pregnancy BMI, abortion/miscarriage history, SUI during pregnancy, mode of delivery |
| Cheng (2022) | China | Retrospective study | Women 42 days after delivery | postpartum health care clinics in 3 hospitals in Fuyang, China | 42 days postpartum | PSUI | According to the Guidelines, “urine overflows when abdominal pressure increases in a variety of degrees, such as laughing, coughing, sneezing or walking, and the urine flow stops immediately when the pressure is stopped” | 360/90(EPV 10–20) | 23/7 | Gravidity、Residence、Occupation during pregnancy、Education、Monthly income、Mode of delivery、Oxytocin |
| Wang (2022) | China | Prospective cohort study | Women over 18 years of age with singleton and full-term pregnancies | the obstetric wards of a tertiary maternity hospital in Hangzhou, a provincial capital city in eastern China | 6–8 weeks after birth | PSUI | International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form (ICIQ-UI-SF) | 830/188(EPV＞20) | 13/7 | Mode of delivery, UI before pregnancy, UI during pregnancy, place of residence, feeding pattern, parity, Maternal age |
| Liu (2022) | China | Prospective cohort study | Singleton pregnant women aged ≥18 years | birth in the Department of Obstetrics and Gynecology at Beijing Friendship Hospital | 6–12 weeks postpartum | PSUI | Questions about urine leakage from activities such as coughing, sneezing, or running in women | 255/105(EPV＞20) | 10/5 | Maternal age, parity, vaginal delivery (mode of delivery), bladder neck descent (BND), angle of internal urethral orifice funnel |
| Baracho (2012) | Brazil | Cross sectional study | primiparous women aged 18–35 years old who underwent a singleton vertex presentation vaginal delivery at term, had no history of abortion, and attended a postpartum visit at 5–7 months after delivery | Dr. David Capistrano's birthing center of the Sofia Feldman Hospital (Belo Horizonte, Minas Gerais, Brazil) | 5–7months postpartum | PSUI | International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form (ICIQ-UI-SF) | 192/32(EPV 10–20) | 15/4 | PFM strength, prior SUI, infant weight, and new onset of SUI in pregnancy |
PSUI: Postpartum stress urinary incontinence; “-”, not reported.
Regarding the predicted outcome, although the measurement tools used varied among the studies, the measurements were made by asking the patients in different ways (telephone, interview or questionnaire, etc.) whether they had involuntary urine outflow during the postpartum period of increased intra-abdominal pressure, and the predicted outcome was the same in all studies.
All studies reported the sample size required to construct the predictive model, which ranged from 192 to 1441, and the number of participants with postpartum stress incontinence ranged from 32 to 431. With the exception of Jelovsek's study [24], all studies reported the number of candidate predictors, and the number of predictors ultimately included in each predictive model ranged from 4 to 13, covering 29 different prenatal, intrapartum, and postpartum factors. The most common predictor was UI during pregnancy (n = 7, 24.13 %), followed by mode of delivery (n = 6, 20.69 %), Maternal age (n = 5, 17.24 %), parity (n = 5, 17.24 %), and UI before pregnancy (n = 4, 13.79 %). Some predictors that were not known antenatally were also included in the existing prediction model, such as newborn weight (n = 3, 10.34 %), duration of the second stage of labor (n = 2, 0.69 %), use of oxytocin (n = 1, 0.34 %), and feeding mode (n = 1, 0.34 %).
The number of events per variable (EPV) in the prediction model, i.e., the number of outcome events relative to the number of candidate predictors, was greater than 10 in all studies (n = 7, 87.5 %) except for the one of Jelovsek [24], which could not calculate the number of events per variable because the number of candidate predictors was not reported.
3.2.2. Derivation of predictive models
For model development, all studies used logistic regression to construct predictive models except Baracho [21] who used Classification and Regression Tree. For the selection of predictors prior to modeling the most common (n = 6, 75 %) was univariate analysis, in Jelovsek's study [24] the selection of predictors was based on the published literature and biological plausibility as well as their availability in the existing predictors was based on the published literature and biological plausibility as well as their availability in the availability of existing labor and pelvic symptom study datasets, the selection of candidate predictors in wang's study [26] (n = 12.5 %) was based on literature review, statistical analysis, and clinical reasoning. Although screening predictors based on univariate significance tests was common. This bias arises when predictors included in the model exhibit spurious associations with outcomes, increasing the risk of overfitting. The most common method (n = 7, 87.5 %) for predicting the identification of the final predictors to be included in the predictive modeling was multivariate logistic regression, with 3 studies (n = 37.5 %) using backward stepwise regression, and the remaining 4 studies (50 %) did not report on the specific method used.
Regarding the treatment of continuous variables, all included studies included continuous variables. Notably, Jelovsek's study [24](n = 1, 12.5 %) did not report the treatment of continuous variables., Leroy's study [28] (n = 1, 12.5 %) divided the continuous variables into two or categorized them. The rest of the studies (n = 6, 75 %) divided some of the continuous variables into two or categorized them. Regarding the treatment of missing data, 3 studies (n = 37.5 %) did not report on the treatment of missing data, and the rest of the studies (n = 62.5 %) excluded participants containing missing data, whereas the simple deletion of participants with missing values may produce ineffective predictive performance when developing and validating predictive models.
7 of the incoming studies (87.5 %) reported the presentation of the predictive model, with six studies using a Nomogram representation and 1 study (n = 12.5 %) demonstrating a Classification and Regression Tree.
3.3. Performance and validation of predictive models
3.3.1. Performance of the predictive model
Table 2 demonstrated the performance and validation of the predictive model. The discriminability of predictive models was usually evaluated using the C-index or the Area Under Curve (AUC, Area Under Curve). In model development, except for Leroy's study [28], 7 studies (87.5 %) reported discrimination metrics with AUC or C-index ranging from 0.680 to 0.850, indicating that most of the predictive models had good discrimination. 6 studies (n = 75 %) reported calibration of the predictive models, and all showed good calibration.
Table 2.
Overview of the information of the included prediction models.
| Author (year) | Continuous variable processing method | Missing data handling | Development methodology | Predictor selection prior to modeling | Predictor selection prior to modeling | Model presentation | Discrimination (AUC/C-statistic) Development/Validation |
Calibration | Internal validation | External validation |
|---|---|---|---|---|---|---|---|---|---|---|
| Jelovsek (2013) | – | Exclusion of cases of missing data | logistic regression | based on the published literature and biologic plausibility as well as their availability in the available CAPS data | logistic regression | Nomogram | Development: antepartum models：0.690 antepartum plus L&D models：0.680 |
Brier score calibration plots | bootstrapping cross-validation |
– |
| Leroy (2016) | Dichotomized or categorized | – | logistic regression | Univariable analysis | logistic regression | – | – | – | – | – |
| Xu (2023) | Continuous variables except for Education、Gravidity、Parity、Perineal lacerations | Exclusion of cases of missing data | logistic regression | Univariable analysis (P < 0.2) | backward stepwise regression | Nomogram | Development:0.850(95%CI:0.74–0.85) Validation:0.830(95%CI:0.79–0.87) |
calibration curve | bootstrapping cross-validation |
Yes, temporal validation |
| Chen (2020) | Continuous variable | Exclusion of cases of missing data | logistic regression | Univariable analysis | backward stepwise regression | Nomogram | Development: Model of primiparous women:0.763（95%C：0.693–0.833） Model of multiparous women:0.783（95 % CI：0.726–0.841） Validation: Model of primiparous women:0.762 (95 % CI, 0.693–0.832) Model of multiparous women:0.781 (95 % CI, 0.723–0.839) |
Hosmer-Lemeshow test | bootstrapping | – |
| Cheng (2022) | Continuous variables except for Age、Gravidity、Education、Monthly income per capita、Number of fetuses、Newborn's weight、 | – | logistic regression | Univariable analysis (P < 0.05) | logistic regression | Nomogram | Development:0.798 (95 % CI: 0.749–0.846) | – | – | – |
| Wang (2022) | Categorized except for Age、BMI before pregnancy | – | logistic regression | based based on literature review, statistical analysis, and clinical reasoning | backward stepwise regression | Nomogram | Development:0.757 (95 % CI: 0.72–0.80) Validation:0.759(95 % CI: 0.70–0.82) |
Hosmer-Lemeshow test、calibration curve | bootstrapping | Yes, temporal validation |
| Liu (2022) | Continuous variables except for Parity | Exclusion of cases of missing data | logistic regression | Univariable analysis (P < 0.05) | logistic regression | Nomogram | Development:0.883(95 % CI: 0.839–0.926) Validation:0.807(95%CI:0.723–0.891) |
calibration curve | – | Yes，− |
| Baracho (2012) | Continuous variables except for Perineal lacerations with suture | Exclusion of cases of missing data | Classification and Regression Tree | Univariate analysis (P < 0.25) | Classification and Regression Tree | Classification and Regression Tree | Development:0.840(95 % CI:0.77–0.92) | – | – | – |
“-”, not reported.
3.3.2. Validation of predictive models
Half of the eight included studies (n = 4, 50 %) developed predictive models that underwent internal validation. Jelovsek and Xu's study [24,27] used bootstrapping and cross-validation for validation respectively, while Chen and Wang's study [22,26] used bootstrapping for validation. Of the 8 studies, only 3 performed external validation, with both Xu [27] and Wang [26] using temporal validation, while Liu's study [25], while reporting external validation, did not report the specific method used to perform external validation.
3.4. Results of quality assessment
Table 3 summarized the risk of bias and applicability of the 8 included studies. All of the included studies were rated as high risk of bias, suggesting methodological problems in the development or validation of the prediction models of the included studies.
Table 3.
PROBAST results of included studies.
|
Author (year) |
ROB |
Applicability |
Overall |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Study type | Participants | Predictors | Outcome | Analysis | Participants | Predictors | Outcome | ROB | Applicability | |||
| Jelovsek (2013) | B | – | – | – | - | – | – | – | - | – | – | |
| Leroy (2016) | A | – | _ | – | - | – | – | – | – | – | - | – |
| Xu (2023) | B | – | – | – | – | - | – | – | – | – | – | – |
| Chen (2020) | B | – | – | – | _ | – | – | – | ？ | ? | – | ? |
| Cheng (2022) | A | _ | – | – | _ | – | – | – | – | – | - | – |
| Wang (2022) | B | – | – | – | _ | – | – | – | – | – | - | – |
| Liu (2022) | B | – | – | – | _ | – | – | – | – | – | – | – |
| Baracho (2012) | A | _ | – | – | _ | – | – | – | _ | – | - | – |
PROBAST = Prediction model Risk Of Bias Assessment Tool; ROB = risk of bias.
A indicates “development only”; B indicates “development and validation in the same publication”.
– indicates low ROB/low concern regarding applicability; - indicates high ROB/high concern regarding application; ? indicates unclear ROB/unclear concern regarding applicability.
3.4.1. Risk of bias for inclusion of studies
In the Participants domain, 3 studies (n = 37.5 %) [21,23,27]were rated at high risk of bias, mainly due to the use of inappropriate data sources, retrospective cohort studies collecting data at a relatively long time from the prediction of the outcome, which may lead to differences in the accuracy or completeness of the information, poorer data quality, and unmeasured predictive factors that may contribute to bias, and non-nested case-control studies are unable to calculate absolute risk, resulting in a high risk of bias due to misestimation of predictive model intercepts or baseline risk. In the Predictors domain, 1 study was rated at high risk of bias, mainly because of non-nested case-control studies, which collect predictors when the outcome is known, the inability to use blinding, and the susceptibility of the measurement process to outcome.
In the Predictors domain, 1 study (n = 12.5 %) [28] was rated at high risk of bias, mainly because of non-nested case-control studies, which collect predictors when the outcome is known, the inability to use blinding, and the susceptibility of the measurement process to outcome.
In the Outcome domain, 4 studies (n = 50 %) [21,23,24,27]were rated at high risk of bias, mainly because the time between study predictor assessment and outcome determination was too long in 2 studies [21,24], and 2 retrospective studies [23,27] and 1 cross-sectional study [21] were unable to control for the quality of predictor assessment. 2 studies [25,28] were rated as unclear, mainly because it was not reported in the study whether the predictor information was clear to the assessor at the time of outcome determination.
In the Analysis domain, all 8 (n = 100 %) of the included studies were rated at high risk of bias in the analysis domain for the following main reasons(1) mishandling of continuous predictors (e.g., transforming age, parturition, education, etc.): into ≥2 categories or using different cut points for categorical variables; (2) failing to articulate how to deal with missing data or mishandling of missing data; and (3) Screening of predictors based on univariate analysis methods; although such screening of predictors based on univariate significance tests is common, predictor selection bias occurs when predictors selected for inclusion in multivariate modeling have a large, but spurious, association with the outcome, potentially increasing the likelihood of overfitting; (4) Some studies did not conduct performance assessments or did not adequately assess the performance of the predictive models (for the predictive model was assessed only for discrimination, not for calibration); (5) some studies did not use internal validation or performed internal validation using a randomized split method that included only data.
3.4.2. Assessment of applicability of included studies
Generally, in the area of applicability, 4 studies (n = 50 %) [21,[23], [24], [25]] had a high risk of applicability. In the field of Participants, the main concern was that some of the studies (n = 3, 37.5 %) [21,23,24] were developed to include only primigravid women or only vaginal births and did not focus on all types of postpartum women, and the predictive models developed and validated in the primigravid or only vaginal birth population may not be applicable to the general population. Regarding in the Predictors domain, the main concern was about predictors, and 2 studies (n = 25 %) [21,25] may limit the push of prediction models because predictors were not easily available. In the domain of Outcomes, the 2 studies (n = 25 %) [21,24] had too long a time interval between predictions of postpartum stress incontinence, which may have passed the high-risk time period for postpartum stress incontinence and may be less applicable.
4. Discussion
4.1. Interpretation
The prevalence of postpartum stress incontinence was high, and several studies have demonstrated that the occurrence of postpartum stress incontinence can be prevented and that prevention is the most effective strategy to minimize its occurrence and associated adverse outcomes [29,30]. Postpartum stress urinary incontinence prediction models allow for early identification of high-risk populations and timely implementation of interventions for pregnant women to reduce the incidence of postpartum stress urinary incontinence. A total of 8 studies and 10 predictive models were included in our systematic review for analysis, and most of the prediction models were more optimistic in terms of differentiation, but the calibration of the included studies was poorly assessed. Existing prediction models have predictions from 6 weeks postpartum to 7 months postpartum, and the new time horizon, which extends the definition of postpartum stress urinary incontinence to 12 months postpartum, challenges existing prediction models, potentially limiting their ability to predict between 7 and 12 months and affecting their accuracy [1]. Therefore, models may need to be updated to cover longer time spans and integrate new variables to improve the comprehensiveness of predictions. In addition, clinical applications may be affected by requiring longer follow-up periods to validate predictions, thus ensuring the validity and clinical relevance of the models in the new time frame.
According to the PROBAST checklist, all of the studies we included were rated at high risk of bias, suggesting significant limitations in the methodology of modeling existing predictive models, which may limit the practical value of predictive models. In addition, we found that all studies included in this systematic review were not strictly reported according to Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) [31]. Notably, Jelovsek and Baracho's study was published before the development of the TRIPOD reporting system and therefore did not follow the TRIPOD guidelines [21,24]. Other studies that were published after the development of the TRIPOD reporting system were also not reported in strict accordance with TRIPOD. This resulted in an opaque process of constructing most of the prediction models, thereby increasing the uncertainty and potential risk of bias in the prediction models.
In our systematic review, there was a high degree of variability in the predictors that were ultimately incorporated into the prediction models due to the differences in the target populations and candidate predictors. In addition to high predictive accuracy, simplicity is equally important for predictive models. The simplicity of a predictive model is mainly reflected in the simplicity of the predictors of the model and the convenience of the evaluation process [32]. In terms of predictors, the higher the number of included predictors, the more likely that overfitting occurs in the process of constructing the model [33], and when there are some predictors in the predictive model that are not easy to obtain, it will limit the generalization of the model [34]. In our systematic review, the number of predictors in the prediction models ranged from 4 to 13, and most of the predictors could be objectively measured and easily collected. The most frequent predictors of postpartum stress urinary incontinence in existing postpartum stress urinary incontinence prediction models were gestational UI (n = 7, 24.13 %), followed by mode of delivery, age at delivery, gestational age, and pre-pregnancy UI, and several studies have confirmed that the above predictors are important risk factors for the development of postpartum stress urinary incontinence [7,35,36]. Some studies ultimately included predictors that may not be helpful in the prevention of postpartum stress urinary incontinence because some indicators are unknown before delivery, such as forceps delivery, oxytocin, degree of perineal tear, breastfeeding, etc. Using them as predictors means that the prediction model is used after delivery, which may be the time when the high-risk period of postpartum stress urinary incontinence has already passed, and is of limited value for the development of prenatal medical care measures. In the future, it is recommended that researchers develop dynamic risk assessment systems that cover the entire period of pregnancy, labor, and postpartum to predict the occurrence of postpartum stress urinary incontinence. In the future, it is recommended that researchers develop a dynamic risk assessment system that covers the entire period of pregnancy, labor and delivery and postpartum period to predict the occurrence of postpartum stress urinary incontinence. In terms of the assessment process, most researchers have transformed predictive models into Nomograms, and Jelovsek et al. have also developed web-based risk calculators [24], which makes the calculation of the prediction model easier and quicker, and the results of the prediction are more readable and easy to be used in the clinical setting [32].
Data sources may affect model performance [18]. Despite the fact that half of the studies used a prospective design, the studies were still rated at high risk of bias, mainly due to poor reporting of the outcome and analysis domains. Outcomes should be measured without knowledge of predictive information; some studies did not consider blinding when determining outcomes and collecting predictors, and the predictive power of models may be overestimated when outcomes are measured without blinding [34]. Secondly, the time interval between predictor assessment and outcome determination is also important; some studies with outcomes determined at a later time according to predictor collection may miss the high-risk time period for disease occurrence, and the performance of the prediction model will be greatly reduced [34].
The predictive models included in this systematic review were modeled using a single approach, with the exception of Baracho's study, which was modeled using Classification and Regression Tree [21], all predictive models were modeled using logistic regression. Machine learning can describe, learn, analyze, and predict data [37], and compared with traditional mathematical models, machine learning has higher sensitivity, specificity, and prediction performance [38]. Researchers at home and abroad have widely applied machine learning in the construction and validation of disease risk prediction models [38,39], but few studies have been reported on the application of machine learning in postpartum stress urinary incontinence risk prediction models. In the future, it is recommended that researchers use machine learning algorithms to construct postpartum stress incontinence risk prediction models to compensate for the shortcomings of traditional prediction models.
Internal validation when constructing predictive models helps to more accurately assess model performance and adjust for overfitting, while external validation is used to test the model's generalizability [40,41]. Of the studies included in this systematic review, only four studies were internally validated and three were externally validated. Most of the studies suffered from inadequate validation, which may affect the reliability of the models and their applicability in different settings. In addition, most of the studies in this systematic review were single-center studies, the target populations included were not unified, the candidate predictors were not the same, and the models were constructed based on maternity in the region, which did not take into account the differences in geography, economic level, education level, and literacy level at the later stage of predicting the model's generalization, which limited the generalizability and accuracy of the models. It is recommended that in the future when constructing the postpartum stress urinary incontinence prediction model, methods such as cross-validation and Bootstrap should be used for internal validation, data outside of the development cohort should be used for external validation of the model, and multicenter study should be used to ensure the generalization ability of the prediction model [34,42].
4.2. Limitations
Our systematic review has some limitations. Firstly, our systematic review included only studies published in English and may have overlooked studies published in other languages. Secondly, because of the heterogeneity of the included studies, we did not perform a meta-analysis. Finally, in terms of the validation of the prediction models, most of the included studies were only internally validated, and only three studies were externally validated, but the lack of large-sample, multicenter external validation, the applicability and stability of the models need to be further validated.
5. Conclusion
A total of 10 prediction models from 8 studies were included in this systematic review, and we found that the postpartum stress urinary incontinence risk prediction models are still in the preliminary stage of development, and most of the prediction models have good predictive performance, but poor modeling methodological quality and high overall risk of bias. However, due to the high overall risk of bias of the prediction models included in the study, it is not recommended to apply the prediction models directly to clinical practice. Clinical practitioners should select available prediction models based on their context and validate them using multicenter, large-sample populations. For future development of postpartum stress urinary incontinence risk prediction models, researchers should follow the PROBAST criteria and TRIPOD reporting guidelines to construct risk prediction models with excellent predictive performance, low risk of bias, and easy clinical application.
6. Explanation of protocol changes
During our actual literature search, we had two researchers searching nine databases, including PubMed, EBSCO, The Cochrane Library, Embase, Web of Science, China National Knowledge Infrastructure（CNKI）, WanFang Data, SinoMed and Technology Journal Database (VIP). In our bias assessment process, we actually three researchers conducted. In the Strategy for data synthesis process, we made a table according to PROBAST tool to synthesize the data. Our study population was included postpartum women. Study types included prospective cohort, retrospective cohort, nested case-control or case cohort studies, cross-sectional designs, randomized trials, and randomized trials. The protocol was changed for this purpose.
Funding
This study was funded by the Start-up Fund Project of the Graduate Student Research and Innovation Support Program, School of Nursing, Binzhou Medical University (HYCX2022⁃013).
Ethical approval
The authors have strictly adhered to ethical standards, ensuring no instances of data fabrication, falsification, plagiarism, misconduct, duplicate publication, redundancy, or any related ethical violations.
Data availability statement
Data included in article/supp. material/referenced in article.
CRediT authorship contribution statement
Liyun Wang: Writing – original draft, Methodology, Data curation, Conceptualization. Minghui Zhang: Methodology, Data curation, Conceptualization. Kaihui Sha: Writing – review & editing, Supervision, Funding acquisition. Yingqiao Qiao: Validation, Investigation, Data curation. Qingqing Dong: Resources, Project administration.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
None.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.heliyon.2024.e37988.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
References
- 1.Doumouchtsis S.K., de Tayrac R., Lee J., Daly O., Melendez-Munoz J., Lindo F.M., Cross A., White A., Cichowski S., Falconi G., Haylen B. An international continence society (ics)/international urogynecological association (iuga) joint report on the terminology for the assessment and management of obstetric pelvic floor disorders. Int. Urogynecol. J. 2023;34(1):1–42. doi: 10.1007/s00192-022-05397-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Subki A.H., Fakeeh M.M., Hindi M.M., Nasr A.M., Almaymuni A.D., Abduljabbar H.S. Fecal and urinary incontinence associated with pregnancy and childbirth. Mater Sociomed. 2019;31(3):202–206. doi: 10.5455/msm.2019.31.202-206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Fitzgerald M.P., Graziano S. Anatomic and functional changes of the lower urinary tract during pregnancy. Urol. Clin. 2007;34(1):7–12. doi: 10.1016/j.ucl.2006.10.007. [DOI] [PubMed] [Google Scholar]
- 4.Rantell A., Veit-Rubin N., Giarenis I., Khullar V., Abrams P., Cardozo L. Recommendations and future research initiative to optimize bladder management in pregnancy and childbirth international consultation on incontinence - research society 2018. Neurourol. Urodyn. 2019;38(Suppl 5):S104–S110. doi: 10.1002/nau.24053. [DOI] [PubMed] [Google Scholar]
- 5.Sangsawang B., Sangsawang N. Stress urinary incontinence in pregnant women: a review of prevalence, pathophysiology, and treatment. Int. Urogynecol. J. 2013;24(6):901–912. doi: 10.1007/s00192-013-2061-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Moossdorff-Steinhauser H., Berghmans B., Spaanderman M., Bols E. Prevalence, incidence and bothersomeness of urinary incontinence between 6 weeks and 1 year post-partum: a systematic review and meta-analysis. Int. Urogynecol. J. 2021;32(7):1675–1693. doi: 10.1007/s00192-021-04877-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Wang K., Xu X., Jia G., Jiang H. Risk factors for postpartum stress urinary incontinence: a systematic review and meta-analysis. Reprod. Sci. 2020;27(12):2129–2145. doi: 10.1007/s43032-020-00254-y. [DOI] [PubMed] [Google Scholar]
- 8.Yang X., Sayer L., Bassett S., Woodward S. The prevalence, associated factors, and impact of urinary incontinence in pregnant and postpartum women in nanjing, China: a cross-sectional study. ASIAN J. UROL. 2023;10(3):337–343. doi: 10.1016/j.ajur.2022.03.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Macarthur C., Glazener C.M., Wilson P.D., Lancashire R.J., Herbison G.P., Grant A.M. Persistent urinary incontinence and delivery mode history: a six-year longitudinal study. BJOG An Int. J. Obstet. Gynaecol. 2006;113(2):218–224. doi: 10.1111/j.1471-0528.2005.00818.x. [DOI] [PubMed] [Google Scholar]
- 10.Macarthur C., Wilson D., Herbison P., Lancashire R.J., Hagen S., Toozs-Hobson P., Dean N., Glazener C. Urinary incontinence persisting after childbirth: extent, delivery history, and effects in a 12-year longitudinal cohort study. BJOG An Int. J. Obstet. Gynaecol. 2016;123(6):1022–1029. doi: 10.1111/1471-0528.13395. [DOI] [PubMed] [Google Scholar]
- 11.Chisholm L.P., Sebesta E.M., Gleicher S., Kaufman M., Dmochowski R.R., Reynolds W.S. The burdens of incontinence: quantifying incontinence product usage and costs in women. Neurourol. Urodyn. 2022;41(7):1601–1611. doi: 10.1002/nau.25007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hägglund D., Walker-Engström M.L., Larsson G., Leppert J. Quality of life and seeking help in women with urinary incontinence. Acta Obstet. Gynecol. Scand. 2001;80(11):1051–1055. [PubMed] [Google Scholar]
- 13.Hammad F.T. Prevalence, social impact and help-seeking behaviour among women with urinary incontinence in the gulf countries: a systematic review. Eur. J. Obstet. Gynecol. Reprod. Biol. 2021;266:150–156. doi: 10.1016/j.ejogrb.2021.10.005. [DOI] [PubMed] [Google Scholar]
- 14.Shim K.H., Choo S.H., Park S.G., Yoo H.J., Choi J.B. Survey on disease insight and prevalence of urinary incontinence in women. Investig. Clin. Urol. 2021;62(5):577–583. doi: 10.4111/icu.20210019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Giugale L.E., Moalli P.A., Canavan T.P., Meyn L.A., Oliphant S.S. Prevalence and predictors of urinary incontinence at 1 year postpartum. Female Pelvic Med. Reconstr. Surg. 2021;27(2):e436–e441. doi: 10.1097/SPV.0000000000000955. [DOI] [PubMed] [Google Scholar]
- 16.Benítez-Andrades J.A., García-Ordás M.T., álvarez-González M., Leirós-Rodríguez R., López Rodríguez A.F. Detection of the most influential variables for preventing postpartum urinary incontinence using machine learning techniques. Digit. Health. 2022;8 doi: 10.1177/20552076221111289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Page M.J., Mckenzie J.E., Bossuyt P.M., Boutron I., Hoffmann T.C., Mulrow C.D., Shamseer L., Tetzlaff J.M., Akl E.A., Brennan S.E., Chou R., Glanville J., Grimshaw J.M., Hróbjartsson A., Lalu M.M., Li T., Loder E.W., Mayo-Wilson E., Mcdonald S., Mcguinness L.A., Stewart L.A., Thomas J., Tricco A.C., Welch V.A., Whiting P., Moher D. The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ Br. Med. J. (Clin. Res. Ed.) 2021;372:n71. doi: 10.1136/bmj.n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Moons K.G., de Groot J.A., Bouwmeester W., Vergouwe Y., Mallett S., Altman D.G., Reitsma J.B., Collins G.S. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the charms checklist. PLoS Med. 2014;11(10) doi: 10.1371/journal.pmed.1001744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Debray T.P., Damen J.A., Snell K.I., Ensor J., Hooft L., Reitsma J.B., Riley R.D., Moons K.G. A guide to systematic review and meta-analysis of prediction model performance. BMJ Br. Med. J. (Clin. Res. Ed.) 2017;356:i6460. doi: 10.1136/bmj.i6460. [DOI] [PubMed] [Google Scholar]
- 20.Wolff R.F., Moons K., Riley R.D., Whiting P.F., Westwood M., Collins G.S., Reitsma J.B., Kleijnen J., Mallett S. Probast: a tool to assess the risk of bias and applicability of prediction model studies. Ann. Intern. Med. 2019;170(1):51–58. doi: 10.7326/M18-1376. [DOI] [PubMed] [Google Scholar]
- 21.Baracho S.M., Barbosa Da Silva L., Baracho E., Lopes Da Silva Filho A., Sampaio R.F., Mello De Figueiredo E. Pelvic floor muscle strength predicts stress urinary incontinence in primiparous women after vaginal delivery. Int. Urogynecol. J. 2012;23(7):899–906. doi: 10.1007/s00192-012-1681-7. [DOI] [PubMed] [Google Scholar]
- 22.Chen L., Luo D., Chen X., Jin M., Yu X., Cai W. Development of predictive risk models of postpartum stress urinary incontinence for primiparous and multiparous women. Urol. Int. 2020;104(9–10):824–832. doi: 10.1159/000508416. [DOI] [PubMed] [Google Scholar]
- 23.Cheng H., Gong F., Shen Y., Ouyang P., Ni R., Gao H. A nomogram model predicting the risk of postpartum stress urinary incontinence in primiparas: a multicenter study. Taiwan. J. Obstet. Gynecol. 2022;61(4):580–584. doi: 10.1016/j.tjog.2022.04.004. [DOI] [PubMed] [Google Scholar]
- 24.Jelovsek J.E., Piccorelli A., Barber M.D., Tunitsky-Bitton E., Kattan M.W. Prediction models for postpartum urinary and fecal incontinence in primiparous women. Female Pelvic Med. Reconstr. Surg. 2013;19(2):110–118. doi: 10.1097/SPV.0b013e31828508f0. [DOI] [PubMed] [Google Scholar]
- 25.Liu W., Qian L. Establishment and validation of a risk prediction model for postpartum stress urinary incontinence based on pelvic floor ultrasound and clinical data. Int. Urogynecol. J. 2022;33(12):3491–3497. doi: 10.1007/s00192-022-05395-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wang X., Jin Y., Xu X., Wang H., Feng S. Development and validation of a predictive model for urinary incontinence postpartum: a prospective longitudinal study. Int. Urogynecol. J. 2022;33(6):1609–1615. doi: 10.1007/s00192-022-05105-9. [DOI] [PubMed] [Google Scholar]
- 27.Xu C., Guo Y., Chi X., Chen Y., Chu L., Chen X. Establishment and validation of a simple nomogram for predicting early postpartum stress urinary incontinence among women with vaginal delivery: a retrospective study. BMC Wom. Health. 2023;23(1) doi: 10.1186/s12905-023-02160-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Leroy L.S., Lúcio A., Lopes M.H. Risk factors for postpartum urinary incontinence. Rev. Esc. Enferm. USP. 2016;50(2):200–207. doi: 10.1590/S0080-623420160000200004. [DOI] [PubMed] [Google Scholar]
- 29.Mørkved S., Bø K. Effect of pelvic floor muscle training during pregnancy and after childbirth on prevention and treatment of urinary incontinence: a systematic review. Br. J. Sports Med. 2014;48(4):299–310. doi: 10.1136/bjsports-2012-091758. [DOI] [PubMed] [Google Scholar]
- 30.Sigurdardottir T., Steingrimsdottir T., Geirsson R.T., Halldorsson T.I., Aspelund T., Bø K. Can postpartum pelvic floor muscle training reduce urinary and anal incontinence?: an assessor-blinded randomized controlled trial. Am. J. Obstet. Gynecol. 2020;222(3):241–247. doi: 10.1016/j.ajog.2019.09.011. [DOI] [PubMed] [Google Scholar]
- 31.Collins G.S., Reitsma J.B., Altman D.G., Moons K.G.M. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod): the tripod statement. BMJ Br. Med. J. (Clin. Res. Ed.) 2015;350(jan07 4):g7594. doi: 10.1136/bmj.g7594. [DOI] [PubMed] [Google Scholar]
- 32.Steyerberg E.W., Vergouwe Y. Towards better clinical prediction models: seven steps for development and an abcd for validation. Eur. Heart J. 2014;35(29):1925–1931. doi: 10.1093/eurheartj/ehu207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hoesseini A., van Leeuwen N., Sewnaik A., Steyerberg E.W., Baatenburg D.J.R., Lingsma H.F., Offerman M. Key aspects of prognostic model development and interpretation from a clinical perspective. JAMA Otolaryngol-Head Neck Surg. 2022;148(2):180–186. doi: 10.1001/jamaoto.2021.3505. [DOI] [PubMed] [Google Scholar]
- 34.Moons K., Wolff R.F., Riley R.D., Whiting P.F., Westwood M., Collins G.S., Reitsma J.B., Kleijnen J., Mallett S. Probast: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann. Intern. Med. 2019;170(1):W1–W33. doi: 10.7326/M18-1377. [DOI] [PubMed] [Google Scholar]
- 35.Hage-Fransen M., Wiezer M., Otto A., Wieffer-Platvoet M.S., Slotman M.H., Nijhuis-Van D.S.M., Pool-Goudzwaard A.L. Pregnancy- and obstetric-related risk factors for urinary incontinence, fecal incontinence, or pelvic organ prolapse later in life: a systematic review and meta-analysis. Acta Obstet. Gynecol. Scand. 2021;100(3):373–382. doi: 10.1111/aogs.14027. [DOI] [PubMed] [Google Scholar]
- 36.Johannessen H.H., Stafne S.N., Falk R.S., Stordahl A., Wibe A., Mørkved S. Prevalence and predictors of double incontinence 1 year after first delivery. Int. Urogynecol. J. 2018;29(10):1529–1535. doi: 10.1007/s00192-018-3577-7. [DOI] [PubMed] [Google Scholar]
- 37.Deo R.C. Machine learning in medicine. Circulation. 2015;132(20):1920–1930. doi: 10.1161/CIRCULATIONAHA.115.001593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Alanazi H.O., Abdullah A.H., Qureshi K.N. A critical review for developing accurate and dynamic predictive models using machine learning methods in medicine and health care. J. Med. Syst. 2017;41(4):69. doi: 10.1007/s10916-017-0715-6. [DOI] [PubMed] [Google Scholar]
- 39.Ramlakhan S.L., Saatchi R., Sabir L., Ventour D., Shobayo O., Hughes R., Singh Y. Building artificial intelligence and machine learning models : a primer for emergency physicians. Emerg. Med. J. 2022;39(5):e1. doi: 10.1136/emermed-2022-212379. [DOI] [PubMed] [Google Scholar]
- 40.Binuya M., Engelhardt E.G., Schats W., Schmidt M.K., Steyerberg E.W. Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review. BMC Med. Res. Methodol. 2022;22(1):316. doi: 10.1186/s12874-022-01801-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Steyerberg E.W., Harrell F.J. Prediction models need appropriate internal, internal-external, and external validation. J. Clin. Epidemiol. 2016;69:245–247. doi: 10.1016/j.jclinepi.2015.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Kent D.M., Nelson J., Upshaw J.N., Gulati G., Brazil R., Venema E., Lundquist C.M., Park J.G., Mcginnes H.L., Maunder R., Paulus J.K., Van Calster B., Steyerberg E.W., van Klaveren D., Wessler B.S. Patient-Centered Outcomes Research Institute (PCORI); Washington (DC): 2021. Using Different Data Sets to Test How Well Clinical Prediction Models Work to Predict Patients' Risk of Heart Disease. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data included in article/supp. material/referenced in article.